Running Zephyr RTOS on Cadence® Tensilica® HiFi 4 DSP_Iuliana Prodan.pdf

1、PUBLICNXP,THE NXP LOGO AND NXP SECURE CONNECTIONS FOR A SMARTER WORLD ARE TRADEMARKS OF NXP B.V.ALL OTHER PRODUCT OR SERVICE NAMES ARE THE PROPERTY OF THEIR RESPECTIVE OWNERS.2023 NXP B.V.J U N E 2 0 2 3Iuliana ProdanRUNNING ZEPHYR RTOS ON CADENCE TENSILICA HIFI 4 DSP#1PUBLICCONTENTHardware overview

2、 of i.MX 8M PlusHiFi4 DSP supportCurrent stateWhats next?Generic Linux and Zephyr communication setupRemoteprocRpmsgMailboxOpenAMPChallengesFuture work2PUBLICHARDWARE OVERVIEW I.MX 8M PLUS3PUBLICHARDWARE OVERVIEW I.MX 8M PLUS4PUBLICHARDWARE OVERVIEW I.MX 8M PLUS Offloading signal processing Migrate

3、neural network workload Leverage 3rdparty software libraries5PUBLICAPPLICATION PROCESSOR HIFI4 DSP INTERACTIONApplication processorloads DSP firmwarestarts the DSPIPC optional during various stagesSecondary CoreApplication processorIPC/SHMZephyrLinuxHiFi4 DSP6PUBLICHIFI4 DSP SUPPORT-CURRENT STATEArc

4、hitectureCPU CoreSoC FamilySoC SeriesSoCDriversBoardXtensa HAL XtensaOptionalMandatorySamples HiFi4 NXP_ADSP IMX8M NXP_IMX8M NXP_ADSP_IMX8M SOF NXP_IMX8M7PUBLICHIFI4 DSP SUPPORT-CURRENT STATEArchitectureCPU CoreSoC FamilySoC SeriesSoCDriversBoardXtensa HAL XtensaOptionalMandatorySamples HiFi4 NXP_AD

5、SP IMX8M NXP_IMX8M NXP_ADSP_IMX8M SOF NXP_IMX8M8PUBLICHIFI4 DSP SUPPORT-CURRENT STATESound Open Firmware(SOF)with ZephyrCustom fw loader&IPC SOF specific APIsApplication processorUser spaceKernel spaceapplicationapplicationsound/soc/custom driverHiFi4 DSP9PUBLICHIFI4 DSP SUPPORT WHATS NEXTGeneric fw

6、 loader&IPCHarness DSP power processingApplication processorUser spaceKernel spaceapplicationHiFi4 DSPfw loaderIPCGeneric frameworkapplicationIPC1 0PUBLICHIFI4 DSP SUPPORT WHATS NEXTGeneric fw loader&IPCOptionalMandatory hello_worldSamples syncronization philosophers openamp_rsc_table what else?Xten

7、sa HAL1 1PUBLICHIFI4 DSP SUPPORT WHATS NEXTOptionalMandatory hello_worldSamples syncronization philosophers openamp_rsc_table what else?Xtensa HALNXP HALExternal Libs-OpenAMP1 2PUBLICGENERIC LINUX AND ZEPHYR COMMUNICATION SETUPAn Introduction to Asymmetric Multiprocessing:When this Architecture can

8、be a Game Changer and How to Survive It-Nicola La Gloria&Laura Nao,Kynetics-ELC 2018-slidesLinux and Zephyr“talking”to each other in the same SoC-Diego Sueiro,Sepura/Embarcados,ELCE 2018-slidesAsymmetric Multiprocessing and Embedded Linux-Marek Novak&Duan ervenka,NXP Semiconductor-ELCE 2017 slidesAs

9、ymmetric Multi Processing with Linux&Zephyr-Arnaud Ferraris,Collabora-linkSecondary CoreApplication processorIPC/SHMZephyrLinuxARM M4ARM M71 3PUBLICGENERIC LINUX AND ZEPHYR COMMUNICATION SETUPGeneric vs specificHow is the application loaded and started on DSP?remoteprocHow does the cores communicate

10、 with one-another?remote processor messaging(rpmsg)mailboxOpenAMPSecondary CoreApplication processorIPC/SHMZephyrLinuxHiFi4 DSP1 4PUBLICREMOTEPROCFramework that allows the different platforms/architectures to control(power on,load firmware,power off)remote processorsOffers services to monitor and de

11、bug the remote processor1 5PUBLICREMOTEPROCimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlInter process communicationSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdpartyrproc ops:start()

12、stop()parse_fw()load()kick()OptionalMandatorydrivers1 6PUBLICREMOTEPROCimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlInter process communicationSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardw

13、are3rdpartyOptionalMandatorydrivers1 7PUBLICREMOTEPROCimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlInter process communicationSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdpartyOption

14、alMandatorydrivers1 8PUBLICdriversREMOTEPROCimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlInter process communicationSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdpartyOptionalMandator

15、y1 9PUBLICdriversREMOTEPROCimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlInter process communicationSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdpartyOptionalMandatory hello_world syn

16、chronization philosopherssamples2 0PUBLICRPMSGA virtio-based messaging bus that allows kernel drivers to communicate with remote processorsCores cooperate using a shared memory-based communicationrpmsgvirtIO/virtqueueshare memoryinter-core interruptsTransport layerMAC layerPhysical layerOpenAMP,rpms

17、g-litevirtIO,vring,virtqueueshare memory,Mailbox,MU2 1PUBLICdriversRPMSGimx_dsp_rprocNXP_HALFw loading and controlremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdpartyOptionalMandatoryrpmsg/virtioIn

18、ter process communicationHiFi4 DSP fwSRAMRX vringTX vringvring buffers2 2PUBLICHardwareRPMSGimx_dsp_rprocNXP_HALFw loading and controlremoteprocInternal peripheral driversUser spaceKernel spaceLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdpartyOptionalMandatoryrp

19、msg/virtioInter process communicationHiFi4 DSP fwSRAMdrivers2 3PUBLICNXP_HALRPMSGimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdpartyOption

20、alMandatoryrpmsg/virtioInter process communicationipm_imxrproc ops:start()stop()parse_fw()load()kick()find_loaded_rsc_table()drivers2 4PUBLICMAILBOXFramework used in inter-processor communication to exchange messages or signals between the host and the remote processorBased on:mailbox controllerthat

21、 is platform dependent;mailbox clientthat oversees the messagesend/receive.2 5PUBLICMAILBOXimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdp

22、artyOptionalMandatoryrpmsg/virtioInter process communicationmailboximx-mailboxdrivers2 6PUBLICMAILBOXimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPha

23、rdware3rdpartyOptionalMandatoryrpmsg/virtioInter process communicationipm_imxmailboximx-mailboxdrivers2 7PUBLICMAILBOXimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlSRAMremoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUA

24、RT communityNXPhardware3rdpartyOptionalMandatoryrpmsg/virtioInter process communicationipm_imxmailboximx-mailboxdrivers2 8PUBLICOPENAMPOpen Asymmetric Multi-processing(OpenAMP)framework provides:life cycle management(remoteproc)inter processor communication capabilities(rpmsg)compatibility with upst

25、ream Linux remoteproc,rpmsg and VirtIO componentsuses Libmetal as an abstraction layer to access shared memoryApplication processorOpenAMPUser spaceKernel spacerprocapplicationapplicationvirtiorpmsgvirtioHiFi4 DSPrpmsg2 9PUBLICdriversOPENAMPimx_dsp_rprocHiFi4 DSP fwNXP_HALFw loading and controlSRAMr

26、emoteprocInternal peripheral driversUser spaceKernel spaceHardwareLinuxApplication processorZephyrHiFi4 DSPMessaging UnitUART communityNXPhardware3rdpartyOptionalMandatoryrpmsg/virtioInter process communicationipm_imxmailboximx-mailboxOpenAMP openamp_rsc_table sample3 0PUBLICGENERIC LINUX AND ZEPHYR

27、 COMMUNICATION SETUPMain-Linuxuboot loads and stars the Linux Kernelremoteproc loads and starts the remote processor(on demand)rpmsg driver creates virtqueues and endpointsnotifies remote processorrpmsg driver waits for name service announcementsend/receive messagesZephyr bootsapp sends fw ready rep

28、lyrpmsg app creates virtqueueswaits for link to be upapp creates endpoints and sends name service announcementRemote-Zephyrsend/receive messagesOptionalMandatory3 1PUBLICGENERIC LINUX AND ZEPHYR COMMUNICATION SETUPMain-Linuxuboot loads and stars the Linux Kernelremoteproc loads and starts the remote

29、 processor(on demand)rpmsg driver creates virtqueues and endpointsnotifies remote processorrpmsg driver waits for name service announcementsend/receive messagesZephyr bootsapp sends fw ready replyrpmsg app creates virtqueueswaits for link to be upapp creates endpoints and sends name service announce

30、mentRemote-Zephyrsend/receive messagesOptionalMandatory VIRTIO_RPMSG_F_NS Enabled,by default Used mostly for demos3 2PUBLICCHALLENGESDocumentation for remoteproc/rpmsg could be improvedLinux remoteprocfor i.MX,applications are written to IRAM which has a limitation of 4bytes writesfor IPC,find_loade

31、d_rsc_table()is mandatoryZephyr IPCshared memory for core communication must be large enough for both vrings and buffersMessaging Unit driver must be correctly initialized,and all interrupts enabledOpenAMPwhen reading the status from resource table structure,the DCache must be invalidated,otherwise the status is never updated3 3PUBLICFUTURE WORKUpstreamAdd or enable new samples in Zephyr using DSP APIBenchmark DSP resultsUse generic loader for other applications(e.g.SOF)3 4PUBLICTHANK YOUQuestions?3 5PUBLIC