Enabling Sound Open Firmware on Arm® Cortex®-a Based Devices Using Zephyr RTOS_Daniel_Baluta.pdf

1、PUBLICNXP,THE NXP LOGO AND NXP SECURE CONNECTIONS FOR A SMARTER WORLD ARE TRADEMARKS OF NXP B.V.ALL OTHER PRODUCT OR SERVICE NAMES ARE THE PROPERTY OF THEIR RESPECTIVE OWNERS.2023 NXP B.V.J UNE 2 0 2 3Daniel BALUTAENABLING SOUND OPEN FIRMWARE ON ARM CORTEX-A BASED DEVICES USING ZEPHYR RTOS#1PUBLICOV

2、ERVIEW-Audio basics&solution overview-Jailhouse support-Linux driver support-Zephyr support2PUBLICANATOMY OF AN EMBEDDED AUDIO SYSTEM3PUBLICSOUND OPEN FIRMWARE SOLUTION OVERVIEWSecondary coreSOF firmware Application processorHost OS driverIPC/SHMFW imagetplg file/lib/firmware/toolsSecondary coreAP4P

3、UBLICNXP I.MX8MP APPLICATION PROCESSOR 5PUBLICNXP I.MX95 APPLICATION PROCESSOR 6PUBLICWHAT IS SOUND OPEN FIRMWARE?Open Source Audio firmware and SDKBSD/MIT licensed firmware,BSD/GPL licensed Linux driversDeveloped initially for Cadence HIFI DSP familyPlatform agnosticSupports multiple hosts platform

4、s(x86 with Intel,AMD and arm64 with NXP,Mediatek)DSP agnosticGeneric OS interfaceToolsIncludes proprietary/open toolchain and libsIncludes a logging system and runtime debugging capabilitiesUses ALSA interface7PUBLICAUDIO STACK ON LINUXAll devices are managed by LinuxAudio apps are using ALSA interf

5、aceTypical HW IPs involvedDigital Audio InterfaceCodecDMA8PUBLICAUDIO STACK ON LINUX WITH DSPOffload audio processingReal time capabilitiesDSP takes over some IPs managementDedicate one core just for Audio processingIntroduce Messaging Unit for IPC9PUBLIC9PUBLICNXP,THE NXP LOGO AND NXP SECURE CONNEC

6、TIONS FOR A SMARTER WORLD ARE TRADEMARKS OF NXP B.V.ALL OTHER PRODUCT OR SERVICE NAMES ARE THE PROPERTY OF THEIR RESPECTIVE OWNERS.2023 NXP B.V.Jailhouse Hypervisor1 0PUBLICOFFLOADING AUDIO ON A SEPARATE CORE(ARM)Use jailhouse for isolationDedicate one ARM core to run the FirmwarePort firmware on ar

7、m641 1PUBLICJAILHOUSE HYPERVISOR-static partitioning hypervisor-splits existing hardware resources into isolated cells-partition a booted system-Linux starts first-loads jailhouse kernel module-enables root cell-creates the inmate cell-loads zephyr.bin-starts inmate cell1 2PUBLIC1 2PUBLICNXP,THE NXP

8、 LOGO AND NXP SECURE CONNECTIONS FOR A SMARTER WORLD ARE TRADEMARKS OF NXP B.V.ALL OTHER PRODUCT OR SERVICE NAMES ARE THE PROPERTY OF THEIR RESPECTIVE OWNERS.2023 NXP B.V.Sound Open Firmware Linux driver1 3PUBLICSOF LINUX DRIVER ARCHITECTURE1 4PUBLICSOF LINUX DRIVER ALSA COMPONENTSMachine driverPuts

9、 together codec,cpu dai into a sound cardPCM driverHelps implementing ALSA operation(pcm,compress,topology,kcontrols)Generic IPC driverAllows to transfer ALSA commands to the FWHandles FW notifications(e.g audio data consumedDSP platform driverImplements IPC transport protocol based on hardware IPs

10、available1 5PUBLICALSA PCM OPSopenLoads the SOF FW and topologyhw_paramsconfigures IPC PCM paramstriggerstream actions:TRIGGER_STARTioctlcopies audio data buffers1 6PUBLICALSA API INTERACTION1 7PUBLICSOF IPC CUSTOM PROTOCOLfirmware loadtopology loadcustom transport protocolshared memory and doorbell

11、 interruptHost AP sends commandsstream,topology,PM,debug,trace,probeSecondary core sendsstream notificationsFW ready1 8PUBLICSOF UTILITIESsof-loggerUser space application that reads the logs from shared memoryrimagetool for firmware image creation(encapsulate elf in a specific binary format)topology

12、 filesdescribes the Audio componentsBased on this Linux kernel sends commands to the FW to create audio topology componentssof-ctlUsed to send custom configuration to SOF FWEncoded usually as a binary blob1 9PUBLIC1 9PUBLICNXP,THE NXP LOGO AND NXP SECURE CONNECTIONS FOR A SMARTER WORLD ARE TRADEMARK

13、S OF NXP B.V.ALL OTHER PRODUCT OR SERVICE NAMES ARE THE PROPERTY OF THEIR RESPECTIVE OWNERS.2023 NXP B.V.Sound Open Firmware with Zephyr2 0PUBLICSOUND OPEN FIRMWARE DEVELOPMENT DIRECTIONPermissive BSD/MIT licensed codeArchitecture agnostic written in C,initially supporting Cadence Xtensa DSPsHIFI2,H

14、IFI3 DSPs used by IntelHIFI4 DSP used by NXP on i.MX8MP,i.MX8QXP,i.MX8QMFeatures selected at runtime using KconfigSupport for user defined Audio scenariosDynamically loaded pipeline topologies at runtimeModular design initially started with XTOSLimitation:it only supports Xtensa architectureGraduall

15、y switching to ZephyrSupport for Xtensa-Support for arm64 and many more architectures2 1PUBLICSOUND OPEN FIRMWARE ON IMX93_EVK_A55 BOARD-Use Jailhouse to isolate one A55 core to run Zephyr-Enable support for imx93_evk_a55 board-Enable SOF application in Zephyr on imx93_evk_a55 board-Use dts overlay

16、to describe SOF memory regions(sram,sai,edma)-Use Kconfig fragment to annotate default board configuration(e.g DCACHE_LINE_SIZE,etc)-Continue work on completely supporting Sound Open Firmware as a native Zephyr app-Use native API for OS primitives-Use native API for platform drivers-End goal:Sound O

17、pen Firmware will only contain Audio logic-Everything else is provided by Zephyr2 2PUBLICENABLE IMX93_EVK_A55 BOARD SUPPORT WITH ZEPHYR2 3PUBLICENABLE IMX93_EVK_A55 BOARD SUPPORT WITH ZEPHYR2 4PUBLICENABLE IMX93_EVK_A55 BOARD SUPPORT WITH ZEPHYR2 5PUBLICENABLE IMX93_EVK_A55 BOARD SUPPORT WITH ZEPHYR

18、2 6PUBLICENABLE IMX93_EVK_A55 BOARD SUPPORT WITH ZEPHYR2 7PUBLICENABLE IMX93_EVK_A55 BOARD SUPPORT WITH ZEPHYRcpuarm,cortex-a55timerarm,armv8-timergicarm,gicclock control module(CCM)iomuxcnxp,imx-iomuxclpuartnxp,imx-lpuart2 8PUBLICSOUND OPEN FIRMWARE APP USING ZEPHYR ON IMX93 BOARD2 9PUBLICSOUND OPE

19、N FIRMWARE ARCHITECTURESOF Audio ComponentsMixerEqualizerSRCVolumeHostDRCDRC DAIBFormerCAdapterNRASRCGeneric Micro Kernel(XTOS,Zephyr)bootIRQswork queuesmemorytimerthreadsschedulermutexexceptionsmsg queuesatomic servicessemaphoresPlatform driversEDMADummyDMAESAISAIIRQSteerSDMAAudio tasksIPCPipelines

20、SOF HW/IPC driverControl/DataLinuxASoC audio driverRTOS/HV audio driverx86 or ARMHost OS driver3 0PUBLICSOF ARCHITECTURE WITH ZEPHYR RTOSSOF Module withZephyr RTOSSOF Audio ComponentsMixerEqualizerSRCVolumeKWDDRCDRC DAIHostCAdapterNRASRCZephyr RTOS(using wrappers)bootIRQswork queuesmemorytimerthread

21、sschedulermutexexceptionsmsg queuesatomic servicessemaphoresPlatform driversEDMADummyDMAESAISAIIRQSteerSDMAAudio tasksIPCPipelinesSOF HW/IPC driverControl/DataLinuxASoC audio driverRTOS/HV audio driverx86 or ARMHost OS driver3 1PUBLICGENERIC MICRO KERNEL SUPPORT3 2PUBLICSOF ARCHITECTURE TRANSITION P

22、HASESOF Module with Zephyr RTOSSOF Audio ComponentsMixerEqualizerSRCVolumeKWDDRCDRC DAIHostCAdapterNRASRCZephyr RTOS(using native APIs)bootIRQswork queuesmemorytimerthreadsschedulermutexexceptionsmsg queuesatomic servicessemaphoresPlatform driversEDMADummyDMAESAISAIIRQSteerSDMAAudio tasksIPCPipeline

23、sSOF HW/IPC driverControl/DataLinuxASoC audio driverRTOS/HV audio driverx86 or ARMHost OS driver3 3PUBLICSOF ARCHITECTURE FINAL PHASESOF Module with Zephyr RTOSSOF Audio ComponentsMixerEqualizerSRCVolumeKWDDRCDRC DAIHostCAdapterNRASRCZephyr RTOS(using native APIs)bootIRQswork queuesmemorytimerthread

24、sschedulermutexexceptionsmsg queuesatomic servicessemaphoresPlatform drivers(using native API)EDMADummyDMAESAISAIIRQSteerSDMAAudio tasksIPCPipelinesSOF HW/IPC driverControl/DataLinuxASoC audio driverRTOS/HV audio driverx86 or ARMHost OS driver3 4PUBLICDIGITAL AUDIO INTERFACE APINew API introduced in

25、 Zephyrinclude/zephyr/drivers/dai.hHigh level audio driver abstractionNXP IPsSynchronous Audio Interface(SAI)Enhanced Serial Audio Interface(ESAI)3 5PUBLICDMA API AND DRIVERSSOF DMA ops vs Zephyr DMA APIIntroduce DMA suspend/resume in ZephyrAdd host-zephyr.c and dai-zephyr.c in SOFPort EDMA and SDMA

26、 driversdma_mcux_edma already in ZephyrUse device tree to describe DMA devices3 6PUBLICCONTINUE ENHANCE SUPPORT FOR SOF IN ZEPHYRMessaging UnitAlready existing in Zephyr,ipm_imx but needs some adaptation to fit SOF IPCInterrupt SteerAdapt to use interrupt controller API in ZephyrClocksMove clock man

27、agement on FW sidePower domainsMove PM domains on FW sideDecouple Firmware from Host OSStandalone run of firmwarePotential for enabling on i.MX RT series3 7PUBLIC3 7PUBLICNXP,THE NXP LOGO AND NXP SECURE CONNECTIONS FOR A SMARTER WORLD ARE TRADEMARKS OF NXP B.V.ALL OTHER PRODUCT OR SERVICE NAMES ARE THE PROPERTY OF THEIR RESPECTIVE OWNERS.2023 NXP B.V.Q&ANXP,THE NXP LOGO AND NXP SECURE CONNECTIONS FOR A SMARTER WORLD ARE TRADEMARKS OF NXP B.V.ALL OTHER PRODUCT OR SERVICE NAMES ARE THE PROPERTY OF THEIR RESPECTIVE OWNERS.2023 NXP B.V.