Sound Open Firmware. Liam Girdwood - Intel

Transcription

1 Sound Open Firmware Liam Girdwood - Intel

2 Sound Open Firmware is an open source audio digital signal processing frmware and driver infrastructure. 2

3 SOF Goals 1 DSP architecture and platform agnostic. Separate architecture and platform code like Linux. Abstract interfaces for architecture APIs (again like Linux). Host AP architecture and host AP OS agnostic. Drivers are generic and not coupled to any host architecture. Nothing in SOF that couples frmmare or lom level driver to any OS. Toolchain freedom Fully support open source tools GCC, Octave, M4, Qemu Also support proprietary tools - Cadence xcc and emulator, Matlab. Permissive licensing of all frmmare and most SDK code. 3

4 SOF Goals 2 Freedom for users to defne nem audio processing pipelines using opensource components and tools. Freedom for users to develop and integrate proprietary audio processing algorithms. All development done in public. Support for several CI systems to best match device and infrastructure under test. 4

5 SDK Birds Eye View 1 SOF Source Code Xtos OS Zephyr Firmmare source OS NuttX OS Android HAL XML(*.xml ) ITT OR Topology fle (*.conf) alsaplg m4 topology fle (*m4) m4 rmbox / trace Tool debugfs Qemu DSP & Host Emulator OR debug SOF Firmware Cadence Emulator Cadence Compiler ELF Image ELF Image Crosstool-NG Tool Chain Topology Metadata /lib/firmware/intel SOF Linux Driver Audio DSP SOF Firmware rimage Tool Signed Image ALSA SoC Topology IPC/Mailbox Unsigned Image Signing Tool *Note: Docker contains only the open source components Legend SOF Source Tool 3 rd Party Tool SDK Binary Image Optional Config Data SDK diagram complete SDK 5

6 SDK Birds Eye View 2 Algorithm Performance Firmmare Criteria source Matlab GNU Octave ELF Image Coefficients Alsa kcontrol SOF Linux Driver Audio DSP SOF Firmware *Note: Docker contains only the open source components SOF Test Bench Test Algorithm. Firmmare source IPC/Mailbox Legend SOF Source Tool 3 rd Party Tool SDK Binary Image Optional Config Data SDK diagram complete SDK 6

7 Firmware Overview Architecture agnostic implementation in C mith some assembler. Permissive BSD/MIT licensed code. Build time selection of features based on target (moving to use Kconfg). Scales domn to ~40kB footprint (text, data and bss). Support for user defned audio processing pipeline topologies. Dynamically loaded pipelines at runtime. Statically stitched into image for host less operation. Allom general compute and signal processing. Not just an audio FW. Image generation tools provided to Convert ELF to device specifc frmmare image formats. Sign frmmare images mith PKCS1o.5 7

8 Firmware Architecture X86 or ARM Host DSP or Discrete Chip mixer mux SOF Audio Components volume EQ ASRC buffer queue AEC/NS speaker protection splitter SRC tone Linux ASoC Audio Driver RTOS/HV Audio Driver SOF HW/IPC driver Control Data Audio Tasks IPC Configurable pipelines Generic Micro Kernel (XTOS, Zephyr, FreeRTOS, NuttX options) boot IRQs timer msg queues schd threads work queues mutex atomic services memory exceptions semaphores Platform Drivers DMA HDA I2C DMIC GNA SPI SSP USB UART GPIO 2 of 2 Firmware diagram complete diagram 8

9 Driver Overview Architecture agnostic. Dual GPL/BSD licensed code. Hardmare and physical IO abstraction, so can mork equally mell mith local MMIO and remote SPI based DSPs. Supports virtualisation through VirtIO backend and frontends (using ACRN). Driver has no hard coded assumptions about DSP pipelines or components. Topologies loaded from FS. Component confgurations can be runtime updated via ALSA kcontrols. 9

10 Driver Architecture ASoC Machine Driver codec integration board integration HW config ASoC PCM Driver topology PCMs Kcontrols DAPM Generic IPC Driver mixer stream PM pipeline IPC Audio Data DSP or Discrete Chip DSP Platform Driver doorbell mailbox IRQ code loader IO PM Physical IO layer can currently be either PCI, SPI, MMIO or VirtIO. 2 of 2 Driver diagram complete diagram o0

11 Pipeline Architecture A pipeline is a set of audio processing components and bufers that share common scheduling. Pipelines can currently be defned using M4 in a fem lines of code. Plans for GUI. m4 topology fle (*m4) M4 Topology fle (*.conf) alsaplg Topology Binary EQ coefficients loaded via ALSA binary kcontrols Standard ALSA volume kcontrols oo

12 Continuous Integration - Travis Travis CI & SOF Source Firmmare source Code m4 topology fle (*m4) Crosstool-NG Tool Chain Topology fle (*.conf) alsaplg Travis Github PR CI ELF Image ELF Image rimage Tool SOF Slim Driver Github Unsigned Image Signing Tool Signed Image Qemu DSP Emulator SOF Source Legend Tool 3 rd Party Tool SDK Binary Image *Note: Docker contains only the open source components Optional Config Data SDK diagram complete SDK o2

13 Continuous Integration - 0day 0day SOF Source Firmmare source Code m4 topology fle (*m4) Crosstool-NG Tool Chain Topology fle (*.conf) alsaplg Travis Github PR CI ELF Image ELF Image rimage Tool SOF Driver Github Unsigned Image alsabat Signing Tool Signed Image SOF Source Legend Tool 3 rd Party Tool SDK Binary Image *Note: Docker contains only the open source components Optional Config Data SDK diagram complete SDK o3

14 SOF Platforms Vendor Platform DSP Status Generic Host PC Host PC Testbench Qemu Host PC Qemu targets Currently only has support for xtensa based devices. Intel Baytrail, Cherrytrail, Brasswell 1 * Xtensa HiFi2EP Upstream, support for upto 6 I2S DAIs. Intel Intel Intel Haswell, Broadwell Apollolake, Geminilake Cannonlake, Whiskylake 1 * Xtensa HiFi2EP Upstream, need I2S integration for some codecs. 2 * Xtensa HiFi3 Upstream, support for I2S, DMIC, HDMI and HDA. 4 * Xtensa Hifi4 Upstream, support for I2S, DMIC, HDMI and HDA. Intel Suecreek HAT on Rasberry PI 2 * Xtensa HiFi4 Upstream core, boot over SPI WiP Xiaomi ARM M4 CEVA Teaklite DSP Upstreaming WiP. o4

15 SOF Processing Components (opensource) Name Generic C available SIMD support Status Volume Yes Xtensa HiFiEP, HiFi3 Upstream, can process 1 8 channels, 16, 24 and 32bit formats. SRC Yes Xtensa HiFiEP, HiFi3 Upstream. 24 and 32bit formats. FIR polyphase optimized M/N and M/N x O/P fractional rate conversions. Configuration tools upstream. FIR Yes Xtensa HiFiEP, HiFi3 16/24/32 bit formats, up to 192 taps. Configuration tools upstream. IIR Yes Planned 16/24/32 bit formats, up to 11 biquads or 22th order. Configuration tools upstream. o5

16 History Year ending Inception Platforms: Bay Trail Features: Qemu Xtensa Firmware open sourced Platforms: Cherry Trail and Braswell Features: Qemu host SOF 1.0 public release Driver open sourced Platforms: Apollo Lake, Haswell, Broadwell Features: BSD/GPL driver, topologies Became Linux Foundation project Driver upstream Platforms: Cannon Lake, Ice Lake, Sue Creek, TeakLite Features: Docker build, SOF on host o6

17 SOF Future Plans Feature Status GDB stub and tunnel. Initial firmware and kernel code in developer repository. Upstream in RPMSG Code complete, now upstreaming to SOF and Linux Loadable firmware modules Initial code in github, still WiP. Upstream in CMSIS RTOS Abstraction layer Work in progress. o7

18 Supporting Organizations Xiaomi o8

19 Firmware Preference? Linux already has a preference for opensource drivers over closed source drivers. Should this preference also be extended to opensource frmmare in the future? Selection criteria mould also have to consider :- Features. Stability. Portabilty. Some opensource frmmare could still contain binary blobs. Regulatory reasons (e.g. comply mith FCC regulations). 3rd party IP (e.g some audio processing algorithms). o9

20 Q & A Visit the booth for demo. 20

21 Qemu Heterogeneous Virtualization 2o

22 Baytrail DSP Architecture Instruction Cache Instruction RAM Xtensa HiFi 2EP core. HiFi EP Core OCP (Upstream) to/from IOSF2OCP bridge 96kB Instruction RAM 168kB Data RAM JTAG JTAG Interrupts (from shim) JTAG Trace Logic Interrupt Must be shared Tangier with IA and Xtensa Source: Sonics VMs VLV HiFi EP Core Data Cache EXT I/F Data RAM PIF PIF2AXI Control & Config signals LPE DMA_00 LPE DMA_01 64 bit AXI Master and Slave LPE Shim 4kB RF Mailbox 32 bit OCP Slave OCP Master Writes OCP Master Reads 32 bit OCP Slave OCP Master Writes OCP Master Reads A U D I O S M X 32-bit APB Slave M/N CLK 32-bit APB Slave M/N CLK 32-bit APB Slave M/N CLK SSP0 SSP1 SSP2 2 * DMACs 3 * I2S ports PCI device from host OS Firmware and host share SHIM registers Mailbox memory. 22

23 Heterogeneous Virtualisation X86 Qemu/KVM Guest OS * Audio DSP driver Codec driver Qemu IO Bridge IO, DMA IRQs Xtensa Qemu SOF Firmmare FW DMA driver FW SSP driver Firmware must be debugged alongside driver. Qemu used to virtualise drivers and firmware together. Host side almost real time. DSP side emulated. Audio Userspace FW Audio mixer VM #0 VM #1 x86 GDB On host xtensa GDB On host 23

24 Heterogeneous IO bridging X86 Qemu/KVM PCI BAR 0 /dev/shm Xtensa Qemu Guest OS * PCI BAR o /dev/shm SOF Firmmare IRQ /dev/mqueue VM #0 VM #1 x86 GDB On host xtensa GDB On host 24

25 Zero Copy Virtual Audio 25

26 Zero Copy Audio Virtualisation SOS buffer PHY pages shared with guests. Audio Bufer (on SOS) SOS bufer SG PHY pages Shared audio bufers SOS allocates SG PHY pages for guest audio buffers. Audio DMA copies directly from/to these buffers. Audio driver IPC virtio Qs Audio Driver Audio DMACs Audio HW Guest N Service OS 26