Quality Assurance for Broadcast Audio Contribution over IP Qualitätssicherung für Hörfunk-Live-Schaltungen über IP-Netze Maxim Graubner, Michael Karle, Thorsten Lorenzen Hessischer Rundfunk, IT Systemservice Hörfunk, {mgraubner, mkarle, tlorenzen}@hr-online.de 26. Tonmeistertagung, November 2010
Overview Introduction Broadcast Audio Contribution Switching to the Packet Quality of Service Mechanisms in the End Devices Concept for the Evaluation of Usability IP Reliability Evaluation EBU N/ACIP Compliance Tests Conclusion 2
Introduction: Broadcast Audio Contribution Exchange of professional audio material normally between remote sites and broadcasting stations explicitly for broadcasting to the radio listener Contribution use cases Bi-directional audio communication e.g. interviews and complex live discussions Outside broadcasts (with return channel) e.g. concerts, sports commentary EBU N/ACIP Specification For interoperability between devices of different manufactures Signaling: SIP/SDP, transport: UDP/RTP (VoIP technology) Coding: lin. PCM, MPEG, ITU-T G.711, G.722, Eapt-X 3
Introduction: Problem Description Professional audio contribution links So far established over circuit-switched networks (e.g. ISDN) Future: packet-switched IP-based networks Certain Quality of Service (QoS) provisioning required No IP-service with ISDN-equivalent QoS available or expensive Variable datarate, higher delay, packet jitter, packet loss How do you achieve user satisfaction using an IP transmission? QoS mechanisms in the end devices to conceal missing network QoS Optimized w.r.t. the subjectively expected quality for a use case 4
Introduction: Problem Description How to achieve user satisfaction with IP transmission? QoS Mechanisms in the end devices to conceal missing network QoS Optimized w.r.t. the subjectively expected quality for an use case Contribution use cases Bi-directional audio communication e.g. interviews and complex live discussions Outside broadcasts (with return channel) e.g. concerts, sports commentary delay sensitive vs. quality sensitive Goal: respective best compromise between e2e-delay and signal-quality! 5
Quality of Service (QoS) Mechanisms in the End Devices Application QoS Tx ACIP device Rx PCM audio stream PCM audio stream Encoder Decoder Packetizer Playout buffer Clock recovery RTP/UDP/IP RTP/UDP/IP Network QoS IP Network variable Bandwidth, Latency, Packet loss 6
Quality of Service (QoS) Mechanisms in the End Devices User satisfaction / Quality of Experience (QoE) Application QoS Tx ACIP device Rx PCM audio stream PCM audio stream Rate control Encoder Decoder Packet loss concealment Packet size control / FEC Packetizer Playout buffer De-jitter control RTP/UDP/IP RTP/UDP/IP Network QoS IP Network variable Bandwidth, Latency, Packet loss 7
Interdependence of Different QoS Parameter at Receiver Network Packet Loss Overall Packet Loss Codec Performance Network Jitter Playout Buffer Perceived Quality Network Delay End-to-end Delay Network Factors Application Factors User Level QoS optimization: compromise between end-to-end delay and signal-quality! 8
Conditions for Reliable Audio Contribution over IP Coding method must provide: Scalability (coding bitrate, audio bandwidth, packet size ) Small coding/decoding delay Packet loss concealment (PLC) Support for a variable playout buffer Standardization of RTP-packetization and SIP/SDP-signalling Codecs designed for reliable IP transmission: MPEG4 AAC (ELD), AMR-WB+, Skype SILK EBU N/ACIP mandatory codecs: G.711, G.722, MPEG-1/2 Layer 2, Linear PCM 9
IP-Codec Evaluation at the Hessischer Rundfunk (hr) Goal: recommendation for a new IP codec infrastructure Test-selection: 20 Devices from different manufactures Candidates derived from the previously performed market survey Requirement: an evaluation concept 10
Concept for the Evaluation of Usability Aspects: - User (subjective) - Measurement - Scientific Foundation: - Requirement profile - hr type test - Network emulation environment IP-Codec Evaluation Concept IP Measurements: - Meaningful - Reproducible - Fair Criteria: - Desired functionality, Ergonomics - Quality and compatibility expectations (e.g. N/ACIP compliance) 11
Evaluation of Usability: IP Reliability Performance tests with direct IP connections Amplification errors, channel swaping, latency and drift, IP jitter For coding methods, IP interface analysis, clock recovery Performance tests with controlled IP Impairments Behaviour concerning: jitter, packet duplication and packet loss For buffer, PLC, FEC Tx-Device IP network emulator Rx-Device Performance tests with emulated IP networks For a realistic picture of transmission quality 12
Evaluation of Usability: IP Network Emulation Audio tests with noise level measurement (German: Störpegelmessung ) Network type const. latency Jitter Max. Packet loss 1. Private Network ("Corporate Network") 20 ms 5 ms 0,01 % 2. Public "Managed Network" 50 ms 10 ms 0,1 % 3. Internet ("good conditions") 100 ms 20 ms 1 % 4. Internet ("bad conditions") 100 ms 30 ms 10 % Tx-Device IP network emulator Rx-Device 13
IP Network Emulator: ZTI NetDisturb 14
IP Reliability Evaluation Results Latency drift of a tested device: 8 ms in 5 min A noise level measurement for network type 2 15
EBU N/ACIP Compliance Tests Infrastructure: SDSL-Internet access Receiver and SIP/STUN-Server from ARD Sternpunkte 16
EBU N/ACIP Compliance Tests: RTP-Audiostream Infrastructure: SDSL-Internet access Receiver and SIP/STUN-Server from ARD Sternpunkte IP packet Coded audio payload Timestamp, codec-infos Port information Address information RTP header UDP header IP header 17
EBU N/ACIP Compliance Tests: SIP/SDP-Signalling Infrastructure: SDSL-Internet access Receiver and SIP/STUN-Server from ARD Sternpunkte Connection establishment and hang up SIP/SDP session IP packet Negotiation of coding parameters Coded audio payload Timestamp, codec-infos Port information Address information RTP header UDP header IP header 18
N/ACIP Compliance Tests: Signalling with SIP/SDP 19
Conclusion Audio Contribution over IP works principally, but problems occur EBU N/ACIP specifies only basics Missing: AAC ELD, FEC, PLC, STUN Further QoS mechanisms in the end devices required So far not widely implemented, not always compatible Usecase-specific optimization necessary to match broadcaster s quality requirements Experiences from the Evaluations Many disappointing results Handling, audio quality, latency, standard s compliance Sophisticated testing before buying always recommended 20
Thank you for your attention. Do you have any questions? 21
Literature S. Church and S. Pizzi. Audio over IP: A Practical Guide to Building Studios with IP. Focal press, 1st edition, Nov. 2009. S. Daniels: 20 Things you should know before Migrating your Audio Links to IP. AES Convention Paper 7651, 126th AES Convention, Mai 2009 EBU Project Group N/ACIP: Audio Contribution over IP - Requirements for Interoperability. EBU Tech 3326rev, Apr. 2008 M. Graubner: QoE Assessment and a Perceptually-Driven QoS Optimization Model for Audio Contribution over IP. Diplomarbeit, TU Darmstadt, Germany, Sep. 2009 A. Raake. Speech Quality of VoIP: Assessment and Prediction. Wiley, Chichester, UK, 2006. P. Rodriguez Altmann und T. Amon: Audio over IP Einblicke. 25. Tonmeistertagung, Nov. 2008, Proceedings 202-208 M. Schnell et al. MPEG-4 Enhanced Low Delay AAC - A New Standard for High Quality Communication. In AES Convention paper 7503, 125th AES Convention, Oct. 2008. 22
QoS Requirements N/ACIP vs. VoIP QoS Parameter ACIP: Interview VoIP End-to-end Delay [ms] < 100 150-400 Audio Bandwidth [khz] 7 12 (20) 3,5-7 IP Datarate [Bit/s] 80k 400k (2M) 20k 80k IP Packetloss - 0,2 % Expected Quality Audio Communication Intelligibility ensured 23