AES67 audio over IP. within SMPTE Peter Stevens. Date of Presentation: 16 th November 2017

Size: px

Start display at page:

Download "AES67 audio over IP. within SMPTE Peter Stevens. Date of Presentation: 16 th November 2017"

Elizabeth Blair
6 years ago
Views:

1 AES67 audio over IP within SMPTE 2110 Date of Presentation: 16 th November 2017 Peter Stevens

2 Topics Introduction to SMPTE Digital Media Standards SMPTE 2110 Relationship to AES67 AES67 History ASE67 Technology Components Interop Plugfests Broadcasting Examples What Next? Summary Acknowledgments to: Andreas Hildebrand of ALC NetworX for adapting some of his ideas and Swedish Radio for their slides and video

3 SMPTE ST 2022 Series Sending digital video over an IP network the Early Years Video formats supported include MPEG-2 and serial digital interface (SDI) Transportation based on RTP (Realtime Streaming Protocol) RFC3550 SMPTE ST FEC for Real-Time Video/Audio Transport Over IP Networks Defines row/column FEC (Forward Error Correction) for IP video streams SMPTE ST Unidirectional Transport of CBR MPEG-2 Transport Streams on IP Networks Encapsulation of MPEG-2 transport streams into IP packets SMPTE ST Unidirectional Transport of VBR MPEG-2 Transport Streams on IP Networks Defines IP packets for variable bit-rate MPEG-2 TS - constrained to have a CBR between PCR messages

4 SMPTE ST 2022 Series Sending digital video over an IP network the Early Years SMPTE ST Unidirectional Transport of Non-Piecewise Constant VBR MPEG-2 Streams on IP Networks Similar to , except removes bit rates constraints SMPTE ST High Bit Rate Media Transport Over IP Networks Expansion of allows larger row/column FEC combinations - support signals > 3 Gbps and beyond SMPTE ST Transport of High Bit Rate Media Signals over IP Networks (HBRMT) Transportation of high bit-rate signals not encapsulated in MPEG-2 transport streams SMPTE ST Seamless Protection Switching of SMPTE ST 2022 IP Datagrams Automatic resilience switching by receiver of identical streams over different routes SMPTE 2022 is an important technology enabling the transition of broadcast systems to IP networking

5 SMPTE ST 2110 Standard Suite Professional Media over managed IP Networks SMPTE ST System Timing and Definitions Defines transport layer & synchronisation (SMPTE 2059, clocks, RTP, SDP, etc) SMPTE ST Uncompressed Active Video Defines payload format for raw video (RFC4175, RTP, SDP, constraints) SMPTE ST Traffic Shaping and Delivery Timing for Video SMPTE ST PCM Digital Audio SMPTE ST AES3 Transparent Transport SMPTE ST Payload definition for SMPTE ST ANC Data

6 SMPTE ST 2059 Standard Suite SMPTE ST Generation and Alignment of Interface Signals to the SMPTE Epoch Alignment points for interface signals (that exist today) Formulae for direct calculation of signals from PTP time Formulae and algorithms for deterministically calculating ST12 Time-address and ST309 date SMPTE ST SMPTE Profile for Use of IEEE-1588 Precision Time Protocol in Professional Broadcast Applications Specific PTP rules required by SMPTE application SMPTE-specific helper metadata Network and SMPTE parameters

7 SMPTE 2110 Relationship Source SMPTE 2110-x Sender Elemental RTP Streams IP Network SMPTE 2110-x Receiver Video Video Video Video Ancillary Audio Encapsulate, Packetise & Timestamp Ancillary Audio Ancillary Audio Accumulate, Decapsulate, & Synchronise Ancillary Audio SDI or non-sdi signals 2110 allows SDI packaged media to be sent over network as groups of individual streams; individual streams not derived from SDI may also be used Received streams can be put back together into SDI, if required, or left as synchronised individual streams Benefit also comes with audio only devices.

SMPTE 2110 Relationship Source SMPTE 2110-x Sender Elemental RTP Streams IP Network SMPTE 2110-x Receiver Video Video Video Video Ancillary Audio Encapsulate, Packetise & Timestamp Ancillary Audio

8 SMPTE 2110 Relationship Source SMPTE 2110-x Sender Elemental RTP Streams IP Network SMPTE 2110-x Receiver Video Video Video Video Ancillary Audio Encapsulate, Packetise & Timestamp Ancillary Audio Ancillary Audio Accumulate, Decapsulate, & Synchronise Ancillary Audio SDI or non-sdi signals Audio SMPTE Receiver Accumulate, Decapsulate, & Synchronise Audio Receiver can take just the audio elements (as ) from the network Or add external IP media sources. All streams can be fully synchronised

9 SMPTE 2110 Relationship SMPTE PCM Digital Audio Digital audio streams shall conform to AES67 An RTP-based transportation specification of PCM digital audio streams over IP Includes SDP (RFC4566) metadata for stream reception and interpretation Mandatory audio requirements for all devices: 48kHz sampling (Media & RTP clocks same as sampling rate) 1ms packet time 1..8 channels per stream 16 & 24 bit depth (L16 - RFC3551 & L24 - RFC3190) Specification includes further provisions Outside mandatory requirements read spec carefully

10 SMPTE 2110 Relationship Some differences between how AES67 works within SMPTE and and operation within the pure audio domain. Synchronisation and Timing PTP: Support of SMPTE required Message rate according to AES-R (AES Media profile) defaultds.slaveonly=true for devices not intended of entering the PTP master state (under consideration at the moment) a=ts-refclk:ptp=traceable and a=tsrefclktsrefclk:localmac=<mac_addr> allowed state RTP clock: offset=0 with respect to media clock/network clock a=mediaclk:direct=0

11 SMPTE 2110 Relationship Protocols Support of RTCP RFC3551 not required by 2110 (must be tolerated) Support of SIP - RFC3261 (or other connection management protocol) not required Redundancy: SMPTE Identical IP source and destination addresses not allowed Optional channel assignment map (SDP) a=fmtp:<payload type> channelorder=<convention>.<order> E.g: a=fmtp:101 channel-order=smpte2110.(51.st) No differences in SDP between and AES67 (apart from channel assignment)

12 Pure AES67 SMPTE 2110 Relationship More Protocols Conformance level support by receivers with respect to AES67 stream formatting Level A 48 khz, 1..8, 1ms Y 48 khz, 1..8, 125 µs 48 khz, 1..64, 125 µs 96 khz, 1..4, 1ms 96 khz, 1..8, 125 µs 96 khz, 1..32, 125 µs Ax Y Y B Y OR Y Bx Y OR Y Y OR Y C Y OR Y Cx Y OR Y Y OR Y Other values are already recommendations in AES67

13 AES67 History EBU Audio Contribution over IP (ACIP) used existing standards from the world of VoIP (Voice over IP) along with broadcast quality codecs to produce a manufacture interoperable standard, usable over the Internet, wide and local area networks. AES67 has used the same model for the development of an interoperable standard for existing and competing professional low latency audio solutions used within campuses, stadiums, event venues and local area IP networks. Use existing standards where possible - don t reinvent the wheel

AES67 History AES67-2015 Standard for Audio Applications of Networks - High-performance Streaming

14 AES67 History AES Standard for Audio Applications of Networks - High-performance Streaming Audio-over-IP Interoperability Originally published on September 11 th 2013 Second Edition published September 21 st 2015

15 AES67 History Interoperable guidelines for professional audio IP Based Low latency Campus and LANs Excludes: Other network types Low-bandwidth media Data compression codecs (ACIP) Low performance WANs and general Internet Methodology could be used for video Mother audio networking protocol installed in products to make use of AES67

16 AES67 History Intended Applications Commercial Audio Installed sound: theatres, stadiums, theme parks, cruise ships Live sound fixed and touring Professional broadcast In-house distribution Inter-facility links on corporate networks BBC R&D used similar RTP/audio for Commonwealth Games in 2014 as an experimental trial OB vehicles Music production Post production

17 AES67 History Candidate AoIP Solutions Date Technology - Manufacturer Transport Synchronisation 2003 Livewire Telos/Axia RTP Proprietary 2005 Wheatnet-IP - Wheatstone RTP Proprietary 2006 Dante - Audinate UDP IEEE N/ACIP EBU RTP Adaptive per stream 2009 Q-LAN QSC UDP IEEE RAVENNA ALC NetworX RTP IEEE AVB - IEEE AVnu Ethernet/RTP IEEE 802.1AS

18 AES67 History OSI Layer A-Net EtherSound Cobranet Livewire, Dante, Application (7) AVB AES67 & RAVENNA Presentation (6) Session (5) RTP RTP Transport (4) UDP UDP Network (3) IP IP Data Link (2) Ethernet Physical (1) Copper Copper/fibre

19 AES67 History Well we do! Now we can all talk to one another We can t talk to one another! IP

20 O Negative of Audio Networking Roland Hemming, Independent Audio Consultant AES 67 RAVENNA AES 67 AES 67 AES 67

21 AES67 Outline High-performance Streaming Audio-over-IP Interoperability Specification Minimal common set of parameters defined for audio in realms of: Audio Coding Payload Format and Sampling rates Packet Time Other requirements: Media Clocks Network Synchronisation Transport Connection Management Network Quality of Service Discovery

22 Core AES67 Technology Components Synchronisation PTP IEEE Media Profile Default Profile AES67 Technology Components

Synchronisation & Media Clocks Local Clocks receive PTP copy Wall Clock PTP GM GPS Sender Toff established and Receiver Roff established Convey SDP to Receiver Relationship Soff established on stream

23 Synchronisation & Media Clocks Local Clocks receive PTP copy Wall Clock PTP GM GPS Sender Toff established and Receiver Roff established Convey SDP to Receiver Relationship Soff established on stream start-up may be random Offset constant throughout stream lifetime conveyed via SDP (a=mediaclk:direct=<offset>) but 0 in ST2110 Sender Local Clock Media Clock Toff RTP Clock PTP copy Soff SDP Stream Clock Stream Data PTP copy RTP Clock Receiver Roff Local Clock Media Clock

24 Synchronisation & Media Clocks Phase accuracy of AES 11 (± 5% of sample period) Deploy PTP-aware switches (BC or TC) Essence data (audio samples or video frames) is related to the media clock upon intake essentially receiving a generation time stamp with respect to the media clock Fixed / determinable latency by configuring a suitable link offset ( playout delay ) Independent stream alignment by comparing and relating the RTP time stamps of individual essence data GM BC BC BC TC TC Node Node Node Node

25 Synchronisation & Media Clocks Link offset describes the latency through a media network Time difference between ingress at the sender and egress at the receiver Determined by receiver setting a buffer value greater than the network latency Receiver buffering to absorb jitter - not too short or long Sender buffer take into account packet time and networks stack/controller

26 Core AES67 Technology Components Synchronisation PTP IEEE Media Profile Default Profile From PTP Local Media Clock Generation AES67 Technology Components

27 Example multicast SDP 8 channel, 24-bit, 48kHz, 1ms packet time v=0 o= IN IP s=stage left I/O c=in IP /32 t=0 0 m=audio 5004 RTP/AVP 96 i=channels 1-8 a=rtpmap:96 L24/48000/8 a=recvonly a=ptime:1 a=ts-refclk:ptp=ieee :39-a7-94-ff-fe-07-cb-d0:0 a=mediaclk:direct=0

28 Example unicast SDP 8 channel, 24-bit, 48kHz, 250μs packet time v=0 o=audio IN IP s=stage left I/O c=in IP t=0 0 m=audio 5004 RTP/AVP 96 i=channels 1-8 a=rtpmap:96 L24/48000/8 a=sendonly a=ptime:0.250 a=ts-refclk:ptp=ieee :39-a7-94-ff-fe-07-cb-d0:0 a=mediaclk:direct=

29 Core AES67 Technology Components 48 samples (6/12/16/192) Max payload 1440 bytes 125/250/333μs 1 or 4ms From PTP RTP/AVP UDP IP Local Media Clock Generation Transport Packet Setup AES67 Technology Components Synchronisation PTP IEEE IPv4 (IPv6) Network unicast/multicast IGMPv2 16/24 bit linear Encoding 48 (44.1/96) khz 1..8 channels Media Profile Default Profile

30 Encoding & Packet Setup Payload format defines audio sample encodings Both L16 & L24 bit linear format at 48 khz sampling mandatory Payload formats from RFC3551 (L16) & RFC3190 (L24) Receivers both, senders either or both At 96 khz sampling, L24 supported by both senders and receivers At 44.1 khz sampling, L16 supported by both senders and receivers Packet time real-time duration of media in packet Determined by sender and sent via SDP 1 ms is the mandatory requirement 48 samples at 48 or 44.1 khz - 96 samples at 96 khz Support by both senders and receivers

31 Encoding & Packet Setup Recommended and required packet times Packet time 125 microseconds Packet samples (48 khz) Packet samples (96 khz) Packet samples (44,1 khz) Notes Compatible with class A AVB transport 250 microseconds High-performance, low-latency operation. Interoperable with class A and compatible with class B AVB transport. 333 microseconds Efficient low-latency operation 1 millisecond Required common packet time for all devices adhering to this standard 4 milliseconds 192 n.a. 192 For applications desiring interoperability with EBU Tech 3326 or transport over wider areas or on networks with limited QoS capability

32 Encoding & Packet Setup Stream channel count maximum number per stream limited by the packet time, encoding format and network MTU Format, sampling rate Packet time Maximum channels per stream L24, 48 khz 125 microseconds 80 L16, 48 khz 250 microseconds 60 L24, 48 khz 250 microseconds 40 L24, 48 khz 333-1/3 microseconds 30 L24, 96 khz 250 microseconds 20 L24, 48 khz 1 millisecond 10 L24, 48 khz 4 milliseconds 2

33 Core AES67 Technology Components 48 samples (6/12/16/192) Max payload 1440 bytes 125/250/333μs 1 or 4ms From PTP RTP/AVP UDP IP Local Media Clock Generation Transport Packet Setup AES67 Technology Components Synchronisation PTP IEEE IPv4 (IPv6) Network unicast/multicast IGMPv2 16/24 bit linear Encoding 48 (44.1/96) khz 1..8 channels Quality of Service DiffServ/DSCP Media Profile Default Profile Clock EF (46) Media AF31 (34) Best Effort DF (0) SIP/SDP (unicast) Multicast (IGMPv2/SDP) Connection Management

34 Connection Management - 1 Exchange of information describing stream characteristics & connection information SDP For multicast connections AES67 doesn t specify transport method of SDP has to be done manually between different systems Different methods are used: RAVENNA RTSP automatic within own domain Axia Livewire RTSP automatic within own domain Dante SAP no manual means for SDP read in or out For unicast connections AES67 uses SIP for connection management and to transport SDP AMWA IS-05 potentially offers connection management for both multicast and unicast

35 Connection Management - 2 RAV2SAP tool a temporary windows application solution for connection between nodes from other manufactures/systems AES70 architecture for system control and connection management - OCA Addresses device control and monitoring only streaming media standards Network Classes are foundation for many media transport networks Specific adjustments may be required to support particular media transport types AES70 Adaptation configuration rules for the network classes AES67 Adaptation specification - in progress - provide media connection management services for devices that implement AES67 media transport.

36 Core AES67 Technology Components 48 samples (6/12/16/192) Max payload 1440 bytes 125/250/333μs 1 or 4ms From PTP RTP/AVP UDP IP Local Media Clock Generation Transport Packet Setup AES67 Technology Components Synchronisation PTP IEEE IPv4 (IPv6) Network unicast/multicast IGMPv2 16/24 bit linear Encoding 48 (44.1/96) khz 1..8 channels Quality of Service DiffServ/DSCP SAP Media Profile Default Profile Clock EF (46) Media AF31 (34) Best Effort DF (0) SIP/SDP (unicast) Multicast (IGMPv2/SDP) Connection Management Discovery Possibilities Bonjour Axia Discovery Protocol WheatNetIP Discovery Protocol NMOS IS-04

37 AMWA Discovery/Connection Management AMWA Advanced Media Workflow Association Networked Media Open Specifications (NMOS) a set of APIs NMOS IS-04 Discovery - discover and register devices as they are connected to the network, enabling their subsequent connection through the NMOS IS-05 Connection Management - allows the configuration of connections between Senders and Receivers Both of the above have been tested for discovery and connection management of AES67/ST devices IS-05 was also tested with some audio manufacturers at a recent AMWA workshop, also part of IBC IP showcase of IS-05 NMOS IS-06 Network Control in progress - viewable network topology, allows creation/retrieval/update/deletion of flows in the network between endpoints. Includes related monitoring and diagnostics.

38 AES67 Interop PlugFests Plugfests organised to confirm interoperability in commercially neutral environment Users benefit from compliant equipment that will connect together regardless of manufacture Three held so far , 2015 & 2017 Increasing numbers with each one Additional new manufacturers appear AES Reports from each Fourth in planning for 2018

39 Munich Oct 2014 Institut für Rundunkunktechnik

40 Munich Oct 2014 Institut für Rundunkunktechnik 10 Manufacturers 16 variable product types - software on a PC to hardware-based FPGA solutions all based on extensions of existing networked-audio products Mark Younge, AES Standards Manager acting as chair providing technical assistance and independent observers

41 Washington DC Nov 2015

42 Washington DC Nov companies - same as Munich plus these: Meinberg Radio Clocks GmbH & Co. KG QSC LLC Wheatstone Corporation Yamaha Corporation 13 products In Munich a number of products based on prefabricated hardware and firmware sub-systems In Washington products were largely-independent implementations of AES67.

43 London Feb 2017 New Broadcasting House

44 London Feb 2017 New Broadcasting House 24 companies - same as Washington plus these: A.R.G ElectroDesign Ltd Audinate Audio-Technica U.S., Inc. Bosch Security Systems Calrec Coveloz Genelec Imagine Communications Riedel Communications Shure Inc. Sonifex Tektronix Thum+ Mahr+ Gmbh 36 products Software implementations on PC to hardware-based FPGA solutions. Products - prefabricated hardware, prefabricated firmware sub-systems and bespoke implementations of AES67 Four NMOS 1S-04 implementations tested in AES67 devices 75% success rate AES67 audio over IP withinyamaha SMPTE 2110

45 London Feb 2017 New Broadcasting House

46 Broadcasting Examples BBC Internet Fit Studio

47 Broadcasting Examples BBC Internet Fit Studio DIRA playout system VM backend and frontend processes Sound card Interface feeding Axia sound card driver process Uses just the mandatory common AES67 settings of L16/L24, 48Khz, 1ms

Broadcasting Examples BBC Wales/Cymru Cardiff Central Square New Build in planning process predominantly IP based Extensive use of AES67 in

SDI & AES3 I/O and Black & Burst 5012 audio sources, 6097 audio destinations (mono) 855 video sources, 1169 video destinations Live IP for

48 Broadcasting Examples BBC Wales/Cymru Cardiff Central Square New Build in planning process predominantly IP based Extensive use of AES67 in combination with 2110 for video/audio AES67 (and Dante) within audio domains Hence PTP (locked to GPS) across network Although some legacy SDI & AES3 I/O and Black & Burst 5012 audio sources, 6097 audio destinations (mono) 855 video sources, 1169 video destinations Live IP for core routing of all video and audio SMPTE 2110 elemental streams ( , -20, -30 & -40) NMOS IS-04 discovery and NMOS IS-05 connection management

49 Broadcasting Examples Swedish Radio NXG Project Simple to produce radio and create ability to broadcast anywhere, anytime Production Content fast from anywhere Contribute live from/to studio anywhere Broadcast live, self op from anywhere Workflow Intuitive on-air equipment Modern studio environment Create conditions & support mobile workflow Engineering Regional & centralised data centre Create scalable end user system System integration ACIP & cloud computing

and ACIP together with an inhouse developed front-end mixer user interface, based on

50 Broadcasting Examples Swedish Radio NXG Project Public Tender Non-technical Based on production scenarios Mutual contract with LAWO in 2014 The project combines AES67 and ACIP together with an inhouse developed front-end mixer user interface, based on touch technology and IT services Deployment 1 st station Q nd station Q4 2017

Broadcasting Examples Swedish Radio NXG Project Regional Data Centre Network audio hub in Gothenburg No local data centres Cost, maintenance

audio codec pool TX on air, STL & processing Centralised Data Centres in Stockholm Private Cloud, VMWare Back end services SIP/CCM environment

51 Broadcasting Examples Swedish Radio NXG Project Regional Data Centre Network audio hub in Gothenburg No local data centres Cost, maintenance access, shared resources, optimise unit storage Mix core engines Network/routers, PTP GM (based on AES67) Playout and recording services IP audio codec pool TX on air, STL & processing Centralised Data Centres in Stockholm Private Cloud, VMWare Back end services SIP/CCM environment PTP GM Cluster buys PTP over Ethernet Network audio hub in Gothenburg In development ? Playout & recording Software based IP audio codec pool

Connectivity LAN/WAN EWAN (Leased Line) Satellite 3G/4G diversity Public Internet QoS

52 Broadcasting Examples Swedish Radio NXG Project Networking With industry AES, EBU, MNA - shows IBC, ISE, NAB IP Any-to-Any!! Have to have this! Connectivity LAN/WAN EWAN (Leased Line) Satellite 3G/4G diversity Public Internet QoS Scheme AES67 in office VRF/VLAN replace internal sound card with virtual card (AES67) PTP Precision Time Protocol Directory Services

Integration & UI via LAWO Mixing/DSP Control onto touchscreens

53 Broadcasting Examples Swedish Radio NXG Project Interoperability AES67 AES70 EBU ACIP II (3326/3368) System Integration & UI via LAWO Mixing/DSP Control onto touchscreens with buttons Group audio sources to increase no. of sources On-air functions on panel

54 AES67 What Next? Latest version AES expected to be published very soon Includes Protocol Implementation Conformance Statement (PICS)

55 AES67 What Next? AES70 for device control, network connections and discovery? Archwave are combining AES67 and AES70 as a solution - AudioLAN2.0. Audio and control streams plus remote control AMWA NMOS APIs: NMOS IS-04 Discovery about to be included as one of the (informative) discovery mechanisms for AES67 Include NMOS IS-05 for connection management? Include NMOS IS-06 (in development) for network control?

56 SMPTE Summary Streams shall conform to AES67 RTP based PCM digital audio only + SDP metadata (RFC4566) Mandatory requirements for all devices (senders and receivers): 48kHz sampling (Media & RTP clocks same as sampling rate) Compliance of media & RTP clocks with sections of Should support 44.1 khz and/or 96 khz sampling 1ms packet time 1..8 channels per stream 16 & 24 bit depth (RFC3551 & RFC3190) Timing & buffering provisions of AES67 Compliance of RTP timestamp with Outside mandatory requirements read & AES67 specs carefully Be aware of differences within and pure AES67 domain

57 Thank you bbc.co.uk/rd

ST2110 & AES67. Commonalities & Constraints. - Andreas Hildebrand RAVENNA Technology Evangelist ALC NetworX, Munich

ST2110 & AES67. Commonalities & Constraints. - Andreas Hildebrand RAVENNA Technology Evangelist ALC NetworX, Munich ST2110 & AES67 Commonalities & Constraints - Andreas Hildebrand RAVENNA Technology Evangelist ALC NetworX, Munich # 1 Andreas Hildebrand, RAVENNA Technology Evangelist more than 25 years in the professional