SYNCHRONISATION AND TIME DISTRIBUTION

THE JOURNAL TJ 22 SÉBASTIEN JOBERT, KENNETH HANN SYNCHRONISATION AND TIME DISTRIBUTION IN MODERN TELECOMMUNICATIONS NETWORKS The past decade has witnessed a race for networks to provide ever faster communications, interconnecting people via applications used every day by billions of users. Radio spectrum utilisation and synchronisation plays a key role here. But now that Ethernet has won the bandwidth and cost per bit wars, how are base stations being synchronised today? For many years, synchronisation was a byproduct of the transmission network, but technology changes in both mobile and fixed networks has transformed synchronisation into an essential service. Requirements are becoming more demanding and the traditional means of delivering network synchronisation are evolving. Time synchronisation enables mobile techniques such as Time Division Duplex, Coordinated Multipoint transmission and reception, and Multicast-Broadcast Single- Frequency Network. When mobile networks are precisely synchronised as shown in Figure 1, performance and throughput can be increased. There is a paradox here: while mobile networks have an increased requirement for synchronisation, transport networks, that were once synchronous and therefore capable of carrying frequency synchronisation at the physical layer, have largely lost that capability. So now that Time Division Multiplexing (TDM) networks are replaced by modern, packet-based, communications networks, how can such a synchronisation service be arranged? In addition to mobile networks, accurate synchronisation is a potential enabler for new paradigms, such as the Internet of Things SÉBASTIEN JOBERT, KENNETH HANN The changing picture. (IoT) and virtualisation techniques. This provided motivation to develop new synchronisation technologies for transport networks. Two main candidates were in the starting blocks ten years ago: Synchronous Ethernet [1] A physical layer technology that inherited the pedigree of the Synchronous Digital Hierarchy (SDH). μs accuracy Mobile Backhaul Network Central Office μs accuracy Common time reference carried over the network Neighboring cells need us accuracy Figure 1: Time synchronisation is essential to some mobile communications technologies. Volume 10 Part 1-2016

INFORM NETWORK DEVELOP SYNCHRONISATION AND TIME DISTRIBUTION IN MODERN TELECOMMUNICATIONS NETWORKS IEEE 1588TM [2] A packet-based protocol technology using timestamps, a rookie in telecommunications networks, but a touted successor to Network Time Protocol. Synchronous Ethernet While TDM networks required a steady rate to avoid slips (corresponding to a buffer that empties or overflows), packet networks such as Ethernet have been designed to work asynchronously, saving on oscillator cost, and removing the need for clock selection and holdover. Although cost reduction was an important factor for network operators, the use of Ethernet broke the timing distribution to the mobile networks [3]. The solution was simple make Ethernet synchronous: the so-called SyncE (Synchronous Ethernet) technology was born! Carrier Ethernet does not require synchronisation as such; the key driver for synchronisation on such networks is only to carry timing up to mobile cell sites. SyncE offers continuity in the synchronisation distribution chain across routers and switches and operators like to use it wherever possible. The standardisation of SyncE was supported by European operators who saw this technology as a way to migrate their networks towards Ethernet while maintaining a link-by-link timing hierarchy fully compatible with SDH (see Figure 2). SyncE ensures that the data bits transmitted at the physical layer are in step with a reference rhythm usually derived from a Primary Reference Clock (PRC). SyncE is therefore a frequency synchronisation solution (although there has been a proposal to expand this to carry also phase/time [4]). Figure 3 illustrates the distinction between frequency In addition to mobile networks, accurate synchronisation is a potential enabler for new paradigms, such as the Internet of Things (IoT) and virtualisation techniques. THE JOURNAL TJ 23

THE JOURNAL TJ 24 SÉBASTIEN JOBERT, KENNETH HANN ESMC frames SyncE aware backhaul network PRC-traceable frequency reference SyncE clock SyncE clock SyncE clock SyncE clock SyncE signal Switch/Router Switch/Router Switch/Router Switch/Router ESMC: Ethernet Synchronisation Messaging Channel Figure 2: Synchronous Ethernet is a link-by-link solution, all switch and router devices must implement a SyncE clock. and phase/time synchronisation. The Synchronisation Status Message that used to be carried as 4 bits in the SDH overhead, can be found in the bowels of SyncE carried via a slow protocol, called Ethernet Synchronisation Messaging Channel. Synchronisation Status Messages indicate when there is a failure in the distribution chain and that the received physical layer signal is no longer traceable to a PRC. Note that, while SyncE does not carry phase/time synchronisation, it is considered a useful technique in combination with IEEE 1588 TM in order to maintain phase/time holdover in case of failure. An example of this is where time is set via the Precision Time Protocol (PTP) (discussed below), but upon losing the PTP connection, time is advanced using ticks from SyncE. Some industries even combine these two standards to achieve incredibly high time accuracy [5]. SyncE is widely implemented in Ethernet telecoms devices and used in many networks, especially for mobile backhaul. The standards defining SyncE are stable; one new performance enhancement to SyncE clocks is currently being defined in ITU-T to reflect the fact that SyncE implementations typically exceed the previous generation of ITU-T requirements by at least one order of magnitude. In the race for mobile, SyncE was first out of the blocks. IEEE 1588 TM PTP and telecoms profiles An alternative to SyncE for delivering both frequency and phase/time synchronisation Figure 3: Difference between frequency and phase/time synchronisation. is known as IEEE 1588 TM which defines PTP. IEEE 1588 TM started out as a mechanism for forming a synchronisation hierarchy across an Ethernet network and was developed for applications ranging from instrumentation and measurement to power and automation. Telecoms was not originally represented. IEEE 1588 TM was first published in 2002 with a second version ratified in 2008 [2] that includes options for telecoms and a mechanism of making specific industry profiles (telecoms profiles are based on this version). A third version is currently under definition. ITU-T decided to define a PTP profile for f 1 = f 2 Frequency synchronisation Phase/time synchronisation frequency distribution in Recommendation G.8265.1 [6]. Later, a second PTP profile was defined for phase/time synchronisation in G.8275.1. A third profile is now under definition in the emerging Recommendation G.8275.2. In order to verify that the PTP options are correctly implemented according to these profiles, the IEEE recently launched a certification programme; the first certification programme in the IEEE s history [7] and it aims to help network operators deploy compliant PTP products. The PTP telecom profiles G.8265.1 and G.8275.1 are currently available in the certification programme, and Volume 10 Part 1-2016

INFORM NETWORK DEVELOP SYNCHRONISATION AND TIME DISTRIBUTION IN MODERN TELECOMMUNICATIONS NETWORKS 25 PRTC (GNSS antenna) GM slave master slave t1 t4 t2 t3 t1 t4 t2 t3 boundary clock Figure 4: Synchronisation transfer across a boundary clock (slave and master back-to-back). PRTC (GNSS antenna) GM slave t1 t4 c1 c4 c2 c3 t2 t3 transparent clock Figure 5: Synchronisation transfer across a transparent clock (timestamps on transiting PTP packets are corrected with the correctionfield value CF). the third PTP profile being defined in G.8275.2 is anticipated in the future. To achieve high accuracy, IEEE 1588 TM relies on hardware techniques to timestamp messages very close to the physical layer on both ingress and egress. This allows PTP Sync messages to carry accurate timestamps from a PTP master port to a PTP slave port. For time distribution, a two-way exchange of messages (e.g. Delay_Req/ Delay_Resp mechanism) is required in order to calculate the propagation delay between a PTP master port and a PTP slave port. Ordinary clocks are defined by IEEE 1588 TM as having a single port that can act either as a master or as a slave. The source of time for a PTP domain is a master clock known as the grandmaster. IEEE 1588 TM defines two other types of clock that are used to build timing networks: Boundary clock A PTP clock with multiple PTP ports that can receive, recover and retransmit the timing reference from other PTP clocks (see Figure 4). A boundary clock maintains locally the PTP time. Transparent clock A PTP clock with multiple PTP ports that computes the residence time that a PTP message spends inside the network node hosting the transparent clock and inserts this information in the correctionfield of the PTP message so that the next PTP clock receiving the message can compensate for this delay (see Figure 5). A transparent clock does not maintain locally the PTP time. The notion of timing support from the network or on-path support usually refers to the use of boundary clocks or transparent clocks in the network. G.8265.1 defines a profile with no support from the network (no boundary clock, no transparent clock). G.8275.1 defines a profile with full timing support from the network (telecom boundary clocks must be supported in all the network nodes, transparent clocks are currently considered for the next revision of the profile). G.8275.2 will define a profile with partial timing support from the network (boundary clocks can be supported in some of the network nodes). These are shown in Figure 6. THE JOURNAL TJ

THE JOURNAL TJ 26 SÉBASTIEN JOBERT, KENNETH HANN Slave The use of various PTP options also characterises the PTP profiles defined in ITU- T; the main differences between G.8265.1 and G.8275.1 are summarised in Table 1. G.8265.1 was first developed when the telecoms network did not support PTP clocks such as boundary clocks. At this time, some network operators were looking for a solution to carry transparently frequency synchronisation over packet-based leased lines to synchronise mobile base stations. This situation was not ideal, because the PTP messages experienced packet delay G.8265.1 PTP clock types Master-only Slave-only PTP unaware backhaul network PTP in end-to-end mode, ITU-T G.8265.1 telecom profile Telecom Time Slave Clock PTP aware backhaul network variation, also called packet jitter, which is known to degrade the synchronisation quality. However, the synchronisation requirements for the targeted mobile networks (Frequency Division Duplex at this time) were relaxed enough to accommodate this solution. This PTP profile is widely used today where the PTP slave function is generally located in the base station or in a cell site gateway, and the PTP master in a dedicated synchronisation device or in a router. The standards are stable now and no revision is currently planned. G.8275.1 Mapping IPv4/IPv6 unicast Ethernet multicast Telecom grandmaster: master-only Telecom boundary clock: boundary clock Telecom time slave clock: slave-only Unicast Slaves negotiate service Not used (fixed rates) negotiation rate using PTP signalling messages Message rates Announce: up to 8 messages Announce: 8 messages (per second) Sync, Delay_Req: up to 128 Sync, Delay_Req: 16 messages messages Table 1: Comparison of main PTP options for G.8265.1 and G.8275.1. Timing reference Timing reference PTP with full timing support from the network, ITU-T G.8275.1 telecom profile Slave T-BC PTP unaware backhaul network T-BC T-BC Boundary Clock T-BC PTP unaware network Master Telecom Boundary Clocks Telecom Grandmaster Master Timing reference G.8265.1-like communication G.8265.1-like communication PTP with partial timing support from the network, ITU-T G.8275.2 telecom profile, under study in ITU-T Figure 6: Status of the IEEE 1588 TM certification programmes for telecoms networks. G.8275.1 was then developed to address the need for accurate phase/time synchronisation, targeting 1μs accuracy. In order to avoid suffering from packet delay variation effects, it was decided to develop an architecture where all the nodes between the telecom grandmaster and the telecom time slave clock support a telecom boundary clock function. This profile clearly targets networks that have been designed from day-one to support accurate phase/time synchronisation. As this standard is quite recent (approved in 2014), the industry is still on the way to implement this new profile. Pre-standard deployments, significantly in China, show that performance targets can be met also in large networks. Major deployments of G.8275.1 are expected in Europe. The telecom boundary clock function is expected to be supported in a wide range of networks. A new revision of the standard is currently in preparation to add support for transparent clock. G.8275.2 aims to address some of the vulnerabilities of Global Navigation Satellite System (GNSS) time sources. One proposal from some American network operators was to use IEEE 1588 TM as a backup to Global Positioning System (GPS) antennas deployed at every base station. As many networks do not support PTP everywhere today, the architecture for this profile employs the partial timing support concept. The notion of assisted partial timing support (A-PTS) refers to the case where GPS is present at the base station during normal operations. (This concept will be further discussed in the next section.) G.8275.2 is still in its early stages and is being published at the ITU-T as we write. Some prestandard solutions have started appearing but with no major deployments so far. Architecture evolution From a centralised PRC architecture to distributed Primary Reference Time Clocks (PRTCs) From the days of SDH, synchronisation has been arranged as a tree structure, with the root of the tree, a Caesium PRC, positioned at the core of the network. The branches are the fixed trunks of high-speed links which Volume 10 Part 1-2016

INFORM NETWORK DEVELOP SYNCHRONISATION AND TIME DISTRIBUTION IN MODERN TELECOMMUNICATIONS NETWORKS 27 Figure 7: G.8265.1 was a rather different application of IEEE 1588 TM. fan out to thousands of mobile base stations at the edge. The synchronisation hierarchy has always been constrained to a tree structure avoiding timing loops even though the networks are typically implemented as rings. Since today the required synchronisation service is typically time (and not frequency only), the PRC is being replaced by the PRTC. Also, the availability of low cost synchronisation solutions in small foot-prints has made it attractive to move the PRTC closer to the edge of the network. Whereas previously, a small country may have had a handful of PRCs shared among all the operators for 3G mobile networks, now a single operator may have a few thousands PRTCs in a Time Division Long Term Evolution network. This architecture reduces the need to transport synchronisation over long network paths and is therefore attractive when very high performance is needed because the time source is now positioned close to the mobile base station. PRTCs have also become much smaller in size. Today, a PTP telecom grandmaster product serving many mobile base stations can be implemented as a small form pluggable module (see Figure 8) has been shackled by reliability conservative design rules, safety margins and Five Nines of network availability. But now, some situations might permit relaxing these requirements. There seems to be a growing trend of taking calculated risks and using solutions that will work under the majority of cases, but cannot be guaranteed by design. One such proposition is the concept of PTS. Let s first look at the motivation, and then examine the risks and rewards of PTS. A modern network can support high accuracy time synchronisation, but the majority of networks have been built over many years and consist of equipment that was not designed for time synchronisation (i.e. does not support a boundary clock or transparent clock function). However, even a legacy network forwards packets. What happens if we carry time synchronisation transparently over such a network in short, will it work? If it works, we have saved the trouble and expense of adding time synchronisation support into the network. If it does not work, however, the quality of wireless transmission will be impacted. The challenge, therefore, is in being able to measure the time accuracy at the remote end point of the network by comparing it to a reference. In practice, the only reference for accurate time measurement is a GNSSbased receiver. So a technician jumps in his GNSS antenna Aggregation switch with pluggable GM white van and drives to the remote site, erects a temporary GNSS antenna and uses some measurement equipment to determine if the time accuracy of the PTS PTP service is within the required limits. This is an echo of the days when the same technician was measuring the quality of the Plain Old Telephone Service as part of the ADSL installation. We can see that just as Internet deployment quickly became technician-free, so the time synchronisation service needs to be automated. So, what if we run the PTS and perhaps measure a small percentage of cases only? This saves significantly on installation costs and, since the networks are built from similar components and structures, it gives a good indication. The risks of this type of survey approach are asymmetric cable installations (250m cable 1μs), asymmetric equipment (may change with load, reset, etc.) and asymmetric routing (forward and reverse paths). The keyword in all of these cases is asymmetric. PTP assumes that the paths are symmetrical and, to the extent that they are not, then an error is introduced. If the total asymmetry can be bounded (by type of equipment, number of hops, fibre installation practices, constrained routing practices), then PTS can offer significant cost savings. But this is not at all a straightforward exercise. The PTP Local Slave Opportunities and challenges with Partial Timing Support (PTS) For decades, the telecoms network engineer PTP Remote Slaves Figure 8: Miniature PTP grandmaster providing time synchronisation at the edge of the network. THE JOURNAL TJ

THE JOURNAL TJ 28 SÉBASTIEN JOBERT, KENNETH HANN "Try G.8275.1 my boy"! PTS Best-effort Beach Figure 9: Time delivery when swimming with asymmetry sharks! question for the operator is: can the risks be bounded within safe limits? Another issue may be that the requirements for time synchronisation are tightening and what is 1.5μs today may be 300ns tomorrow. One European operator had the view that taking such a risk with PTS would be like swimming with sharks (Figure 9). There may be some applications where PTS can be used to benefit. The next section provides an example. Assisted Partial Timing Support (A-PTS) Combining PTS and GNSS On the edge of the network there is a high reliance on direct GNSS which has become the de-facto solution for mobile networks requiring time synchronisation. Recently, susceptibility of GNSS receivers to jamming has become a major concern and has led to a proposal to combine a PTS solution with local GNSS. This solves the main problem that we already identified T-GM M S A-PTS Clock Figure 10: A-PTS provides resiliency against local GNSS failures. Volume 10 Part 1-2016

INFORM NETWORK DEVELOP SYNCHRONISATION AND TIME DISTRIBUTION IN MODERN TELECOMMUNICATIONS NETWORKS 29 with PTS that of needing a reference to check or calibrate the PTP path. So, the time derived from PTP can be compared with the time from GNSS. Any fixed offset can be measured and compensated, which in turn allows PTP to provide accurate time should the GNSS fail as shown in Figure 10. Forward looking synchronisation for the IoT There is much speculation as to the synchronisation requirements of the devices and applications making up the collective IoT and the benefits of having high accuracy time available to the billions of devices making up the IoT. Clearly if time is ubiquitously available, there can be huge changes and improvements in the way things are done for example, programming languages with a concept of time, programme counters in central processing units where real-time at the nanosecond level is available, cyberphysical systems where events happen at precise time and allow for distribution with precise control. However, all this tends to assume that precise time is pumped across the Internet at high accuracy, with reliability and security and for free. On the other hand, if time is such a valuable and prestigious service, and could be made available, wouldn t there be users ready to pay a reasonable price for the service? Such a premium-time service would have to provide a better quality of service compared with a time being carried transparently over a data connection and could be provided by network operators, perhaps in co-operation with national physics laboratories. AUTHORS CONCLUSIONS The major synchronisation technologies of SyncE and IEEE 1588 TM address different use cases. SyncE offers traditional frequency distribution and takes a supporting role in providing time holdover for PTP systems. IEEE 1588 TM is still under development in multiple standards bodies, ABOUT THE AUTHORS Sébastien Jobert Sébastien is a telecommunications expert involved for over 10 years in the standardisation of ITU-T Synchronous Ethernet and IEEE 1588 TM technologies. He is co-author of a book and of several patents. He has been the editor of various international standards and test suites in ITU- T, IEEE and MEF and led the first certification program of IEEE s history. Sébastien was senior director of engineering for a certification laboratory based in the Silicon Valley (Iometrix), and a distinguished network expert with Orange, where he focused on QoS, cloud and mobile technologies. Kenneth Hann Kenneth is Senior Director Oscilloquartz, Finland. He was founder and CEO of Time4 Systems with 30 years experience in telecommunications from Nokia, Martis and Tellabs. For the past decade Kenneth has focused on innovative synchronisation solutions for packetnetworks and has been actively involved in the development and standardisation of both synchronous Ethernet, and IEEE 1588 TM. Kenneth has a number of synchronisationrelated patents. His daughter Tanja draws nice clock characters. with specific telecoms profiles defined in ITU-T standards. Combined together, they can achieve accurate and reliable time synchronisation. Applications are now waiting. There s a whole world still to synchronise! ITP INSIGHT CALL Want to talk to the author? To discuss this article and its content, join in the ITP Insight Call on 1 June. To book onto the call visit: https://www.theitp.org/calendar/ REFERENCES 1. Ferrant, J-L., et al. Synchronous Ethernet: A Method to Transport Synchronization. IEEE Communications Magazine. Sep 2008 2. IEEE Std 1588 TM -2008. IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems. Jul 2008 3. Ferrant, J-L., et al. Synchronous Ethernet and IEEE 1588 TM in Telecoms: Next Generation Synchronization Networks. ISTE - Wiley, ISBN 978-1-84821-443-9. Jun 2013 4. Hann, K., Jobert, S., and Rodrigues, S. Synchronous Ethernet to Transport Frequency and Phase/Time. IEEE Communications Magazine. Aug 2012 5. Moreira, P., et al. White rabbit: Subnanosecond timing distribution over Ethernet. International Symposium on Precision Clock Synchronization for Measurement, Control and Communication. Oct 2009 6. Ferrant, J-L., et al. Development of the First IEEE 1588 TM Telecom Profile to Address Mobile Backhaul Needs. IEEE Communications Magazine. Oct 2010 7. Eidson, J.C., et al. IEEE-SA Conformity Assessment Program for IEEE 1588 TM in Mobile Networks. IEEE-SA White Paper. Nov 2014. ABBREVIATIONS A-PTS GNSS GPS IoT PRC PRTC PTP PTS SDH TDM Assisted Partial Timing Support Global Navigation Satellite System Global Positioning System Internet of Things Primary Reference Clock Primary Reference Time Clock Precision Time Protocol Partial Timing Support Synchronous Digital Hierarchy Time-Division Multiplexing THE JOURNAL TJ