IEEE-1588 Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems John Eidson July 16, 2003 john_eidson@agilent.com
Overview Objectives of 1588 A bit of history Where is it being used? Comparison with other protocols What the 1588 standard defines Technical overview Prototype results Page 2
Objectives of 1588 Sub-microsecond synchronization of real-time clocks in components of a networked distributed measurement and control system Intended for relatively localized systems typical of industrial automation and test and measurement environments. Applicable to local areas networks supporting multicast communications (including but not limited to Ethernet) Page 3
Objectives of 1588 (continued) Simple, administration free installation Support heterogeneous systems of clocks with varying precision, resolution and stability Minimal resource requirements on networks and host components. Page 4
A bit of History Status of IEEE 1588 Approved by the IEEE-SA Review Committee on 12 Sept. 2002 Published on November 8 2002 Available from the IEEE http://standards.ieee.org Page 5
Members of the IEEE-1588 working group Volker Arlt-Lenze AG Scott Carter-GE Medical John Eidson-Agilent Richard Hambly-CNS Systems Bruce Hamilton-Agilent Steve Jennings-Bosch- Rexroth Robert Johnson- Telemonitor Bill Kneifel-KUKA Jurgen Knopke-Lenze Kang Lee- NIST Judah Levine-NIST Anatoly Moldovansky- Rockwell Ed Powers-Naval Observatory Jim Read-Hyperfine Richard Schmidt-Naval Observatory Steve Smith-ORNL Joe White-NRL Stan Woods-Agilent Page 6
Where is it being used? (presentations scheduled for Sept. 24 workshop) NETWORK DEVICES: IEEE 1588 and Network Devices: Dirk S. Mohl, Hirschmann Electronics Boundary Clock implementation: Øyvind Holmeide, Managing Director, OnTime Networks AS. Extending IEEE 1588 to fault tolerant synchronization with a worst-case precision in the 100 ns range: Nikoaus E. Keroe, Oregano Systems Page 7
Page 8
Where is it being used? (presentations scheduled for Sept. 24 workshop) TELECOMMUNICATIONS: Proposal for IEEE1588 use over Metro Ethernet Layer 2 VPNs: Glenn Algie, Senior Advisor, Wireless Technology Labs, Nortel Networks INDUSTRIAL AUTOMATION: Impact of Switch Cascading on Time Accuracy: Prof. Thomas Mueller, University Wintherthur, Suisse, Karl Weber, SIEMENS Automation and Drives Consequences of Redundant Structures PTP: Ludwig Winkel, SIEMENS Automation and Drives Application of 1588 for Distributed Motion Control: Kendal R. Harris, Sivaram Balasubramanian and Anatoly Moldovansky, Rockwell Automation Page 9
HZx Wrappers Courtesy of SIG Pack Systems AG Page 10
Control of multiple robots Page 11
Mulit-stand Printing Press 60mph = 1 inch/millisecond = 1 mil/microsecond Page 12
Where is it being used? (presentations scheduled for Sept. 24 workshop) MILITARY FLIGHT TEST: Time Correlation using network based data acquisition on-board a Military Test Vehicle: Jiwang Dai, Ph.D, Senior Software Engineer, L3 Communications Telemetry East, Thomas DeSelms, Senior Network Systems Engineer, Veridian Engineering, and Edward Grozalis, Senior Engineer, L3 Communications Telemetry East POWER SYSTEMS: Implementation of IEEE Std.-1588 in a Networked I/O Node: Mark Shepard, GE Drives & Controls, Inc., Salem, VA Page 13
Page 14
Comparison with other protocols 1588: Target is groups of relatively stable components, locally networked (a few subnets), cooperating on a set of well defined tasks. NTP: (Network Time Protocol, RFC 1305). Target is autonomous systems widely dispersed on the Internet. GPS: (Satellite based Global Positioning System of the US Department of Defense): Target is autonomous, widely dispersed systems. TTP(www.ttpforum.org), SERCOS (IEC 61491): Target is tightly integrated, usually bus or specialized TDMA network based closed systems. Page 15
1588 NTP GPS TTP SERCOS Spatial extent A few subnets Wide area Wide area Local bus Local bus Communications Network Internet Satellite Bus or star Bus Target accuracy Submicrosecond Submicrosecond Few milliseconds Submicrosecond Submicrosecond Style Master/slave Peer ensemble Client/server Distributed Master/Slave Resources Small network message and computation footprint Moderate network and computation footprint Moderate computation footprint Moderate Moderate Page 16
1588 NTP GPS TTP SERCOS Latency correction Yes Yes Yes Configured No Protocol specifies security No Yes No No No Administration Self organizing Configured N/A Configured Configured Hardware? For highest accuracy No RF receiver and processo r Yes Yes Update interval ~2 seconds Varies, nominally seconds ~1 second Every TDMA cycle, ~ms Every TDMA cycle, ~ms Page 17
1588 Defines: Descriptors characterizing a clock The states of a clock and the allowed state transitions 1588 network messages, fields, and semantics Datasets maintained by each clock Actions and timing for all 1588 network and internal events Page 18
Critical physical specifications A suite of messages for monitoring the system Specifications for an Ethernet based implementation Conformance requirements Implementation suggestions Page 19
Timing Latency & Fluctuation Application layer Network protocol stack msecs of delay and fluctuation Application layer Network protocol stack Physical layer < 100 nsecs of delay and fluctuation Physical layer Repeaters & Switches: fluctuations ~100ns to usec Routers:fluctuations ~ms Repeater, Switch, or Router Page 20
Reducing Timing Latency & Fluctuation Within a node: Make timing measurements as close to the physical layer as possible to eliminate protocol stack and operating system fluctuations. Use statistical techniques to further reduce residual fluctuations from PHY layer, network and repeaters and switches. Routers: Use transfer devices (1588 boundary clocks) to reduce router latency and fluctuations Page 21
Selecting a Master Clock-Single Subnet Self-configuring based on clock characteristics Information contained in Sync messages All clocks run an identical Best Master Clock algorithm Master clock Typical slave clock Repeater or Switch Repeater or Switch =1588 code & hardware Page 22
1588 Characterization of Clocks Based on primary source of time, e.g. GPS, local oscillator Accuracy Variance Preferred set membership Type: Boundary clock (spans subnets) or ordinary clock UUID Page 23
1588 Timing Related Messages Multicast only within a subnet Four types of timing messages: Sync, Follow_Up, Delay_Req, Delay_Resp Issuing and response to these messages dependent on the state of each clock Page 24
Detection of Sync messages Application layer Network protocol stack Optional Sync and Delay_Req message detector Physical layer e.g. MII interface in Ethernet Page 25
IEEE-1588 Code Network protocol stack & OS Sync detector & timestamp generator Physical layer msecs of delay and fluctuation Master clock sends: 1. Sync message 2. Follow_up message < 100 nsecs of delay and fluctuation Repeaters & Switches: ~100ns to usec Routers: ~ms Repeater, Switch, or Router Page 26
uprocessor-a estimated sending time = 9:59.99AM precise sending time = 10:00AM SMD-A measures Sync message precise sending time = 10:00AM Master Clock Master Clock uprocessor-a precise sending time = 10:00AM SMD-A 1588 Synchronization Sync message estimated sending time = 9:59.99AM network Clock correction circuits Clock Follow_Up message precise sending time = 10:00AM network 0:05 Slave Clock uprocessor-b estimated sending time = 9:59.99AM receipt time = 10:05AM SMD-B measures Sync message receipt time = 10:05AM Slave Clock uprocessor-b estimated sending time = 9:59.99AM precise sending time = 10:00AM receipt time = 10:05AM SMD-B Step 1: Sync message sent from master to slaves All Sync message detectors (SMD) note the time on the local clock this message appears at the SMD Step 2: Follow-Up message containing precise sending time sent from master to slaves All slaves compute offset and correct their clock Page 27
Synchronization computation offset = receipt time precise sending time one way delay (for a Sync message) one way delay = {master to slave delay + slave to master delay}/2 (assumes symmetric delay) master to slave delay = receipt time precise sending time (for a Sync message) slave to master delay = Delay_Req receipt time -precise sending time (of a Delay_Req message) Page 28
Sync messages: Issued by clocks in the Master state Contain clock characterization information Contain a estimate of the sending time When received by a slave clock the receipt time is noted Can be distinguished from other legal messages on the network For best accuracy these messages can be easily identified and detected at or near the physical layer and the precise sending (or receipt) time recorded Page 29
Follow_Up messages: Issued by clocks in the Master state Always associated with the preceding Sync message Contain the precise sending time as measured as close as possible to the physical layer of the network When received by a slave clock the precise sending time is is used in computations rather than the estimated sending time contained in the Sync message Page 30
Delay_Req messages: Issued by clocks in the Slave state When received by the master clock the receipt time is noted The slave measures and records the sending time Can be distinguished from other legal messages on the network For best accuracy these messages can be easily identified and detected at or near the physical layer and the precise sending (or receipt) time recorded Page 31
Delay_Resp messages: Issued by clocks in the Master state Always associated with a preceding Delay_Req message from as specific slave clock Contain the receipt time of the associated Delay_Req message When received by a slave clock the receipt time is noted and used in conjunction with the sending time of the associated Delay_Req message as part of the latency calculation Page 32
1588 Multiple Subnet Topology =1588 code & hardware Grand Master Clock Repeater or Switch Typical Slave Clock GPS Repeater or Switch Boundary clock Router Repeater or Switch Repeater or Switch Only Slave Port of Boundary Clock Typical Master Port of Boundary Clock Page 33
Multiple Subnet Synchronization &Master Clock Selection Boundary clocks do NOT pass Sync, Follow_Up, Delay_Req, or Delay_Resp messages Within a subnet a port of a boundary clock acts just like an ordinary clock with respect to synchronization and best master clock algorithm The boundary clock internally selects the port that sees the best clock as the single slave port. This port is a slave in the selected subnet. All other ports of the boundary clock internally synchronize to this slave port. Page 34
Boundary clocks define a parent-child hierarchy of master-slave clocks. The best clock in the system is the Grand Master clock. If there are cyclic paths in the network topology the best master clock algorithm reduces the logical topology to an acyclic graph. Page 35
1588 Management Messages Management messages are multicast ONLY within a subnet. Boundary clocks, not routers, transfer management messages between subnets. Management messages allow appropriate query and update of selected database information maintained by each clock. Page 36
1588 Time Scales The time base in a 1588 system is the time base of the Grand Master Clock. All other clocks synchronize (perhaps via boundary clocks) to the grand master. The Grand Master Clock time base is implementation dependent. If the Grand Master Clock maintains a UTC time base, the 1588 protocol distributes the appropriate leap second information to the slaves. Page 37
Basic Uses of Explicit Time in Measurement and Control Time stamping (relative or absolute) Time stamping system events for debugging Time stamping data to allow correlation with other data or events Coordination of measurement: (sampling and triggering) Coordination of action (time based behaviors) Timed execution scripts Temporal mutual exclusion: (time slots) Page 38
Measurement setup to test prototypes of an earlier version of 1588 Agilent 5372A F&T Analyzer 1 PPS signal Clock-A Clock-B Agilent Technologies prototype clock 1-4 Ethernet repeaters Page 39
Allan frequency deviations for test oscillators 10-7 10-8 CTS CB3LV Allan Deviation 10-9 10-10 10-11 10811D 10-12 10-1 10 0 10 1 10 2 10 3 10 4 Averaging Time, seconds Page 40
Switch Connection Frequency 70 60 50 40 30 20 10 0 Inexpensive oscillator, fast loop s = 140 ns -490-420 -350-280 -210-140 -70 0 70 140 210 280 350 Frequency 70 60 50 40 30 20 10 0-460 -395-330 10811D, fast loop s = 138 ns -265-200 -135-70 -5 60 125 190 255 320 385 Offset-nanoseconds Offset-nanoseconds Frequency 80 60 40 20 0 Inexpensive oscillator, slow loop -390-325 -260-195 -130-65 0 65 130 195 260 325 390 Offset-nanoseconds s = 123 ns Frequency 100 80 60 40 20 0-260 10811D, slow loop -210-160 -110-60 -10 40 90 140 190 240 290 340 Offset-nanoseconds s = 92 ns Page 41
Repeater Connection 120 Inexpensive oscillator, fast loop s = 75 ns 200 10811D, fast loop s = 46 ns Frequency 100 80 60 40 20 Frequency 150 100 50 0-235 -200-165 -130-95 -60-25 10 45 80 115 150 185 0-150 -125-100 -75-50 -25 0 25 50 75 100 125 Offset-nanoseconds Offset-nanoseconds Frequency 120 100 80 60 40 20 0 Inexpensive oscillator, slow loop -300-255 -210-165 -120-75 -30 15 s = 80 ns 60 105 150 195 240 Frequenc 300 250 200 150 100 50 0-220 -185 10811D, slow loop s = 42 ns -150-115 -80-45 -10 25 60 95 130 165 200 Offset-nanoseconds Offset-nanoseconds Page 42
Direct Connection Frequency 250 200 150 100 50 0 Inexpensive oscillator, fast loop -145-125 -105-85 -65-45 -25-5 15 35 55 75 Offset-nanoseconds s = 34 ns Frequency 250 200 150 100 50 0-80 -65-50 -35-20 10811D, fast loop -5 10 25 40 Offset-nanoseconds s = 39 ns 55 70 More Frequency 140 120 100 80 60 40 20 0 Inexpensive oscillator, slow loop -320-270 -220-170 -120-70 -20 30 80 130 180 230 Offset- nanoseconds s = 76 ns Frequenc 600 500 400 300 200 100 0-35 -25 10811D, slow loop -15-5 5 15 25 35 45 Offset- nanoseconds s = 15 ns Page 43
Seconds tick deviations: connection via a repeater Page 44
Seconds tick deviations: connection via a repeater nanoseconds Page 45
For further information: John Eidson, chair 1588 committee, john_eidson@agilent.com September 24, 2003 IEEE1588 Workshop http://ieee1588.nist.gov Page 46