Developing deterministic networking technology for railway applications using TTEthernet software-based end systems Project n 100021 Astrit Ademaj, TTTech Computertechnik AG
Outline GENESYS requirements - railway Time-triggered communication TTEthernet SW based implementation of the TTEthernet Conclusion ARTEMISIA Association Title Presentation - 2
GENESYS GENeric Embedded SYStems Instruction how to build your embedded systems architecture GENESYS: is a reference architecture template providing specifications and requirements to design a cross domain embedded systems architecture. architecture style supports a composable, robust and comprehensible, component based framework with strict separation of computation from message based communication distinguishes between 3 integration levels: Chip Level (IP cores communicate via a deterministic Network-on-a-Chip) Device Level (Chips communicate within a device) System Level (Devices communicate in an open or closed environment) ARTEMISIA Association Title Presentation - 3
GENESYS and the railway domain Safety-critical applications in the railway domain require deterministic communication networks robustness and composability are key issues. GENESYS architecture style supports a composable, robust and comprehensible, component based framework with strict separation of computation from message based communication distinguishes between 3 integration levels:. System Level (Devices communicate in an open or closed environment) ARTEMISIA Association Title Presentation - 4
TTEthernet TTEthernet is a suitable candidate to implement the integration at the system level For the railway domain and not only Composability/determinism and robustness are key issue Enables a cost effective implementation of design diversity. ARTEMISIA Association Title Presentation - 5
Time-Triggered communication Predictability Time-Triggered Communication Scalable Composability Properties at the component level remain unchanged after integration TT communication architectures suitable Reduce testing and certification efforts Easy implementation of fault-tolerance mechanisms Robustness Fault containment Error containment Replication of components ARTEMISIA Association Title Presentation - 6
What time-triggered systems need Any Time-Triggered System must have two key properties: a notion of time in case of a distributed system: a GLOBAL notion of time, available to each node in the system a schedule (when to do what) in case of a distributed system: a GLOBAL schedule or CONSISTENT parts of a GLOBAL schedule available to each node in the system ARTEMISIA Association Title Presentation - 7
Example: Time-Triggered vs Event- Triggered Transportation cars and taxis are event-triggered: they go whenever they are needed buses and trains are time-triggered: they go according to a fixed schedule Advantage of the event-triggered approach: very flexible Advantage of the time-triggered approach: very predictable When would you prefer a time-triggered solution? ARTEMISIA Association Title Presentation - 8
Nondeterminism under Peak Load Peak load can corrupt established system properties The communication network fails to provide the properties established in functional verification timing, latencies, error rates increase massively The control functions fail to respond to external influences (e.g. steering commands) in time unspecified or unacceptable behavior occurs for complex electronic systems with high availability, reliability, and safety requirements peak load scenarios must be avoided by design! ARTEMISIA Association Title Presentation - 9
Addressing Peak Load in Critical Systems How can you ensure that a system stays reliable under load? A time-triggered system uses the same amount of resources and provides the same amount of throughput all the time In cases of low load, this performance is wasted But in cases of high load, unexpected loads or faults, no peak occurs throughput no peak load 100 % ideal system time-triggered system load requirements wasted area ARTEMISIA Association Title Presentation - 10
Composability with a Time-Triggered communication Properties established at the component level are maintained after the system integration the properties of the communication schedule are defined by the system integrator before implementation and integration the schedule is distributed consistently to all nodes all nodes can communicate only according to this schedule Integration does not change anything it only completes the communication pattern which was incomplete for each subsystem. ARTEMISIA Association Title Presentation - 11
Composability with a Time-Triggered communication Communication schedule as designed by the system integrator - contains one spare (green) slot Communication schedule for Subsystem A only a R cd hk sop ab Rt fg hk so a R cfg hk sop ab Rt fg hk so a cd hk ab fg hk a cfg hk ab fg hk Communication schedule for Subsystem B only R Rt R Rt sop so sop so Communication schedule for Subsystem C can be added without affecting A and B ARTEMISIA Association Title Presentation - 12
Fault Tolerance and design diversity Fault-tolerance is implemented by replication of system components Two or more components perform the same services in parallel and provide their output simultaneously to mask failures of one of them. Tolerance against design failures design diversity Use different specification to implement the same service ARTEMISIA Association Title Presentation - 13
What is TTEthernet? A TT communication system, which integrates real-time and non real-time traffic into a single communication infrastructure integrate traffic with different characteristics (requirements) in a flexible way switched topology Support application with different criticality requirements data acquisition, multimedia, real-time control app., safety-critical applications. ARTEMISIA Association Title Presentation - 14
What is TTEthernet (2)? In principle we can enable any carrier protocol with time-triggered technology, but there are some good reasons for using Ethernet. Ethernet is a well-established open-world standard Scalable. Bandwidth (10 Mbit/s, 100 Mbit/s, 1Gbit/s, 10Gbit/s) COTS Ethernet hardware is low cost. Existing tools can be leveraged cost-efficient monitoring tool (e.g., Wire Shark) for maintenance and configuration (ssh, web servers, ). ARTEMISIA Association Title Presentation - 15
TTEthernet Topology Consist mainly of TTE-Switches, TTE- End Systems (ES) and Standard Ethernet End Systems TTE switch Eth TTE switch TTE TTE TTE TTE TTE Eth ARTEMISIA Association Title Presentation - 16
TTEthernet features Time-Triggered comm systems are deterministic, composable and scalable real-time comm. network compatible with IEEE 802.3. Standard Ethernet traffic does not affect the properties of the real-time traffic. End Systems HW based (dedicated chip/component) or SW based The software-based TTEthernet software based implementation uses COTS Ethernet controllers showcasing that TTEthernet can be implemented on any Ethernet compliant hardware. thus providing a cost-efficient and flexible technology implementation, allowing the usage of design diversity ARTEMISIA Association Title Presentation - 17
Ethernet Standard IEEE 802.3 IEEE 802.3 addresses the lowest layers of the ISO/OSI reference model, some higher layers are represented by other IEEE 802 parts. TTEthernet performs services transparently within the Data Link layer, using all IEEE 802.3 services without modification. 7 6 5 4 3 2 1 Application Presentation Session Transport Network Data Link Physical ISO/OSI layer model architecture, NM, layers above (TCP,UDP,IP) Logical Link Control (IEEE 802.3 LLC) Media Access Control (IEEE 802.3 MAC) Physical Layer (IEEE 802.3 PHY) 10BaseT 100BaseTx 1000BaseCX ARTEMISIA Association Title Presentation - 18
TTEthernet Traffic Classes TTE-frames - compatible to the standard Ethernet frame format. Destination MAC address is use to identify the frames, where the first 4 bytes represent the critical traffic marker (cluster ID), the last 2 bytes the critical traffic identifier (denoted also as message ID). Schedule ID VLID TTEthernet traffic classes Time-Triggered - TT (hard real-time) configuration required Rate-Constraint RC Best Effort - BE (or Event-triggered ET, or background - BG) ARTEMISIA Association Title Presentation - 19
Virtual Links End-Systems exchange frames through Virtual Links (VLs) A Virtual Link defines a unidirectional path from one End-System to one or more destination End-Systems VL 1 ES ES Network ES ES VL 2 ARTEMISIA Association Title Presentation - 20
TTEthernet TT traffic class Time-Triggered (TT) used for periodic exchange of messages sending instant is triggered by the time - statically configured schedule constant transmission delay and small and bounded jitter networks can be utilized fully (close to maximum) due to the possibility of strictly deterministic communication scheduling each TTE frame is transmitted by the end system at a certain time the switch expects the frame from the transmitter within a certain time interval (window) this provides an implicit bus guardian functionality: TTE traffic received outside of the expected time interval is discarded switch forwards the frame to the receivers (end systems or other switches) at certain times - these times can be different for each port! receivers receive the frame with well-defined latency and minimal jitter Best Effort (BE) Rate-Constraint (RC) ARTEMISIA Association Title Presentation - 21
TTEthernet scheduling configuration Senders have a defined transmit schedule Switches have an acceptance schedule for incoming data Switches have a forwarding schedule per port VL ID 1 2 3 8 Sender a @ 07:30 a @ 09:00 b @ 10:00 b @ 11:15 Receiver(s) b @ [07:40-07:50]; d @ [8:20-8:30] c @ [10:30-10:40]; c @ [10:20-10:30]; e @ [10:25-10:35],f @ [10:30-10:40] a @ [11:30-11:40]; f @ [11:30-11:40] a g VL ID 3 8 Time @ 10:00 @ 11:15 b 3 c d e f ARTEMISIA Association Title Presentation - 22
TTEthernet Traffic Classes Time-triggered - TT Best Effort- BE Best effort traffic - BE (also denoted as event-triggered or background traffic) usually are used for sporadic exchange of event information sending instant is driven by an event transmission delay unknown BE messages are stored in the switch messages in the queue are processed on the FIFO order BE messages shall be transmitted when communication medium is free of TT traffic fully compatible with Ethernet standard Rate-Constraint RC ARTEMISIA Association Title Presentation - 23
TTEthernet Traffic Classes Time-Triggered (TT) Best-Effort (BE) Rate-Constraint (RC) - AFDX RC traffic class is defined by its End-to-end transmission latency, BAG bandwidth allocation gap Jitter. BAG defined the maximum amount of bytes (or frames) per time interval. RC traffic can be shaped within an End-System, in order to ensure BAG times. Different priorities ARTEMISIA Association Title Presentation - 24
Fault Isolation Restricted access for configured VL Traffic Filtering Firewalling At Switch and ES VL not configured at Switch Host ES Switch VL not configured at ES ES Host ES Host ARTEMISIA Association Title Presentation - 25
SW based TTEthernet solution TTEthernet End System Protocol Stack can be implemented in any general purpose computer that has a standard Ethernet interface. supporting not only the features of TT communication systems for predictability, composability, robustness but also the flexible way for design diversity as different ES can be implemented into different targets. ARTEMISIA Association Title Presentation - 26
SW architecture TTEthernet TTEthernet core protocol is HW and OS independent TTE_API - message handling - status and diagnosis - control and configuration HW_API (low-level API) - for the Ethernet controller - API_ETH_CTRL - API for the HW timer - API_HW_TIMER ARTEMISIA Association Title Presentation - 27
SW based TTEthernet core TTEthernet core is HW and OS independent. It contains: Initialization, Start-up Dispatching/scheduling of action points according to the configuration Clock synchronization TTE message transmission and reception. BG message transmission and reception. Task execution. Error handling It provides the API functions to the host application It uses the low-level API functions for the Ethernet (reading incoming messages and triggering the start of transmission) and timer unit. Implements the TTE state machine, which is triggered by the timer interrupts (timer driver). ARTEMISIA Association Title Presentation - 28
Ethernet and Timer driver Ethernet driver contains the functions for: Ethernet controller initialization and configuration Allocation of Ethernet buffers and buffer descriptor for transmission (TX) and reception (RX) Frame handling Frame transmission Managing RX buffer descriptors (frame reception is handled automatically by the Ethernet HW unit) Raising interrupts on frame reception Timer driver Configuring one programmable timer with timer interrupt. Timeout function implementation Ethernet and Timer drivers are HW and OS dependent Porting of SW based TTEthernet is equivalent with the development of these two drivers ARTEMISIA Association Title Presentation - 29
Middleware layer Linux example Host app. ET traffic Host app. TT traffic TT-Ethernet core protocol Fast Ethernet Controller driver HW timer driver HW System without OS support System with OS support ARTEMISIA Association Title Presentation - 30
Middleware layer To support the usage of existing operating systems mechanism Linux ( communication ET messages (background eth0 device driver TT messages eth1 device driver for all TT messages eth 1, eth 2, eth 3, eth n device driver for each TT messages Char device file for each TT message ARTEMISIA Association Title Presentation - 31
Performance 1.6 GHz CPU Intel ATOM, 1 GB RAM, 0.5 MB cache, Standard Linux OS Cluster cycle: 3 ms, 1 Sync msg/cycle Dummy application sending dummy TTE data with length of 1,500 bytes Measurement with Linux command top TTE Messages Bandwidth CPU Utilization Configuration 1 2 (1,500 bytes each) 8 MBit/s 1 % Configuration 2 6 (1,500 bytes each) 24 MBit/s 2 % Configuration 3 10 (1,500 bytes each) 40 MBit/s 3 % Configuration 4 15 (1,500 bytes each) 60 MBit/s 3 % ARTEMISIA Association Title Presentation - 32
Block print Approx 12 KLOC TTE-Core and configuration requires 20 KB memory Minimum of 20 KB necessary for Ethernet buffers memory Ported to different targets With no OS small memory 64 KB ENEA OSE OS Standard Linux Linux with RT extension Industrial PC (100 Mb/s and 1Gb/s) EeePC (100 Mb/s) ARTEMISIA Association Title Presentation - 33
Summary SW based TTEthernet cost effective way for implementing deterministic communications systems Design diversity Software based TTEthernet will be ported in the HW target used in the railway industry and it will be used for investigation of robustness services in the course of the INDEXUS project by using a railway app. ARTEMISIA Association Title Presentation - 34
INDEXYS: http://www.indexys.eu Thank you for your attention Astrit ADEMAJ, Senior TTEthernet Project Engineer TTTech Computertechnik AG Tel: +43 1 585 34 34 986 Mail-to: astrit.ademaj@tttech.com