End-to-end Real-time Guarantees in Wireless Cyber-physical Systems

End-to-end Real-time Guarantees in Wireless Cyber-physical Systems Romain Jacob Marco Zimmerling Pengcheng Huang Jan Beutel Lothar Thiele RTSS 16 - IoT and Networking Session December 1, 2016

Predictability is key! 2

A Cyber-physical System goes beyond a real-time wireless network Predictability Real-time guarantees > Deadlines are met Wireless network Low-power Multi-hop Network deadline 3

A Cyber-physical System goes beyond a real-time wireless network Predictability Real-time guarantees > End-to-end deadlines are met Buffer management > No buffer overflow Actuator Sensor Wireless network Low-power Multi-hop Controller Network deadline End-to-end deadline 4

A Cyber-physical System goes beyond a real-time wireless network Predictability Real-time guarantees > End-to-end deadlines are met Buffer management > No buffer overflow Local interference Communication Application 5

Design goals Blink Bolt DRP Predicatability Network Device System Real-time guarantees Buffer management Adaptability Efficiency Composability 6

First piece A predictable and adaptive wireless network Actuator Sensor Wireless network Low-power Multi-hop Controller 7

First piece A predictable and adaptive wireless network Actuator Sensor Wireless network Low-power Multi-hop Controller 8

State-of-the-art wireless protocols are predictable or adaptive Splash, RAP Efficient Adaptive > not Predictable WirelessHART Predictable Efficient > not Adaptive 9

Blink A real-time, reliable and [1] adaptive wireless protocol Adaptive Reliable Real-time Based on Glossy > Flooding primitive Average 99.97% reception rate > Multiple testbeds > Tested up to 94 nodes Online scheduling > EDF-based Lazy Scheduling [1] Zimmerling M. et al., Adaptive Real-time Communication for Wireless Cyber-physical Systems To appear in ACM Transactions on Cyber-Physical Systems, 2016 10

Blink A real-time, reliable and [1] adaptive wireless protocol Abstraction Low-power Wireless Bus (LWB) > MAC protocol Communication TDMA-based in rounds > Sleep between rounds > Wireless yields sync! Variable! Node 1 Node 2 T net t t Node N C net M slots t [1] Zimmerling M. et al., Adaptive Real-time Communication for Wireless Cyber-physical Systems To appear in ACM Transactions on Cyber-Physical Systems, 2016 11

Blink A real-time, reliable and [1] adaptive wireless protocol Abstraction Low-power Wireless Bus (LWB) > MAC protocol Communication TDMA-based in rounds > Sleep between rounds > Wireless yields sync! Variable! Node 1 Node 2 Node N C net T net Sched(n+1) M slots t t t [1] Zimmerling M. et al., Adaptive Real-time Communication for Wireless Cyber-physical Systems To appear in ACM Transactions on Cyber-Physical Systems, 2016 12

Design goals Blink Bolt DRP Network Device System Real-time guarantees Buffer management na Adaptability Efficiency Composability na 13

Second A dual-processor architecture piece to mitigate local interference Actuator Sensor Wireless network Low-power Multi-hop Controller 14

Second A dual-processor architecture piece to mitigate local interference Actuator Sensor Wireless network Low-power Multi-hop Controller 15

Bolt A dual-processor architecture [2] to mitigate local interference Application Tasks Application Processor (AP) Receive Buffer BOLT API flush read write BOLT Incoming Outgoing BOLT API write flush read Communication Processor (CP) Receive Buffer Wireless Comm. Tasks Real-time behavior Formally verified Implemented, tested, deployed Efficient μw > sleep mw > active Composable Hardware/Software free composition [2] Sutton F. et al., Bolt: A Stateful Processor Interconnect, SenSys 15, 2015 16

Design goals Blink Bolt DRP Network Device System Real-time guarantees Buffer management na Adaptability Efficiency Composability na 17

Design goals How can we fill the blanks Blink Bolt DRP Network Device System Real-time guarantees Buffer management na Adaptability Efficiency Composability na 18

Third piece Distributed Real-time Protocol (DRP) Blink Bolt DRP Network Device System Real-time guarantees Buffer management na Adaptability Efficiency Composability na 19

DRP is based on three main concepts Communication is constrained within registered flows only Global requirements are splited across distributed components Interaction between components is based on contracts 20

DRP is based on three main concepts Communication is constrained within registered flows only Global requirements are splited across distributed components Interaction between components is based on contracts 21

Communication is constrained within registered flows only Flow i F i = ( n i s, n i d, T i, J i, D i ) Source Destination Min. release interval Jitter End-to-end deadline Release model Sporatic with jitter 22

DRP is based on three main concepts Communication is constrained within registered flows only Global requirements are splited across distributed components Interaction between components is based on contracts 23

Global requirements are splited across distributed components Real-time guarantee DRP : D i s + D i net + D i d = D D i s D i net D i d Source Wireless protocol end-to-end deadline : D Destination 24

DRP is based on three main concepts Communication is constrained within registered flows only Global requirements are splited across distributed components Interaction between components is based on contracts 25

Interaction between components is based on contracts > Node Network Contract CONSTRAINTS ON LOCAL SCHEDULES > Max resource demand Min service delivered > Predictability Performance A A B B 26

Example Scheduling of Communication Processors (CP) Real-time Participate in rounds guarantees > T net Blink schedule Flush before rounds Write after rounds > T net Blink schedule Buffer Flush Bolt regularly s management > T f Comm. Processor schedule 27

Example Scheduling of Communication Processors (CP) How can we guarantee that all tasks are schedulable C f C net C w T net Participate in rounds Flush before rounds Write after rounds Flush Bolt regularly t T f s t 28

Example Scheduling of Communication Processors (CP) How can we guarantee that all tasks are schedulable C f C net C w T net Variable! Participate in rounds Flush before rounds Write after rounds Flush Bolt regularly t T f s Variable! t 29

Example Scheduling of Communication Processors (CP) How to guarantee that all tasks are schedulable C f C net C w T net Variable! Participate in rounds Flush before rounds Write after rounds Flush Bolt regularly t T f s Variable! Conflict! t 30

Example Scheduling of Communication Processors (CP) How to guarantee that all tasks are schedulable C f C net C w T net Variable! Participate in rounds Flush before rounds Write after rounds Flush Bolt regularly t Solution T f s Variable! TDMA approach C CP C f + C net + C w > T net k C CP t 31

Example Scheduling of Communication Processors (CP) How to guarantee that all tasks are schedulable C f C net C w T net Variable! Participate in rounds Flush before rounds Write after rounds Flush Bolt regularly t Solution T f s Fixed to min TDMA approach C CP C f + C net + C w t > T f s C CP > T net k C CP 32

Example Scheduling of Communication Processors (CP) How to guarantee that all tasks are schedulable C f C net C w T net Variable but constrained Participate in rounds Flush before rounds Write after rounds Flush Bolt regularly t Solution T f s Fixed to min TDMA approach C CP C f + C net + C w t > T f s C CP > T net k C CP 33

+ Predictability of Network + Predictability of Devices + DRP contracts = System predictability Worst-Case Ultimately Analysis End-to-end latency Buffer size Can we set local parameters such that = f( Local parameters ) End-to-end deadline Memory space Flush interval d Flush interval T f net Network deadline D i T f s If YES System predictability by design 34

Design goals Blink Bolt DRP Network Device System Real-time guarantees Buffer management na Adaptability Efficiency Composability na 35

Design goals Blink Bolt DRP Predicatability Network Device System Real-time guarantees Buffer management na Adaptability Efficiency Composability na 36

From there, adaptability is one (close) step away Can we set local parameters such that ADMISSION TESTS End-to-end latency Buffer size End-to-end deadline Memory space 37

From there, adaptability is one (close) step away Example Communication Processor ADMISSION TESTS Depends only on local parameters! 38

Adaptability is achieved via a distributed registration scheme end-to-end latency D i f(t f s, D i ) g(t f d ) AP s BOLT CP s BLINK CP d BOLT AP d Packet flow PHYSICAL LAYER T i, J i D i LOGICAL LAYER REGISTRATION SCHEME Compute D i request : T i, J i D i, D i Bolt buffer and YES Network YES Local Y local memory OK schedule OK memory OK YES Bolt buffer and T f d OK ADMISSION TESTS Abort Register flow and update schedule nack ack NO NO NO NO nack nack nack Abort Abort Abort Abort Register flow ack Register flow and update schedule ack Register flow ack Register flow and update schedule YES update : T f d 39

Design goals Blink Bolt DRP Predicatability Network Device System Real-time guarantees Buffer management na Adaptability Efficiency Composability na 40

The detailed system analysis allows for performance optimization Minimal admissible end-to-end deadline D min = δ const f + T min + T min + δ const d g + T f,min AP s BOLT CP s BLINK CP d BOLT AP d Packet flow end-to-end latency D min 41

The detailed system analysis allows for performance optimization Minimal admissible end-to-end deadline D min = δ const f + T min + T min + δ const d g + T f,min AP s BOLT CP s BLINK CP d BOLT AP d Packet flow Bolt delays end-to-end latency D min Source delay > Message release by the source > Available for communication 42

The detailed system analysis allows for performance optimization Minimal admissible end-to-end deadline D min = δ const f + T min + T min + δ const d g + T f,min AP s BOLT CP s BLINK CP d BOLT AP d Packet flow end-to-end latency D min Network delay: Waiting time > Message available > Message processed by the network 43

The detailed system analysis allows for performance optimization Minimal admissible end-to-end deadline D min = δ const f + T min + T min + δ const d g + T f,min AP s BOLT CP s BLINK CP d BOLT AP d Packet flow end-to-end latency D min Network delay: Transmission > Message processed by the network > Message transmitted to the destination node 44

The detailed system analysis allows for performance optimization Minimal admissible end-to-end deadline D min = δ const f + T min + T min + δ const d g + T f,min AP s BOLT CP s BLINK CP d BOLT AP d Packet flow end-to-end latency D min Destination delay: Bolt > Message transmitted to the destination node > Message available in the Bolt queue 45

The detailed system analysis allows for performance optimization Minimal admissible end-to-end deadline D min = δ const f + T min + T min + δ const d g + T f,min AP s BOLT CP s BLINK CP d BOLT AP d Packet flow end-to-end latency D min Destination delay: Application > Message available in the Bolt queue > Message retrieved by the destination application 46

The detailed system analysis allows for performance optimization Minimal admissible end-to-end deadline D min = δ const f + T min + T min + δ const d g + T f,min Given Packet size 32 Bytes C net 1 s d T f,min 0.1 s > D min 3.46 s Max data rate 29.7 Bps per flow 47

Design goals Blink Bolt DRP Predicatability Network Device System Real-time guarantees Buffer management na Adaptability Efficiency Composability na 48

Using flooding primitives enables the design of both adaptive AND predictable Wireless Cyber-Physical Systems 49

End-to-end Real-time Guarantees in Wireless Cyber-physical Systems Romain Jacob Marco Zimmerling Pengcheng Huang Jan Beutel Lothar Thiele RTSS 16 - IoT and Networking Session December 1, 2016

Simulation correlates closely with the analysis Typical simulation trace result 49