CS 856 Latency in Communication Systems

Transcription

1 CS 856 Latency in Communication Systems Winter 2010 Latency Challenges CS 856, Winter 2010, Latency Challenges 1

2 Overview Sources of Latency low-level mechanisms services Application Requirements Latency Measurement Summary & Classification CS 856, Winter 2010, Latency Challenges 2

3 Low-level Mechanisms Memory Access (Hierarchy) Bus Contention Disk Access Asynchronous Processing CPU Scheduling Network Transmission packetization, processing, queueing, propagation CS 856, Winter 2010, Latency Challenges 3

4 Services System Calls Transport Messaging Remote Service Invocation Encryption CS 856, Winter 2010, Latency Challenges 4

5 Application Requirements Web Surfing Telecommunication Interactive Games Streaming User Interfaces Algorithmic Trading Monitoring & Control High-performance Computing CS 856, Winter 2010, Latency Challenges 5

6 Low-level Mechanisms CS 856, Winter 2010, Latency Challenges 6

7 Memory Access memory hierarchy registers O(1) cycles cache O(10) cycles RAM O(100) cycles relevant for effective CPU throughput e.g. routers: per-packet processing budget very small! CS 856, Winter 2010, Latency Challenges 7

8 Bus Contention multiple access & arbitration similar to link layer medium access control (later) CS 856, Winter 2010, Latency Challenges 8

9 Disk Access physical characteristics head movement, speedup/slowdown disk rotation I/O transfer via bus CS 856, Winter 2010, Latency Challenges 9

10 Asynchronous Processing cost of context-switching (and/or preemption) e.g. network stack socket queue network queue interface multi-threading send side: multiplexing with other traffic receive side: traditional interrupt levels OS and application programming CS 856, Winter 2010, Latency Challenges 10

11 CPU Scheduling control of asynchronous processing priorities & deadlines, real-time waiting time on ready queue -> latency CS 856, Winter 2010, Latency Challenges 11

12 Network Transmission transmission serialization of data: (file size / link speed) packetization: discrete units of data cost at each node vs. cut-through processing medium access control & back-off processing cf. memory latency cf. asynchronous processing CS 856, Winter 2010, Latency Challenges 12

13 Network Transmission (2) queueing well-studied queueing theory packet scheduling: average vs. worst-case propagation determined by fundamental physics e.g. global communication: Toronto Sydney ~10msec propagation (linear distance only) e.g. inter-planetary communication... CS 856, Winter 2010, Latency Challenges 13

14 Synchronous Networks e.g. SONET, CDMA no medium access control no queueing but: fixed upper bound rate cf. transmission delay CS 856, Winter 2010, Latency Challenges 14

15 Services CS 856, Winter 2010, Latency Challenges 15

16 System Calls blocking system calls typically I/O operations cf. asynchronous processing CS 856, Winter 2010, Latency Challenges 16

17 Transport network transmission signalling reliability timeout retransmission CS 856, Winter 2010, Latency Challenges 17

18 Messaging transport signalling encoding / decoding might include segmentation & reassembly network stack process interaction (system calls) CS 856, Winter 2010, Latency Challenges 18

19 Remote Service Invocation messaging and transport 2-way request / reply interaction e.g. name lookup e.g. web services (aka RPC) CS 856, Winter 2010, Latency Challenges 19

20 Transcoding e.g. encryption, video transcoding computational overhead offloading (e.g. special-purpose chip) throughput vs. latency gains? CS 856, Winter 2010, Latency Challenges 20

21 Application Requirements CS 856, Winter 2010, Latency Challenges 21

22 Web Surfing browsing - user expectations responsiveness: a few seconds 8- resp. 4-second rules commercial web services (B2B) retail: shopping, banking, payments avoid duplicate transaction do not hit reload! messages... online auctions, brokering, etc. -> much faster? CS 856, Winter 2010, Latency Challenges 22

23 Telecommunication telephony call setup and control authentication & authorisation e.g. cell phone roaming data human perception of interactivity good: one-way msec, end-to-end! e.g. packetization: 64 kbit in 40x 200-byte packets/sec 25 msec packetization vs. header overhead (cf. ATM) CS 856, Winter 2010, Latency Challenges 23

24 Telecommunication (2) videoconferencing similar delay bound than audio (also: lip-sync) need ~25 frames/sec for continuous presentation => upper bound on packetization collaborative work e.g. document editing instantaneous interaction locking and event ordering important CS 856, Winter 2010, Latency Challenges 24

25 Interactive Games event distribution / roundtrip FPS, Car Racing similar to video/telephony simulate continuous action trick human brain RT Strategy seconds ok? event order relevant ignore/overwrite old events CS 856, Winter 2010, Latency Challenges 25

26 Streaming startup delay several seconds ok cf. VCR, DVD live vs. stored content? no difference? latency variation => playout buffer CS 856, Winter 2010, Latency Challenges 26

27 User Interfaces asynchronous: multi-threaded or event-based need instantaneous response show at least brief immediate effect (button click) and/or drive continuous presentation e.g. video presentation + interactivity CS 856, Winter 2010, Latency Challenges 27

28 Algorithmic Trading receive electronic market data compute price quotes feed back into the market everybody else does the same speed matters, i.e., fast reaction to market data... at high volumes CS 856, Winter 2010, Latency Challenges 28

29 Monitoring & Control examples monitor factory automation monitor nuclear power plant control anti-lock brake system (ABS) reaction speed matters tight control loop CS 856, Winter 2010, Latency Challenges 29

30 High-Performance Computing distributed computation compete with integrated systems price/performance trade-off remote memory access communication and synchronization overhead CS 856, Winter 2010, Latency Challenges 30

31 Wrap-Up CS 856, Winter 2010, Latency Challenges 31

32 Latency Measurement round-trip latency easy to measure with local clock accuracy, statistics? one-way latency difficult requires synchronized clocks derive from round-trip? symmetry? CS 856, Winter 2010, Latency Challenges 32

33 Summary & Classification fundamental physical limitations propagation speed and latency contention: bus, MAC, router, etc. latency vs. throughput synchronous better for latency asynchronous better for throughput average vs. worst-case considerations CS 856, Winter 2010, Latency Challenges 33

34 Summary & Classification performance modelling memory and communication latencies chip, board, chassis, cluster, data center latency classes? machine (ns) computation human subconscious (ms) interactive human conscious (s) response time asynchronous (m) backup human asynchronous (h) CS 856, Winter 2010, Latency Challenges 34