Better Quality, Lower Delay: Improving Realtime Video by Co-designing the Codec and the Transport

Transcription

1 Better Quality, Lower Delay: Improving Realtime Video by Co-designing the Codec and the Transport Sadjad Fouladi, Emre Orbay, John Emmons, Riad Wahby, Keith Winstein Stanford University

2 Outline Introduction Design & Implementation End-to-end Measurement of Realtime Video Evaluation & Results 2

3 Realtime video: intelligent cameras

4 Realtime video: video conferencing

5 Realtime video: remote control

6 Realtime video in a nutshell Encoder transport Decoder 6

7 Many things rely on realtime video Realtime video applications require low delay (and good quality). Many of these application may be on a variable link. 7

8 The problem: current systems do not work well on variable links 1. Transport tells encoder what bitrate to target [tens of milliseconds pass...] 2. Encoder delivers next frame 3. Transport stuck sending what encoder created Encoder overshot the bitrate blows up delay Network changed blows up delay 8

9 Enter Salsify Salsify introduces: A video-aware congestion control. A network-aware video codec. Salsify s traffic more closely match the network s evolving capacity. Salsify consistently achieved higher video quality and lower delay than Hangouts, Skype, FaceTime, and WebRTC. 9

10 Outline Introduction Design & Implementation End-to-end Measurement of Realtime Video Evaluation & Results 10

11 Three problems with current realtime video systems Problem 1: No data on how big of a frame we can send without blocking. Video-aware congestion control Problem 2: The current video codecs cannot hit the exact frame size. Providing transport with multiple options Problem 3: It's not possible to mix-and-match different encoding streams. Functional video codec 11

12 1) No data on how big of a frame we can send without blocking. Video-aware congestion control Salsify s transport protocol estimates the frame size that it thinks the network can handle without dropping or excessively queuing. 12

13 1) No data on how big of a frame we can send without blocking. Video-aware congestion control frame i frame i+1 Sender Receiver t₁ t₂ grace t₃ t₄ t₅ period The receiver treats each coded video frame as a packet train intended to probe the network. Receiver maintains moving average of packet inter-arrival times, pausing inference during grace period. 13

14 2) The current video codecs cannot hit the exact frame size. Providing transport with multiple options 100 KB original 75 KB 40 KB 10 KB Transport late-binds the decision of which version to send. 14

15 3) It's not possible to mix-and-match different encoding streams. Functional video codec Why can't we just run multiple encoder instances with different quality levels? 15

16 Existing video codecs only expose a simple interface encode([,,..., ]) frames[1:n] decode(frames[1:n]) [,,..., ] 16

17 The decoder is an automaton frame frame frame state state state state 17

18 We cannot mix-and-match multiple video streams A1 B1 C1 A2 B2 C2 A3 B3 C3 18

19 We cannot mix-and-match multiple video streams A1 B1 C1 A2 B2 C2 A3 B3 C3 19

20 Video codec in explicit state-passing style encode(state, image) frame, image)) decode(state, frame) (state, image) state frame state frame state frame state 20

21 3) It's not possible to mix-and-match different encoding streams. Functional video codec By using the functional video codec, we can encode the options based on the current state. 21

22 1. Transport to Encoder: Give me the next frame. 22

23 2. Encoder to Transport: Here s a menu of N different encodings. better same worse 23

24 3. Transport to Encoder: I sent option #3. Save that state, restore it to all N encoders, then go back to step 1. 24

25 2. Encoder to Transport: Here s a menu of N different encodings. better same worse 25

26 3. Transport to Encoder: I sent option #1. Save that state, restore it to all N encoders, then go back to step 1. 26

27 2. Encoder to Transport: Here s a menu of N different encodings. better same worse 27

28 3. Transport to Encoder: I sent option #1. Save that state, restore it to all N encoders, then go back to step 1. 28

29 2. Encoder to Transport: Here s a menu of N different encodings. better same worse 29

30 3. Transport to Encoder: I sent option #2. Save that state, restore it to all N encoders, then go back to step 1. 30

31 Outline Introduction Design & Implementation End-to-end Measurement of Realtime Video Evaluation & Results 31

32 End-to-end measurement of realtime video Reproducible input video Reproducible network traces Run unmodified version of the system-under-test Target QoE metrics: Frame delay (from the camera at the sender, to the screen at the receiver) Image quality 32

33 33

34 34

35 Outline Introduction Design & Implementation End-to-end Measurement of Realtime Video Evaluation & Results 35

36 Network throughput (synthetic link) 36

37 Network throughput (synthetic link) 37

38 Network throughput (synthetic link) throughput (Mbps) Salsify Skype 1 WebRTC time (s) 38

39 Frame delay (synthetic link) 39

40 Network trace (Verizon LTE) 16 Salsify 15 video quality (SSIM db) Skype WebRTC Better FaceTime Hangouts Salsify (no late binding) video delay (95th percentile ms) 40

41 Network trace (AT&T LTE) 16 Salsify 15 video quality (SSIM db) WebRTC Better Hangouts FaceTime Skype video delay (95th percentile ms)

42 Network trace (T-Mobile UMTS) 14 Salsify 13 video quality (SSIM db) WebRTC Hangouts Skype FaceTime Better video delay (95th percentile ms) 42

43 Takeaways Salsify introduces: A video-aware congestion control. A network-aware video codec. Salsify s traffic more closely match the network s evolving capacity. Salsify consistently achieved higher video quality and lower delay than Hangouts, Skype, FaceTime, and WebRTC. excamera@cs.stanford.edu 43

44

45 End-to-end measurement of realtime video Sender Receiver Network Simulator AV.io Video System Measurement System 45

46

47