GamingAnywhere: An Open- Source Cloud Gaming Testbed

Transcription

1 1 GamingAnywhere: An Open- Source Cloud Gaming Testbed Chun-Ying Huang, De-Yu Chen, Cheng-Hsin Hsu, and Kuan-Ta Chen ACM Multimedia 2013 OSS Competition, Barcelona, Spain

2 2 Cloud Gaming is Hot Cloud gaming is expected to lead the future growth of computer games: 9 times in 6 years [CGR] T5-Labs [CGR]

3 3 What is Cloud Gaming Real-time game playing using light-weight clients

4 4 Selling Points of Cloud Gaming Gamers perspectives: Frees gamers from indefinitely upgrading their computers Eliminates setup overhead and compatibility issues for trying out a game, as everything needed will be settled in data centers by game operators Enables gamers to play games anywhere, anytime Game manufacturers perspectives: Allows developers to support more platforms Reduces the production cost Prevents pirating

5 5 Limitations of Existing Services OnLive demands for 5 Mbps for reasonable quality OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting up 3 data centers (CA, VA, TX) Only people who live in 1000 mile radius from a data center can play the game and more We, researchers, have tons of ideas to improve cloud gaming services, but all cloud gaming systems are proprietary and closed

6 6 Solution GamingAnywhere is the first cloud gaming platform for researchers, service providers, and users

7 7 Design Objectives Extensibility Components can be implemented as modules Features can be easily expanded Portability Cross-platform ready Configurability Exposes as many configurations as possible Openness It has been released to the public since April 2013

8 Cloud Gaming Scenario 8

9 9 System Architecture The client and the server, with many components Implemented by leveraging open-source packages Game Server Game Client Running the selected game Game console Agent Process/Thread Replay User Inputs (Keyboard, Mouse,...) Decode Input Events (Customized Protocol) Audio / Video Capturer Audio / Video Encoder RTSP / RTP / RTCP Audio / Video Player Audio / Video Decoder RTSP / RTP / RTCP User Inputs (Keyboard, Mouse,...) Encode Input Events (Customized Protocol) Control Flow Data Flow Internet

10 10 Implementation Overview GamingAnaywhere core is implemented as a library With several modules and executable Server components Video source (modularized, platform dependent) Audio source (modularized, platform dependent) Encoders (modularized, currently w/ ffmpeg) Replayer (modularized, platform dependent) RTSP/RTP server (in library, w/ ffmpeg) Client components RTSP/RTP client (using live555) Decoders (in library, w/ ffmpeg) Controller (using SDL) Renderer (using SDL)

11 11 License GamingAnywhere adopts 3-clause BSD license Less restrictive than GPL Libraries used by GamingAnywhere Library License Library License ffmpeg LGPL zlib zlib glibc LGPL lame LGPL libasound2 LGPL libvpx 3-clause BSD libsdl zlib opus 3-clause BSD libx11 LGPL x264 GPLv2 live555 LGPL libva MIT

12 12 GamingAnywhere Screenshots Server Client #2 Client #1 Client #3

13 14 Performance Evaluation Compare against OnLive and StreamMyGame Environment setup Sample results are presented

14 Response Delay Measurement For Closed Systems (OnLive and SMG) sec Response delay Screen Response delay (RD) = Network delay (ND) + Processing delay (PD) + Playout delay (OD) Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-Laung Lei, Measuring the Latency of Cloud Gaming Systems, ACM Multimedia 2011

15 16 t 0 (Key event sent) t 1 (Key event received) t 4 (Frame displayed) Menu screen shown Client t 3 (Frame received) Server t 2 (Frame sent) Menu frame Network delay (ND) : network RTT Processing delay (PD) : t 2 t 1 t 3 t 0 ND Playout delay (OD) : t 4 t 3

16 17 GA Has The Lowest Response Delay With GA: For a strict 100 ms RD requirement, ND can be as long as 52 ms From Barcelona to Greenland (~4000 km)

17 18 In 6 months Web: 16,103 visits, 9,689 unique visitors Forum: 53 topics, 178 posts

18 Visitor Distribution 19

19 20 Conclusion GamingAnywhere is the first open cloud gaming platform May be used by researchers, engineers, and gamers We hope it will stimulate more studies on cloud gaming, Codecs for cloud gaming Usability study for cloud gaming interfaces Cloud gaming in heterogeneous networks 3D/Stereoscopic cloud gaming and many more! We need your participation!

20 21 QUESTIONS? Join us at

21 BACKUP 22

22 23 Server Overview Threads Shared buffers Desktop/Game RTSP server thread Object owner (1n) handle clients (2a) write audio frames Audio buffer (4a) read audio frames (1a) audio capture Audio source (3a) wake up encoder Audio encoder (5a) encode and send (1v) video capture Video source (3v) wake up encoder Video encoder (5v) encode and send Data Flow Connections (RTSP/RTP/RTCP) (2v) write a video frame Video buffer (4v) read a video frame (2i) replay input events Input replayer (1i) receive input events Control Flow Connections

23 24 Process Video Frames w/ a Shared Buffer A shared buffer for a single screenshot The writer could be blocked by readers

24 25 Process Video Frames Process a video frame 1. Video source: Capture a frame 2. Converter: Convert from RGB format to YUV format 3. Encoder: encoding and packetization

25 26 Process Video Frames in Parallel Suppose the targeted inter-frame delay is t The response delay may greater than t frame capture + color space conversion + encoding It could degrade encoding bitrate Process in parallel Video source (frame capture) F1 F2 F3 F4 Color space converter (RGB to YUV) F1 F2 F3 Video encoder F1 F2 F3 t t+ t t+2 t t+3 t

26 27 Process Audio Frames Each audio encoder has an audio source buffer A queue for captured audio frames

27 28 Client Overview Threads Game Interaction Buffers SDL Rendering Input Events Object owner (3rv) decode and render video frames (3ra) decode and render audio frames (callback) (1i) receive input events Video buffer (2rv) buffer an encoded audio frames RTSP client thread (1r) receive encoded A/V frames Audio buffer (2ra) buffer encoded audio frames Main thread (2i) send input events Data Flow Connections Control Flow Connections

28 29 Video Playout Buffering The 1-frame buffering strategy Based on the RTP marker bit An H.264 frame can be split into different numbers of packets The marker bit (with a value of 1) indicates the last packet of a frame An example: 30KB frame transmitted in a 50Mbps network would have a buffering time of 30KB*8-bit/50Mbps =~ 5ms Frame #1 Frame #2 Frame #3 Frame #4 Frame #5 M M M M M RTP Packets time

29 30 Performance Evaluation (Cont d) Audio: The LAME encoder Video: The x264 encoder One of the VideoLan projects With the below default parameters $r is the bitrate variable, default 3Mbps --profile main --preset faster --tune zerolatency --bitrate $r --ref 1 --me dia --merange 16 --intra-refresh --keyint 48 --sliced-threads --slices 4 --threads 4 --input-res 1280x720

30 31 Delay Decomposition of GA Processing delay Memory lock + copy RGB to YUV x264 encoder Packetization (RTP packets) Playout delay 1-frame buffering S/W decoder SDL rendering

31 32 GA Incurs Low to Moderate Network Loads Configured to capture at 50 fps with a bitrate of 3 Mbps

32 GA Provides Superior Video Quality 33