GamingAnywhere: An Open Cloud Gaming System

Transcription

1 1 GamingAnywhere: An Open Cloud Gaming System Chun-Ying Huang, Cheng-Hsin Hsu, Yu-Chun Chang, and Kuan-Ta Chen ACM MMSys 13, Oslo, Norway

2 2 Tough Life of Gamers Games are becoming way too complex The overhead of setting up a game is significant Often locked on a specific computer Games may not be software or hardware compatible with some computers Computer hardware constantly demands upgrading

3 3 Cloud Gaming Real-time game playing via thin clients Streaming PC Game servers Streaming Streaming Laptop Internet Mobile

4 4 Selling Points of Cloud Gaming Gamers perspectives: Frees gamers from indefinitely upgrading their computers Eliminates setup overhead and compatibility issues for trying out a game, as everything needed will be settled in data centers by game operators Enables gamers to play games anywhere, anytime Game manufacturers perspectives: Allows developers to support more platforms Reduces the production cost Prevents pirating

5 5 Cloud Gaming is Hot Cloud gaming is expected to lead the future growth of computer games: 9 times in 6 years [CGR] T5-Labs [CGR]

6 6 Limitations of Existing Services OnLive demands for 5 Mbps for reasonable quality OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting up 5 data centers (CA, VA, TX, IL, GA) Only people who live in 1000 mile radius from a data center can play the game and more We, researchers, have tons of ideas to improve cloud gaming services, but all cloud gaming systems are proprietary and closed

7 Solutions: GamingAnywhere is the first cloud gaming platform for researchers, service providers, and users 7

8 8 Design Objective Extensibility Components can be implemented as modules Features can be easily expanded Portability Cross-platform ready Configurability Exposes as many configurations as possible Openness It will be released to the public soon

9 Cloud Gaming Scenario 9

10 10 System Architecture The client and the server, with many modules Implemented by leveraging open-source packages Game Server Game Client Running the selected game Game console Agent Process/Thread Replay User Inputs (Keyboard, Mouse,...) Decode Input Events (Customized Protocol) Audio / Video Capturer Audio / Video Encoder RTSP / RTP / RTCP Audio / Video Player Audio / Video Decoder RTSP / RTP / RTCP User Inputs (Keyboard, Mouse,...) Encode Input Events (Customized Protocol) Control Flow Data Flow Internet

11 11 Implementation Overview GamingAnaywhere core is majorly implemented as a library With several modules and executables Server modules Video source (platform dependent) Audio source (platform dependent) Encoders (various encoders via ffmpeg) Replayer (platform dependent) RTSP/RTP server (using ffmpeg) Client modules RTSP/RTP client (using live555) Decoders (via ffmpeg) Controller (using SDL) Renderer (using SDL)

12 12 Server Components Threads Shared buffers Desktop/Game RTSP server thread Object owner (1n) handle clients (2a) write audio frames Audio buffer (4a) read audio frames (1a) audio capture Audio source (3a) wake up encoder Audio encoder (5a) encode and send (1v) video capture Video source (3v) wake up encoder Video encoder (5v) encode and send Data Flow Connections (RTSP/RTP/RTCP) (2v) write a video frame Video buffer (4v) read a video frame (2i) replay input events Input replayer (1i) receive input events Control Flow Connections

13 14 Processing Video Frames Process a video frame 1. Capturer: Capture game frames (T capture ) 2. Converter: Convert from RGB format to YUV format (T convert ) 3. Encoder: Encoding and packetization (T encode )

14 15 Improving Game Frame Throughput Overall frame throughput 1/(T capture + T convert + T encode ) frames per second If (T capture + T convert + T encode ) is 40 ms, the maximum frame throughput is 25 fps A pipelined design for parallelism Improved frame throughput: 1/max(T capture, T convert, T encode ) If max(t capture, T convert, T encode ) is 20 ms, the frame throughput can achieve 50 fps Video source (frame capture) F1 F2 F3 F4 Color space converter (RGB to YUV) F1 F2 F3 Video encoder F1 F2 F3 t t+ t t+2 t t+3 t

15 17 Client Overview Threads Game Interaction Buffers SDL Rendering Input Events Object owner (3rv) decode and render video frames (3ra) decode and render audio frames (callback) (1i) receive input events Video buffer (2rv) buffer an encoded audio frames RTSP client thread (1r) receive encoded A/V frames Audio buffer (2ra) buffer encoded audio frames Main thread (2i) send input events Data Flow Connections Control Flow Connections

16 18 Video Playout Buffering The 1-frame buffering strategy An H.264 frame can be split into different numbers of packets The marker bit (with a value of 1) indicates the last packet of a frame Assuming 30 KB frames transmitted in a 50 Mbps network buffering time = 30 KB * 8 bits / 50 Mbps =~ 5 ms Frame #1 Frame #2 Frame #3 Frame #4 Frame #5 M M M M M RTP Packets time

17 19 GamingAnywhere Screenshots Server Client #2 Client #1 Client #3

18 LIVE DEMO 20

19 A demonstration of GamingAnywhere system. There are four devices in the photo. One game server (left-hand side laptop) and three game clients (an MacBook, an Android phone, and an ipad2). 21

20 22 Performance Evaluation Compare against OnLive and StreamMyGame Environment setup

21 23 Performance Evaluation (Cont d) Audio: The LAME encoder Video: The x264 encoder One of the VideoLan projects With the below default parameters $r is the bitrate variable, default 3Mbps --profile main --preset faster --tune zerolatency --bitrate $r --ref 1 --me dia --merange 16 --intra-refresh --keyint 48 --sliced-threads --slices 4 --threads 4 --input-res 1280x720

22 25 t 0 (Key event sent) t 1 (Key event received) t 4 (Frame displayed) Menu screen shown Client t 3 (Frame received) Server t 2 (Frame sent) Menu frame Network delay (ND) : network RTT Processing delay (PD) : t 2 t 1 t 3 t 0 ND Playout delay (OD) : t 4 t 3

23 26 GA Has Lower Response Delay Low response delay

24 27 Delay Decomposition of GA Processing delay Memory lock + copy RGB to YUV x264 encoder Packetization (RTP packets) Playout delay 1-frame buffering S/W decoder SDL rendering

25 28 GA Incurs Low to Moderate Network Loads Configured to capture at 50 fps with a bitrate of 3 Mbps

26 GA Provides Superior Video Quality 29

27 30

28 31 Conclusion GamingAnywhere is the first open cloud gaming system GamingAnywhere outperforms OnLive and StreamMyGame 66% 90% of processing delay reduction 3 19 db video quality improvement We hope it will stimulate more studies and applications on cloud gaming Server resource scheduling QoS/QoE measurements and provisioning Mobile cloud gaming and much more!

29 32 QUESTIONS? Thanks for your attention!