Network Programming. Simulation Engines 2008 Chalmers University of Technology. Markus Larsson

Transcription

1 Network Programming Simulation Engines 2008 Chalmers University of Technology Markus Larsson Simulation Engines 2008, Markus Larsson 1

2 Networked games Most games are played against some form of opponent Last lecture we spoke about computer controlled opponents This lecture we will discuss human controlled opponents Human opponents often allows a lot more interesting gameplay Massively Multiplayer Online Games are interesting examples Simulation Engines 2008, Markus Larsson 2

3 Example: Counterstrike One of the most successful online games ever Hundreds of thousands of active players Simulation Engines 2008, Markus Larsson 3

4 Example: World of Warcraft Ridiculously successful MMOG, released by Blizzard on November 23rd, accounts were created during the first day Simulation Engines 2008, Markus Larsson 4

5 Networking issues Regardless of whether we want to create a small scale networked game such as Counterstrike or a huge MMOG like WoW the following issues need to be looked in to Which network architecture is suitable for our game? Client/server or peer to peer (or a combination)? What kind of network protocols are useful for a computer game? How will we handle the issues of high lag (latency), low bandwidth, and packet loss that exist on the Internet? The type of game naturally affects which kind of network programming techniques we will use. What differs from a small scale and a massive scale multiplayer game? Chalmers has many interesting courses on networking programing in general and Internet programming in particular Simulation Engines 2008, Markus Larsson 5

6 Fundamentals Packet Network message. All information sent over a network is split up into indivisible packets. Packets can be lost and may have to be resent depending on the protocol. Latency Network lag or latency is the delay associated with the time between sending a message from a source and receiving it at the destination. Bandwidth Maximum information rate that can be sent over a particular network connection (remember to distinguish bandwidth from latency!) Simulation Engines 2008, Markus Larsson 6

7 Fundamentals Packet loss Most networks are not perfect, and packets might be lost due to corrupted links or collisions. Packet loss is a measure of how high ratio of all packets are lost in transmission. Protocol Data format specification for embedding packets as well as routines for reliability, acknowledgment, transfer, etc. Network architecture Topology of a particular network (or subset of a network) involved in a particular system or protocol instance. Most common examples in this context include peer to peer (P2P) and client server (C/S) architectures Simulation Engines 2008, Markus Larsson 7

8 Terms more related to games Lobby service Players often need a virtual meeting place outside the game itself where they can meet, chat, and form games. This place is often called a lobby, and a lobby service is a middleware allowing for the creation of such lobbies. Dead reckoning The imperfect nature of the Internet means that games will be plagued by high latency and high packet loss. This means that we will often be unable to send frequent updates of the object state to the players participating in a game. In order to avoid jerky in game behaviour, each client extrapolates the position of the objects given their old position and velocity. Security There are lots of malicious users on the Internet and we must take care to protect our game as well as the players from these. Security in multiplayer games is a whole lecture in itself (maybe even a whole course!), but we will just talk briefly about this subject Simulation Engines 2008, Markus Larsson 8

9 Protocols Definition A protocol is a convention or standard that controls or enables the connection, communication, and data transfer between two computing endpoints Simulation Engines 2008, Markus Larsson 9

10 Protocols A protocol typically defines the following Detection of the underlying physical connection (wired or wireless), or the existence of the other endpoint or node Handshaking Negotiation of various connection characteristics How to start and end a message How to format a message What to do with corrupted or improperly formatted messages (error correction) How to detect unexpected loss of the connection, and what to do next Termination of the session or connection Simulation Engines 2008, Markus Larsson 10

11 Protocol families Of the protocol families listed here, we are mostly interested in the last one, Internet protocol suite Simulation Engines 2008, Markus Larsson 11

12 Socket programming The standard way of doing Internet programming is to make use of a network socket library Berkeley sockets originated with the 4.2BSD system in 1983 The Windows implementation of the BSD Socket API is called WinSock and includes a number of extensions specific to network sockets on Windows Definition of Socket A socket can be used in computer networking to form one end of a bi directional communication link between two programs Simulation Engines 2008, Markus Larsson 12

13 Socket programming The following header files are involved in C programming with BSD sockets under UNIX (similar on Windows) sys/socket.h Definitions for the most basic of socket structures sys/types.h Basic data types associated with structures within the API netinet/in.h Definitions for socketaddr in and other base data structures sys/un.h Definitions and data type declarations for SOC UNIX streams sys/select.h Definitions for the use of the select family of functions Simulation Engines 2008, Markus Larsson 13

14 Socket programming A socket is an endpoint in a two way communication link between two parties. An Internet (TCP/IP) socket is (explicitly or implicitly) bound to a specific address (hostname and portnumber) and can be connected to another socket with another address Potentially on another computer Simulation Engines 2008, Markus Larsson 14

15 Internet adress Internet addresses are 32 bit numbers, usually represented as four bytes on the form xxx.yyy.zzz.www They are also identified by a 16 bit port number Normal connection sockets do not care about the exact address, but a server usually binds a specific server socket to a pre defined address Such a server socket listens for inbound connections and then creates a new socket for the actual client server communication Only one socket can be bound to a specific address at any time Port numbers below 2000 are ordinarily reserved Port numbers below 1024 need Root access to bind HTTP is port 80, FTP is port 21, Telnet is port 23, etc Simulation Engines 2008, Markus Larsson 15

16 Socket tutorial A brief overview Simulation Engines 2008, Markus Larsson 16

17 Socket tutorial: Server code int sockfd, newsockfd, portno, clilen, n; struct sockaddr_in serv_addr, cli_addr; char buffer[256]; sockfd = socket(af_inet, SOCK_STREAM, 0); // (1) bzero((char *) &serv_addr, sizeof(serv_addr)); serv_addr.sin_family = AF_INET; serv_addr.sin_port = htons(port_number); serv_addr.sin_addr.s_addr = INADDR_ANY; bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)); // (2) listen(sockfd, 5); // (3) clilen = sizeof(cli_addr); newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen); // (4) bzero(buffer, 256); n = read(newsockfd, buffer, 255); // (5) printf("message: %s\n", buffer); Simulation Engines 2008, Markus Larsson 17

18 Socket tutorial: Client code int sockfd, portno, n; struct sockaddr_in serv_addr; struct hostent *server; sockfd = socket(af_inet, SOCK_STREAM, 0); // (1) server = gethostbyname(server_name); bzero((char *) &serv_addr, sizeof(serv_addr)); serv_addr.sin_family = AF_INET; bcopy((char *) server->h_addr, (char *) &serv_addr.sin_addr.s_addr, server->h_length); serv_addr.sin_port = htons(port_number); connect(sockfd, &serv_addr, sizeof(serv_addr)); // (2) fgets(buffer, 255, stdin); write(sockfd, buffer, strlen(buffer)); // (3) Simulation Engines 2008, Markus Larsson 18

19 Managing several clients The client/server examples only support one client and one server In a real system, we will want to continue listening to other connections on the connect socket while simultaneously handling communication with the already connected clients as well There are two approaches to this Input/Output multiplexing Switch the communication sockets to be non blocking and use the select() system call to manage input and output from a number of sockets in a sequential way (select() allows you to wait for input/output on a set of sockets). Introduce concurrency Implement concurrency in our server program so that different parts of the server can handle the clients and the connections simultaneously. The traditional UNIX way is to fork() child processes off the main server process to handle clients; this is however not very practical for a game server where clients co exist in the same virtual world, so here it is better to assign individual threads to users Simulation Engines 2008, Markus Larsson 19

20 UDP datagrams The previous example used TCP (Transmission Control Protocol) for the transfer TCP is a reliable stream based protocol that guarantees ordered delivery of all information sent over the socket Bad connectivity and high packet loss might produce a large overhead when resending and reordering packets UDP (User Datagram Protocol) is an unreliable message based protocol where information is sent in variable sized chunks called datagrams UDP datagrams have no guarantee of delivery, but this is sufficient for non critical game information In order to create an UDP socket, we specify SOCK DGRAM instead of SOCK STREAM when creating our socket. In addition, we use the sendto() and recvfrom() calls for sending and receiving datagrams, respectively Simulation Engines 2008, Markus Larsson 20

21 Final words about sockets It often makes sense to have two connections per client A TCP socket for game critical information (messages, deaths, etc) A UDP socket for non critical information (position updates, etc) TCP like functionality can be manually recreated using UDP Simulation Engines 2008, Markus Larsson 21

22 Network architectures The network architecture of a game specifies the topology of the hosts involved in the game and their communication structure and protocol Typically, game developers will implement their own game protocol on top of TCP/UDP and design it to fit the network architecture selected Client/Server Star network with a central server through which all clients (hosts) communicate. The server may or may not be an active participant in the game itself. Client/Server cluster Similar as C/S architectures, except the server load is shared among a cluster of interconnected computers instead of a single one. Peer to Peer No central server, all peers communicate (more or less) directly with each other. Truly distributed system Simulation Engines 2008, Markus Larsson 22

23 Client/Server A client/server multiplayer game has two kinds of entities Clients Servers Benefits The normal computers playing the game over the network The coordinating hosts storing the universal game state and arbitrating the actions of individual clients Relatively easy to implement Synchronization and consistency easy Cheating can be controlled by the server Communication is minimized Drawbacks All communication goes through the server Usually requires planning in the design stages of a game, adding it last will be difficult Single point of failure; If the server goes down, the whole game goes down Simulation Engines 2008, Markus Larsson 23

24 Client/Server Simulation Engines 2008, Markus Larsson 24

25 Peer to peer A peer to peer networked game has no single server, but rather the world state is shared among the various peers which are all Benefits No single point of failure, no bottleneck Simple implementation Easy to build on top of an existing game Drawbacks Lots of communication Synchronization and consistency can become difficult Scalability is poor without an extremely well thought out design Easy to cheat on clients Firewalls can be a major pain in the *** Simulation Engines 2008, Markus Larsson 25

26 Peer to peer Simulation Engines 2008, Markus Larsson 26

27 Advanced P2P techniques There are many obvious drawbacks to P2P mechanisms for multiplayer games, here are a few possible solutions Poor scalability Clearly, not everyone in the world needs to know about every event that affects other peers, especially if they are not geographically close. Use an interest management technique to filter out relevant information. Costly communication Use techniques for broadcast and multicast (group communication) to cut down on communication costs (problem: not supported by all network hardware, must be implemented in software). Easy to cheat Involve a trusted (or randomly selected) third party to arbitrate all critical transactions (i.e. ask C whether A hit B). Can be difficult (but not impossible) to solve. Many of these areas are still being researched and very few commercial games use peer to peer communication Simulation Engines 2008, Markus Larsson 27

28 Break 15 minutes Simulation Engines 2008, Markus Larsson 28

29 Network programming for games An extremely basic multiplayer game loop might look like this 1. Receive messages from other peers (or the server). 2. Process messages, updating local copy of game state accordingly. 3. Send message about my status to other peers (or server). 4. Render the frame and perform other world updates. In the following slides, we will look at the issues that effect these steps as well as take a peek at middleware solutions that can be of assistance Simulation Engines 2008, Markus Larsson 29

30 A few issues The internet The Internet is dreadful for multiplayer gaming simply due to his high lag, low bandwidth, and high packet loss. We need to come up with ways to provide a playable game despite this. Encryption We may have to encrypt our network data to avoid eavesdropping by malicious users. Firewalls Many players are behind firewalls on the Internet, yet still want to participate in multiplayer games. Denial of service Players can be denied access to a game, usually through flooding Simulation Engines 2008, Markus Larsson 30

31 Lag, bandwidth and packet loss Three fundamental issues that we must face as network programmers Limited bandwidth The data transfer rate over a network is limited and certainly magnitudes smaller than that within the internal memory bus of a computer. Packet loss Networks are unreliable and packets are often lost in transit. High latency The delay between sending and receiving a message over the network can be long, especially for games played over the Internet Simulation Engines 2008, Markus Larsson 31

32 Managing bandwidth Static data Caching Transmit information once and thereafter use a integer identifier for it Initialization Transmit static information once when the client connects, never during actual game play Data compression Packing Use an algorithm to tightly pack messages; encode booleans as bits, floats as fixed point numbers, etc Compression Use a data compression scheme (i.e. Huffman coding) to compress messages Relevance filtering Define a metric to decide whether a specific event is relevant or not for a specific client Simulation Engines 2008, Markus Larsson 32

33 Managing bandwidth Message prioritization Important messages should not be clogged down by less important messages; implement a priority scheme to allow client programmers to specify the importance of a message. Incremental updates Game objects rarely change their whole state at the same time. We cannot rely on incremental updates indefinitely; sooner or later, packet loss will ensure that the players view of the game objects will drift from their true state. Once in a while we need to synchronize game object state Simulation Engines 2008, Markus Larsson 33

34 Managing packet loss TCP does a good job of ensuring reliable message transfer, but the considerable overhead associated with the protocol is not a desirable property for a real time networked simulation Similarly, the totally unreliable UDP protocol is also not suitable for a real time simulation where we may have to ensure message delivery for some messages Two basic solutions Use two connections per client, one TCP and one UDP. This is a workable solution that is used by a few games, but which results in slightly more complex network code than for just one connection. Implement our own protocol on top of UDP that provides us with the features we need Simulation Engines 2008, Markus Larsson 34

35 Our own UDP based protocol We have already remarked that there are different kinds of network data Important and less important Because of this, it makes sense to provide different policies in our network protocol Guaranteed order Messages are required to be guaranteed delivery at the destination in the same order they were sent. This is the strongest transmission policy and is essentially the same as TCP. Guaranteed Messages are required to be delivered at the destination, but not necessarily in the order they were sent. Unguaranteed Messages have no requirements on delivery; they are processed by the destination if they arrive and are not resent if they do not Simulation Engines 2008, Markus Larsson 35

36 Managing latency Latency (or lag) is the delay from the sender sending a message to the receiver actually receiving it Caused by hardware and the limited speed of light In network gaming we are often mostly interested in roundtrip times (the time for server >client >server) Roundtrip times of 250 milliseconds are not uncommon All clients involved in a network game see an outdated version of the world; only the server knows the real version Simulation Engines 2008, Markus Larsson 36

37 Managing latency Interpolation Interpolation is used perform smooth transitions between old and new object state (usually position and velocity) to avoid visible pops. Extrapolation We take our best guess at the new state of an object given its past state and predict future behaviour Positional state is easily extrapolated by adding the velocity vector times the time difference to the old position vector If we know the acceleration vector of the object, we can update the velocity vector as well Client side prediction Player controlled entities move unpredictably and cannot be easily extrapolated In some cases, it is possible to forward the input commands of one player to observing players so that they can more accurately predict the player's behavior Simulation Engines 2008, Markus Larsson 37

38 Security and encryption We have virtually no control over which networks our data is sent, and in many cases we send sensitive information (i.e. credit card information etc.) Symmetric encryption Two parties communicating over a shared but insecure connection and sharing a common and secret key The key is used for both encoding and decoding the data so that no malicious eavesdroppers can read it Requires secure key exchange Simulation Engines 2008, Markus Larsson 38

39 Security and encryption Assymetric encryption Also known as public key cryptography, this scenario also involves two parties on an insecure connection, but here each user has a public and a private key. A user s public key is used to encode messages for a that user so that only the user himself can decode it using his private key. Computationally much more expensive than symmetric encryption Very useful for key exchange for symmetric encryption Message authentication In order to detect malicious tampering with messages, we can tag each message with a message authentication code (MAC) generated using a secure cryptographical hashing function Digital signatures and certificates Certificates and signatures allow for the verification of the public key of a user to avoid Man in the Middle attacks Simulation Engines 2008, Markus Larsson 39

40 Distributed objects In a game networking layer it might be a good idea to raise the abstraction level from sockets and packets to distributed objects One useful metaphor for implementing distribut,ed objects is the RPC mechanism (Remote Procedure Call) Allows nodes to remotely execute methods in an object over the network Allows us to implement our game objects as first class objectoriented distributed entities living on the network and which support method invocations Also allows programmers to specify properties for different parts of the state of a game object to control updates and relevance filtering For example, a client in a first person shooter can receive a network object reference to a Player object residing on the server, allowing it to call methods such as Player.Shoot(), Player.Run(), Player.Duck(), etc Simulation Engines 2008, Markus Larsson 40

41 Existing game networking APIs We will take a brief look at two existing game network engines OpenTNL The open source version of the Torque Network Layer Quazal Net Z A commercial multiplayer engine Simulation Engines 2008, Markus Larsson 41

42 Torque Network Layer The Torque Network Layer (TNL) and OpenTNL is the network layer on the Torque engine UDP based delivery notification connection protocol Two phase connect for prevention of IP spoofing attacks Client puzzle system for server CPU depletion Denial of Service protection ECC key exchange and certificate authentication AES symmetric encryption with SHA 256 message authentication Efficient packet streaming architecture for consistent bandwidth utilization Bit level compression for optimal bandwidth utilization Huffman encoded strings and common substring skipping Simple and efficient event and RPC (remote procedure call) framework Extensible master server framework Simulation Engines 2008, Markus Larsson 42

43 Quazal Net Z Quazal Net Z API is a commercial multiplayer engine for games Used in Company of Heroes, Supreme Commander, Hitman, etc The key feature is that it deals with distributed game objects described using DDL (Data Description Language) Client programmers only have to worry about integrating Net Z objects into their game code Simulation Engines 2008, Markus Larsson 43

44 Quazal Net Z Duplicated objects Decentralized, distributed objects describing all the network relevant information about that game object Duplicated object migration Duplication master objects can migrate from one station to another DObject & session fault tolerance When a station fails, any distributed objects will migrate to another station to ensure that the object survives the fault Dead reckoning Net Z s dead reckoning (data extrapolation) reduces bandwidth usage by avoiding sending unnecessary updates across the network Latency masking techniques Application topology Objects are known globally but programmers can dictate who controls what Synchronized user inputs Functionality for to propagating and synchronizing user inputs across the network Simulation Engines 2008, Markus Larsson 44

45 Massively Multiplayer Online Games The rise of the Internet led to the development of text only multiplayer games known as MUDs MUDs, modern computer graphics and conventional single player games eventually combined to spawn a whole new generation of MMOGs Some of the first commercial 3D MMOGs include Meridian 59, Everquest and Asheron's Call Nowadays people tend to think about games such as WoW, Eve Online, Lineage These games show potential for a lot of business but bring a whole new array of network issues Simulation Engines 2008, Markus Larsson 45

46 Special issues for MMOGs Network architecture With thousands of simultaneous connected users, the issue of which network architecture to choose is difficult: a single server can quickly become a bottleneck. Cluster architectures are common. Security and cheating Competitive online games are very welcome targets for malicious users who want to gain an upper hand. Therefore, we must work hard to minimize possibilities of tampering with the client or the network traffic to cheat. Reliability, patching and updating The more players we have connected to our server, the more unhappy customers we will get if the server goes down (planned or not). We have to deal with issues how to update (or patch ) the game as well as the clients Simulation Engines 2008, Markus Larsson 46

47 Special issues for MMOGs Relevance filtering Managing the interest of players and only sending them information that is relevant to them is vital in a game with thousands of connected users. Operating system limits In some cases, we may for instance have to increase the total number of open sockets in the operating system to support a huge online game Simulation Engines 2008, Markus Larsson 47

48 Example: Quazal Eterna Eterna is Quazal's middleware framework for MMOGs Based on the same principles as Net Z, but has been adapted to support large scale network applications Duplication spaces Dynamically configurable objects that partition the virtual world based on a publish/subscribe mechanism. Configurable network topology Control over how information is sent over the network, including restrictions on where Duplicated Objects are created, where they reside, who may discover them, and who may modify them Simulation Engines 2008, Markus Larsson 48

49 Example: Quazal Eterna Load balancing Duplicated objects migration allowing for exploiting the server cluster s available processing power and reduce bandwidth usage via workload distribution. Fault tolerance Automatic migration of duplicated objects to ensure their survival. Hierarchical messaging Individual stations (server or client) to update all of their duplicas using a hierarchy of intermediate stations Simulation Engines 2008, Markus Larsson 49

50 Summary Multiplayer functionality in computer games has become more and more important as connectivity increases There are lots of issues plaguing networked games that we have to take into account; network programming for games is (much) more than sockets and packets Different network architectures are suitable for different kinds of games The three major factors of network programming are low bandwidth, packet loss, and high latency We also have to consider aspects of security and encryption Massively multiplayer online games (MMOGs) introduce a whole new set of issues that we have to deal with to create a successful game Simulation Engines 2008, Markus Larsson 50