Multicast Tree Reconfiguration in Distributed Interactive Applications

Transcription

1 Multicast Tree Recofiguratio i Distributed Iteractive Applicatios Carste Griwodz 1,2, Kut-Helge Vik 1, Pål Halvorse 1,2 1 IFI, Uiversity of Oslo, Norway 2 imula Research aboratory, Norway {griff, kuthelv, paalh}@ifi.uio.o Abstract Commuicatio i highly iteractive distributed applicatios, such as massive multiplayer olie games, ca ofte be performed efficietly usig multicast, i.e., applicatio level multicast. However, i applicatios with a very dyamic group maagemet, the multicast tree will have frequet chages, ad i applicatios that have striget latecy requiremet, this operatio eeds to be fast. Curret multicast approaches either have o otio of recofiguratio, they do ot care about tree recostructio latecy or wrogly assume that this is a fast, atomic operatio. I this paper, we have focused o dyamic recofiguratio ad have tested differet ways for a ode to joi a tree. Our results show that this is a importat issue for the class of highly iteractive distributed applicatios. I. INTRODUCTION arge improvemets i computer techology have eabled highly iteractive distributed applicatios such as distributed virtual eviromets, massive multiplayer olie games (MMOGs) ad computer supported cooperative work. These applicatios ofte iclude several types of media ragig from text to cotiuous media ad may have very striget requiremets with respect to the quality of the cliet data playout. I the MiMo project, we aim for better system support for such applicatios, i.e., tryig to make more efficiet use of the available resources by offerig mechaisms like predictio, area-of-iterest (AoI) maagemet, group commuicatio, aggregatio, replicatio, etc. I particular, we look at MMOGs due to the mix of differet media, the striget latecy requiremets, the dyamic cliet groups ad the fact that it has become a popular, fast growig, multi-millio idustry with a very high user mass. Today, MMOGs are icreasig i size ad complexity, supportig hudreds or thousads of cocurret players [1], ad they typically iclude a mixture of game situatios from role-playig games, first perso shooter games ad real-time strategy games. Players i the game move aroud ad iteract with other players, seemigly as if they were located ext to each other. Frequetly, may players iteract with each other ad the same object i the game world, these are the said to share a AoI or to be withi each others AoI. If the game is large eough, the majority of players will ot share a AoIs. Today, most MMOGs apply a cetral server approach for collectig ad processig game evets geerated by players, ad poit-to-poit commuicatio for the distributio of game state updates. They have few, if ay, mechaisms to optimize evet distributio. This approach is applicable because games are desiged with extremely low badwidth for players access etworks i mid. Eve these thi idividual data streams cotribute to game server scalability problems if the umber of players that share a AoI is high, forcig game desigers to prevet game situatios that ca lead to such clusterig. It would therefore be desirable to distribute the game evets efficietly, such that 1) the perceived game quality is above a satisfactory threshold regardless of system capacity ad geographical distace ad 2) the available resources are utilized efficietly. By groupig players accordig to their AoI, group commuicatio may be a efficiet meas to achieve this. ackig workig etwork layer multicast i the Iteret, we cosider applicatio layer multicast (AM) ad ivestigate group commuicatio algorithms appropriate for MMOGs to eable efficiet evet distributio. I this paper, we focus o the cost of multicast tree recofiguratio. There is a lot of existig work o multicast group maiteace. Both olie (dyamic trees) ad offlie (static trees) algorithms exist, but oly olie algorithms are applicable, sice players AoI chages frequetly. I fact, AoIs i a moder game ca be etirely player-specific ad have a adaptive size (as opposed to, e.g., room-sized AoIs), which implies that the membership of a group iteractig with a particular object ca be extremely volatile. Curret olie mechaisms do ot cosider tree recostructio latecy or wrogly assume that this is a fast atomic operatio. We have therefore tested several ways of recofigurig (joiig) a tree ad show that this is a importat issue for the class of highly iteractive distributed applicatios. The approaches tested do ot yet address the problem of o-trivial leave operatios where three or more eighbors are ivolved. We share this problem with much of the existig work, ad recogize from the preseted results that this is problematic. II. APPICATION EVE MUTICAT A MMOG may have players located all aroud the world. Thus, choosig a suitable group commuicatio style is importat, both i terms of improvig the service to the users ad optimizig the resource utilizatio of the system. Multicast provides efficiet meas for group commuicatio where the groups are typically orgaized i a tree structure. Oe implemetatio of multicast is IP multicast, but it is ot fully deployed i the Iteret ad lacks features like address filterig ad group membership cotrol. The alterative is AM. Approaches add group membership cotrol, ad make

2 it easy to support high level fuctioalities. Compared to IP multicast, AM is ecessarily less efficiet i terms of latecy but is easier to deploy. I highly iteractive distributed applicatios, especially MMOGs, each user frequetly both seds ad receives data from other users i the same group. I this sceario, we have three importat, but cotradictory, challeges: 1) The maximum delay betwee ay pair of odes must ot exceed a give threshold. 2) The total overall cost of trasmittig data i the multicast tree should be miimized. 3) Multicast tree recofiguratio must be supported olie, ad this operatio must be fast. I literature, a lot of differet algorithms have bee proposed, but curret multicast approaches have isufficiet support for fast recofiguratio which is required i a MMOG sceario due to for example players joiig ad leavig a group (eterig aother room), liks becomig overloaded (too may players sharig a lik), odes goig dow (a player quits), reachig maximum tree ode-to-ode latecy (icreased latecy o a lik due to cogestio), etc. As a first step towards a appropriate multicast algorithm i the MMOG sceario, we therefore look at the cost of olie multicast tree recofiguratio. We compare approaches for performig joi operatios, ad are also evaluatig tree cost if o additioal recofiguratio is performed. III. REATED WORK The costructio of low-cost, delay-bouded multicast trees has bee ivestigated several times before, ofte as teier tree heuristics. However, to the best of our kowledge, existig mechaisms do ot meet all of the required properties from sectio II. Recetly, some delay bouded, may-to-may multicast algorithms are proposed [2], [3], but these (as may oe-to-may approaches) assume scearios where all group members start the sessio at the same time. Olie multicast algorithms support dyamic tree maipulatios [4] [9]. Typical operatios iclude joi ad leave, ad some allow olie rearragemet of the multicast tree. A joi operatio is typically performed usig the shortest path [10], [11] or the delay costraied miimum cost path [12] [14]. This is cheaper tha tearig dow the tree ad re-buildig it from scratch, but it will probably give a larger cost icrease of the tree. A remove operatio deletes a ode from the multicast tree. A leaf ode is trivially removed. Deletig a ode with a degree of 2 results i two subtrees that for example are coected usig the least cost path [15] or least cost delay boud path [13]. If the ode has a larger degree ( 3), the ode is ofte kept i the tree for routig purposes, a approach that we follow as well. Additioally, to reduce the cost of the tree, some algorithms allow periodic tree rearragemet. The cost of the tree may be periodically calculated i the backgroud, ad a complete rearragemet operatio may be triggered based o some threshold value. It has bee proposed to divide the tree ito regios where the quality of the regio is associated with the umber of chages that have bee made without re-optimizig the regio [10], [12]. Existig approaches assume that recofiguratio operatios are atomic, i.e., that ew joi ad remove operatios do ot occur before previous rearragemet has bee completed. MMOGs do ot allow such assumptios. It is critical that tree recofiguratio is fast, i particular ode joi operatios as ew odes should receive the data o time. It is also vital that leave operatios keep the multicast tree itact for all remaiig odes. Rearragig the tree while the data is flowig may cause members to loose data if the operatio is ot hadled appropriately. Existig iteractive applicatios such as ACTIVE [16] perform joi operatios more quickly, but the performace of the operatio has ot bee give particular cosideratio, either. IV. IMUATION AND REUT To uderstad the time that is cosumed by differet approaches for quickly joiig a multicast group, ad the worst-case delays i the resultig overlay etworks, we use simulatio. I our simulatio, we distiguish odes that represet Iteret routers (the backboe etwork) ad odes that represet computers participatig the distributed game (cliets ad servers). We use the topology geerator BRITE [17] to create backboe etworks as graphs with 1000 odes. The lik delay betwee eighborig odes is uiformly distributed betwee 2 ms ad 50 ms. The servers ad cliets are placed i the etwork by attachig them radomly as leaf odes to the backboe etwork. Oe to three leaf odes are coected to each of the backboe odes, ad oe of them is maually selected as the game server. This is meat to model that the game is deployed idepedetly from ay etwork provider. We do t assume support for multicast either. ice we for ow are oly cocered with group maiteace ad ot data exchage, we igore badwidth i this simulatio. To compare topological effects, we have ru all simulatios with the backboes odes arraged i both flat ad hierarchical topologies. The hierarchical topologies orgaize them 10 etworks of 100 odes, respectively. A. Group membership Group membership i our simulatios is ot modeled after the behavior of ay actual applicatios, where odes are usually active members of a group for a loger while, ad where they are members of several groups at the same time. Rather tha that, our simulated cliets remai members of a group withi the game oly for a very short time, ad chage to aother group afterwards. This time is short with respect to the maximum ed-to-ed delay i our topologies, which is approximately 800 ms, a value that is take from real-world MMOG traces. We do this to ivestigate how well joi algorithms perform whe group membership is frequetly outdated. The cetral server is ot member of ay group. It provides a lookup service that allows cliets eterig a group to locate

3 2.JOIN 4.PING 5.PONG 6. CONNECT (first) REQUET 3.INVITE 2.JOIN 4.PING 5.PONG 6. CONNECT (best) REQUET 3.INVITE 3.PING 4.PONG 5.CONNECT (first) REQUET 3.CONNECT (closet address) REQUET (a) coect first with ivitatio (b) coect best with ivitatio (c) coect first without ivitatio (d) coect to closest address Fig. 1. Protocols variatios group members. Every group has a leader, which is the ode that has the smallest maximum delay to all other group members. Wheever a group leader determies that oe of its eighbors has a smaller maximum delay, it hads over group leadership. The ew leader iforms the cetral server of the chage. To ivestigate the performace of the algorithms both i equally sized ad heavily uequally sized groups, we create as may empty groups as the simulatio has cliet odes. To achieve equal group sizes, we let odes perform a radom walk betwee odes, where those are arraged i a square. To achieve uequally sized groups, we assig groups Zipfdistributed popularities. B. Protocol variatios Figure 1 shows the approaches for joiig groups that we have ivestigated i this work. The letter marks the server, the group leader, ad the cliet that tries to ewly joi a group. I all cases, a ode that jois a group cotacts a server first. However, this membership ad leadership iformatio will be outdated frequetly because average membership duratio is short with respect to the ed-to-ed delay i the system. I the sceario coect first with ivitatio (figure 1(a)), the server is otifyig the group leader of the itetio to joi, which will the sed o ivitatio to the ew ode that icludes the addresses of all group members. The ew ode cotacts all of those odes (PING). As soo as it receives a aswer (PONG) from oe of the odes, it coects itself to the overlay etwork formed by the group members at that ode. We expect that this approach would create deep graphs with a high worst-case delay iside groups but fast joi operatios. The sceario coect best with ivitatio (figure 1(b)) is early the same, but the joiig ode waits for aswers from all cotacts odes util it chooses to coect to the ode that has the lowest worst-case latecy to other group members. We expect that this approach would create wide graphs with slow joi operatios but a low worst-case delay iside groups. The sceario coect first without ivitatio (figure 1(c)) resembles the first sceario, but the server is sedig group membership iformatio to the joiig ode, istead of delegatig that operatio to the group leader. While this avoids the latecy that is added by cotactig the group leader first, the group membership iformatio at the server is usually less upto-date tha the group leaders, which may make it ecessary to retry the joi operatio more frequetly. The sceario coect to closest address (figure 1(d)) is the simplest oe. It is similar to the previous approach, but the joiig ode uses the etwork address to estimate physical distace, ad coects to the ode with the most similar oe. We expect very fast joi operatios but a rather high worstcase delay iside groups from this approach. No ode i ay of these approaches collects eough topological iformatio aywhere to be able to compute a optimal tree for the group. I fact, odes kow oly their eighbors ad the worst case delays achieved through these eighbors - all other iformatio becomes outdated too quickly because membership duratio is frequetly shorter tha the maximum delay betwee pairs of group members. C. Evaluatio We have compared the four variatios of our protocol by simulatio. To evaluate them, we use joi latecy ad the tree cost as performace metrics. Joi latecy is a issue accordig to challege 3 i sectio II, i.e., how fast a player ca resume the game-play after the AoI has bee chaged such that a group membership chage is required. Tree cost addresses challege 2, i.e., the total cost i terms of the sum of all edge delays idicatig the overall system performace. The figures show here use etworks with 1000 odes. Other etwork sizes show the same treds, but more sparsely populated etworks do ot show particular chages for the most popular groups. The time that a ode stays i a group is oly partly relevat for efficiecy of the multicast tree that is created accordig to our algorithm. I geeral, all of our approaches will lead to the same multicast tree o all time scales. However, i case of log ed-to-ed delays ad short membership duratio, the cotact iformatio that a joiig ode must retrieve from a cetral etity ( i figure 1) will frequetly be outdated. imilarly, o sigle ode i a multicast tree ca safely kow all other group members at a give time. To provoke this situatio, we have chose 100 ms for our simulatio. This is a frequetly used update frequecy i cetral server games, which limits also the frequecy of dyamic AoI chages.

4 e+07 1e+06 request, ivite, pig, coect first request, ivite, pig, coect best request, create, pig, coect first request, create, coect closests ip percet sum of delays (ms) request, ivite, pig, coect first request, ivite, pig, coect best request, create, pig, coect first request, create, coect closests ip 40 group umber (by popularity, 300 of 1000 total, every 5th show) group umber (by popularity, 300 of 1000 groups, every 5th show) (a) Autoomous system (a) Autoomous system e+07 1e+06 request, ivite, pig, coect first request, ivite, pig, coect best request, create, pig, coect first request, create, coect closests ip percet 55 sum of delays (ms) request, ivite, pig, coect first request, ivite, pig, coect best request, create, pig, coect first request, create, coect closests ip 40 group umber (by popularity, 300 of 1000 total, every 5th show) group umber (by popularity, 300 of 1000 groups, every 5th show) (b) Flat etwork (b) Flat etwork Fig. 2. Percetage of time spet eterig a group, average of 10 rus Fig. 3. um of all edge delays i a group, average of 10 rus I the figures, we show the results for Zipf-distributed group popularities: a small X value i the graph represets a large groups, while a large X value represets a small group. The graphs i figure 2 show the percetage of the time i a group that is used performig the steps described i sectio IV- B. Figure 3 shows the average cost of the overlay etwork coectig all odes i a group, where the cost is the sum of all lik delays. 1) Joi time: As expected, the approach that jois the ode that is closest by its etwork level address speds the highest percetage of its time actually i the group, while the other approaches sped more time performig the steps of figure 1. Eve for the case of groups that are upopular ad therefore frequetly foud empty, this is still true. I geeral, however, the algorithms perform similarly for those odes because early all time is cosumed i cotactig the cetral server for the iitial referece to the group. It is more remarkable that the more complex joiig approaches are also early as fast as coectig to closest address whe groups have a large umber of members. The reaso is that the joiig ode is offered may odes to coect to, some of which are boud to be close to it. The coect step itself is therefore fast. For the groups with a medium umber of odes, o the other had, coect first/best with ivitatio shows large delays. The problem here is that joi operatios are likely to trigger a leader chage i the group. ice the cetral server seds ivitatios oly to the group leader, these are frequetly refused because the cetral server s iformatio is already outdated. This, i tur, requires that the cetral server seds aother ivitatio to the ew leader, which is time-cosumig. 2) Group cost: Coect best with ivitatio achieves cosistetly the cheapest graphs (i terms of the sum of all uidirectioal delays) whe all groups have the same popularity. The created graphs are also cheapest i the Zipf sceario whe groups have medium or low popularity. However, whe the group popularity is high, this is ot the case aymore. The reaso for this is that we use the same leave

5 operatio that is described i sectio III, ie. we do ot remove a leavig ode from the multicast tree before the umber of its direct eighbours drop below three. Coect with ivitatio leads to wide graphs, while coect without ivitatio leads to deep graphs. Wide graphs are boud to have may odes with more tha three eighbors. Nodes close to the group leader ca therefore rarely leave the tree whe they decide to leave the group, ad coectios through them cotribute to the overall cost of the tree. 3) ummary: Our results show that the time to joi a multicast group ca be sigificat ad the chose protocol ca have large effects o the total cost. Additioally, ote that we oly have simulated simple protocols lookig at joi time. If calculatios to check for goals 1 ad 2 i sectio II should be performed before joiig, the joi operatio will cosume cosiderably more time. However, the fast joi operatios lead to a degradatio i tree costs for groups of medium group size. This implies that our further work o recofiguratio must support the removal of ier odes of a tree whe they leave groups. V. CONCUION I this paper, we have looked at the importace of reducig the multicast tree recofiguratio latecy i highly iteractive applicatios. We cosidered games i particular because there are example for cetral server games that beefit from extremely high group dyamicity, but we expect that a icreasig umber of distributed applicatios will require dyamic group membership. I this paper, we have compared several approaches for performig joi operatios, ad performed leave operatios oly whe the umber of ode eighbours is or drops below three. However, to give a complete view of this issue, several more tests must be performed, e.g., compariso with optimal tree costs, further protocol variatios, etwork variatios ad cost factors. Our results idicate that the recofiguratio operatio is a potetial bottleeck for highly iteractive applicatios. Our ogoig work icludes a further ivestigatio of olie recofiguratio mechaisms (icludig group leave operatios) i combiatio with differet treealgorithms, i particular teier tree algorithms, to be able to fid a proper trade-off of the cotradictig goals stated i sectio II. [6] Xiaohua Jia. A distributed algorithm of delay-bouded multicast routig for multimedia applicatios i wide area etworks. IEEE/ACM Trasactios o Networkig, 6(6): , December [7] Girish Kumar, Nishit Narag, ad C. P. Ravikumar. Efficiet algorithms for delay-bouded miimum cost path problem i commuicatio etworks. I Proceedigs of Iteratioal Coferece o High Performace Computig, Cheai (Madras), Idia, December [8] Aa Hac ad Kelei Zhou. A ew heuristic algorithm for fidig miimum-cost multicast trees with bouded path delay. Iteratioal Joural of Network Maagemet, 9: , [9] ih-chyau Wuu, og og i, ad hig-chyi hiao. Costructig delay-bouded multicast trees i computer etworks. Joural of Iformatio ciece ad Efieerig, 17(3): , [10] Fred Bauer ad Auja Varma. ARIE: A rearrageable iexpesive edge-based o-lie teier algorithm. I Proceedigs of the Joit Coferece of the IEEE Computer ad Commuicatios ocieties (IN- FOCOM), pages , a Fracisco, CA, UA, [11] Ehud Aharoi ad Reuve Cohe. Restricted dyamic teier trees for scalable multicast i datagram etworks. IEEE/ACM Trasactios o Networkig, 6(3): , Jue [12] riram Raghava, G. Maimara, ad C. iva Ram Murthy. A rearrageable algorithm for the costructio of delay-costrait dyamic multicast trees. IEEE/ACM Trasactios o Networkig, 7(4): , August [13] Tawfig Alrabiah ad Taieb Zati. Delay-costraied, low-cost multicast routig i multimedia etworks. J. Parallel Distrib. Comput., 61(9): , [14] Mehrdad Parsa, Qig Zhu, ad J. J. Garcia-ua-Aceves. A iterative algorithm for delay-costraied miimum-cost multicastig. IEEE/ACM Trasactios o Networkig, 6(4): , [15] Berard M. Waxma. Dyamic teier tree problem. IAM J. Discrete Math., 4: , [16] eslie. iu, Rahul Hampole, Beomjoo eo, ad Roger Zimmerma. Active: A low latecy P2P live streamig architecture. I Proceedigs of PIE/ACM Coferece o Multimedia Computig ad Networkig (MMCN), a Jose, CA, UA, Jauary [17] Alberto Media, Aukool akhia, Ibrahim Matta, ad Joh Byers. BRITE: Uiversal topology geeratio from a user s perspective. Techical Report BUC-TR , Bosto Uiversity, Jauary REFERENCE [1] eo ag-mi Whag ad Jee Yeo Kim. The olie game world as a product ad the behavioral characteristics of olie game cosumers as role player. I DIGRA 2005, Vacouver, Caada, Jue [2] Chor Pig ow ad Xueya og. O fidig feasible solutios for the delay costraied group multicast routig problem. IEEE Trasactios o Computers, 51(5): , [3] Hug-Yig Tya, Jeifer C. Hou, ad Bi Wag. May-to-may multicast routig with temporal quality of service guaratees. IEEE Joural of elected Areas i Commuicatios, 52(6): , [4] H. alama, Y. Viiotis, ad D. Reeves. A efficiet delay costraied miimum spaig tree heuristic. I Proceedigs of the 5th Iteratioal Coferece o Computer Commuicatios ad Networks (ICCCN), October [5] George N. Rouskas ad Ilia Baldie. Multicast routig with ed-to-ed delay ad delay variatio costraits. IEEE Joural of elected Areas i Commuicatios, 15(3): , 1997.