Analysis of BitTorrent and its use for the Design of a P2P based Streaming Protocol for a Hybrid CDN

Transcription

1 Analysis of BitTorrent and its use for the Design of a P2P based Streaming Protocol for a Hybrid CDN Karl-André Skevik, Vera Goebel, Thomas Plagemann {karlas,goebel,plagemann}@ifi.uio.no Department of Informatics University of Oslo Abstract Peer-to-peer (P2P) based networks have several desirable features for content distribution, such as low costs, scalability, and fault tolerance. However, they fail to provide guarantees for content delivery. In order to combine the desired features of classical Content Distribution Networks (CDNs) and P2P based networks, we propose a hybrid CDN structure with a P2P based streaming protocol in the access network. Our proposal is based on an empirical analysis of BitTorrent. In our BitTorrent measurements we made two important observations: (1) clients with high bandwidth connections leave the system shortly after they have downloaded the file, and (2) clients that are unable to accept incoming connections, likely because they are behind a firewall, suffer from a significant reduction in download speed. Our design attempts to address these problems by structuring the streaming protocol in a way which makes it possible to discourage freeloaders, and by incorporating a proxy based structure which can avoid performance problems due to firewalls. The proxy based structure also makes it possible to incorporate caching, which has often been identified as lacking in P2P networks. This structure will also make further reduction in bandwidth usage possible by having clusters automatically created at the lowest levels of the hierarchy. 1 Introduction Content Distribution Networks (CDNs) represent an area which has been the subject for research in many years, but where only recently the basis for widespread deployment has become available, with increasing availability of broadband Internet connections among private consumers. More powerful computers have also made more efficient video compression techniques possible. Most CDN related research has focused on streaming issues and QoS support, which are important properties and are also considered in our work, however to 1

2 improve the foundations for deployable CDNs we focus our work especially on the following four properties: Low cost. The solution should be easily deployable for the provider and not consume a significant amount of resources as with large scale CDNs. This should also result in lower cost of content for users. Fault tolerance. The solution should be without any single point of failure, with regard to both operation and content. This would mean that clients should be able to download content without requiring the presence of a central server once a download has been initiated. Content should also be duplicated, so multiple download locations exist for a given data file. Scalability. The solution should be scalable with regards to number of users and amount of content. Ideally being able to support ten or hundred thousand users. Replication of data, which is beneficial to fault tolerance, allows load distribution and improves scalability. Content quality. There should be a clearly identifiable source for content, with a provider guaranteeing for the consistency of the content. We regard Peer-to-peer (P2P) networks as an interesting basis for a CDN system, due to the potential for both scalability and fault tolerance in a P2P network. A desirable feature of typical CDNs and the WWW which is often lacking in P2P networks is the ability to quickly and independently of any central organization set up a server which is universally accessible. Use of caches in CDNs can significantly reduce network load, and is a feature which is lacking in many P2P networks. Downloading from the official address of the content provider also guarantees for the authenticity and quality of the content, in clear contrast with content found on typical P2P networks. A big problem with CDNs is their potentially high cost, which makes it desirable to spread the cost between users in a way similar to that which is done in P2P networks. We propose a hybrid solution which aims to combine the best features of CDNs and P2P networks. Our solution also extends the operation of caches to provide proxy functionality, in order to avoid a firewall related performance issue which is quantified in the measurements. The remainder of this paper is structured as follows. Section 2 describes related research, while Section 3 gives a general overview over BitTorrent. Our measurements are described in Section 4 and their results in Section 5. Our proposed design is detailed in Section 6. Section 7 gives our conclusions. 2 State of the Art Work related to this paper generally falls into two groups; other CDN systems and P2P network analysis work. We start by looking at the first. 2

3 Chaining [1] uses multicast streaming of VoD data. Clients cache data for later multicasting to late-comers, resulting in a transmission chain starting from the main server. Application-level multicast [2] [3] [4] [5] [6] is an alternative to standard multicast which does not require infrastructure support, but might result in duplicate packets being transmitted on the same physical link. The general techniques often improve fault tolerance, scalability, and reduce load on the main server. End System Multicast [2] for example, uses application-level multicast; clients forward data to other nodes, making streaming of broadcast data possible. A general problem with application level multicast applications is that they as the name indicates are often designed primarily for distribution of live content, even if some also can be used for Content on Demand (CoD) streaming. There are several systems for streaming CoD among, using P2P networking techniques, scalability and redundancy is usually achieved as a result, and load is spread among participants. In P 2 VoD [7] clients cache recently received content and can forward it to new clients if the sufficient outgoing bandwidth is available. Nodes are grouped into generations, based on cache content. Failures are handled locally without involving the content server. [8] supports QoS, and is meant for closed environments such as large companies, universities and hotels. Gnustream [9] is built upon the Gnutella [10] P2P file distribution network. P2Cast [11] uses the patching technique for P2P based streaming. DirectStream [12] uses a directory server for storing content and keeping information about clients. PALS [13] uses adaptive layered streaming. Promise [14] is a streaming system built upon the P2P network Collectcast [15]. Collectcast monitors the network and peer status to select the best peers. None of these systems appear to take firewalls into consideration, or use dedicated caching. Techniques for downloading content from multiple mirror sites such as [16] face similar challenges as receiver driven streaming in P2P networks. MDC [17] also encodes the requested data, and sends it from multiple servers in a CDN over different paths, resulting in increased fault tolerance. A similar strategy for receiver driven streaming from multiple mirror servers using FEC is given in [18]. There are many different P2P based file distribution networks such as KaZaA, edonkey2000 and Gnutella, but these networks have file searching and indexing as a fundamental feature, which is not desired in our system. Research on the behavior of P2P networks and the potential benefits of caching are presented in [19] [20]. BitTorrent was also analyzed in [21], which examined the lifetime of a torrent, measured over five months, and found it to be very efficient and to handle flash crowds well. 3 BitTorrent BitTorrent is a P2P based file distribution application that seems to achieve most of the goals for CDNs we have stated earlier, i.e., it seems to be easily 3

4 deployable, independent of other servers, fault tolerant, and scalable. It differs from many other P2P networks in that it doesn t implement functionality such as searching. This functionality can be obtained through integration with the WWW. In BitTorrent, a group of clients cooperate in the distribution of a set of data, usually a single file, thereby spreading the load of serving the data among the participants. The information required to participate is stored in a file which can be downloaded with a normal web browser. The fact that the actual data is not downloaded via HTTP, but rather a P2P protocol, can be made transparent from the viewpoint of the user. BitTorrent divides a file into pieces of 256KB which are exchanged between hosts. The protocol is designed in a way which discourages freeloaders, by having the nodes prefer peers from which data has been received; the download speed of a host will be reduced if the upload speed has been limited. This ensures that content will be spread among hosts and improves reliability. Each host informs new clients of the pieces it currently has, and sends notifications when new pieces are received. A BitTorrent session starts with a single server and the file which is being served or seeded. A client which has downloaded the entire file also serves as a seed for the file until the user aborts the application. The initial seeder does not have to be available all the time; after duplicates of all pieces exist, the original seeder can withdraw. The only result will be that one less seeding host exists; the other hosts can continue the session and serve new clients without any problems. Having many seeds is beneficial, since these hosts contribute to the reliability of the session by serving data, and they do not place any load upon it by downloading. 4 Measurements To make the measurements the standard BitTorrent client [22] was modified to store the protocol interaction between clients to disk. No data was downloaded, but maintaining connections to as many clients as possible was attempted, while the interaction between the other clients was examined. This method does not give a perfect overview of all events; sometimes connections are lost, and the information reported by the clients might not be correct, but it gives an indication of the behavior of the clients in the session. The results which were obtained also correspond to that which would be expected based on usage of the application. This data was combined with the output from a crawler [21], which regularly retrieved the list of peers from the tracker. An application called TorrentSniff was used to get the number of seeds and downloaders from the tracker. When a client first connects to the tracker, the tracker appears to do a reverse connect to the client. If it succeeds, the new client is added to the list of potential peers, otherwise it is not added. This test ensures that the peer list contains hosts which should be able to accept incoming connections. A subset 4

5 of this peer list, is returned to clients upon request. Adding hosts which are unable to accept incoming connections would result in all clients wasting time trying to connect to hosts which are unable to accept connections. While the actual operation of the tracker is unknown, the observed behavior matches the description above. Examination of the traffic between the client and the tracker showed a reverse connection from the tracker when the client was not behind a firewall, and the IP address of this client was shortly after returned in the peer list received by another client. Connecting to the network from a host which was behind a firewall never resulted in it being returned on the peer list. The crawler was used to regularly request the list of peers from the tracker. Even though the peer list is expected to contain only hosts which allow incoming connections, the crawler was sometimes unable to connect to some of the hosts in the list. This is most likely due to the host becoming unavailable or the user ending the download session, and not firewall related. The peer information returned from the tracker can be used as a basis for determining if a given peer is behind a firewall or not; a host which has been returned in the peer list can be assumed to have been open at some point, otherwise it is most likely behind a firewall. This is not entirely reliable though, temporary network problems might cause the reverse connection from the tracker to fail, resulting in a misclassification of the host. The modified BitTorrent client attempts to detect misclassified hosts by reverse connecting to the hosts it receives a connection from. In summary, the client communicates with other clients through a connection which it either initiates itself, by using the peer list received from the tracker, or the client receives a connection from another host. Hosts which are behind a closed firewall are only able to make outgoing connections to hosts which are not similarly disadvantaged. We have monitored several BitTorrent sessions, including RedHat torrents and MandrakeLinux torrents, over several months. In all measurements, we execute the modified client and the crawler on a PC in our Lab and on a PC in France at Institut Eurecom to verify our results. Both PCs could accept incoming connections. We used the earlier measurements to improve our monitoring and analysis tools and to eliminate as many sources for errors as possible. The results we present in the following section are derived by monitoring a MandrakeLinux-10.0 torrent from the 31th of March 2004 to the 6th of May. These results are also representative for results of our earlier monitoring sessions. 5 Results This section examines the results of the measurements, which have focused on how long clients have been connected to the network, and the effects of firewalls. 5

6 seedtime (hours) average client download speed (Mbps) Figure 1: Seedtime 5.1 Seedtime The time a host serves as a seed and its relationship to the download speed of the host is shown in Figure 1. The seedtime starts at zero once the entire file has been downloaded by a client. One might expect that a host which quickly downloads a file will quit early, since the user is still likely to be sitting by the machine, while for hosts with a low download speed, the seedtime will be high, since many days can have passed since the download was started. The plot does to a certain extent fulfill these expectations; being somewhat L shaped. The majority of long seeding hosts achieved low average download speed. Clients leaving early would negatively impact the scalability of a P2P based streaming network. 5.2 Firewall effects Figure 2 shows the number of hosts for different host types to which the client is connected to at a given time. The open and firewalled lines show the number of hosts which are able to accept incoming connections and those which are not. The misclassified hosts are the ones which are thought to have been incorrectly placed in the firewalled category. The tracker total value is for reference and gives the total number of hosts as determined by the tracker. The tracker reports a higher number of hosts in the session than the client is connected to. One reason for this is likely that the client is somewhat aggressive in trying to determine when a host is no longer connected to the network. The client was changed to regularly send messages to all connected hosts, to avoid 6

7 hosts tracker total total hosts open firewalled total seeds misclassified time (days) Figure 2: Host overview having TCP connections time out. The tracker does most likely not do anything similar. The plot showing the total number of hosts reported by the client is similar; the client is most likely connected to the hosts which are actively participating in the session. The increase in the number of hosts after day seven corresponds to the beginning of the easter holiday period. The number of open hosts is generally higher than the number of firewalled hosts. There are only a few misclassified hosts. There are fewer seeds than downloading hosts. The total number of connected hosts is high at the beginning, but falls towards the end. Figure 3 shows the average download speed of different host types. There is a clear difference. Everything passes through the open hosts, which can potentially communicate with all the hosts in the session, and which as a result enjoy high download speeds. Finding a way around this would seem to be desirable. The difference appears to be related to the number of hosts in the session, and it decreases as the total number of hosts falls towards the end (see Figure 2). The figure also shows a low average bandwidth for misclassified hosts. However, as can be seen in Figure 2, the number of misclassified hosts is very low, so no definite conclusion can be drawn. In general however, the average download speed for all clients is quite high, being around 200Kbitps. The total bandwidth usage of the same session is shown in Figure 4. The total bandwidth usage for all hosts is also high, at times exceeding 50M bitps, which would represent a significant load, if only one server had been used. The number of misclassified hosts is quite low, and they contribute little to the total 7

8 average download speed (Kbps) time (days) open all hosts firewalled misclassified Figure 3: Firewall impact on download speed total bandwidth usage (Mbps) total hosts open firewalled misclassified time (days) Figure 4: Total bandwidth consumption 8

9 cumulative 90 percent completed clients time (days) total hosts open firewalled Figure 5: Cumulative downloads load. The open hosts receive more data than the hosts behind a firewall. Very little data is exchanged at the end when few hosts remain. The difference in total download speeds also results in a higher number of downloads, as can be seen in Figure 5. Completion of 90% is chosen instead of 100% since the completion of a download might not be observed if the client quickly leaves the session. That the number of completed downloads is higher for hosts which are not behind a firewall is not unexpected, since this represents a majority of the hosts, but as can be seen in Figure 6, the average download speed is higher even on a per host level. There appears to be no significant difference between the number seeds which are open or closed. Figure 7 shows a histogram over how much of the total file which was completed when the clients of either host type was last seen by the client. The biggest differences are at the ends of the graph. Hosts which are behind a firewall are more likely to disconnect early, perhaps because they achieve poor download performance. In a streaming network the speed difference between these two hosts types might be large enough to prevent streaming for hosts which are behind a firewall. 5.3 Observations The average download speed in the observed network lies at roughly 0.2M bitps, 0.3Mbitps for open hosts. The average speed was seen to vary between BitTorrent sessions measured. [21] found average speeds in the range from 0.5M bitps 9

10 time (days) open/closed total speed open/closed seeds open/closed hosts Figure 6: Per host difference ratio hosts (percent) completed (percent) firewalled open all hosts Figure 7: Completed at disconnect time 10

11 to 0.6M bitps, for a similar session. Take into consideration that the maximum download rate for Digital Subscriber Line subscriptions are often between 0.5M bitps and 2M bitps, and the performance achieved with P2P based networking is evidently promising with regards to streaming, even without any special support for this. Downloading a file with a file sharing application is a typical background task performed with a reduced amount of resources, while video streaming is typically an exclusive foreground task, because watching the video is the main or sole activity of the user. Therefore, it can be assumed that more resources, including bandwidth, might be available for streaming. However, streaming requires ordered delivery of blocks. While making good use of a client while it is downloading, the BitTorrent protocol provides no incentive for a client to seed after completing a download. Especially clients which achieve high download speeds are quick to leave the network, as can be seen in Figure 1. While many clients seed for a long time, they are often the ones which achieved lowest download speed. Keeping clients connected after they have completed a download would improve the reliability of the network. Being able to accept incoming connections might positively affect the speed with which data is downloaded as firewalls make an impact on download speed. While this observation is obvious and not new, the difference appears to be so large that it must be taken into consideration in a streaming network, where clients must be able to achieve a certain minimum speed to watch the content while it is being downloaded. Large amounts of data is being exchanged; P2P networks appear to be effective at spreading the load of content distribution, and thus reducing the cost for the content provider. With many duplicates of the file available, the fault tolerance of the session also increases; sessions can continue even if the content provider becomes unavailable. In [21] it was also shown that BitTorrent is capable of handling flash crowds. A P2P based structure would appear to provide significant potential for cost reduction, fault tolerance and scalability, but some way of reducing the impact of firewalls and encouraging longer connectivity would be beneficial. 6 Proposed architecture This section describes a proposed design for a P2P based streaming system, which attempts to address the firewall and seedtime problems in the measurements. 6.1 P2P based streaming A significant amount of traffic in P2P based file-sharing networks comes from the exchange of files of dubious legality. As such, many P2P networks have been designed to obscure the identity of its users. When the goal is efficient file 11

12 Main Server WAN P2P SCC LAN P2P LHC Client app Client host Figure 8: Streaming P2P network structure distribution, pieces of a file does not need to be downloaded sequentially. This allows the use of techniques which improve the distribution of pieces. Exchanging downloaded pieces and preferring hosts from which data has previously been received to discourage freeloaders is also possible [22]. In a P2P network designed for streaming, the design goal is the timely delivery of a data stream to a client. Data will in most cases be accessed sequentially, as opposed to the random access which is possible in a file distribution network. The result is that a P2P based streaming network will likely be less suited for the distribution of illegal files than a P2P based file distribution network; a file distribution network can do this with greater anonymity has a wider range of anti-freeloader techniques available. 6.2 Hybrid CDN structure We propose a hybrid structure which shares some properties with the structure of the WWW, where a content provider places files on a server. These files are presented to the user by an application which caches content locally, with connections optionally made through a site cache. This structure also satisfies the desires of ISPs [23] by allowing support for caching at the ISP level. The P2P based streaming network consists of several different elements, as seen in Figure 8. At the leftmost part is the client application which presents the content. This can be an application for displaying video. The client application sends file and seek requests to the host cache on the local host (LHC). How the data is retrieved from the network is not visible to the application, it only receives a sequential data stream from the requested offset in a file. The data stream might be retrieved from many different peers. The LHC communicates with the proxy/site content cache (SCC) and receives the data stream from the SCC and the LHC of other clients on the same network. It also sends data it already has in its cache to other clients on the same site. The SCC communicates with other similar caches, other external clients and the main content server. The operation of the SCC and the LHC is essentially identical, but the SCC will likely have more diskspace available for storing content, and will actively try to complete files which are only partially requested by a client. It is also more likely than the LHC to have full access to the Internet. The content providers server provides the same protocols and functionality as the LHC and the SCC. The main difference is that it has a full collection of files and does not receive files from other peers. 12

13 This structure allows caching of data at site or ISP level, and sharing of data among LAN clients. 6.3 Protocol overview The protocol is similar to a subset of the HTTP [24] protocol. It is also expected to be used with the WWW in a way similar to BitTorrent, by presenting general information about the files to the user via a browser which starts a special application for doing the actual download. The general structure is similar to that used in BitTorrent, but the operation is different. Two different, but similar protocols are used; one between the application and the LHC, and another between the different SCCs and the main server. The former mainly consists of file requests based on an URI, to which a continuous data stream is returned. The latter is based upon the transmission of fixed size blocks. Three commands are used; GET, to request data, IGOT, to report the reception of data, and HEAD to retrieve information such as filesize from the content server. The communication between the application and the LHC is simplest and only uses the GET command. Each file is identified by an URI, such as spp://spp.example.com/file. mpg. To receive a stream of this file, an application sends a GET request (GET /file.mpg SPP/1.0 ) to the LHC. Seeking in the file can be achieved by setting the start value in a Range option given with the command to the desired byteoffset. This option can contain two numeric offset values. Communication between different SCCs, and the main server is more complicated, and based on the exchange of fixed size blocks. Requests are made to the main server and between SCCs for individual blocks, using the Range option in the GET command. A block request might result in the block data itself or a redirect message. A proxy or server will if necessary serve a block request itself. The server maintains a list of hosts which have successfully downloaded different blocks in a file. When alternate download locations for a block exist, the server can reply to a request with a redirect message containing a list of other hosts with the block, to spread the load. New requests are then made to hosts from this list, until a host which is able and willing to serve the block is found. During reception the client will calculate a checksum based on the received data. When the whole block is received it will send a IGOT command (IGOT /file.mpg SPP/1.0 ) to the host it originally contacted. This command specifies the checksum (e.g Checksum: 8c5f8c65c407653d449 ), and the address of the host it downloaded the data from (Peer: ), if different from the host it sends the IGOT command to. The block checksum does not protect the initial downloader against transmission errors, to achieve this, a stream encoding technique can be used in addition. Rather, the block checksum is used to ensure that corrupt data is not propagated by caches, and to prevent against a malicious host claiming to have downloaded more data than it actually has. 13

14 Server 2. GET block 0 3. block 0 data SCC 1. GET block 0 4. block 0 data Host A LHC B LHC C LHC D Client app Figure 9: Block request Server 1. IGOT block 0 2. CKSUM OK SCC 3. IGOT block 0 4. CKSUM OK LHC A LHC B LHC C LHC D Client app Figure 10: Confirmation of block reception A IGOT command with a correct checksum tells the server or SCC that this block can be found on at least these two locations, and that the host specified as a peer was willing to serve it. This information can be used as a basis for priority calculation in the server and SCC; serving data to other hosts would improve a nodes priority. This priority can be used to provide incentives by using the priority during resource allocation. The start of a possible protocol exchange can be seen in Figure 9. All requests are made for the same file. A request for block 0 of a file on Server is made by LHC B to SCC. This block is not located at SCC, or known to be located in the local network, so the SCC makes the request directly to the Server which returns the block data. The block data is then forwarded to Host B from the SCC, as early as possible, before the entire block is downloaded. When the entire block is received by SCC, it sends an IGOT command to Server (see Figure 10), which confirms that it was received correctly. The same is done by LHC B, which also receives a confirmation of the checksum it 14

15 SCC 2. GET block 0 3. redirect to Host B 4. GET block 0 LHC A LHC B 1. GET file 6. file data Client app 5. block 0 data Figure 11: Application request and cache communication calculated during block reception. Server now knows that a copy of block 0 has been successfully received by SCC, which again knows that LHC B also has a copy. At some point in time after this interaction has occurred, a user on LHC A makes a request for the file (see Figure 11). As a result of this, the LHC makes a request for block 0 from SCC, which redirects the client to LHC B. The same request is made once more, this time to LHC B, which returns the data, which is then forwarded to the user application. After the block has been downloaded completely, this is reported to SCC by LHC A in a way similar to step 3 and 4 in Figure 10, except that LHC A specifies LHC B as the source. The information which is transmitted includes the checksum LHC A has calculated during block reception, that the block was downloaded from LHC B. The reply from the LHC indicates that the block was correctly received. The SCC now knows that another copy of the block exists, that it still exists at LHC B, and that LHC B has served a block to another node. The request for block 1 will usually have been made by LHC A before block 0 has been completely downloaded, to ensure that the client application will never have to wait for data. 6.4 Benefits The basic structure described above, with four different levels, is not mandatory. The client application could implement the P2P protocol itself, and communicate directly with the main server, but using the structure above has several benefits. Implementing the P2P operation in the LHC on the client host simplifies the application, and should make it easier to serve blocks to other peers even when the application is not in use by the user, thus encouraging longer participation in the network. A single copy of the LHC would also be beneficial on a multiuser machine. As an alternative for small sites, the application can also connect directly to the SCC, and receive the data stream from it directly. Using a common cache for a site has several benefits. Analysis of one P2P 15

16 network for file distribution found that traffic was concentrated around a small number of large files, indicating that caching would be beneficial [19] [20]. Use of a proxy also automatically creates clusters which can share data directly at the lowest level of the distribution tree, reducing the load on the SCC; the network administrator can ensure that communication between local hosts is done in an efficient manner, without guesswork from the clients. For communication between different SCCs and the main server this will still be an issue, but the number of nodes is highest at the lowest level. The use of a proxy also solves the problem of limited node availability due to firewalls. Most nodes are able to make outgoing connections, but the increasing use of firewalls usually prevents incoming connections. This severely limits the number of nodes between which communication can occur. Using a SCC and giving it full access to the Internet eliminates this problem. The installation of a cache for streaming P2P networks should be more defensible since the content is expected to be less problematic with regards to copyright than for P2P file-sharing networks, in addition to reducing the bandwidth consumption from the service. Having an officially administered cache also gives the option of controlling from which content providers local users can connect to. Having semipermanent open peers should also increase the reliability of the network, since the the LHCs are likely to have large disks available for storing files, and are always on. The main alternative to providing a proxy for solving the firewall problem is to preserve the current situation. Opening the firewall would represent a security risk for large networks, and private P2P users will likely already have opened it, if possible. Some way of tricking a firewall might be possible, but probably not legal. For these reasons we choose the use of a proxy as an integrated, but not mandatory element in the design. In sum, the structure should increase the number of nodes for all clients, and as a result the total storage space in the network, improving scalability and fault tolerance, while reducing cost for the provider. The structure has the potential for making efficient use of network capacity, but would benefit from long connectivity in the clients, which the measurements in Section 5.1 found to be often be lacking. The protocol is designed to make the introduction of incentives for staying connected possible, while still keeping a simple client interface. The IGOT command can be used to provide disincentives to freeloaders. The priority of a client can be used in the allocation of finite resources at the SCC or the main server. The total number of connections to the server and the stream bandwidth are possible candidates. In the most extreme case a host busy error message can be returned. The LHC also has the option of using information about hosts from which it has received data when it decides if it should fulfill a request. By building this information over time, from the download of a multiple files, each client has further incentives to leave the LHC running for an extended period of time, if not permanently. We also consider content quality to be important. No knowledge apart from 16

17 the URI is needed before a download can be started. With the use of a WWW server and URI for identification, the origin of the content should be easy to determine, since the use of server addresses for the WWW is familiar to most people. 6.5 Potential problems A potential problem with the design is the possibility of increasing the complexity of the system by using a proxy based structure. The scalability of the system might also suffer if the SCC becomes a hot-spot or goes down. Subtrees in the structure are expected to continue to function; clients can still request already downloaded blocks from the SCC if the content server goes down. The popularity of requested files is also a factor; caching would provide less benefit if a large number of different files with uniform popularity are requested. Caching of continuously streamed content, such as traditionally broadcasted TV would not be suited. The memory usage and resource consumption in the SCC and main server as a result of protocol interaction must also be measured. 7 Conclusion Broadband Internet connectivity in an increasing number of private households, and powerful compression techniques for high quality video, makes the deployment of streaming CDN systems increasingly realistic. In this paper, we have presented our design for a P2P based CoD system. We have measured the BitTorrent P2P network and found the need for a streaming system for additional freeloader protection and a way to limit the potentially negative consequences of firewalls on download speeds. Our system should be a quickly deployable, low cost system with scalability and fault tolerance, for streaming of content from a content provider. Ongoing and future work includes simulation and implementation of this design, to determine to which extent it fulfills our goals, to identify minimum resource requirements, and find suitable values for variables such as the block size. Initial simulations indicate that the behavior of the system is as expected, and especially provides benefits when firewall effects are taken into consideration, but work on this is still ongoing. 8 Acknowledgments We would like to thank Ernst Biersack, Pascal Felber, Guillaume Urvoy-Keller and Mikel Izal Azcárate at Eurecom for fruitful discussions and provision of their crawler for our measurements. 17

18 References [1] S. Sheu, K. A. Hua, and W. Tavanapong, Chaining: A generalized batching technique for video-on-demand systems, in Proceedings of IEEE International Conference on Multimedia Computing and Systems (ICMCS 97), June [2] Y.-H. Chu, S. G. Rao, and H. Zhang, A case for end system multicast, in ACM SIGMETRICS Santa Clara, CA: ACM, June [3] M. Castro, P. Druschel, A. Kermarrec, A. Nandi, A. Rowstron, and A. Singh, Splitstream: High-bandwidth content distribution in a cooperative environment, [4] D. A. Tran, K. A. Hua, and T. T. Do, A peer-to-peer architecture for media streaming, IEEE JSAC Special Issue on Advances in Overlay Networks, [5] H. Deshpande, M. Bawa, and H. Garcia-Molina, Streaming live media over a peer-to-peer network, Stanford University, Tech. Rep., August [6] V. Padmanabhan, H. Wang, P. Chou, and K. Sripanidkulchai, Distributing streaming media content using cooperative networking, [7] T. T. Do, K. A. Hua, and M. A. Tantaoui, P2vod: Providing fault tolerant video-on-demand streaming in peer-to-peer environment, School of Electrical Enginneering and Computer Science, University of Central Florida, Tech. Rep., [8] C. Loeser, M. Ditze, and P. Altenbernd, Architecture of an intelligent quality-of-service aware peer-to-peer multimedia network, in Proc. of the 7th World of Multiconference on Systemics, Cybernetics and Informatics, July [9] X. Jiang, Y. Dong, D. Xu, and B. Bhargava, Gnustream: a p2p media streaming prototype, in Proceedings of IEEE International Conference on Multimedia & Expo, July [10] M. Ripeanu, Peer-to-peer architecture case study: Gnutella network, University of Chicago, Tech. Rep., [11] Y. Guo, K. Suh, J. Kurose, and D. Towsley, P2cast: peer-to-peer patching scheme for vod service, in Proceedings of the twelfth international conference on World Wide Web, 2003, pp [12], A peer-to-peer on-demand streaming service and its performance evaluation, in Proceedings of 2003 IEEE International Conference on Multimedia & Expo,

19 [13] R. Rejaie and A. Ortega, Pals: peer-to-peer adaptive layered streaming, in Proceedings of the 13th international workshop on Network and operating systems support for digital audio and video, [14] M. Hefeeda, A. Habib, B. Botev, D. Xu, and D. B. Bhargava, Promise: Peer-to-peer media streaming using collectcast, Proc. of ACM Multimedia 2003, November [15] M. Hefeeda, A. Habib, D. Xu, B. Bhargava, and B. Botev, Collectcast: A peer-to-peer service for media streaming,, ACM/Springer Multimedia Systems Journal, October [16] J. W. Byers, M. Luby, and M. Mitzenmacher, Accessing multiple mirror sites in parallel: Using tornado codes to speed up downloads, in INFO- COM. New York, NY: IEEE, Mar. 1999, pp [17] J. Apostolopoulos, T. Wong, W. Tan, and S. Wee, On multiple description streaming with content delivery networks, [18] T. Nguyen and A. Zakhor, Distributed video streaming over the internet, in Proceedings of SPIE Conference on Multimedia Computing and Networking, January [19] N. Leibowitz, M. Ripeanu, and A. Wierzbicki, Deconstructing the kazaa network, [20] K. P. Gummadi, R. J. Dunn, S. Saroiu, S. D. Gribble, H. M. Levy, and J. Zahorjan, Measurement, modeling, and analysis of a peer-to-peer filesharing workload, in Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP-19), ser. Bolton Landing, NY., October [21] M. Izal, G. Urvoy-Keller, E. Biersack, P. Felber, A. A. Hamra, and L.Garces-Erice, Dissecting bittorrent: Five months in a torrent s lifetime, in Passive and Active Measurements 2004, April [22] B. Cohen, Incentives build robustness in bittorrent, in Workshop on Economics of Peer-to-Peer Systems, June [23] G. Hasslinger, Peer-to-peer networking from the view of internet service providers, Peer-to-Peer-systems and -Applications seminar, March [24] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee, Hypertext transfer protocol http/1.1, RFC2616, June