Query Adaptation Techniques in Temporal- DHT for P2P Media Streaming Applications

Transcription

1 Query Adaptation Techniques in Temporal- DHT for P2P Media Streaming Applications Abhishek Bhattacharya, Zhenyu Yang, and Deng Pan School of Computing and Information Sciences Florida International University, Miami, FL, USA. ABSTRACT Peer-to-Peer (P2P)-based approach for on-demand video streaming systems (P2P-VoD) characterized by asynchronous user-interactivity has proven to be practical and effective in recent years with real-world Internet-scale deployment (Huang, Li, & and Ross, 2007). Current state-of-art P2P-VoD systems employ tracker server for discovering content suppliers which poses scalability and bottleneck issues. Temporal-DHT is a structured P2P based approach which can efficiently accommodate the large number of update operations with the continuous change of user s playing position and supporting asynchronous jumps (Bhattacharya, Yang, & Zhang, Temporal DHT and its Application in P2P-VoD Systems., 2010). We propose different query adaptation strategies based upon content popularity distributions and shortage bandwidth ratios which are proved to be effective in improving the performance of P2P streaming system by deriving certain optimized solutions. We formulate valuable optimization problems in the context of a P2P-VoD system such as minimization of query search cost, server bandwidth consumption, and a joint cost-load framework. We provide optimized solutions that achieve the best result for the above mentioned optimization objectives. We show extensive simulation studies under various scenarios of search cost, streaming quality, and other associated factors in a dynamic network environment where users are free to asynchronously join/leave the system. Keywords: Multimedia Information Systems, Video Streaming, Distributed Hash Tables, Optimization, Peer-to-Peer Systems, Video-on-Demand. INTRODUCTION Gnutella, Napster, etc. are some of the first-generation unstructured systems that started the P2P revolution, followed by the more efficient structured approaches such as Distributed Hash Tables (DHT) represented by Chord (Stoica, Morris, Karger, Kaashoek, & Balakrishnan, 2001), CAN (Ratnasamy, Francis, Handley, Karp, & Shenker, 2001), Pastry (Rowstron & Druschel, 2001), and a suite of similar systems which is based upon similar principle. Web caching, distributed storage, etc. are some of the earlier applications supported by P2P approach, followed recently by the more popular ones such as file sharing e.g., BitTorrent (Qiu & Srikant, 2004), multicasting e.g., Narada (Chu, Rao, & Zhang, 2000), and live streaming e.g., CoolStreaming (Zhang, Liu, Li, & Yum, 2005), PPLive (Hei, Liang, Liu, & Ross, 2007), AnySee (Liao, Jin, Liu, Ni, & Deng, 2006), etc. The potential advantage of P2P-based applications is mainly associated with the fact that peers share their resources such as processing power, storage, and bandwidth to help each other in searching/distributing content, thereby alleviating the server load. The

2 management and distribution of multimedia content is particularly critical with respect to P2P applications and imposes more importance to the Internet traffic which is largely dominated by the ever-growing bandwidth-hungry multimedia data. On-demand streaming can be enormously benefited from the application of P2P techniques as revealed in a recent study (Yann, Fu, Chiu, Lui, & Huang, 2008). We advocate a DHT-overlay based approach to address the challenging problem of efficient content discovery in On-demand system with asynchronous user interactivity. DHT overlays are already proved to be stable substrate with nice characteristics such as scalability, decentralized control, self-organizing, and resilience to network/peer dynamics. Incorporating DHT in one-demand streaming systems is not a trivial issue since it will generate a flurry of update operations with the continuously changing playback position of the user. The framework for Temporal-DHT (Bhattacharya, Yang, & Zhang, Temporal DHT and its Application in P2P-VoD Systems., 2010) addressed this issue of accommodating a large number of update operations by exploiting the temporal dynamics of the content for estimating the current playing position of the peers automatically. Temporal-DHT combines the advantages of both the approaches of cache-relay and static-cache. Cache-relay based approach has a high streaming efficiency due to buffer-overlap relation between parent and child peers, whereas, on the contrary static-cache based approaches are more adapted for supporting dynamic and synchronous operations such as random jumps by avoiding the dependency on playing position between peers. Temporal-DHT employs a skilful integration of static and dynamic buffer management schemes to handle the request dynamics and streaming efficiency in a seamless fashion. We can describe Temporal-DHT as an augmented version of generic DHT semantics by incorporating the query reformulation, TTL filtering, and access workload self-profiling techniques. The initial Temporal-DHT framework involved a static query reformulation mechanism without considering the possible effects of content popularity distributions and other related factors which are common phenomenon s in present day P2P systems. The concept of popularity awareness is generally employed for optimizing certain objectives such as search cost or server/peer load factor utilizing the content popularity ratios. One of the important intentions is to reduce the search cost of more popular contents since they are queried more frequently which will eventually help to improve the overall performance of the system (Rao, Chen, Fu, & Wang, 2010). It has already proved to be highly useful in web-caching and file-sharing systems where the data objects are typically characterized with different popularity ratios (e.g., some popular files are downloaded with a higher frequency or some popular web pages are accessed more frequently). Different studies reported that web requests in Internet are highly skewed with a Zipf-like distribution (Yiu, J, & Chan, 2007) with typical characteristics of a few objects having a very high popularity, a medium number of objects with average popularity, followed by a long tail with a huge number of objects with very low popularity. Zipf-distributions are universally used for modeling popularity in various scenarios. The generic approach to deal with this kind of skewed popularities is to cache the data objects at the various intermediate relay nodes in the query resolution path which will eventually help to reduce the number of search hops for the popular queries. This type of caching should be adaptive under dynamic popularity scenarios (popularity of data objects change with time) since there is an associated trade-off relation between the higher performance due to lower search complexity and the cost for caching the data objects at the intermediate nodes. In the context of media streaming applications, caching is not a

3 reasonable choice since it does not make sense to continuously cache large media-sized objects at the intermediate nodes which consumes a lot of network bandwidth. Various proposals such as VMesh (Yiu, J, & Chan, 2007) employs a popularity-based content storage mechanism where the cached segments are continuously replaced in accordance with the recent content popularity distribution. Continuous replacement of the cached segments to adapt to the dynamic popularity variations is one of the downside of this kind of mechanism which consumes large network bandwidth and thereby rendering this as a heavy-weighted technique. As already mentioned, current approaches for dealing the popularity skew is replication whereby the less popular objects are replaced with more popular objects in a dynamic fashion. This technique consumes excessive bandwidth for keeping the cache updated (i.e., proportional to content popularities) and so we pursue a different approach of query resolution adaptation. Distinct from other approaches where the query resolution adaptation depends on the replication/caching strategies, our method avoids the expensive method of replication by adopting a range query adaptation technique. This is possible due to the availability of a range query reformulation technique inherently present in a Temporal-DHT framework where the generic exact-match DHT prefix routing is augmented with a range query and the range query span is dependent on the object update interval. The initial Temporal-DHT framework assumed a fixed value for the object update interval thereby rendering increased search cost with respect to popularity skewness of the content (Bhattacharya, Yang, & Zhang, Temporal DHT and its Application in P2P-VoD Systems., 2010). There exists a tradeoff relation between the performance benefits of decreasing search cost and the increased cost of update operations i.e., if we intend to minimize the search cost then we need to decrease the update interval which will trigger more number of update operations thereby increasing the messaging overhead. Due to this situation, it is essential to find an efficient solution that optimizes certain performance objectives and then perform the adaptations based on the optimization solutions. In this context, we address the following problems: P1: How to minimize the search cost with a given threshold constraint of update interval? P2: How to minimize the server load with a given constraint of available outbound bandwidth and update interval? P3: How to jointly minimize the search cost-server load with given constraints of available bandwidth, update interval, messaging overhead? P1 is addressed in (Bhattacharya, Yang, & Pan, Popularity Awareness in Temporal-DHT for P2P-based Media Streaming Applications, 2011) and in this paper we undertake P2 and P3. We present formulations for optimization objectives of P1, P2, P3, and present practical solutions for them which will help to develop techniques to perform dynamic adaptation of the object update intervals in the context of a Temporal-DHT with varying popularity distributions. To summarize, our contributions are as follows: (a) We incorporate the notion of popularityawareness in the context of Temporal-DHT with different performance objectives for optimizing the search cost, server load, update interval, and messaging overhead in a dynamic fashion under varying conditions; (b) We formulate the problems P1, P2, P3, in a representative manner and propose solutions to achieve the objectives; (c) We implement the three adaptation strategies in a Temporal-DHT based P2P Video-on-Demand system model and provide extensive simulation

4 studies to show the effectiveness of the adaptive query resolution strategies in a media streaming scenario and the performance benefits associated with the optimization of P1, P2, P3. The rest of the paper is organized as follows: We present some basic background stuff related to DHT and Temporal-DHT in the next section. In the following section, we present the detailed adaptation mechanisms and the optimization problems P1, P2, P3, and the various solutions with its interpretation in the Temporal-DHT framework. We analyze our simulation studies in the following section. The next section summarizes related work from the literature followed by the section for conclusion. RELATED WORK The general trend in dealing with popularity skews is caching and replication, where the queried data objects are cached or replicated in the intermediate relay nodes or some strategic nodes near the query originator. The typical problem in this domain mainly involves in the placement strategies of replicas or cached objects to reduce the search cost for the more popular objects. Web-caching systems are benefited from these techniques since the web-based objects typically follow a Zipf-like popularity distribution. CFS (Dabek, Kaashoek, Karger, Morris, & Stoica, 2001) is a cooperative file system over Chord DHT which caches the popular objects along the lookup path towards the home node where the popular objects are originally stored. PAST (Rowstron & Druschel, Storage management and caching in PAST, a large-scale, persistent peerto-peer storage utility, 2001) is a storage system over Pastry DHT where the search for some object is redirected to the nearest replicas of the targeted object. One of the more technique is proposed in Beehive (Ramasubramanian & S, 2004) where it replicates the object copies to all the nodes that have at least l common prefixes matching with object hash ID where l is defined as the replication level. Replication was proposed in (Cohen & Shenker, 2002) to optimize search efficiency where the number of replicas of an object is kept proportional to the squareroot of the object popularity. A square-root topology for unstructured P2P networks was proposed in (Cooper, 2005) where the in/out degree of a peer is proportional to the square-root of the node popularity. PRing/PCache (Rao, Chen, Fu, & Wang, 2010) presented a replica placement strategy for web-caching systems with data objects having skewed popularities in both deterministic and randomized structured P2P networks. They gave detailed analytical results with closed form optical solutions for different resource optimization objectives. LAR (Gopalakrishnan, Silaghi, Bhattacharjee, & Keleher, 2004) proposed a lightweight, adaptive, and system-neutral replication framework that maintains low access latencies and good load balance even under higly skewed demand. Now, let us discuss some replication/caching strategies specifically for multimedia data object: VMesh (Yiu, J, & Chan, 2007) uses a static-cache based DHT overlay for P2P VoD streaming where the cached objects are continuously refreshed with different video segments and this segment replacement strategy is proportional to the probability of the derived segment popularities. (Tan & Massoulie, 2011) proposed optimal content placement strategy and request acceptance policy for P2P-VoD systems which jointly maximize uplink bandwidth utilization. Statistical modeling is proposed in (Zhou, Fu, & Chiu, 2011) to derive relationship among storage capacity, number of videos, number of peers, server load, which is later used for a replication algorithm that balances load among all the peers for both deterministic and random demand models, and both homogenous and heterogeneous upload bandwidth distribution. (Wu & Lui, 2011) presented mathematical models and optimization framework for understanding the impact of popularity on server load where they argued the conventional wisdom of proportional replication strategy to be non-optimal and expanded the

5 design space by deriving passive replacement and active push policies based on optimal replication ratios. (Tewari & Kleinrock, 2006) advocated to tune the number of replicas in proportion to the request rate of the corresponding content, based on a simple queuing formula from the standpoint of load on network links. Investigations for content placement in P2P-VoD systems were conducted in (Kyoungwon, et al., 2007) in the context of both queuing and loss models. (Wu & Li, 2009) used dynamic programming to derive the optimal replication strategy for P2P-VoD system where the peers have homogenous upload capacity. BACKGROUND MODEL DESCRIPTION We introduce the following notations to describe our model of Temporal-DHT based P2P-VoD system as follows: P is the set of participating peers, p i such as { }. N is the number of peers in the system. is the upload capacity of peer i. is the download capacity of peer i. S is the media server with an outbound bandwidth of. C is the video stream such that { } is made up of M chunks or segments. D is the size of one chunk or segment in MB. d is the video data rate in Kbps required to maintain for uninterrupted streaming. is the playtime of each video segment is a dynamic/random buffer with size of k segments i.e., kd MB. is a static/sequential buffer of size b segments i.e., bd MB. T is the publish interval i.e., where z is a system defined parameter. TTL is the Time-to-Live which indicates the freshness index for each indexing record. / are the successor/predecessor pointers in the content space. A novel conceptual augmentation of the traditional DHT semantics for indexing content with temporal dynamics provide considerable savings in messaging overhead is proposed in one of our earlier work as the framework for Temporal-DHT (Bhattacharya, Yang, & Zhang, Temporal DHT and its Application in P2P-VoD Systems., 2010). The proposed framework has two distinctive properties: (a) Application-level Characteristics: DHT takes a more active role by exposing the internal behavior of the application which allows for a chance to better service the dynamic needs of the application by advocating a proactive design approach; (b) Data Transiency: Unlike traditional DHT, the stale indexing records are flushed off from the system at a periodic interval and the predictive temporal dynamics is exploited for effective query resolution in Temporal-DHT. represents a 3-tuple indexing record in a typical Temporal-DHT with and indicating contains at and this record will be flushed off from the system at. Temporal-DHT can accommodate both static (like traditional DHT indexing records) and dynamic indexing records within the same framework by initializing the value of TTL to z (for dynamic case) and (for static case). Temporal-DHT exploits the technique of lazy updations by allowing certain degree of inconsistencies in the indexing structure which enable the record to update in a coarser granularity i.e., predefined constant periodic interval T. To allow this kind of inconsistency relaxation, the query resolution mechanism of the traditional

6 DHT is augmented by employing query reformulation and TTL filtering techniques by taking hint from the dynamics of content workload. Next, we try to illustrate the underlying idea with the help of an intuitive example: Referring to Fig: 1(a) and suppose k=1, we notice that VoD peer ( ) perform an update operation at time ( ) with z=4 which is represented as an indexing record. After each time interval, the buffer slides by one segment and the next update is performed by peer ( ) at time ( + *4) with the record <,, + *4, *4>. During the time interval [ ] where δ is a very small time unit signifying already loaded in of but the Temporal-DHT update is not yet performed, the traditional exact-match query resolution will fail to return as a result in this framework due to the allowed inconsistency. For effectively returning as a result, we transform the exact-match query resolution with a range query reformulation of <q, q-z>. Fig: 1(b) depicts an illustrative example with playing buffer of VoD peers and sliding over the section of video stream during the time interval [ ] ( ) with z=6. The accurate query resolution formalization is given in Theorem 1 taken from (Bhattacharya, Yang, & Zhang, Temporal DHT and its Application in P2P-VoD Systems., 2010). Further proof details and TTL filtering schemes are covered in (Bhattacharya, Yang, & Zhang, Temporal DHT and its Application in P2P-VoD Systems., 2010). Figure 1: (a) Temporal-DHT content linkage and updates and (b) Range query reformulation and buffer sliding.

7 Theorem 1: Given the playback buffer of size k and the publish interval z, a peer that searches for dynamic segment needs to perform a range query of at most k+z segments. There are also some other distinctive features associated with Temporal-DHT which will be briefly discussed as follows: A content based overlay is initiated for supporting in-order access and range-query resolution by maintaining pointers with respect to the semantic sequential relationship i.e.,. This content linkage pointers can also support short random jumps as long as the number of routing hops (which can be easily calculated in this case using content distance i.e., jump from to segment is 3) is less than O(log N) in DHT generic routing (Yiu, J, & Chan, 2007). An overarching framework was proposed as a Temporal-DHT based mesh (TDHTM) which can seamlessly integrate the power of asynchronous interactivity support with static cache based indexing and smooth in-order streaming efficiency with dynamic cache based indexing. A combined static-dynamic buffer management scheme is employed in TDHTM where the static or segments are indexed with TTL= (kept constant throughout the peer s lifetime) and the dynamic or segments are indexed with TTL=z (keeps changing with player viewing position by buffer sliding after each segment playback). Static indexing involves a one-time publication of indexing record at initialization, and query processing follows the generic DHT based exact-match resolution mechanism, whereas dynamic indexing is concerned with the publication of indexing records in a periodic interval of T with augmented Temporal- DHT based range query resolution technique. Moreover, TDHTM also employs access workload self-profiling at the client end for adaptive content distribution by dynamic switching between random seek mode (handled by static indexing) and continuous playback mode (handled by dynamic indexing). We provide a high-level overview of the algorithm in pseudocode as follows: Peer joins the system and initializes the temporal DHT by deriving the finger table Randomly selects b segments for filling followed by search and download. Publish static indexing records of content. Fix / pointers by joining the content overlay. Accepts user s request of starting video segment and searches static/dynamic indexing records. Peer fills dynamic buffer by invoking temporal DHT queries and download video segments from neighbors with available. At each gossip time interval, exchange (send/receive) messages with neighboring peers for segment query/access popularity information. Peer computes the access/query popularity index values for each segment and set the update interval according to the proportionality of popularity indexes. After each time interval of, publish a dynamic indexing record of the representative segment in to the temporal DHT. During each segment playback, search and download the next segment from tree parent or neighbors and perform buffer sliding of. Peer leaves the system by informing the temporal DHT or no information through failures.

8 FRAMEWORK DETAILS We now present detailed optimization strategies for different objectives by incorporating content popularity and other resource management techniques in the context of a Temporal-DHT based P2P-VoD system model. Search Cost Let us analyze the cost of a search query in the Temporal-DHT framework which will be obviously more than the generic DHT cost due to the range query reformulation and so it is one of the important metric for optimization. A typical Temporal-DHT query is composed of two sections: (1) a basic exact-match generic DHT query with prefix routing executed through the pointers in the finger table, (2) a range query reformulation performed by the linear traversal of the content overlay by moving forward/backward directions with the help of / linkage pointers. The query cost in a generic DHT search between any pair of source and destination is given by O(log N) (Stoica, Morris, Karger, Kaashoek, & Balakrishnan, 2001). Let the i th node in Chord DHT have a node ID i in the hash identifier space, then the k th entry in the finger table points to the successor node of ID where 1 k log N and therefore, the distance travelled by a routing hop is given by for 1 x log N and the query is forwarded to the node in the x th entry from the finger table. This can be generalized to the fact that the query can traverse at most half of the remaining distance between the source and destination in the identifier space in each routing hop. The range query cost can be derived from Theorem 1, where it was stated that the search span can traverse at most for k+z segments. The time complexity for the range search can be equated to O(k+z) since each sequential segments can be reached by a single application hop from each other which is facilitated by the content based overlay. So, the total cost (in terms of messaging) to search any content in the Temporal-DHT framework represented as number of hops is given as follows: Search Cost Optimization We formulate the problem of search cost by MIN-SEARCH as follows: MIN-SEARCH: Minimize the total query cost ( threshold constraint of update interval ( ). ) in terms of lookup hops with a given This problem involves in maximizing the performance benefits of the Temporal-DHT framework by associating the cost with the messaging complexity required for query resolution which is crucial in conserving valuable network bandwidth. In our initial proposal (Bhattacharya, Yang, & Zhang, Temporal DHT and its Application in P2P-VoD Systems., 2010), we had a fixed value of z which essentially fails to realize the query load skew due to time-varying popularity of individual segments in a video stream. This will essentially generate a higher total search cost of the system mainly contributed by the large number of popular query segments. Suppose, the total query set for a P2P session is defined by { } where { } and denotes the set of segments with similar ID ( ) for each value of i. Now, we can define popularity index ( ) of as follows:

9 The total search cost for a single Temporal-DHT query is as derive before. Now, given certain query popularity distribution function as, then the total search cost H for M data objects can be represented as follows: The optimization objective is to minimize the value of H. We derive our solution by exploiting the theorem stated in (Rao, Chen, Fu, & Wang, 2010) as follows: Theorem 2: Let the cost of each update operation be denoted as (i.e., ) and the total number of update operations as L, then it is observed that for ; H is minimized when. From Eq: 10 in (Rao, Chen, Fu, & Wang, 2010) and adapting for our scenario we have: Now, let us substitute and in Eq: 4 for H as follows: It is interesting to note that the term denotes the entropy of the query popularity index. This is in accordance with our intuition that the expected popularity distribution skewness will play a crucial role in the cost optimization objective. It can be observed that considering the entropy of query popularity is a sound measure for modeling the skewness distribution. Thus, we can notice that the total search cost H depends upon N, k, L, M, z, Entropy ( ). In our framework, {N, k, M} are kept fixed, and our goal is to minimize the value of H by adapting z with respect to. We define the estimated update interval adaptation for segment as follows: where is the estimated popularity index for segment using the number of received requests and derived in a later section. Popularity of Video Segments Popularity models are typically based on Zipf distribution which is usually derived from the popularity of web objects in the Internet. It is generally true that in a video stream, some portions are more popular than other as evident from previous studies which will essentially generate a skewed query pattern by overwhelming the system with more popular queries. If all the video segments are ranked in the descending order if their popularities, then the popularity index of the i th segment ( ), can be denoted as follows:

10 where α is a Zipf constant. We assume that the segment popularities are linked to the user request distribution which is reasonable since the more popular segments are requested by a larger number of users with a high probability. We model the VoD query distribution as follows: A peer initializes from a randomly selected segment and start to watch the video from that point. The user continues to play in a normal sequential playback mode for a random time period with an exponential distribution of mean seconds. Then, the process goes on repeating by jumping to another random segment and remaining in normal playback mode for a certain period. This process continues until the user leaves the system. The life of a peer in the system is given by an exponential distribution with mean. Our main objective is to derive the segment popularities based on the above user access model in a typical VoD system where peers randomly join/leave. Let be the state when a peer is accessing segment i. The average time of a peer staying in the system or its expected life period can be denoted as. The peer plays the media in a sequential in-order mode by traversing from to with a probability of. The average number of sequential segments accessed by the peer during this phase is (geometric summation series). The random jump probability from any segment i to another non-sequential segment j is defined as follows: We can formulate the one-step transition probability function from to for any { } as follows: { Let us denote as the probability of segment i at time x such that since the peer starts from a random point which is typically evenly distributed among all the segments. Suppose at the end of time slot t, the peer still stays in the system with probability which follows an exponential distribution. Thus, the expected access probability for segment i is given as follows: It is possible to calculate the access probabilities of each segment from the above equations provided the one-step probability function is known. The one-step probability function is typically random in nature due to asynchronous user access patterns whereby a peer can jump to any position at any time. This is unlike any static distribution pattern which is typically assumed to study various theoretical properties. Moreover, it also considers the knowledge of global information at each peer to make optimal decisions. Hence, we take a practical approach of popularity estimation in a distributed fashion suitable for realistic conditions as described in the next section.

11 Query Popularity Estimation We use a distributed averaging algorithm for estimating query popularity in a dynamic and decentralized fashion without any static assumption of load distribution. The average number of queries received from a set of distributed peers is utilized for estimating the popularity indices by exploiting the algorithm proposed in (Yiu, J, & Chan, 2007). We provide a brief description of the algorithm as follows: Each node exchange messages with r randomly connected nodes. Assume node i have a local value of and the objective is to estimate the average value of all over the network. The value can be conceptualized as which represents the total number of queries for video segment in the P2P system. A local dynamic variable which is initialized with a value of is also maintained at each peer. Each node periodically communicates with its set of random neighbors and performs a set of action as follows: (1) Node i send its local value to Node j, (2) Node j update its local value to where 0 < <1 is a local parameter. Node j also sends back the value ( ) to Node i, (3) Node i updates its local value to ( ). The central idea behind the algorithm is based on alternate increment and decrement operations of the same value in two neighboring nodes which helps to conserve the sum of all the values in the system and approaching closer to the global average value after each update. This technique can also be extended to cope with peer dynamics where each node i maintains a variable for each neighbor j, which accumulates all the changes made to j. On detection of failure of node j, node i performs which helps to conserve the total sum of all values. The above distributed algorithm can be utilized to keep track of the total number of requests from different peers which is used for the calculation of. Each peer maintains an array for each indicating its access to. If receives a request for, then it sets, otherwise it remains as. A peer also maintains another set of local variables which stores the frequency of received requests for. The averaging algorithm is then executed to exchange and update the value of continuously with its neighboring nodes. The information gets propagated through each peers neighborhood and thereby converge to a local value which can be assumed to be a good approximation of the global popularity distribution for. Now, it is trivial to compute the estimated popularity of from its local set of average values as follows: Server Load Optimization An efficient P2P-VoD system will tend to minimize the upload bandwidth traffic of S for reducing the total operating cost. The upload bandwidth consumption of S depends on various factors but some important of them are: (1) peer scheduling policies, and (2) content replication strategies. We employ a practical peer scheduling strategy where a peer initially strives to locate and download data from other peers already in the system, and only when other peers cannot supply due to content/bandwidth bottleneck, the request is redirected to S. A peer scheduling strategy involve two design issues: (1) peer seeking to download needs to decide which peers to request for data (suppliers); (2) peer seeking to upload needs to decide which peer for fulfilling its request (provider). Both these decisions are based on a queue of requests since there will be a list of suppliers and providers and the choice need to be made with priorities. We incorporate an

12 intuitive approach for scheduling and assigning priority to requests based on node capacity. Node capacity is a function of the node s access bandwidth, processing power, disk speed, etc. This strategy will ensure fair load sharing among the different node heterogeneities. They can be calculated locally at each node and the information is propagated to the decision making peer by piggybacking on request messages. Assume to be the set of peers that currently hold segment in buffer. Obviously, and. The expected upload bandwidth consumption of server S can be expressed as follows: ( ) [ (( )) ] where is the server bandwidth consumption with respect to segment (in other words it can be conceptualized as the number of segments downloaded from server S); is the peerassisted bandwidth throughput provided by all the other peers in the set who currently hold segment in buffer (in other words it can be conceptualized as the number of segments downloaded by peer from other peers in set currently holding in buffer); is the maximal upload bandwidth from all peers that can contribute to. For solving the above problem, the notion of shortage bandwidth with respect to segment i is defined as follows: ( ) and we denote [ ] where can be defined as the expected shortage bandwidth in the peer-set which is actually the gap between the demand bandwidth and the available bandwidth supported by peerset. Thus, we can obtain the following: Our objective is to find an adaptation strategy such that the average upload bandwidth consumption U of the server S can be minimized. The shortage bandwidth can be efficiently calculated in an iterative way as follows: (( ) ) { where ; since the first peer entering do not have any suppliers and have to download the segment from server. Based on this framework, we can show the impact of popularity indices of each segment and shortage bandwidth on the server upload bandwidth consumption. To model the number of peers ( ) currently holding in buffer, we assume the Zipf-based popularity distribution as already defined before. Let a random variable, denote the

13 probability for segment having number viewers ( number of peers possessing segment in buffer ) as follows: ( ) Thus, the average upload bandwidth consumption of server S can be derived as follows: { [ ( ) Now, the model of server load optimization can be formulated as follows: ; In general it is difficult and also not a practical approach to find a closed form solution of this optimization problem which will require global knowledge not inherently present for scalable P2P systems. Rather, we define the following practical and distributed solution where the estimated update interval adaptation is based upon shortage bandwidth proportionality ratio which can be derived as follows: We can estimate the denominator by exploiting the distributed averaging algorithm as discussed before. This strategy of update adaptation based upon shortage bandwidth proportionality ratio is found to produce good result as shown later in experimental evaluation. Search Cost-Server Load Joint Optimization Now, we define the joint optimization problem of search cost and server load in a single function as follows: { ]} ( )} { [ ( ) ]} and the objective is: ( ). We derive the joint optimization solution by a linear combination of their respective solutions as follows: where and are their respective weightage values and can be tuned to find the implications and we study them in experimental evaluation. Both the denominators can be estimated using the previous distributed averaging algorithm piggybacked in the same communication message. EXPERIMENTAL EVALUATION We present extensive simulation results to validate our models and evaluate the performance of our adaptation strategies with the help of different system properties. We implemented a discrete event-driven simulator for various P2P operations in C++. All the P2P events such as media

14 playback, random jump, and peer join/leave/failure are simulated by events scheduled at respective times. Chord (Stoica, Morris, Karger, Kaashoek, & Balakrishnan, 2001) is used as the base DHT due to its simplistic construction and provable performance guarantees. The following adaptation strategies are evaluated: STATIC: No query adaptation strategy employed; SP: Query adaptation based upon proportionality ratio of segment popularities; ;. SB: Query adaptation based upon shortage bandwidth proportionality ratio; ;. SP-SB: Joint query adaptation based upon linear combination of popularity and shortage bandwidth proportionality ratios; ;. Some of the simulator details are as follows: The underlying network topology generated using GT-ITM (Zegura, Calvert, & Bhattacharjee, 1996) consist of 15 transit domains, each with 25 transit nodes, and each transit node connected to 10 stub domains, each with 15 stub nodes. We randomly place the server in a transit node and peers in the stub nodes. The latency of each link is computed in proportional to the Euclidean distance between the nodes. For each point in the plot, we repeated the placement and simulation for 10 times to mitigate the effect of randomness. The number of peers (N) in the system is varied from 256 to We model the user arrival process as a Poisson distribution with an inter-arrival time λ=1 sec. The peer lifetime is modeled as an exponential distribution with an expected mean of 30 mins. The peer upload bandwidth ( ) is randomly distributed between 250~1000 kbps and the video data rate is d=500 Kbps. The user request pattern or the segment popularities follow a Zipf distribution with different values of α. Each segment size (d) is set to be 3.84 MB which corresponds to one minute of video length. The total viewing length of the video stream is 128 mins and each simulation session is set for 2 hrs. Other parameters are: =4 Mbps, =500 Mbps; k=5, b=4. Query Resolution Cost The query resolution cost is measured by the number of lookup hops required for returning the result set. It is calculated by the average of lookup hops initiated through the entire query set by all the peers in the system in a simulation session. It is an important P2P performance metric which enables to control the query messaging overhead and also improve the chances of the request to be served within the deadline. Figure 2 illustrates the performance of different adaptation strategies with respect to the average number of lookup hops for various peer populations. Some of the fixed parameters for this plot are: α=1.0; ;. It can be observed that adaptation strategies (SP/SB/SP-SB) provide considerable performance gains compared to no-adaptation strategies (STATIC). Among the adaptation strategies, SP provides slight improvement over SB which is justified since SP is specifically geared to minimize the lookup cost, but the joint adaptation technique (SP-SB) performs slightly better among all the variants.

15 Avg # of lookup hops Avg # of lookup hops STATIC SP 8 6 SB 4 2 SP-SB Number of peers (N) Figure 2: Plot of average number of lookup hops for different number of peers in the system. Now, we study the effect of skewness degree of the popularity distribution (i.e., variation of α) with respect to the different strategies. The performance metric is kept the same (i.e., query resolution cost in terms of average lookup hops) and the skew degree (i.e., α) is varied from 1.0 to 5.0 for N=406 peers plotted in Figure 3. The no-adaptation strategy fairs badly with increasing α as expected. SP performs better than SB/SP-SB which is justified since the adaptation is tuned in proportion to the estimated query popularities (i.e., ). SP performs better with increasing value of α, which indicates that is able to capture the variation of skew (i.e., α) STATIC 15 SP 10 SB Zipf constant (α) SP-SB Figure 3: Plot of lookup cost for different popularity skew degree (Zipf constant ) with N=4096. Request Rejection Ratio

16 % Request Rejection The next performance metric for our experimental study is request rejection ratio which is defined by the ratio of the number of requests fulfilled with respect to the total number of requests initiated by all the peers in a P2P system. There are various types of requests made by the peers at different time but we will be restricting ourselves only to the requests dealing with content/bandwidth and do not consider any requests for control parameters for effective performance evaluation since they are not a major focus in this framework. Figure 4 plots the variation of request rejection rate for different population of peers in the P2P system. The noadaptive strategy generates the highest rejection rate since the large number of popular queries traverse through a long query path which invokes request rejections and high query overhead. SP-SB and SB strategies give good performance values since they adapt the query resolution based on popularity and available bandwidth. The increase in peer population does not seem to have a high influence on rejection rate which also renders the strategies (SB/SP-SB) to be scalable. Next, we study the variation of request rejection with change in popularity models with different skew ratios. Figure 5 plots the request rejection rates for the different adaptation strategies with varying α from 1.0 to 5.0. In accordance to the previous discussions, the STATIC strategy is not able to control the high request rejections in the system and it is observed that it grows quite significantly with increasing α. It is not a scalable solution and in a system with 4096 peers, the rejection rate increase from 48.47% (α=1.0) to 60.39% (α=5.0) which is not a desirable property. SP-SB strategy performs the best and even with increase in α it is able to considerable reduce the request rejection rate. SB performs significantly better than SP which suggests that available bandwidth based adaptation is better suited to minimize the request rejection rate as compared to popularity based adaptation STATIC SP SB SP-SB Number of peers (N) Figure 4: Plot of request rejection rate for different system size at α=1.0

17 Number of server streams % Request Rejection STATIC SP SB Zipf constant (α) SP-SB Figure 5: Variation of request rejection rate with different popularity skew (α) for N=4096. Server bandwidth consumption We study the server load or upload bandwidth consumption which is one of the most important concerns for content providers with respect to varying system parameters. Figure 6 illustrates the variation of different adaptation strategies for varying system sizes on the server bandwidth consumption where the popularity model is kept constant at α=1.0. The no-adaptation strategy (STATIC) does not perform quite well and generate a steep increase of server load from (N=256) to (N=4096). The shortage bandwidth based adaptation strategy (SB) performs the best with a higher control on server load from (N=256) to (N=4096) rendering it to be a scalable solution. Joint adaptation (SP-SB) also performs considerably better than SP which infers that the request popularity based adaptation does not help to reduce the server load to a significant extent. Instead SB and SP-SB strategies are more preferred for the conservation of server bandwidth STATIC 400 SP SB SP-SB Number of peers (N)

18 Figure 6: Server stress for varying user population with α=1.0 Next, we study the effect of varying segment popularity skew and its influence on the server load generated by the four variants of adaptation technique as depicted in Figure 7. Note, that the range of X-axes values are magnified to fit in the (max, min) range for studying the properties of each curve in a higher granularity. We make the following observations: (1) STATIC increases consistently with higher values of α and the average rate of increase for every interval (i.e., increase of α=0.5) is with the least in α= [ ] interval (value=1.7078); (2) SP performs better than STATIC but still generate a considerable server load and the curve is somewhat invariant to change of α with an average value of ; (3) The performance of SP-SB is better than the above two and is able to lower the server consumption to a considerable extent. The curve is initially invariant to α till 3.5 but after that it consistently shows a better performance in lowering the server load and thus can be concluded that this adaptation strategy performs well after α=3.5; (4) SB performs the best in terms of conserving server bandwidth and a closer look at the curve shows that it is able to consistently drop the server load till α=3.5 after which it seems to stay at a constant level (α>3.5). So, it can be concluded that the best operating point for SB strategy is α 3.5. Streaming Quality Though all the above metrics are important in a P2P system performance point of view, but streaming quality is a more relevant parameter from a user-centric perspective for improving Quality of Experience (QoE). Streaming Quality can be defined from various context, but here we consider playback continuity which can be defined as the number of segments that are received within deadline and used for continuous playback divided by the total number of segments that can fit in its lifetime. Figure 8 plots the result for different values of α from 1.0 to 5.0 with N=4096 peers. The general observations are: no-adaptive (STATIC) strategy has the lowest playback continuity index; performance of SP and SB are within comparable limits; SP- SB produces the best result with consistently high continuity. Note that, the range of X-axes is made to fit in the (max, min) range to get a closer look. Now, let us take a detail look in each of the respective curves as follows: (1) STATIC undergoes a consistent drop in continuity as the value of α varies from 1.0 to 5.0 with an average index of which means that it is unable to download 25% segments within deadline due to content/bandwidth deficiency; (2) SP improves the continuity index from (α=1.0) to (α=2.5) but after α=2.5 it consistently drops to (α=5.0) and thus its performance is reasonable till α=2.5; (3) SB have an initial drop of continuity index from (α=1.0) to (α=3.0), but improves consistently in the later part till (α=5.0) and thus its performance starts to enhance post α=3.0; (4) SP-SB generates the best streaming quality among all the four techniques and consistently improves the continuity from (α=1.0 ) to (α=3.0) after which it saturates and levels off without any further possible improvements.

19 Continuity Index Number of server streams STATIC SP Zipf constant (α) SB SP-SB Figure 7: Server load for different popularity ratios (α) with N= STATIC SP SB Zipf constant (α) SP-SB Figure 8: Playback continuity index variation for different popularity ratios with N=4096. Finally, we study the inter-relationship of weights and in the joint adaptation strategy SP-SB and their effects on streaming quality for a fixed segment popularity distribution (α=5.0) illustrated in Figure 9. From the figure it can be observed that the optimal weight ratios are.4 and where it generates the highest continuity index of We have also experimented with different popularity distribution skew (i.e., different values of α) and the results follow similar trend with similar optimal weights and. These values are obtained with our experimental assumptions for synthetic workload pattern and we do not claim that these are universally optimal. More informed values can be derived by experimenting with real network traces and the operating point can be dynamically adjusted in real and dynamic environments.

20 Continuity Index α_sp α_sb = 1.0 α_sb = 0.8 α_sb = 0.6 α_sb = 0.4 α_sb = 0.2 α_sb = 0.0 Figure 9: Plot of streaming quality variation with different values of and for α=5.0 CONCLUSION Query adaptation is important in the context of Temporal-DHT, especially when the popularity distributions are skewed. We formulated optimization problems to address search cost and server load. We derived practical optimized solutions which can help to adapt the query resolution mechanism for dealing with popularity skewed distribution of content. The essential objective is to minimize the search cost and server load which are important performance parameters in the context of a P2P-VoD system. The basic mechanism involves the adaptation of object update interval for minimizing the search cost and server load under dynamic changing content popularity distributions. We also showed distributed approaches for estimating the adaptation parameter ratios reliably. Simulation results demonstrated the effectiveness of the proposed techniques for improving various performance indicators such as search cost, request rejection rate, server bandwidth consumption, and streaming quality. REFERENCES Bhattacharya, A., Yang, Z., & Pan, D. (2011). Popularity Awareness in Temporal-DHT for P2Pbased Media Streaming Applications. IEEE International Symposium on Multimedia, (pp ). Bhattacharya, A., Yang, Z., & Zhang, S. (2010). Temporal DHT and its Application in P2P-VoD Systems. IEEE International Symposium on Multimedia, (pp ). Chu, Y., Rao, S., & Zhang, H. (2000). A case for end system multicast. Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, (pp. 1-12).