AN ABSTRACT OF THE THESIS OF. Arul Nambi Dhamodaran for the degree of Master of Science in

Transcription

1

2 AN ABSTRACT OF THE THESIS OF Arul Nambi Dhamodaran for the degree of Master of Science in Electrical and Computer Engineering presented on September 12, Title: Fast Data Replenishment in Peer to Peer Networks Abstract approved: Thinh Nguyen Peer-to-Peer (P2P) based distributed storage systems have gain much popularity in recent years. These systems rely greatly on the data redundancy to be robust under network dynamics, i.e., the dynamics of peer entering and departing the network. Hence, it is important to implement mechanisms for maintaining a certain level of data redundancy at all times in the network. One such mechanism is called distributed data replenishment which attempts to repair data due to a peer failure or departure in a distributed manner. Such distributed data replenishment schemes make use of the well-known Repetition code, the Reed Solomon code, or recently the Random Linear Network Coding techniques. However, these schemes do not consider bandwidth associated with peers during the replenishment. In this thesis we explore techniques for fast replenishment by taking into consideration the bandwidth capacities of peer links. Specifically, we formulate the problem of fast replenishment via linear programing framework. Our simulation results indicate that the proposed fast replenishment significantly outperforms the current approach under many typical network scenarios.

3 c Copyright by Arul Nambi Dhamodaran September 12, 2011 All Rights Reserved

4 Fast Data Replenishment in Peer to Peer Networks by Arul Nambi Dhamodaran A THESIS submitted to Oregon State University in partial fulfillment of the requirements for the degree of Master of Science Presented September 12, 2011 Commencement June 2012

5 Master of Science thesis of Arul Nambi Dhamodaran presented on September 12, 2011 APPROVED: Major Professor, representing Electrical and Computer Engineering Director of School of Electrical Engineering and Computer Science Dean of the Graduate School I understand that my thesis will become part of the permanent collection of Oregon State University libraries. My signature below authorizes release of my thesis to any reader upon request. Arul Nambi Dhamodaran, Author

6 ACKNOWLEDGEMENTS I would like to begin by expressing my sincere gratitude to my advisor, Prof. Thinh Nguyen, for his guidance and continuous encouragement, without his support this thesis would not have been possible. Working with him was enjoyable and refreshing. I would also like to thank Prof. Bella Bose for his support and inspiration, which helped me a great deal during my thesis work. I would like to thank all my friends and colleagues at Oregon State University for their friendships and useful discussions. My special thanks to my friends/roommates whose support helped me get through these two years with such ease and making every moment of it enjoyable and wonderful. I would like to thank my parents G. Dhamodaran and Kayali Dhamodaran for their constant support and encouragement. They have thought me the joy of love, the importance of family and the value of knowledge. They have stood by me in every one of my decisions and have always been a pillar of strength and support, without which I would not have reached this far.

7 TABLE OF CONTENTS 1. INTRODUCTION Distributed Storage Networks Data Replenishment Existing Methods Page Repetition Code Based Replenishment scheme Reed Solomon code based replenishment scheme Random Linear Network Coding based replenishment scheme Our Strategy BACKGROUND AND RELATED WORK Network Coding Linear Network Coding Peer to Peer Data Streaming Mesh Topology Linear Optimization Linear Programming The Simplex Method BANDWIDTH BASED TRAFFIC FLOW OPTIMIZATION FOR P2P NET- WORKS Introduction Fast Data Replenishment Scheme Bandwidth Based Optimization scheme General Case Bandwidth Based Optimization Linear Optimization for a N-node network node Bandwidth Based Optimization Example Optimization Function

8 TABLE OF CONTENTS (Continued) 3.5. Replenishment Time Analysis Page 4. SIMULATION RESULTS AND PERFORMANCE EVALUATION Simulations Results for Bandwidth Based Optimization Replenishment Time Analysis FUTURE WORK Bandwidth Based Optimization for Proportional Mixing Integer optimization of packet flow in P2P networks CONCLUSIONS BIBLIOGRAPHY

9 LIST OF FIGURES Figure Page 1.1 Simple P2P Network Simple illustration of Random Linear Network Coding (a) Represents the existing data replenishment method (b) Represents the proposed Fast Data Replenishment Technique Simple Butterfly Network illustrating the network coding technique Random Linear Network Coding example (a) Fully Connected Mesh Topology. (b) Partially Connected Mesh Topology Flow graph of the Simplex Algorithm (a) Packet Flow in Direct Method. (b) Packet Flow in Bandwidth Based Optimization Method N-node Mesh Network with one destination node (a). Packet flow of packet type P (1) using the existing methods. (b). Packet flow of packet type P (1) using bandwidth based optimization method Node Bandwidth Based Optimization Node Bandwidth Based Optimization Example Node Simulation Example Replenishment time for a 10 node network Bandwidth Based Optimization for Proportional Mixing Example of Integer based packet flow optimization

10 LIST OF TABLES Table Page 3.1 Packet Flow in a 4 node network with and without optimization Mean of Rate of packet flow with and without optimization for uniform distribution (R C < R D ) Mean of Rate of packet flow with and without optimization for uniform distribution (R C > R D ) Variance of packet flow with and without optimization for uniform distribution (R C < R D ) Variance of packet flow with and without optimization for uniform distribution (R C > R D ) Mean of Rate of packet flow with and without optimization for exponential distribution (R C < R D ) Mean of Rate of packet flow with and without optimization for exponential distribution (R C > R D ) Variance of packet flow with and without optimization for exponential distribution (R C < R D ) Variance of packet flow with and without optimization for exponential distribution (R C > R D )

11 FAST DATA REPLENISHMENT IN PEER TO PEER NETWORKS 1. INTRODUCTION 1.1. Distributed Storage Networks The distributed storage networks are computer networks in which the data is stored in several different nodes or peers. This is in stark contrast with traditional storage system where data is stored in a centralized location. Distributed storage network offer several advantages over traditional storage systems. First, due to its distributed design, a distributed storage system helps avoid the bottle neck failure which can be catastrophic in centralized storage systems. Second, if designed correctly, the storage capacity of distributed storage systems can also be increased incrementally, often with small costs, i. e., leading to high scalability in both capacity and cost. On the other hand, it is more complex to manage distributed storage systems as storage nodes are geographically far apart. This is especially true for the recently emerging Peer-to-Peer (P2P) based storage systems where peers can enter and depart the systems almost freely. As a result, data might disappear from the networks permanently or temporarily due to peer dynamics. Therefore, a major research effort in distributed storage system is to develop mechanisms for keeping the data in a P2P-based storage network in spite of the highly dynamic nature of P2P networks. One scalable mechanism is termed data replenishment. [7],[10].

12 Data Replenishment In the he last few years there has been an exponential increase in the use of Peer to Peer (P2P) Networks for sharing of information, video and audio files in the Internet. The P2P networks are distributed data storage systems where the data that is stored in them is replicated across multiple nodes or peers in different locations. This data replication across multiple nodes in a P2P Network helps avoid one of the main problems in a centralized storage network known as the bottleneck problem. [4],[5]. As peers in the P2P network can arrive and depart at any given time, now when a peer departs the P2P network the data stored in the peer is lost temporarily if the peer rejoins the network later, or permanently if the peer does not rejoin the network or if the data stored in that network is deleted. FIGURE 1.1: Simple P2P Network The files stored in such distributed storage networks will remain alive as long as there

13 3 is one full copy of the file present in the P2P network [14], [15]. In order to ensure that the files in the P2P network remain alive we must ensure that the data redundancy levels of the files stored in the P2P network always remain above one. The data redundancy levels are critical because, if the data redundancy level of a file goes below one then the file will no longer be recoverable and the information stored in the file is lost. In order to keep the files stored in the P2P networks active we must maintain the data redundancy levels of all the files stored in the network be above one, this can be achieved by constantly replenishing the lost data Existing Methods In the last few years various techniques have been created to improve the performance of the peer to peer networks. Data replenishment is one factor that plays a significant role in improving the performance of P2P networks by replenishing the data lost due to peer departure thus helping maintaining the data redundancy levels. Some important P2P data replenishment techniques are as listed below. Repetition code based replenishment scheme Reed Solomon code based replenishment scheme Random Linear Network Coding based replenishment scheme Repetition Code Based Replenishment scheme The Repetition code based replenishment scheme is the simplest and most straightforward replenishment technique. In this replenishment scheme, data is replicated across multiple peers. Specifically, suppose a file is divided into two parts A and B, and N peers are used to store the file. Then, half of the peers can store parts A and the remaining half

14 4 store parts B. In this case, the redundancy level or the ratio of the total data storage to the file size is N/2. Now, if a peer leaves the network, and a new peer joins the network, it will randomly connect to one of the existing peers and download its data. In this way, data is replenished. However, if the replenishments are repeated over many times, there will be a chance that there is only one identical part of a file that will remain in the network. Thus, a file is permanently irrecoverable since the file was originally composed of two parts. Note that one can change the redundancy level by dividing the file into an appropriate number of parts. For example, if a file is divided into three parts, and each third of peers store a different part of the file, then the redundancy level is N/ Reed Solomon code based replenishment scheme The Reed Solomon code based replenishment scheme is one of the data replenishment scheme in distributed storage networks, which employs standard channel coding techniques for data replenishment in P2P networks. In this scheme the data file of size L bits is divided into three equal parts of size L/3 bits. These three parts are then channel coded to produce K codewords. Each peer stores one such codeword with a redundancy level in such a type of network will be N/3. In such a type of coding technique only 3 unique codewords are required to reconstruct the original file [11], [12]. When a node leaves the network the new node that is joining the network will be replenished with the data from it s neighboring nodes. Here the new peer joining the network will randomly connect to other peers in the network and download any one codeword from any of the peers for replenishment of the data. If this process keeps repeating eventually the data file will not be recoverable when one set of the three set codewords is lost completely. The Reed Solomon code based technique performs better then the repetition code based technique. Also the data recoverability in such a system is longer.

15 Random Linear Network Coding based replenishment scheme The Random Linear Network Coding (RLNC) based replenishment scheme is very much similar to the Reed Solomon code based replenishment scheme where the file is first divided into there parts and then coded into K codewords, where each codeword is a Random Linear combination of the three parts of the original file [2],[8],[9]. Each peer or node in the network stores one such codeword, the data redundancy levels in the network is the same as the Reed Solomon code based technique N/3. M = x 1 A + x 2 B + x 3 C (1.1) The codeword generation using the random linear network coding technique is as given in the equation 1.1 where A, B and C are the three parts of the data file and x 1, x 2 and x 3 are the coefficients, the codeword M is a random linear combination of the file parts of A, B and C. Here the coefficients x i s are elements drawn uniformly at random intervals from a finite field. In this method since the coefficients x i s are known, we can say that if a peer has at least three independent codewords then that peer will be able to recover the original file. The figure given above shows the basic operations of random linear network coding technique where A, B and C are the source nodes and D is the destination node. S 1 and S 2 are the two parts of the data file. σ 1 and σ 2 are the segments of the data file parts S 1 and S 2 respectively and α 1 and α 2 are their coefficients. Now when a peer leaves the network and a new peer joins the network. The new peer that is joining the network will randomly connect to two or more of its neighboring nodes and instead of downloads the data packets from just one of its neighbors as is the case in the above two data replenishment schemes, it downloads data from all the neighboring nodes that it connects with and codes the downloaded data packets using the random linear network coding technique. By using this scheme we can prolong the duration for which the file will remain active in the network longer than the other two replenishment

16 6 FIGURE 1.2: Simple illustration of Random Linear Network Coding schemes mentioned above Our Strategy All of the above mentioned replenishment methods mainly focus on the different strategies for improving the data replenishment rates in P2P networks by using different coding techniques to achieve better replenishment rates, but none of these techniques focus on reducing the replenishment time to improve the performance of the P2P networks and in doing so it also increases the probability of successful data replenishment. Replenishment techniques that are commonly used today, all transfer the encoded data packets directly from the existing source nodes to the newcomer destination node and the new data is generated at the destination node only. Hence in this method the

17 7 total data transfer speed is essentially limited by the slowest link connecting the source nodes to the destination new node. This problem can be clearly explained by using the simple example that is given below. Consider a simple scenario in which a new node N connects with two existing source nodes A and B to download the data stored in these source nodes. The destination node then generates its own data by linear network coding the data downloaded form the source nodes A and B. Here the bandwidths between the three nodes of the network are R AN, R BN and R AB as shown in the figure below. FIGURE 1.3: (a) Represents the existing data replenishment method (b) Represents the proposed Fast Data Replenishment Technique. The figure 1.3(a) shows the packet flow route that is chosen by all the existing replenishment schemes mentioned above, here we can clearly see that all the existing replenishment methods only choose the most direct path between the source nodes to the destination node N and have no consideration of the bandwidth available between the source nodes A and B. This essentially limits the transmission speed of the nodes by the slowest link that is connecting the source nodes A and B to the destination node N. In our proposed method we also route some additional data packets through the link between the nodes A and B to reach the destination node N through the other source

18 8 node utilizing this unused bandwidth between the peers to increase the throughput and reduce the data replenishment time, thus improving the overall performance of the P2P Networks by increasing the probability of a successful replenishment before any of the nodes involved in the replenishment can depart the network.

19 9 2. BACKGROUND AND RELATED WORK 2.1. Network Coding Network coding is a traffic flow management technique introduced in the recent years, which has gained wide spread usage ever since it was first introduced by Ahlswede et. al., in the year 2000 [1]. Network coding is a technique where packets are encoded together at intermediate nodes and transmitted instead of simply forwording the packets of information that they receive from the source nodes towards the destination. Thus increasing the overall performance of the network and also reduce the congestion rates by reducing the traffic flow through the network. Another interesting fact about network coding is that using this technique we can achieve the maximum network capacity using some form of the random network coding techniques, while this is not usually possible with the traditional store and forward routing. [2]. The butterfly network given above is a simple example to illustrate the performance of network coding. In this example two packets A and B are to be sent to the two destination nodes as shown in the diagram. In the traditional approach we just send the two packets to each of the destination nodes, which requires four transmissions to complete the data transfer. In the network coding approach the packets A and B are sent to the destination nodes 1 and 2 directly and then a third packet A + B (network coding of A and B) is sent to the destination nodes thus reducing the number of required transmissions to only three transmissions to complete the data flow.

20 10 FIGURE 2.1: Simple Butterfly Network illustrating the network coding technique Linear Network Coding Traditionally all store and forward devices like the router, a node in P2P Networks or a node in an ad-hoc network will just relay the information that it has received to the destination node or another intermediate node. In network coding this node combines the information of the outgoing packets that it has received with the information packets that this node has generated or has received from another node and then forwards these new packets as outgoing packets in route to the destination node. Where the outgoing packets are a linear combination of the original packets present in that node. The linear network coding result of n bit packets will also have n bits only. [2]. The encoding and decoding process in the linear network coding scheme can be explained as given below. Assume that there are n original packets A 1, A 2,..., A n. which are generated by one or more source nodes in the network. Now each intermediate node present in the network generates new packets to forward by using linear network coding

21 11 to combine the original packets[16], [17]. m X i = g ij M j (2.1) j=1 The equation given above represents the encoding function in the linear network coding scheme where X i is the encoded packet and m is the number of original packets present in that node. g i1, g i2,..., g im are the coefficients which have been chosen from a finite field F n. The encoding node then includes the information about the coefficients (g ij ) in the header of these new packets and forwards these new packets to its neighbors. These encoding can be done recursively (i.e. the encoding process can be done on packets that have been already encoded). This operation can be repeated at any number of intermediate nodes present in the network and it will not affect the decoding process in the destination node. The decoding process of the linear network coding scheme is simple. Here let us assume that the destination node has already received a set of n packets. The decoding scheme retrieves the original packets by solving these n sets of linear equations. m (X i = g ij M j, i = 1, 2,...n) (2.2) j=1 The equation given above is used in the decoding processed where M i is unknown. That is the above given linear system has n encoded packets and m unknown variables. In order to recover all of the original data the receiver must have at-least received m linearly independent encoded packets (n m), i.e., the number of received packets must be at-least as large as the number of original packets. The most important aspect of this linear network coding scheme is how to select the linear combinations at each intermediate node of the network. A simple way to solve this problem is to let each node in the network select the coefficients uniformly at random over the field F n [3], which is completely independent and decentralized as shown in Figure

22 If the size of this field is large enough then the probability of the node selecting a combination that is linearly dependent will be very small and hence it is negligible. Hence by using this algorithm, the receiver will get enough number of independently encoded packets with high probability and small redundancy. That is it has a high probability of successfully decoding the encoded packets and retrieving the original data file. FIGURE 2.2: Random Linear Network Coding example.

23 13 The figure given above illustrates a simple example of random linear network coding in multicast scenario, where X 1 and X 2 are the original packets transmitted from the source node S. These data packets are received at the destination nodes E and F with network coding being performed at node B. The coefficients α i s are randomly chosen elements of a finite field F n Peer to Peer Data Streaming Peer to Peer system are raising in popularity with the increase in the number of internet users. The main reason for the increase in the usage of P2P systems is because of their ability to deliver large amounts of data at a significantly reduced cost. The most popular application of the P2P systems is the Bit-Torrent system [4]. In this system a Bit- Torrent file is partitioned into multiple distinct pieces. These pieces are then distributed among the peers present in the network to increase the receiving throughput as compared to the use of the centralized storage system in which all the data is stored in a single server which sends the pieces to multiple receivers which essentially limits the maximum speed at which the data can be sent. In P2P systems, since multiple peers store data a peer can receive multiple pieces of data from different peers in one time slot. In simple terms the Bit-Torrents can be viewed as a multi-sender multi-receiver system. The P2P systems also represent a simple scalable and cost effective alternative to traditional media streaming services enabling extended network coverage in the absence of IP multicast or expensive content distribution networks (CDN). The main drawback in using P2P systems for media streaming is that they do not provide any guaranteed support to the streaming services, hence these streaming systems must rely on self organized and adaptive network architectures to meet the stringent quality requirements of these systems.

24 Mesh Topology The Mesh topology is a simple network topology in which the nodes present in the network are self organized into a directed mesh. There are two types of mesh topologies a) Fully connected mesh topology and b) Partially connected mesh topology. The fully connected mesh topology is the one where every node in the network is connected to every other node that is present in the network as shown in the figure 2.3.(a) given below. The second type is the partially connected mesh network. In this type of a mesh network every node in the network is connected to one or more nodes present in that network as shown in the figure 2.3.(b) given below. FIGURE 2.3: (a) Fully Connected Mesh Topology. (b) Partially Connected Mesh Topology.

25 Linear Optimization Linear optimization is a mathematical modeling technique, which is used to obtain the optimal solution in resource allocation problems such as Production planning, optimizing packet flow in a network etc., The formulation of a linear programming problem requires a set of well-defined decision variables, with an objective function and a set of constraints. The objective function represents the aim or goal of the problem (i.e., decision variables), which has to be determined from the problem. Generally the objective in most cases would be to either maximize or minimize the decision variables based on the given set of constraints. The constraints are the set of limitations, and bounds that the variables cannot exceed Linear Programming Linear Programming is the process of transforming a real life problem into a mathematical model, which contains variables representing decisions that can be examined and solved for an optimal solution using algorithms. Linear programming problems are mainly concerned with the maximization or minimization of a linear objective function containing many variables subject to linear equality or inequality constraints. The General formulation of a linear programming problem is as given below n f(x) = c j x j = c T x (2.3) j=1 where c T x = c 1 x c n x n Subject to: n a ij x j b i for i = 1, 2,..., m. (2.4) j=1 x j 0 for j = 1, 2,..., n

26 16 here x j - variables that are to be optimized (i.e., Minimized or Maximized) c j - cost associated with each variable x j x j 0 is the non negativity constrain (i.e., the value of the variables cannot go below zero) a ij - coefficients of the functional constraints b i - amount of resources available The most Commonly used linear programming method is the Simplex method. Goerge B Dantzig is credited with creating the first solutions to linear programming problems using the Simplex Method in the year 1947 [5] The Simplex Method The simplex algorithm is an optimization algorithm, which is used to operate on the linear programs of the standard form. The simplex algorithm basically tackles linear programming problems that are too large to be handled by conventional means. This method simply uses the iterative process systematically to find out the maximum or minimum values defined by the objective function. The simplex method not only yields the optimal solution but also gains other valuable information to perform economic and what if analysis.

27 FIGURE 2.4: Flow graph of the Simplex Algorithm. 17

28 The basic operations of the simplex algorithm for solving linear optimization problems is given below: 18 The linear optimization problem given is first formulated into a system consisting of the objective function (OF) and its corresponding functional constraints. Here the objective function defines whether the given problem is a minimization problem or a maximization problem and optimizes it by altering the variables based on the functional constraints. The above defined system is then formulated into an initial tableau (i.e., in a matrix format in the form Ax = b). Calculate the initial best feasible solution (BFS). We then find the pivotal column in the initial tableau (it is the column with the most negative value in the objective function). If there are no negatives then the initial best feasible solution is the optimal solution. Find the ratios between the non-negative entries in the right hand side and the positive entries in the pivot column. If there are no positive entries, then there is no optimal solution for the problem. Now we find the pivotal row in the initial tableau (it is the row with the smallest non-negative ratio.) Here zero is also considered as a non-negative ratio. Now if identify the pivot element where the pivot row and pivot column meet. Return to step 4 and continue this operation until there are no more negatives in the bottom row. In the algorithm given above the first three steps are actually preliminary steps, which are used to setup the problem to be solved using the simplex algorithm. The

29 19 pivotal operation mentioned in the algorithm above is a geometrical operation of moving from one basic feasible solution to its adjacent feasible solution. This process is repeated until the optimal solution is found. The flow chart given below illustrates the functioning of the simplex algorithm.

30 3. BANDWIDTH BASED TRAFFIC FLOW OPTIMIZATION FOR P2P NETWORKS Introduction In the recent years the use of Pear to Pear (P2P) networks for information and video sharing in the internet has increased exponentially. This exponential growth has increased the need to improve the performance of these networks greatly. The P2P networks being a distributed storage and streaming network where peers share not only bandwidth but also storage, and computational resources between them. Given the nature of the P2P networks where peers can join or leave the network at any given time, results in the loss of data that is stored in these peers temporarily or permanently. Such data losses would result in the file becoming unusable. Hence it is important to develop techniques and protocols to help maintain sufficient redundancy levels, so that when a file is requested it will be available in the network irrespective of which peer leaves or joins the network. All the existing techniques for maintaining the data redundancy levels are some well known data replenishment techniques like the Random Linear Network Code, the Reed Solomon code and the Repetition code. All the above mentioned data replenishment schemes operate based on packet manipulation to ensure data replenishment by reducing the amount of traffic flow between the source and the destination node links, and maintain the redundancy levels, which in turn results in ensuring the data availability in the network for a prolonged period of time. However all the current replenishment mechanisms focus only on the techniques on how to replenish data to reduce the regeneration traffic, and to increase the lifetime of a piece of data stored in the system. Here, the data flow in these replenishment schemes is always the direct path between the source nodes and the destination node irrespective of the bandwidth capabilities associated with the peer links. Our scheme focuses on a fast

31 replenishment technique where the bandwidth capabilities of peer links are also taken into consideration and the traffic flow is optimized using the linear programming framework Fast Data Replenishment Scheme In a distributed storage network, to ensure the data reliability and availability, we need to minimize the data replenishment time to as little as possible. The less the replenishment time, more replenished data and higher the probability of a successful data replenishment before any of the peers involved in the replenishment processes can leave the network. Fast data replenishment results in the data file being preserved in the system without any data loss and it also increases the duration for which the file will remain active in the network. All the techniques involved in the data replenishment processes achieve it by reducing the network traffic flow between the peers and the newcomer destination node. Among all the different techniques replenish data in the P2P network the random linear network coding technique is the best as it is able to replenish the data much faster, compared to all the other data replenishment techniques. However even in the random linear network coding based replenishment scheme, the replenished data is always transferred directly from the existing peers to the newcomer destination node and the new data is generated at the destination node only. Therefore, the maximum number of packets that can flow through the peer links in these methods are limited by the slowest link that connects the source nodes to the newcomer destination node. In all distributed storage networks (specifically P2P Networks) there exists more than one path between the source and the destination nodes. In this work, we want to exploit any such alternate paths between the source and the destination nodes via other source nodes or peers and find an optimal packet flow route in which the data

32 22 replenishment is done in the shortest possible time Bandwidth Based Optimization scheme Distributed storage networks always have more than one path that connects the source and the destination nodes, but all the existing replenishment algorithms use the most direct route from the source to the destination irrespective of the bandwidth capabilities of the peer links found in the alternate routes. The replenishment time for all these replenishment schemes is limited by the link with the least bandwidth connecting the source and destination nodes directly. In the bandwidth based optimization method we exploit the unused bandwidth that exists between the peer links in the alternate paths between the source and the destination via other source nodes to improve the replenishment time. FIGURE 3.1: (a) Packet Flow in Direct Method. (b) Packet Flow in Bandwidth Based Optimization Method The bandwidth based traffic flow optimization problem consists of the source nodes and the newcomer destination node where all the nodes in the network are connected to every other node in the network (fully connected mesh network). Here the source nodes are the nodes that are already present in the network and have some data stored in them.

33 23 The newcomer destination node is the new node that is joining the network and does not have any data stored in it. To maintain the data redundancy levels in the distributed storage networks (P2P Networks) it is important to replenish the data from the source nodes to the newcomer destination node in the fastest possible time. Our optimization technique being a traffic flow regulation technique, it is independent of all the other replenishment schemes that are currently being used. This is one major advantage because this technique exploits the full bandwidth capacities of a distributed storage network by regulating the traffic flow through the bandwidths of the peer links to improve the replenishment time instead of focusing only on the direct link between the provider and the destination node as in the case of all existing data replenishment schemes. The bandwidth based optimization problem can be explained clearly by the following example. Consider the scenario that a newcomer destination node N, which connects to two existing source nodes A and B to replenish data by downloading the information that is stored in these nodes. It then generates its own data by using random linear network coding on the downloaded data from nodes A and B. Assuming link bandwidths between source and destination nodes are as shown in Figure 3.1. In the random linear code based replenishment scheme or any other data replenishment scheme that is currently being used, the destination node receives data from each source node directly as in Figure 3.1(a). In this scheme, the replenishment speed depends on the minimal edge connecting the destination node to the source node, which is 30KB/s on link A N. Thus, the bandwidth bottleneck on link A N limits the actual transmission rate during the replenishment process to 30KB/s only. Now if we allow the source node B to download data from source node A, and combine it with its own stored data then send to the destination node N. Using this updated scheme, the bottleneck link is A B with 40KB/s. It increases the actual transmission rate during the replenishment process to 40KB/s. We can clearly see

34 24 the improvement in replenishment speed if we utilize the available bandwidth between providers, which will reduce the replenishment time as well. The traffic flow method that is shown in Figure 3.1.(b) is optimal because it use the excess unused bandwidth between the source node A and source node B to transmit additional data to the destination via whichever node that can afford additional traffic flow, without essentially limiting the traffic flow between the source node to the destination node to the slowest node connecting the source nodes and the destination node as in the case shown in Figure 3.1.(a). By transmitting data via other source node instead of just transmitting the data directly from the source node to the destination node we greatly increases the throughput and reduce the replenishment time as it makes better utilizes the distributed bandwidth among the source and the destination nodes. For experimental purposes the packet size of the data packets used in bandwidth based optimization technique is kept very small and hence the delay time for the packet to reach the destination node via an intermediate node is proportionally small and hence neglected. In our model we have limited the traffic flow between the source and the destination is limited to a single hop transmission because multi-hop transfer greatly increases the packet loss rate the packet error rate and the propagation delay time. These factors negate any of the performance enhancements resulting due to the implementation of the multi-hop transmission scheme. The other constraint that we have applied in our bandwidth based traffic flow optimization scheme is that the destination node must have equal number of packets of all packet types at any given time. This equality constraint is implemented to enable the use of Random Linear Network Coding scheme to further improve the performance of the P2P network. If the equality constraint is not met, then the Random Linear Network Coding scheme will perform the mixing based on the smallest number of packets of a particular type. We will discuss this further in Section 3.3.1

35 General Case Bandwidth Based Optimization The general case bandwidth based optimization technique to optimize the traffic flow for an N node mesh network with one destination node and I source nodes (where I = N 1). In this type of a network every node is connected to every other node in the network with bi-directional communication capabilities between the nodes. Now we optimize the traffic flow in this network using linear optimization techniques such that equal number of packets of all packet types is received at the destination at any given time. FIGURE 3.2: N-node Mesh Network with one destination node.

36 26 The general case N-node mesh network given in Figure 3.2. consists of I source nodes (where I = N 1 and one destination node. The link R ij represents the maximum rate of traffic that can flow between the nodes i and j, and P (1) to P (i) represent the packet types present in the source nodes 1 to I respectively. The main objective of this bandwidth based optimization scheme is to receive equal number of packets of all packet types at the destination node at any given time. All the existing packet flow algorithms and replenishment schemes take the paths R 1N to R IN (most direct path between the source and the destination) as the preferred route for the traffic flow from the source nodes 1 to I to the destination node N respectively, without even considering all the other alternative paths that are available, their unused bandwidth capabilities and their potential to significantly reduce the replenishment time. The bandwidth based optimization scheme for fast data replenishment focuses on exploiting the bandwidth capacities of such alternate paths where the packet flow is routed via other source nodes (Peers) to significantly reduce the time required for data replenishment by increasing the throughput and in doing so increasing the probability of successful replenishment before any of the nodes involved in the replenishment leaves the network. Since this optimization scheme utilizes the alternate paths to reach the destination node via other source nodes we use the linear optimization technique to optimize the traffic flow between these nodes subject to certain constraints. The Packets that are traveling in the alternate path can travel through a maximum of one source node to reach the destination. This constrain is enforced to keep the packet loss rates and the packet delays to a minimum acceptable range. The second constrain is that the packet flow between the nodes is limited by the maximum available bandwidth R i,j between those nodes. The regulation of the packet flow in this type of a network using the linear optimization technique subject to the constraints given above for a N-node mesh network is explained below. The Figure 3.3 shows the difference in the packet flow of packet P (1) from its

37 source node 1 to its destination node N using the existing methods and the bandwidth based optimization method. 27 FIGURE 3.3: (a). Packet flow of packet type P (1) using the existing methods. (b). Packet flow of packet type P (1) using bandwidth based optimization method. The packet flow of packet type P (1) from the source node 1 to the destination node N takes the most direct path between the source and the destination in all of the existing methods which in this case is the path through the interconnect link R 1N and hence the data transfer rate of packet type P (1) is limited. But using the bandwidth based optimization technique we can increase the data transfer rate of packet type P (1) by sending these packets by exploiting the unused bandwidth available in the alternate paths where the packets are transferred via other source nodes to the destination node as shown in Figure 3.3.(b).

38 Linear Optimization for a N-node network The traffic flow in the bandwidth based optimization scheme is regulated using the linear optimization technique. Linear optimization is a mathematical modeling technique, which is used to find the optimal route for traffic flow in the given network, subject to certain constraints. The main parameters that we deal with in the bandwidth-based optimization technique are listed below: R i,j represents the maximum number of packets that flows between the nodes i and j per time slot. and P (1) i,j Actual number of packets of type P (1) that flows between the nodes i and j per time slot. P (2) i,j Actual number of packets of type P (2) that flows between the nodes i and j per time slot.... P (I) i,j Actual number of packets of type P () that flows between the nodes i and j per time slot. Where I = N-1 Here: P (x) i,j Represents the actual number of packets of type P (x) that flows between the nodes i and j per time slot. Where x = 1,2,3,..,I Goal: Maximize the number of received packets per time slot for the packet type with the mini-

39 29 mum number of packets. This objective is important as Random Linear Network Coding replenishment technique uses the same number of packets of different types for mixing. Thus the rate that can be generated from packet mixing is the minimal overall packet type rate. Constraints: The number of packets per time slot traverses a link cannot exceed the link capacity. These packets can take a maximum of one hop to reach the destination node. Based on the constrains given we can formulate our problem into a max-min optimization problem as shown in the equation (3.1). Optimization: max P (x) in [min( P (1) in, P (2) in, P (3) in,..., P () in, )] (3.1) The above equation can be re-written as: Where x = 1, 2, 3,..., n 1. max P (x) in [min( P (x) in )] (3.2) constraints: P (1) ij 0; P (2) ij 0; P (3) ij 0;... ; P () ij 0; and R ij x P (x) ij ; P (1) Xj P (1) jn ; P (2) Xj P (2) jn ; P (3) Xj P (3) jn ;... ; P () Xj P () jn ;

40 30 Where X - is the source node which contains all the packets of type X. N - is the final destination node Note that the total number of packets of type j is P j in. So the formulation in (3.1) is intuitively clear as it describes the maximization of the minimal rates over all the packet types. Furthermore, the constraints above are simply capacity constraints. We note that the problem above does not have the canonical form of a linear programming problem. However it can be easily transformed into multiple linear programming problems. Each can be solved separately, the results then can be combined. The main idea here is to pretend sequentially that each of the sum in the min is the actual minimum value. If it is, then it must be smaller than any other sum. We then use this fact as additional constraints. If there are n nodes, then we will have n different linear programming problems. We note that the objective for each problem will only have one max expression, and is of linear programming form. For optimization: Case(1) max... P (1) in P (1) in P (1) in P (1) in (Subject to all the constraints given above) And P (2) in ; P (3) in ; P () in ;

41 31 Case(2) max... P (2) in P (2) in P (2) in P (2) in (Subject to all the constraints given above) And P (1) in ; P (3) in ; P () in ; Case(3) max... n 1 P (3) in P (3) in P (3) in P (3) in (Subject to all the constraints given above) And P (1) in ; P (2) in ; P () in ;... Case(N-1) max... P () in P () in P () in P () in (Subject to all the constraints given above) And P (1) in ; P (2) in ; P (N 2) in ;

42 32 Here we choose the maximum value case as the optimal path for traffic flow, because it will produce better traffic flow routes as compared to the other cases without violating any of the constraints listed above node Bandwidth Based Optimization Example To fully understand the potential and benefits of the proposed bandwidth based traffic flow optimization using linear optimization scheme, let us consider a simple 4 - node P2P network as an example. In this example the nodes are mesh connected (i.e., every node in the network is connected to every other node in the network) which is as shown in the Figure 3.4 give below. Here the nodes 1, 2 and 3 are the source nodes and the node N is the destination node. Our goal here is to transmit the data in the source nodes to the destination node in the fastest possible time. The only constrain for this transmission being that the destination node must have equal amount of data from all the three source nodes at any given time. In all of the existing data replenishment schemes the traffic flow for such type of a network with the given constrain is essentially limited by the slowest link that is connecting the source nodes and the destination node N (i.e., min(r 1N, R 2N, R 3N )). But using the bandwidth based optimization scheme we can achieve much better throughput.

43 33 FIGURE 3.4: 4-Node Bandwidth Based Optimization. Let R 1N = 40kbps, R 2N = 60kbps and R 3N = 80kbps and R 12 = R 13 = R 32 = 50kbps. In this network the interconnect links are bi-directional full duplex links with equal bandwidth available in both directions. Now for this given scenario let us compare the performance of the existing replenishment methods to that of the bandwidth based replenishment scheme. For the network given above traffic flow using the existing replenishment techniques is essentially limited by the slowest link connecting the source and the destination nodes, which is 40kbps in this case. But using the bandwidth based optimization technique for data replenishment we can achieve a throughput of 60kbps as the traffic also flows in the alternate paths R 12, R 13 and R 23 which are ignored in all of the existing replenishment techniques. Hence by exploiting this unused bandwidth we increase the amount of traffic flow by 1.5times and proportionally reduce the replenishment time thus greatly enhancing the probability of a successful data replenishment.

44 34 FIGURE 3.5: 4-Node Bandwidth Based Optimization Example Optimization Function The Linear optimization of the traffic flow pattern in a 4 node network is formulated as given below: Optimization function: max[min( 3 P (1) i4, 3 P (2) i4, 3 P (3) i4 )] (3.3) Constraints: P (1) ij 0; P (2) ij 0; P (3) ij 0; and R 1N 3 x=1 P (x) ij ; P (1) 12 P (1) 24 ; P (1) 13 P (1) 34 ; P (2) 12 P (2) 14 ; P (2) 23 P (2) 34 ; P (3) 13 P (3) 14 ; P (3) 23 P (3) 24 ;

45 35 For optimization: Case(1) max P (1) i4 P (1) i4 P (1) i4 3 3 (Subject to all the constraints given above) and P (2) i4 ; P (3) i4 ; Case(2) max P (2) i4 P (2) i4 P (2) i4 3 3 (Subject to all the constraints given above) and P (1) i4 ; P (3) i4 ; Case(3) max P (3) i4 P (3) i4 P (3) i4 3 3 (Subject to all the constraints given above) and P (1) i4 ; P (2) i4 ; The matrix representation of the above given optimization problem can be written in the form Ax b as given below for the optimization constraints given in case(1):

46 P (1) 12 P (1) 13 P (1) 14 P (1) 24 P (1) 34 P (2) 12 P (2) 14 P (2) 24 P (2) 23 P (2) 34 P (3) 13 P (3) 14 P (3) 23 P (3) 24 P (3) R 14 R 24 R 34 R 12 R 13 R 12 R 23 R 13 R Based on the matrix given above we can obtain the optimal traffic flow routes by using the linear programming. The results of the optimal traffic flow pattern of the 4 node network is as given in the table 3.1 below. This table clearly shows that the packet flow rates of the 4 node network with optimization produces better results as compared to that of all the existing data replenishment methods, thus resulting in faster replenishment.

47 37 List of variables Packet Flow With optimization P (1) P (1) P (1) P (1) P (1) P (2) P (2) P (2) P (2) P (2) P (3) P (3) P (3) P (3) Packet Flow Without optimization P (3) TABLE 3.1: Packet Flow in a 4 node network with and without optimization Replenishment Time Analysis In the bandwidth based optimization technique that we discussed above we neglected the propagation delay (i.e., the time taken for a packet to flow from one node to another) because in this technique we consider the packet size to be very small and hence the delay is very small thus neglected. For the packet delay analysis we fix the packet size to be significantly large which in turn increases the propagation delay.

48 38 All the existing data replenishment algorithms will have low propagation delay because the traffic flow in these scheme is sent directly from source to the destination. The bandwidth based optimization scheme however tries to exploit the additional bandwidth that is available between the source nodes thus increasing the propagation delay, because the packets reach the destination by flowing through other source nodes. That is in this method the packet is to flow from node A (source node) to node N (destination node) via node B then the node B has to wait until it has received the full data packet from the node A before it can begin to transmit that packet to the destination node N. This wait time is called as the Propagation delay time. This delay time increases as the size of the data packet being transmitted is increasing thus resulting in the need for the time based optimization scheme. In this Replenishment time analysis we mainly focus on the size of the packet and the resulting propagation delay. Once the delay is computed for traffic flow between all the nodes in the network we use the linear optimization technique to find the optimal traffic flow route between the source nodes and the destination.

49 4. SIMULATION RESULTS AND PERFORMANCE EVALUATION 39 In this chapter, we show the simulation results that demonstrate that the bandwidth based traffic flow regulation using linear optimization techniques produces better replenishment rates as compared to the various replenishment schemes that are currently in use. Our Replenishment scheme also increases the probability of successful replenishment of the data before any of the nodes involved in the replenishment can leave the network Simulations Results for Bandwidth Based Optimization The Simulation setup is implemented using the MATLAB s Linear Optimization tool. Here we simulate a N node P2P network that consists of N 1 source nodes (existing nodes) and one destination nodes (newcomer nodes). Since data replenishment in P2P networks are done between neighbors where all neighboring nodes are interconnected our P2P network has a fully connected mesh network. The simulator complies with all the traffic flow constraints mentioned above. The bandwidth of each link present in the topology of the P2P network that is being simulated is chosen at random using uniform distribution or exponential distribution. In our simulations we implement a 3 node network to a 10 node network and compare the traffic flow rates between these networks for various bandwidth values. The simulations results of a 3 - node network to a 10 node network whose bandwidths were chosen at random between the limits specified below for R C and R D respectively. where R C - represents the interconnect links between the source nodes R D - represents the interconnect links between the source nodes and the destination node

50 40 The functioning of the simulator can be explained using a 3-node P2P network where node A and node B are the source nodes and the node N is the destination node, where R C represents the link R AB and R D represents links R AN and R BN as shown in the Figure 4.1. Now we select the bandwidths of the interconnect links in a random uniform distribution or random exponential distribution. This process is repeated for several different bandwidth values as illustrated in the table below. The simulation is repeated for the 4-node to 10-node networks. The tables given below illustrate the results for the traffic flow patterns in a 3-node to a 10-node networks. FIGURE 4.1: 3-Node Simulation Example.

51 41 The tables 4.1 and 4.2 given below shows the mean rate of packet flow for a 3-node to 10-node P2P network with and without optimization for R C < R D and R C > R D respectively. Here the R C and R D values are drawn randomly using uniform distribution. These tables clearly show that the packet flow rates are better in the traffic flow with optimization method than the existing methods. R C = packets per time slot and R D = packets per time slot No. Of Nodes Mean Rate of Packet Flow With optimization Mean Rate of Packet Flow Without optimization TABLE 4.1: Mean of Rate of packet flow with and without optimization for uniform distribution (R C < R D ) R C = packets per time slot and R D = packets per time slot No. Of Nodes Mean Rate of Packet Flow With optimization Mean Rate of Packet Flow Without optimization TABLE 4.2: Mean of Rate of packet flow with and without optimization for uniform distribution (R C > R D )

52 42 The tables 4.3 and 4.4 given below show the variance in packet flow for a 3 to 10 node P2P network with and without optimization for R C < R D and R C > R D respectively. The R C and R D values are drawn randomly using uniform distribution. The variance levels for the packet flow with optimization as compared to that of packet flow without optimization shows reduced variance in the traffic flow with optimization. R C = packets per time slot and R D = packets per time slot No. Of Nodes Variance of Packet Flow With optimization Variance of Packet Flow Without optimization TABLE 4.3: Variance of packet flow with and without optimization for uniform distribution (R C < R D ) R C = packets per time slot and R D = packets per time slot No. Of Nodes Variance of Packet Flow With optimization Variance of Packet Flow Without optimization TABLE 4.4: Variance of packet flow with and without optimization for uniform distribution (R C > R D )

53 43 The tables 4.5 and 4.6 given below show the mean rate of packet flow for a 3 to 10 node P2P network with and without optimization for R C < R D and R C > R D respectively. The R C and R D values are drawn randomly using exponential distribution. These tables clearly that the packet flow rates are better in the traffic flow with optimization method than the existing methods. R C = packets per time slot and R D = packets per time slot No. Of Nodes Mean Rate of Packet Flow With optimization Mean Rate of Packet Flow Without optimization TABLE 4.5: Mean of Rate of packet flow with and without optimization for exponential distribution (R C < R D ) R C = packets per time slot and R D = packets per time slot No. Of Nodes Mean Rate of Packet Flow With optimization Mean Rate of Packet Flow Without optimization TABLE 4.6: Mean of Rate of packet flow with and without optimization for exponential distribution (R C > R D )

54 44 The tables 4.7 and 4.8 given below show the variance in packet flow for a 3 to 10 node P2P network with and without optimization for R C < R D and R C > R D respectively. The R C and R D values are drawn randomly using exponential distribution. The variance levels for the packet flow with optimization as compared to that of packet flow without optimization show that the packet flow with optimization has greater variance indicating higher change in the traffic flow rates for different bandwidth values of the network. R C = packets per time slot and R D = packets per time slot No. Of Nodes Variance of Packet Flow With optimization Variance of Packet Flow Without optimization TABLE 4.7: Variance of packet flow with and without optimization for exponential distribution (R C < R D ) R C = packets per time slot and R D = packets per time slot No. Of Nodes Variance of Packet Flow With optimization Variance of Packet Flow Without optimization TABLE 4.8: Variance of packet flow with and without optimization for exponential distribution (R C > R D )

55 Replenishment Time Analysis The replenishment time analysis computes the average time taken for successful replenishment of data with and without optimization. The Figure 4.2 given below shows the difference in the replenishment time for traffic flow in a 10 node network with and without optimization, for a fixed packet size. The results given below clearly indicate the performance enhancement in the replenishment time of the 10 node network with optimization. FIGURE 4.2: Replenishment time for a 10 node network