Performance analysis of Bloom filterbased

Transcription

1 Aalto University School of Science Degree Programme of Computer Science and Engineering Sajjad Rizvi Performance analysis of Bloom filterbased multicast Master s Thesis Espoo, June 18, 2011 Supervisor: Instructor: Professor Tuomas Aura, Aalto University Professor Peeter Laud, University of Tartu András Zahemszky, M.Sc., Ericsson Research

2 Aalto University School of Science Degree Programme of Computer Science and Engineering Author: Sajjad Rizvi Title: Performance analysis of Bloom filter-based multicast ABSTRACT OF MASTER S THESIS Date: June 18, 2011 Pages: 87 Professorship: Data Communication Software Code: T-110 Supervisor: Instructor: Professor Tuomas Aura Professor Peeter Laud András Zahemszky, M.Sc., Ericsson Research The advancement of technologies such as IPTV, online gaming and content delivery networks require efficient bandwidth utilization for scalability and resource conservation. IP multicast, an elusive solution, is a thoroughly explored research area. The stateful nature of IP multicast and other problems such as complexity of management make it a questionable solution for deployment on global scale. As an alternative, multicast based on Bloom filters, a space and time efficient data structure, has been proposed as a promising technique for stateless multicast routing. However, the inherent problem of false positives in the Bloom filters causes the packets to be forwarded on some unnecessary network paths. In addition, poor choice of parameters such as the size of the Bloom filter may induce network overhead. The overhead increases with the number of receivers, which limits the potential of Bloom filter based multicast as a scalable solution for large multicast groups. In this work, we simulate a Bloom filter based multicast protocol to analyze its limitations and to explore the behavior of different parameters related to Bloom filters. We propose novel techniques for reducing the network overhead by splitting the multicast delivery tree in a way that increases the scalability of the protocol. The simulation results show that Bloom filter based multicast can be highly scalable for a multicast network where there is a large number of multicast groups with relatively small number of receivers per group. Our multicast tree splitting methods make it more scalable for a larger number of receivers per group while maintaining the network overhead below a certain maximum level. Moreover, the simulation results provide us with insights about the choice of good parameters for Bloom filters. Keywords: Language: Multicast, Bloom filters, routing, network overhead English 2

3 Acknowledgements I am especially thankful to my instructor, András Zahemszky, for his dedication, patience and guidance throughout this thesis work. Tuomas Aura, my supervisor, is an impressive personality whose close supervision allowed me to learn a lot of things and made me finish this work in time. My other supervisor, Peeter Laud, his help and guidance alway amazes me and I always admire his support. I will always be thankful to my supervisors and my instructor. I am thankful to my family whose constant support is always the source of inspiration. I am also thankful to all of my friends who made my two years memorable and enjoyable. Especially, Ameer Shah, Raheel Farhat, Mudassir Abbad, Sanja Šćepanović and Jabed Morshed for their friendship, love and care. It am also thankful to our administrative staff, especially Soili Adolfsson, for her presence and willingness to help in any matter. Espoo, June 18, 2011 Sajjad Rizvi 3

4 Abbreviations and Acronyms AS Autonomous system GBF Group Bloom filter GID Group ID GUID Globally Unique Indentifier HT Hierarchical divisive Tree IP Internet Protocol IPv4 Internet Protocol version 4 IPTV Internet Protocol Television ISP Internet Service Provider LIPSIN Line Speed Publish/Subscribe Inter-Networking MHT Multiple Hierarchical divisive Tree MRT Multiple Random Tree MST Multiple Single Tree MVPN Multicast Virtual Private Network NS3 Network Simulator 3 PIM Protocol Independent Multicast PIM-SM Protocol Independent Multicast Sparse Mode PIM-SSM Protocol Independent - Source Specific Multicast RBF Receiver Bloom filter RID Receiver ID RT Random Tree ST Single tree TTL Time To Live 4

5 Contents Acknowledgements 3 Abbreviations and Acronyms 4 1 Introduction 10 2 Bloom filter-based multicast Bloom Filter Bloom filter-based multicast protocol Joining a multicast group Leaving a multicast group Distributing data to a multicast group Controlling the network overhead Random tree splitting Hierarchical divisive tree splitting Creating the Bloom tree Splitting the Bloom tree A quick enhancement Implementation and Simulation Simulator Dataset Group membership model Implementation Main module Topology module Packet structure and semantics Protocol stack Network interface layer Bloom filter forwarding layer

6 Upper layer Bloom filter Logger Experiments Running the simulations Simulation results How many hash functions? Single multicast Tree Single multicast tree per network interface Random Tree splitting Random tree splits per network interface Hierarchical divisive tree splitting Hierarchical divisive tree splits per network interface Putting it all together Discussion Overhead due to Bloom filter length Choosing the right parameters Are the results absolute? Should we unicast or multicast? Should we vary the parameters of the Bloom filter Merging of the Bloom filters depending on the similarity of the set-bit positions Fast inspection of joined group on the receiver node Duplication of received data Conclusions 73 A Node degree distribution 79 B Results of simulation 81 C Calculation of optimal number of hash functions k 83 6

7 List of Figures 2.1 An illustration of 8-bit Bloom filter; Links 1 and 2 are added in the Bloom filter whereas Link 3 is a false positive. The practical size of Bloom filters is usually large A network of 4 nodes and 3 links A sample forwarding table of 8-bit link-id and k=3 set bits per ID; the practical size of a link-id is usually relatievly large Permutation and reverse permutation of 8-btis Bloom filter Process of joining multicast group Process of data distribution on multicast group An example of multicast delivery tree for one group. A is the source node and E, F, G and H are the receivers. Link E2 shows a false positive Random tree splitting (a) and an improved version of the random splitting (b). Two trees are shown in each figure. The same packet is sent twice for each (shown by different arrow lines and colors) Illustration of the creation of the in-memory Bloom tree. There are five receivers E,F,G,H and I, and A is the source node A Bloom tree to illustrate HD-tree splitting. The resultant trees are (A1,B1,B2,C1,D1), (A1,B3,E1) and (A1,B3,E2,I1,I2) The source node with multiple interfaces. The Bloom filters received from different interfaces must not be merged Long tail of number of receivers Formats of the packets used in the protocol. Note some optional fields in the JOIN and DATA packet formats Protocol stack Flow of events and actions in Bloom filter forwarding layer. The dotted line box shows the process which is only used in case of hierarchical divisive tree splitting

8 4.5 Flow of events and actions in Upper layer. The dotted line box shows the process which is only used in case of hierarchical divisive tree splitting Network overhead for different values of k for various sizes of the Bloom filter (m) in single multicast delivery tree, dataset 3257, (500 groups) Network overhead and fill factor as a function of the number of receivers for various Bloom Filter lengths, k = 4, dataset 3257 (500 groups); the dots at the bottom show the number of links, n = (m/k) ln 2 in respective colors Network overhead as a function of fill factor for various sizes of Bloom filters, k = 4, dataset 3257 (500 groups) Network overhead as a function of number of receivers for single tree (ST ) and multiple single tree (MST ) for various Bloom filter sizes, k = 4, dataset 3257 (500 groups) Network overhead as a function of number of receivers for single tree (ST ) and multiple single tree (MST ) for various Bloom filter sizes, k = 4, dataset 3257 (500 groups) Fill factor as a function of number of receivers for single tree (ST ) and multiple single tree (MST ) for various Bloom filter sizes, k = 4, dataset 3257 (500 groups) Random tree split, overhead and tree count for various tree split thresholds, m = 32 and 48 bytes, k = 4, dataset 3257 (500 groups) Random tree split, overhead and trees count for various tree split thresholds, m = 64 and 128 bytes, k = 4, dataset 3257 (500 groups) Random Tree (RT) and Single Tree (ST) network overhead, thr = 50%, k = 4, dataset 3257 (500 groups) Random Tree (RT ), number of trees, thr = 50%, k = 4, dataset 3257 (500 groups) Multiple Random Tree (MRT ) and Random Tree (RT ) network overhead, thr = 50%, k = 4, dataset 3257 (500 groups) Multiple Random Tree (MRT ) and Random Tree (RT ), number of trees with increasing number of receivers, thr = 50%, k = 4, dataset 3257 (500 groups) Multiple Random Tree (MRT ) and Random Tree (RT ), average fill factor of tree splits, thr = 50%, k = 4, dataset 3257 (500 groups)

9 5.14 Random Tree (RT ), Multiple Random Tree (MRT ), Single Tree (ST ) and Multiple Single Tree (MST ) network overhead, thr = 50%, m = 32 bytes, k = 4, for two datasets (500 groups) Hierarchical divisive Tree (HT) splitting, network overhead, thr = 50%, k = 4, dataset 3257 (500 groups) Hierarchical divisive Tree (HT ) splitting, number of trees with increasing number of receivers, thr = 50%, k = 4, dataset 3257 (500 groups) Multiple Hierarchical divisive Tree (MHT ) splitting and single Hierarchical divisive Tree (HT ) splitting, network overhead with increasing number of receivers, thr = 50%, k = 4, dataset 3257 (500 groups) Multiple Hierarchical divisive Tree splitting, number of trees with increasing number of receivers, thr = 50%, k = 4, dataset 3257 (500 groups) Multiple Hierarchical divisive Tree (MHT), average fill factor with the increasing number of receivers, thr = 50%, k = 4, dataset 3257 (500 groups) Multiple Hierarchical divisive Tree (MHT ) splitting, network overhead, thr = 50%, k = 4, dataset 1239 (500 groups) Multiple Hierarchical divisive Tree (MHT ), Multiple Random Tree (MRT ), Multiple Single Tree (MST ) and multiple unicast, network overhead with increasing number of receivers, thr = 50%, k = 4, for two datasets (500 groups) Network overhead for various values of k with 32 Bytes Bloom filter for MHT splitting method, for two datasets (500 groups) Multiple Hierarchical divisive Tree (MHT ) splitting, data bytes overhead when 1476 bytes of data is multicast with 30-byte header in addition to Bloom filter size, thr = 50% (500 groups) An example of packet duplication; B gets packets from both trees, A,B,C and A,B,D A.1 Node degree distribution among the nodes for dataset AS A.2 Node degree distribution among the nodes for dataset AS

10 Chapter 1 Introduction An increasing demand for group communication applications for user generated content, online gaming services and the live streams through Internet such as IPTV are creating a need of IP multicast deployment on large scale. IP multicast provides an efficient solution for point-to-multipoint communication. However, IP multicast has been the subject of research since its proposal in 1990 [10] and it is still not widely deployed due to numerous problems [9, 11]. One of these problems in the current IP multicast protocols (e.g., SSM [4] and PIM-SM [12]) is the scalability of the number of multicast groups in the network. The IP multicast protocols store forwarding tables and states within the routers for each multicast group to which these routers forward packets. The size of these tables increases with the number of multicast groups and receivers. However, the routers have limits on the size of forwarding tables or, in other words, on the size of the multicast network. Among many proposed solutions, Bloom filter [5] based packet forwarding has been proposed as a scalable solution for IP multicast at network level [14, 21, 22, 30]. Bloom filter is a space efficient data structure to store large number of items in small space. BloomCast [27, 28] and LIPSIN [14] are the proposals which uses in-packet Bloom filters for data forwarding decisions. A Bloom filter is packed in the packet header of each data packet. This Bloom filter contains efficiently encoded information (ID) of the links on which the packet needs to be forwarded towards the receivers. Thus, the packets are source routed and the routing nodes do not need any state. However, the packets are also forwarded to some wrong links due to the false positives which are an inherent limitation of the Bloom filter data structure. The false positives may cause several packet forwarding anomalies such as loops within the network [28]. The packets forwarded to wrong links generate unwanted network traffic which results in network overhead as compared to multicast protocols. This network overhead causes the wastage of computational and 10

11 CHAPTER 1. INTRODUCTION 11 communication resources such as processing, memory, bandwidth and ultimately energy. The network overhead depends on the false positive rate which, in addition to another variable, is a function of the fill factor, a parameter of the Bloom filter data structure. Therefore, the network overhead caused by the false positives can be controlled by keeping the fill factor under a defined limit. Keeping the fill factor within a limit implies an upper bound on the number of receivers participating in that multicast group because the fill factor increases with the number of receivers. Thus, Bloom filter based multicast is also limited in scale but with a different perspective than traditional IP multicast. That is, it has no limit on the number of multicast groups but an upper bound on the number of receivers which can participate in a single multicast group. A possible way to cater for this scalability limitation of the Bloom filter based forwarding is to divide the receivers into more than one Bloom filter [28]. This allows us to limit the fill factor for each Bloom filter and the false positive rate. If 10 receivers, for instance, fill the Bloom filter with 70% set bits, dividing the receivers in two Bloom filters will reduce the fill factor to 35% per Bloom filter. However, dividing the single Bloom filter into multiple Bloom filters essentially means creating multiple multicast delivery trees having different sets of receivers, rooted at the same source node. This results in forwarding the same data over the redundant links between the multiple delivery trees, which also increases the network overhead. Thus, we limit the network overhead caused by false positives but, on the other hand, increase the network overhead due to redundant links in the multiple delivery trees. We can minimize the overhead due to redundant links by splitting the multicast tree in such a way that the the number of common links is minimal. This will result in increased scalability of Bloom filter based multicast. In this thesis, we implement and simulate the Bloom filter based packet forwarding protocol and analyze the protocol s scalability limitations by running simulations over realistic network topologies of AS level datasets. We also propose novel methods for splitting the multicast delivery trees in order to reduce the false positive rate and to increase the scalability. The major contributions of this work are the implementation of Bloom filter based forwarding protocol with bit-permutations to stop loops in the network caused by false positives (proposed by Särelä et al. [28]), its simulation with existing topological datasets to analyze the scalability of the protocol, formulation and implementation of novel methods and analysis of the scalability of these tree splitting methods. The rest of the thesis is arranged as follows. Chapter 2 gives the overview and the information necessary to understand the Bloom filter based multicast

12 CHAPTER 1. INTRODUCTION 12 protocol. The scalability limitations of the Bloom filter based multicast and the methods to increase the scalability are presented in chapter 3. The details of the simulation and implementation are described in chapter 4. Chapter 5 discusses the simulation results. The lessons learned in the study and some open issues are discussed in chapter 6. Finally, the chapter 7 concludes the present work.

13 Chapter 2 Bloom filter-based multicast Multicast is a one-to-many communication model. In multicast communication, packets are destined for a set of receivers who are members of the multicast group. The multicast has the advantage of efficient network usage since the data is not unnecessarily duplicated and packets are not sent on redundant links of the network. Several IP based multicast protocols have been proposed. Among others, protocol independent multicast (PIM) is a family of multicast routing protocols. These routing protocols save information about each multicast group in the routing nodes and thus cannot scale to a very large number of groups. Source-routed multicast based on Bloom filters (a space efficient data structure) has been proposed as scalable solution for multicast. In Bloom filter based multicast, the state is not saved in the routers. Instead the delivery tree is encoded in the packet header as a Bloom filter and the group management is done by the source node. Thus the routing nodes do not have any extra burden of an information management. Therefore, Bloom filter based multicast can scale to a large number of groups. The network nodes just forward packets based on the ready information in the packet header. In this section, first we introduce the Bloom filter data structure and then we discuss the Bloom filter based multicast protocol. 2.1 Bloom Filter A Bloom filter is a space efficient data structure to store a large set of items. It is an array of bits initialized to zero where the number of bits define the length of the Bloom filter. In order to add an item in the Bloom filter, at most k bits are set to 1 in the bit array. The indexes of the bits to set are defined by k hash functions independently. The input of these hash functions is the item to be added and the output of each hash function is a bit-position 13

14 CHAPTER 2. BLOOM FILTER-BASED MULTICAST 14 Figure 2.1: An illustration of 8-bit Bloom filter; Links 1 and 2 are added in the Bloom filter whereas Link 3 is a false positive. The practical size of Bloom filters is usually large to be set to 1 in the Bloom filter. The subsequent addition of items may overlap the set-bit positions. Since no other information about the added items is stored, the overlapping of the bits by the addition of different items does not allow the deletion of items from the Bloom filter. Therefore, the items can be added in the Bloom filter but cannot be deleted. To query the membership of an item in the Bloom filter, the corresponding k bits are checked, as defined by the output of the hash functions for that item. If all the bits are set, the item is considered to be present in the Bloom filter. However, it may happen that some of those bits were originally set by another item. Thus, the checking may falsely report the presence of an item in the Bloom filter. This is called a false positive. That is, the item has not been added in the Bloom filter but the membership query reported it to be there. The false positive rate of the Bloom filter is a function of two variables, the number of hash functions k and the fill factor of the Bloom filter, where the fill factor is the percentage of set bits in the filter. Thus, the false positive rate is given by the formula fpr = ρ k where k is the number of hash functions and ρ is the fill factor of the Bloom filter. As we increase the fill factor by adding more items, the probability of false positives increases. Therefore, a limited number of items can be added into the Bloom filter to keep the false positive rate under a certain limit. In this thesis work, we use Bloom filters to store the IDs of the links in the multicast delivery tree. Thus the Bloom filter is the set of the link IDs and we test for presence of specific link IDs for packet forwarding decisions. Figure 2.1 shows an 8-bit Bloom filter where two links, link-1 and link-2 are added to the Bloom filter. These two links set the bits 2, 6 and 5, 8 respectively.

15 CHAPTER 2. BLOOM FILTER-BASED MULTICAST 15 Link-3 is falsely reported to be present in the Bloom filter (a false positive) since the bit-positions given by the hash functions are set to 1. In the next section, we discuss the multicast protocol which uses such Bloom filters to distribute multicast data to the receivers of the multicast group. 2.2 Bloom filter-based multicast protocol BloomCast [27, 28] is a Bloom filter based multicast protocol which is originally derived from LIPSIN [14]. We modify BloomCast for our study about the Bloom filter based multicast and its network overhead. The Bloom filter based multicast is similar to the source specific multicast (PIM-SSM) [13] in the sense that there is only one source node per group. However, unlikely PIM-SSM, no information about the multicast groups is stored in the routing nodes. In the Bloom filter based multicast protocol, it is assumed that any node can be a source or a receiver of multicast data in the network. The protocol demands slightly higher computational and memory resources on the node which is sending data to the multicast group. Moreover, it is assumed that IP based packet routing is available during the process of joining and leaving the multicast group. However, when data is sent by the source of multicast data, IP routing is not used. Instead, Bloom filter based forwarding is used. In addition to previous assumption, it is also assumed that the packets always follow the shortest path between the nodes when unicast through IP based routing system. Furthermore, the security and reliability of protocol is not considered in present work. The reliability requirements are discussed by Zahemszky et al. [29]. The focus of this thesis work is to analyze the limits of network in terms of scalability and efficiency when data is multicast through Bloom filter based packet forwarding. We begin the description of protocol with the introduction of entities involved in the protocol: Receiver node: It is the network node which subscribes to a multicast group. These nodes join and leave the multicast network by sending JOIN and LEAVE messages, respectively, to the publisher of data. We will use the term receiver or subscriber interchangeably to refer to the receiver node. Source node: It is the network node which sends the data to the receiver nodes in a multicast group. We will use the term source or publisher

16 CHAPTER 2. BLOOM FILTER-BASED MULTICAST 16 Figure 2.2: A network of 4 nodes and 3 links. interchangeably for the source node. Like PIM-SSM, each multicast group has only one source which can multicast data. Multicast group: A multicast group is a set of a source node and receiver nodes which subscribe to the source of a group. It is uniquely identified by the pair Group ID and source ID (like in the case of PIM-SSM). When a source node sends data to a multicast group, it is received by all the receivers which subscribed to that group on that source. Group ID: It is a unique identifier for a multicast group. It is used to uniquely identify a multicast group and to multicast data to the group. No strict restriction on the type and format of a group ID is imposed. It can be a small unique number like GUID [TODO: reference for GUID] or a string based topic e.g., However, the size of the ID is restricted due to packet length format. Node ID: It is a unique identifier like the group ID. It is used to uniquely identify source and receiver nodes. Again, no restrictions are made on the type and format of the ID but its size is fixed. Data packets: The packets which are sent by the source node to the receiver nodes on multicast group are called data packets. These packets contain the actual data which is multicast to the group. Link ID: Nodes in the network are connected through interfaces and links. Each end of a link is connected to an interface. The link is given a unique ID and is represented by a Bloom filter containing a single element. The size (in bits or bytes) of this ID and the number of set bits, k, depends on the configuration of the protocol and is predefined for each device. Moreover, the interface on which a packet is received is called the incoming interface and the corresponding link is called incoming link.

17 CHAPTER 2. BLOOM FILTER-BASED MULTICAST 17 Figure 2.3: A sample forwarding table of 8-bit link-id and k=3 set bits per ID; the practical size of a link-id is usually relatievly large Forwarding table: Each node has several interfaces and each interface is connected to a link. The combinations of interface numbers and link IDs are saved in a table in each node which is termed as forwarding table. A sample forwarding table is shown in the figure 2.3. Permutation table: The bits of the Bloom filter contained in the packet are permuted (before forwarding towards the source) during subscription to multicast group and reverse permuted (after receiving from the upstream) during publication of data. The permutation and reverse permutation of the bits is defined by a permutation table. This permutation table defines which bit is permuted to which place. It works exactly like cryptographic P-boxes. Figure shows the process of permutation and reverse permutation of a Bloom filter. Each node creates a permutation table during its bootstrapping process. The permutation table entries are created at random and different nodes have different permutation tables. We start our protocol description with the scenario where a source node is ready to listen for the subscription requests for a certain multicast group. The receiver node has information about its own IP address and node ID, the IP address and node ID of the source node and the group ID of the multicast group 2. The receiver node sends a subscription request to the source node for the specific multicast group. The source node receives the subscription requests from the receiver, adds the receiver node into the multicast group and sends multicast data to all the subscribed receivers for that multicast 1 Although, permutation and reverse permutation tables are shown separately in the figure, the process of permutation and reverse can be implemented with one table only. 2 The management and distribution of group IDs as well as IP addresses is not considered in current study. The problem of getting the group ID for the multicast group and finding the IP address of source is left for further research

18 CHAPTER 2. BLOOM FILTER-BASED MULTICAST 18 Figure 2.4: Permutation and reverse permutation of 8-btis Bloom filter group. When a receiver wants to leave a multicast group, it sends a message to the source node which removes the receiver from the group. Now we will discuss how the nodes join and leave the network and how the data is published on the multicast group. 2.3 Joining a multicast group A receiver joins a multicast group by sending a JOIN packet to the source node. This JOIN packet contains the Group ID of the group for which the receiver wants to receive data packets, the ID of the receiver node, and an empty Bloom filter. For illustration, we denote the Group ID as GID, receiver node ID as RID and Bloom filter as RBF. The receiver adds the GID, RID and an empty RBF of a predefined size in the packet and the packet is then sent through IP routing over an upstream link towards the destination (i.e., the source node). 3 The receiver also saves the GID into a list to keep track of the multicast groups it has joined. When the JOIN packet is received by the next hop, it adds the link ID of the incoming link into the RBF contained in the packet. The addition of the link ID into the received Bloom filter (RBF) is a simple process of bit-wise ORing the RBF with the link ID of the incoming interface. After updating the RBF with an added link ID of incoming interface, the RBF bits are permuted according the permutation table (depicted in figure 2.4(a)) and the packet is again forwarded to the next hop upstream towards the source node. No information is saved on the forwarding node. The next hop again adds the incoming link ID into the RBF contained in the packet, permutes 3 As mentioned earlier, during the joining to and leaving from multicast group, next hop is deteremined using the IP routing and the IP address of the source node.

19 CHAPTER 2. BLOOM FILTER-BASED MULTICAST 19 Figure 2.5: Process of joining multicast group the RBF bits and forwards the packet. In this way, the Bloom filter in the packet collects the link IDs of the traversed links. Thus, an RBF represents the reverse path between the source and the receiver, and represents a branch of the multicast delivery tree maintained at the source node. The process of adding link IDs and forwarding the packet continues till the packet is received by the source node. The source node also adds the link ID of the incoming interface into the RBF, permutes the RBF bits and saves the RID (receiver ID) and RBF into the group membership table of GID (group ID). A group membership table is identified by Group ID and consists of a list of pairs, receiver ID, receiver Bloom Filter. This group membership table is used to maintain the membership of the receiver nodes. Figure 2.5 illustrates the group joining process without permutations. Let s consider that A is the source node and C and D are the receiver nodes. First C sends a JOIN packet to A to join the multicast group GRPA. When B receives this packet, it adds the link ID B1 into the Bloom filter and forwards the packet to A. Node A receives the packet, adds the incoming link A1 into the Bloom filter and saves the receiver ID (RID) and receiver Bloom filter (RBF) into the group membership table shown on the left in figure 2.5. Similarly, C joins the network and this RBF value is the union of B2 and A1 when the packet reaches at A.

20 CHAPTER 2. BLOOM FILTER-BASED MULTICAST Leaving a multicast group Leaving the multicast group is a very simple process. When a receiver wants to leave a multicast group, it unicasts a LEAVE message to the source node. The LEAVE message contains the receiver ID (RID) and the group ID (GID). There is no need for bit-permutations during this process. The packet is routed through the IP routing to the source. When the source receives the packet, it removes or disables the corresponding entry of the receiver in the group membership table. It calculates the group Bloom filter again by combining the RBFs of the remaining receivers. 2.5 Distributing data to a multicast group As described in the previous section, the source node maintains a group membership table which contains the Bloom filters (RBFs) received from the receiver nodes. The source node combines these RBFs using bitwise ORing to create a group Bloom filter (GBF). Thus, this GBF contains the links IDs of all the links on which data has to be sent to reach all the group members Essentially, it can be said that the GBF contains the multicast delivery tree. GBF is added into the data packet along with the data to be distributed and the group ID (GID) for which the data is destined. When a node receives a data packet, it performs the reverse permutation and then checks whether any one of its interfaces exists in the GBF. It performs bitwise AND operations between GBF and each of its interface s link IDs one by one. If the result of a bitwise AND is equal to the corresponding link ID, the packet is forwarded on that interface. Obviously, the packet is never sent back on the link on which the node has received the data packet. Since any node can be a receiver, each node also checks if it is a receiver of this data packet or not. It does this by searching for the GID in the list of joined multicast group IDs 4. Returning to our pictorial illustration, figure 2.6 shows the process of multicasting without permutations. Source node A creates a data packet which contains a group Bloom filter, GBF, and data to be sent to the receivers. It checks interface A1 against the GBF, finds it in the GBF, and forwards the packet on A1. Similarly, node B forwards the packet on links B1 and B2. When C and D receive the packet, these nodes find the GID in the list of joined groups and extract the data for processing. The packet is 4 Keeping in view the line-speed processing requirements, this checking can be disabled on the routers which never join any multicast group.

21 CHAPTER 2. BLOOM FILTER-BASED MULTICAST 21 Figure 2.6: Process of data distribution on multicast group not forwarded further as no link ID except the one on the incoming interface matches in the GBF. It is to be noted that a data packet can also be forwarded on a wrong link due to false positives caused by the Bloom filter data structure. In the next chapter we will learn how to keep the false positive rate low to make the protocol practically usable.

22 Chapter 3 Controlling the network overhead In the previous chapter, we learned about the multicast protocol in which the multicast data packets are routed depending on the link IDs in the data structure called the Group Bloom filter (GBF). This Bloom filter has the multicast tree encoded in it as a set of links. Consider the example of a multicast delivery tree shown in figure 3.1. Node A represents the source node whereas the nodes E, F, G and H are the receiver nodes. The GBF of this multicast tree contains 7 links as shown in the figure. In this chapter, we will explore the network overhead and the scalability limitations of the Bloom filter based multicast. In addition, we will discuss some techniques for reducing the network overhead during multicast of data. We start our discussion from the explanation of the network overhead caused by the number of packets 1 sent during data delivery. In the example shown in the figure 3.1, the total number of links in the multicast delivery tree is 7. Therefore, in ideal case, there should not be more than 7 packets sent between the nodes when the source node sends one data packet over the network. In order to explain precisely, if A multicasts one data packet, ideally, there should be one data packet from A to B, then a copy of the packet from B to C and further to F, another copy from B to G and two packets from B to E to H. This makes 7 packets in total. Thus, in ideal case, there should as many packets (tpkts) as the number of links (tlinks) added into the Bloom filter (without redundancy) when one data packet is multicast from the source to the receivers. In reality there are more packets than total links due to the false positives caused by Bloom filters. If there are more packets than tlinks sent over the network during the data distribution, then there are some extra packets and these extra packets make the network overhead. We define this network overhead (ovh) as: 1 Overhead caused due to the size of protocol headers in the packet is discussed in the chapter 6. 22

23 CHAPTER 3. CONTROLLING THE NETWORK OVERHEAD 23 Figure 3.1: An example of multicast delivery tree for one group. A is the source node and E, F, G and H are the receivers. Link E2 shows a false positive. ovh = tpkts tlinks tlinks (3.1) In our example, let us say node I also receives the packet due to a false positive on node E, then the network overhead will be 14%. So, there can be large network overhead if the probability of false positives is high in the multicast delivery tree. The effect of false positives is discussed comprehensively by Särelä et al. [28]. In their study, the false positive rate (fpr) is defined as: fpr = ρ k (3.2) where k is the number of bits per item set in the Bloom filter and ρ is the fill factor (the ratio of all set bits to the total number of bits in the Bloom filter). Since the false positive rate is strongly linked to the network overhead, we want to keep it as low as possible. This can be achieved by keeping the fill factor below 50% because the probability of false positives increases if the fill factor exceeds 50% [17]. This limits the effective number of links (n) which can be added into a Bloom filter of a given size if we want to optimize the multicast for minimum network overhead. Given the number of hash functions k and the size of Bloom filter m, the number of links n that can be added into the Bloom filter with ρ 50% is [28]:

24 CHAPTER 3. CONTROLLING THE NETWORK OVERHEAD 24 n = (m/k) ln 2 (3.3) Thus, the scalability of our multicast network depends on three competing variables; the size of the Bloom filter m, the number of hash functions k, and the number of links n. The values of m, n and k define the fill factor ρ of the Bloom filter. For instance, if we assume m = 256 bits and k = 8 bits, then we can add approximately n = 22 links into the Bloom filter while keeping ρ 50%. This limit can be reached with a few receivers only. If we assume 5 links per receiver, we can add only 4 receivers into a 256-bits Bloom filter. Therefore, in order to increase the scalability or to make room for more links, either we have to increase the size of Bloom filter m or decrease the number of hash functions k. If we decrease k, it increases the probability of false positives [17] and we already know that false positives may create unexpectedly large network overhead [28]. Another way around is to increase the size of the Bloom filter m at the cost of increased packet size. For instance, if the fill factor of a packet reaches 50%, the size of the Bloom filter can be increased to decrease the fill factor and, as a result, more links can be added. Another way to keep the fill factor below 50% (which keeps the false positive probability low), is to split the multicast delivery tree in two or more trees at the cost of increased number of packets sent over the network. For instance, if a Bloom filter has the capacity of 100 links and we have 200 links to add into the Bloom filter, instead of adding 200 links into one Bloom filter which increases the probability of false positives, we can add 100 links into one Bloom filter and 100 into another one. In this way, there will be very less network overhead due to false positives. On the other hand, there will be some added network overhead due to the same data packet sent multiple times over the same links (targeted for different receivers sets). Referring again to the same figure 3.1, let s say the Bloom filter has the capacity of 4 links to keep the fill factor below 50% and we divide the multicast delivery tree into two trees. We merge the Receiver Bloom filters (RBFs) received from nodes F and G into one Bloom filter and the RBFs from nodes E and H into another Bloom filter. Thus, we have two multicast delivery trees (splits of single tree) or Group Bloom filters (GBFs) for one multicast group and the fill factor for both the GBFs remain ρ 50%. This will keep the false positives probability low but on the other hand there will additional packets due to common links in both trees. In our particular example, there are 7 links in the ideal case but now we have 9 packets for data delivery (without false positives) and the resultant network overhead is

25 CHAPTER 3. CONTROLLING THE NETWORK OVERHEAD 25 28%. In chapter 5, we will see that the network overhead due to multiple delivery trees can be significantly lower than the network overhead due to the false positives in the single multicast delivery tree. Notice that this network overhead depends on the number of common links in the split multicast tree. Thus, the overhead can be controlled by devising effective tree splitting methods. Those receivers should be merged into the same tree which share the most common links and which do not exceed the fill factor limit of the merged Bloom filter. In subsequent sections of this chapter, we will discuss the novel tree splitting methods. 3.1 Random tree splitting As discussed above, to increase the scalability of the network while keeping the false positive rate low, we split the multicast tree into two or more trees and keep the fill factor of Group Bloom filter below a certain threshold (typically 50%). The splitting of the multicast tree essentially means combining the receivers into different sub-groups, each sub-group making one multicast tree. One way to combine the receivers is to do it randomly. For instance, we pick nodes on first-come-first-served basis. Let us say, if a Bloom filter has capacity of 10 receivers for a given fill factor limit, the first 10 receivers are combined into one Bloom filter and a new Bloom filter is created for new joining receivers. One Bloom filter represents one multicast tree. In our example shown in figure 3.2, we use artificially short m = 32 bit Bloom filter and k = 4 hash functions, which implies that the filter has the capacity of n = 5 links to keep the fill factor ρ 50%. The source A combines the nodes E and F, which, let us say, fill the Bloom filter up to 50%. Now, no more links can be added into the Bloom filter. So, A creates another Bloom filter and combines the nodes G and H into it. Now, there are two delivery trees, (A1,B1,C1,B3) and (A1,B2,D1,B3,E1), as shown in the figure 3.2(a). Links A1 and B3 are common to both trees. Now, A has to send the same data packet twice, once for each tree. This increases the network overhead. The original single delivery tree consisted of 7 links but now there are 9 links, 4 for (A1,B1,C1,B3) and 5 for (A1,B2,D1,B3,E1). This increases the network overhead (without false positives) by 28%. However, if we combine the nodes more carefully such that the different trees have fewer common links, the network overhead can be reduced. For instance, if we combine the links from the nodes F and G into one Bloom filter and the links from the nodes E and H into the other (shown in figure 3.2(b)), the total number of links will be 8, 5 for (A1,B1,C1,B2,D1) and 3

26 CHAPTER 3. CONTROLLING THE NETWORK OVERHEAD 26 (a) An example of random tree splitting. Single tree is split into two trees, (A1,B1,C1,B3) and (A1,B2,D1,B3,E1) by combining (E,F) and (G,H) for respective tree. (b) Better than random tree splitting. Now only A1 is common between the two trees, (A1,B1,C1,B2,D1) and (A1,B3,E1). Figure 3.2: Random tree splitting (a) and an improved version of the random splitting (b). Two trees are shown in each figure. The same packet is sent twice for each (shown by different arrow lines and colors). for (A1,B3,E1). In this way, the only common link is (A1) and the network overhead is 14%. This implies that we can improve the tree splitting compared to the random algorithm. This, we are going to discuss in the next section. 3.2 Hierarchical divisive tree splitting Continuing from the above discussion, random tree splitting may create trees which have unnecessarily many common links. This will result in redundant path in different trees and in increased network overhead. We can avoid redundancy of links in different trees if Bloom filters from those receivers are merged into the same tree which share the most common links. This is the general idea behind our Hierarchical divisive tree splitting algorithm, in which the trees are created on the basis of a common path between the nodes. However, the fill factor for each Group Bloom filter is kept below a given threshold. For this purpose, the source node needs to know the exact shape of the tree. First, we will discuss how this tree is created at the source node and then we will discuss how to split the tree.

27 CHAPTER 3. CONTROLLING THE NETWORK OVERHEAD Creating the Bloom tree In hierarchical divisive tree splitting, the source needs to know the reverse path of each receiver, i.e., the list of hops from the source to the receivers. The exact reverse path cannot be extracted from the received Bloom filter. However, a JOIN packet can collect the node IDs of each hop during the joining process. When a node receives a JOIN packet, it appends its node ID to the packet header before forwarding it to the next node. When the packet reaches the source, the source has a list of the hops traversed by the packet and the link IDs in the Bloom filter. From this information, the source can make an in-memory tree rooted at the source node, illustrated in figure 3.3. We call this tree the Bloom tree. Each node in the Bloom tree has a Bloom filter which is the merger of the Bloom filters of the child nodes. 2 The Bloom filters for the leaf nodes are the ones which are received in the JOIN packets. Note that an intermediate node can also be a receiver but, in this tree, its Bloom filter is still the union of the Bloom filters of its child nodes. In figure 3.3(d), the Bloom filter of node E, (A1 B3 E1 E2), is the merger of the Receiver Bloom filters (RBFs) of nodes H and I; the merger of (A1 B3 E1) and (A1 B3 E2). The process of tree creation is shown in the figure 3.3 on the next page. Before we continue with the discussion of splitting the Bloom tree, its necessary to explain a caveat in the process of node ID collection in the JOIN packet. The collection of hop information may increase the packet size. But how much can the packet size grow? As the average number of hops in the Internet is estimated to be around 22 nodes by 2011 [8], the list of hops may increase to 88 bytes (if we use 4-byte node-id) in addition to 1 byte for the number of hops in the packet (the JOIN packet does not contain any data bytes). If we assume 500 bytes for packet headers and the maximum transmission unit (MTU) of the network to be of 1500 bytes (RFC 894), a packet can store 250 hops, which is fair enough to reach from the source to the receiver. 3 Therefore, this way of collecting the node IDs in the packet is practically feasible. 2 Since, we have assumed that the JOIN packet always follows the shortest path towards the source, there cannot be any loops in the graph. 3 From the security perspective, this collection of node IDs can have problems. Due to some reason, intentional or non-intentional e.g., loops in the network, the packet can traverse hundreds of hops and when the packet reaches the MTU of the network, the network device may divide the packet in fragments. And if the packet still keep traversing the nodes, there are possibilities of Denial of Service attacks.

28 CHAPTER 3. CONTROLLING THE NETWORK OVERHEAD 28 (a) State of Bloom tree when the first node F joins the multicast group. (b) Node E joins the group. Its Bloom filter is merged with those of all of its ancestor nodes (A and B). (c) Nodes H and I also join the group and their Bloom filters are merged with those of their ancestor nodes. (d) Node G joins the group. Note the updated Bloom filters of nodes A and B. Figure 3.3: Illustration of the creation of the in-memory Bloom tree. There are five receivers E,F,G,H and I, and A is the source node Splitting the Bloom tree During the creation of the Bloom tree, there can be the nodes whose fill factor exceeds the fill factor limit (e.g., node A in figure 3.4). Therefore, those nodes need to be split. We start to check for splits from the root node. If the fill factor of the Bloom filter of the root exceeds a defined threshold, let s say 50%, the node needs to be split. To split the node, a merge and test operation is done among the children of that node. In the merge and test operation, two siblings are merged and then tested if the merger exceeds the threshold or not. If it does not exceed the threshold, the nodes are kept merged. Otherwise, the nodes are not merged.