A Visualized Parallel Network Simulator for Modeling Large-scale Distributed Applications

Transcription

1 Eighth International Conference on Parallel and Distributed Computing, Applications and Technologies A Visualized Parallel Network Simulator for Modeling Large-scale Distributed Applications Siming Lin 1,2, Xueqi Cheng 1 and Jianming Lv 1,2 1. Institute of Computing Technology, Chinese Academy of Sciences, Beijing 2. Graduate School of the Chinese Academy of Sciences, Beijing {linsiming, lujianming}@software.ict.ac.cn, cxq@ict.ac.cn Abstract Large-scale distributed systems, with thousands or even millions of nodes, produce complex and dynamic behaviors. Packet-level simulation is necessary to test and analyze these systems, such as grids, peer-to-peer (P2P) applications as well as worm and DDoS containment systems. However, the current network simulators are not convenient for application layer simulation. We present the NSME, a visualized parallel simulator that allows researchers to simulate their applications in a large virtual network with most details transparent to them. A hierarchical routing is used to enhance the fidelity of simulation and reduce the memory cost of network construction. A variation of CMB algorithm is implemented for parallel synchronization. We show the functions of NSME by three applications: 1) a self-similar background traffic model, 2) a Slammer worm spreading model, and 3) a P2P live streaming system, which demonstrate its effectiveness for simulating any large-scale distributed applications. The performance tests show that the memory cost of NSME is distinctly lower than NS2 and the parallel efficiency can reach about 60% in any of the above applications. 1. Introduction Grids and peer-to-peer (P2P) applications have become successful paradigms of distributed computing. On the other hand, as a special type of distributed systems, worm and DDoS attacks have become serious threats to the Internet. These systems are usually very large, and their behaviors are very complex and dynamic, which makes it unpractical to directly analyze or test their efficacy on the Internet. Packet-level network simulation is an available and low-cost approach to reach this goal. Through objectoriented programming, each host, router, link and packet are simulated independently, which makes them have their own attributes and interactive behaviors. By constructing realistic topologies and implementing necessary protocols, approximately real behaviors and data can be produced in a simulation system. At present, some well known network simulators, such as NS2 [1], OPNET [2] and GTNetS [3], have been proposed. However, these simulators are originally designed for communication protocols testing, such as MPLS, IP routing and TCP/IP. They are not convenient for application layer simulation, since they mostly focus on the details of lower layers of the network so that the functions of application layer are greatly simplified. In addition, these simulators are difficult to use for the researchers working with distributed applications. To configurate a simulation task, they have to know well the details of lower layers, simulation parameters, and even parallel computing, which is excessive to such researchers. In this paper, we propose a visualized parallel simulator NSME for dedicated simulation of largescale distributed applications. In our previous work [4], a sequential version was developed as Network Security Modeling Environment based on NS2. We keep its name for unification, but entirely redesigned its architecture. Now, it consists of two parts. One is a GUI management tool, XNSME, used to construct network topologies, deploy virtual applications, configurate simulation tasks, and review data traces. The other is a parallel simulation development kit, LibNSME, which is a C++ static library provided to researchers for virtual network programming. The LibNSME is also a core to construct the parallel simulation engine, which is totally transparent to researchers. The main features of NSME include: 1) application layer oriented simulation, 2) Internet hierarchical routing, 3) large-scale network topologies, 4) transparent parallel characteristics, and 5) visualized simulation management. We will illustrate its effectiveness by simulating three distributed applications. The performance evaluation shows the /07 $ IEEE DOI /

2 memory cost of the NSME is evidently lower than NS2, and the parallel efficiency of the above simulation instances can reach about 60%. The remainder of this paper is organized as follows. Section 2 provides related work. Section 3 describes the architecture of NSME. Section 4 describes the implementation in detail. Section 5 demonstrates three NSME simulation instances. Section 6 evaluates the performance. Section 7 gives some conclusions and future directions. 2. Related work There are many different simulators developed for network research. Most of them are based on the principle of discrete-event simulation. One of the well known commercial simulators is the OPNET. It has an abundant network component library and a visualized environment. However it is expensive. One of the widely used simulators is NS2 developed at UC Berkeley. It is a typical packet-level network simulator implemented in C++. It meanwhile is designed as an Otcl language parser. The new modules, so-called agents, can be described using both C++ and Otcl. Although NS2 is widely adopted, there are some limitations on it. First, it does not have the functions related with real IP addresses. Second, its agent mechanism makes it inconvenient to configure a simulation task, such as the TCP connection behaviors. The above simulators can only simulate the networks composed by thousands of nodes. In addition, since they are both sequential programs, the performance of simulation is deeply limited. Parallel and distributed network simulation has been studied for many years. Fujimoto, Riley and their team at Georgia Tech are contributive to the research of parallel discrete-event simulation [3, 5, 6]. They developed an extensive parallel library, libsynk. Based on it, they built a parallel simulator GTNetS. Furthermore, by adding this library, they extended a parallel version of NS2, called PDNS. However, to the best of our knowledge, the automatic configuration for task mapping is still deficient in it. The GloMoSim [7] is another well known parallel simulator developed by Lokesh Bajaj, Mineo Takai and their team at UCLA. Different from the above simulators, it was designed for simulation of large hybrid networks including wireless, wired, and satellite based communications. Unfortunately, there are very few tools convenient for the simulation of distributed applications. The grid researchers realized the situation and designed some grid simulators. The SimGrid [8], developed at UCSD, Task management / mapping is a tool for the simulation of application and resource scheduling. The MicroGrid [9] is an emulator for modeling virtual grid environment, on which the grid applications using Globus middleware can be executed. However, these toolkits are not universal for any other distributed applications. Also, their capability of simulating large networks is limited. 3. Architecture 3.1. Overview Graphic User Interface Log files Dump files Log files Dump files Log files Dump files Parallel synchronizations Review / data collection SimEngine SimEngine SimEngine Figure 1. NSME Architecture The NSME is designed for universal packet-level simulation of large-scale distributed applications. Its two-layer architecture is shown in Figure 1. The function of its upper layer is global management of simulation tasks, and the function of lower layer is the execution of actual simulation. More precisely, the global management includes constructing virtual network topologies, deploying virtual applications, partitioning/mapping subtasks for parallel execution, configurating data traces, and reviewing the data produced during simulation. We have developed a graphic user interface, XNSME, to integrate these management functions. In the lower layer, the actual simulation is undertaken by parallel engines, which are synchronized by certain conservative algorithms [6]. The NSME provide such parallel engines by a Visual C++ static library, LibNSME, whose details of implementation are transparent to the users. In their view, the LibNSME is only a programming interface to develop virtual applications. In addition, the NSME has extensive capability of data tracing. It can trace not only the normal statistics, such as link bandwidth, lost rate, and host state, but also the detailed packet loads in tcpdump format [4]. 340

3 (a) Inter-AS level (b) Inter-Area level (c) Intra-Area level Figure 2. The snapshots of the NSME network topology 3.2. Application layer oriented virtualization In the simulation space modeled by the NSME, there are two levels of virtualization, the virtual network infrastructure and the virtual applications. To researchers on distributed applications, the network infrastructure is usually expected to be a black-box. We have encapsulated most of the basic behaviors of lower layers of the network. We model hierarchical routing mechanism for such virtual network, which makes it homologous with the real Internet. A virtual network is also composed by Autonomous Systems (AS), and in each AS, multi-areas OSPF routing is employed at present, which means each AS consists of many OSPF areas. Consequently, there are three types of virtual routers, AS Border Routers (ASBR), Area Border Routers (ABR) and Internal Routers (IR). We also model the behaviors of virtual LANs and hosts. A virtual LAN is an abstract object that denotes a subnet segment. It consists of virtual hosts that share a virtual broadcast link. A virtual LAN can be connected to an IR by appointing its default gateway. Figure 2 shows the snapshots of a virtual network instance. Although the topology of a virtual network is complex, its configuration is quite simple. With the visualized tool XNSME, a virtual network can be constructed rapidly. The only parameters that need to be set are the bandwidth and delay of a link or a LAN. The IP addresses of each router interface and host will be automatically assigned. A virtual host can be treated as a virtual computer with a simple operating system installed. Each virtual application can be defined as a C++ class derived from the base class CApplication which provides some basic member functions, such as entry function (main), timer and pseudorandom number generator. We implement a dynamic TCP/IP protocol stack for each virtual host. Based on this, the LibNSME provides a set of socket base classes for virtual network programming, such as CRawSocket, CUDPSocket and CTCPSocket. They are very similar with the base class CAsyncSocket in Visual C++ MFC. Based on these socket classes, the requirements for developing virtual network programs with TCP, UDP or even raw IP communication can be satisfied. The virtualization of network applications makes the programming style in the NSME the same as that in the real world except the socket APIs, which means when the researchers are developing a prototype system they do not have to care where their system will be deployed. They are kept away from the excessive details of simulation Visualized Management Although the OPNET is a visualized simulator, it can only simulate a small or medium network. There are very few simulators providing an effective GUI. It is because a large topology with more than thousands of nodes is difficult to display on screen. Moreover, a simulation task is traditionally configured by certain script languages, which is a flexible but inefficient method. We solve the above two problems by using hierarchical display and object-oriented component library. Hierarchical display: Corresponding to the organization of virtual network, the display is defined in three levels, inter-as, inter-area and intra-area. (See Figure 2) Object-oriented component library: We define all visualized components, such as routers, links, LANs, OSPF areas and ASs, in object-oriented way. Basically, there are two types of component classes, statistics classes and sample classes. A statistics class allows including statistical behaviors and attributes, such as the class name, bandwidth, delay, or number and degree distribution of its member classes. A sample class can be automatically generated by the former in terms of its statistical information. By 341

4 can be assigned with IP addresses that have the same prefixion. In each virtual host, a mini TCP/IP protocol stack is designed elaborately, which allows a virtual program to access the underlying network transparently and dynamically. Its details are shown in Figure Task Mapping Figure 3. The graphic user interface combination and reusing of sample classes, a large virtual network can be constructed rapidly. Figure 3 shows the GUI of the XNSME. There are some other functions provided by it. Deployment of virtual applications: With a given probability, each virtual host can be assigned a virtual task ID, which is corresponding to a set of virtual applications implemented with the LibNSME. Task mapping: The XNSME adopts the METIS tool [10] to partition a given virtual network. It then generates the corresponding profiles to each computer that will execute this simulation. Review of data traces: The data traces can be reviewed in graphs when a simulation finishes. The tcpdump files generated by simulation can be further analyzed by other professional tools, such as Ethereal. 4. Implementation 4.1. Routing and TCP/IP protocol stack We implemented a hierarchical routing mechanism to forward virtual packets. In each normal OSPF area, there are many IRs, several ABRs or even ASBRs. A shortest path first (SPF) algorithm is used for computing the routes between them. All ABRs in an AS make up of a bone area (Area 0) for forwarding packets between different normal areas. Similarly, all ASs in the whole virtual network are connected by the ASBRs. A SPF based configurable BGP routing has been implemented. In each virtual router, a radix tree route table, the popular structure in real routers, has been implemented, which makes it capable of assigning real IP addresses in our virtual network. For reducing the size of route table in each ASBR or ABR, our implementation supports route summary in BGP and OSPF. Each router and host in the same AS or Area For improving the parallel performance, load balance have to be taken into account. More precisely, a simulation task should be mapped into many subtasks. Since the load of simulation is correlative with the traffic of the simulated network, a popular approach of task mapping is to partition such network topology into N subnets and to simulate them with different computers (or logical progresses, LPs) in parallel. With an assumption that the load on each virtual node is approximately equal, a partitioning algorithm should be capable of dividing a graph into N almost equal-size subgraphs such that the sum of the weight of the edges connecting nodes in different subgraphs is minimized. Considering the bandwidth of a virtual link can reflect the maximal amount of the events that need to be handled in one second of simulation time, and the delay of a virtual link can affect the value of lookahead of the synchronization algorithm, we define the weight of a virtual link as follows: α bandwidth weight =, where α and β are the β delay adjustment factors. Based on the definition of weight, we partition a simulation task with a graph partitioning tool, the virtual application layer virtual socket layer create TCP socket TCP listen tcp socket UDP socket raw socket socket tcp socket out_point udp_demuxer dport_bbtree icmp_handler ip_local_deliver tcp_demuxer sip_bbtree 7240 sport_bbtree dport_bbtree rawip_filter in_point Figure 4. NSME protocol stack 342

5 METIS [10]. The number of subtasks is decided by the list of real IP addresses of the computers undertaking this simulation. The remote virtual links between different computers can be configured automatically Parallel scheduling and synchronization To guarantee logical validity, all events should be scheduled according to their temporal order. For a sequential simulation, a non-descending local event list (LEL) is competent. However, for a parallel one, the synchronization of virtual time among the LEL and other external event lists (EEL) has to be considered. There are two types of synchronization algorithms, conservative and optimistic. The former guarantees the correct temporal order of events by asynchronous or synchronous negotiation, such as CMB algorithm [6]. The latter aggressively advances simulation time until receiving events with incorrect timestamps, and then invokes a serial of rollback operations. Considering the cost of rollback operations in optimistic method, we implemented a variation of CMB algorithm which uses null messages to avoid dead lock. The CMB is very simple, however, the number of null messages exchanging among different LPs is excessive, which greatly reduces the effective utilization rate of I/O and CPU. To reduce the number of null message, in our algorithm, any LP does not send a null message to one of its neighbors unless it is blocked or its local virtual clock is greater than the expected time of the neighbor. Each expected time can be updated independently when receiving any null or event messages from the corresponding neighbor. Each LP computes expected time T e for each its neighbor according to the following equation: () i Te = min{ f ( LEL),min{ f ( EELi)}} lookaheadi where function f gets the minimal timestamp of an event list. If the event list is empty, f returns the halt time. The value of lookahead i is equal the delay of the virtual link from the neighbor i. 5. Modeling distributed applications 5.1. Modeling the network topology Before simulating these different applications, we construct a virtual Internet. All virtual applications presented in this section will run on it. With the visualized tool XNSME, we construct a campus network, a benchmark topology mentioned in [11], as a normal OSPF area with one ABR assigned in it. We extend the number of hosts in each LAN in such an area, and then construct an AS consisting of 12 such areas. Finally, a virtual Internet is generated with 10 such ASs. There are total 3600 routers and active hosts. For convenient description, the bandwidth and delay of all inter-as links are set as 10Gbps and 1ms respectively, those of inter-area and intra-area links are 1Gbps and 1ms. The bandwidth and delay of each LAN are 100Mbps and 0.5ms. It is important to note that the NSME is capable of constructing any scale of virtual networks with the combination of three routing granularities. The memory cost to maintain a virtual network will be discussed in section Modeling the self-similar traffic For convictively simulate distributed applications, it is necessary to accurately simulate the background traffic. There are many researches on the modeling of network traffic. It is well known that the network traffic is self-similar and a large number of ON/OFF traffic generators with heavy-tailed distribution can synthesize such traffic [12]. The heavy-tailed distribution can be defined as P[ X > x] x α, as x, 0< α < 2. The Pareto distribution is a typical heavy-tailed distribution. Its cumulative distribution function is F( x) = P[ X x] = 1 ( k/ x) α To each generator, there are iterative two periods, ON or OFF. It sends packets at fixed bit rate in the ON period, and sleeps in the OFF period. The ON and OFF periods have independent Pareto distributions. The degree of self-similarity they can generate is described by Hurst parameter H. H = (3 min{ α1, α2}) / 2, where α 1 and α 2 are heavy tail parameters of the ON and OFF periods. We programmed a virtual application to undertake the function of an ON/OFF generator. It uses a raw socket to send packets in the ON periods, with the destination IP addresses and ports randomly assigned. We deploy the application on each virtual host. The simulated traffic on a certain Inter-Area link is shown in Figure 5 (a). It is very similar with the real traffic, captured from CAIDA [13], shown in Figure 5 (b) Modeling the spread of the Slammer worm The simulation of worm behavior is helpful for developing corresponding countermeasures. We used NSME to simulate the Slammer worm that has a very fast propagating speed with UDP transmission and not concerning whether the targets exist or not. Our virtual application acts as both a worm and a vulnerable host. It selects a target randomly and sends it a single attack packet at a scan rate R s. If the target is vulnerable and has not been infected, it will be infected after receiving 343

6 Traffic (Mbps) generated traffic Time (second) Time (second) (a) Traffic generated by NSME (b) Traffic from CAIDA Figure 5. The comparison of self-similarity traffic the attack packet. After a lurk period T l, it will break out. In addition, we add some active behaviors to each host. At each virtual second, a host has a probability P u to upgrade its safe measures, which will makes it immune to the worm. It also has another probability P r to reinstall its operating system, which will make it temporarily cured from the infection. However, its vulnerability still exists. We then deploy the traffic applications and the worm applications in the network. The infected rate of the network is shown in Figure 6. It is obvious that if active measures, such as firewalls and safe upgrade, are actively taken, the effect of worms will be reduced Modeling a P2P live streaming system The P2P live streaming applications are popular in the Internet. It is necessary to analyze the performance of such systems. We designed a simple P2P live streaming system based on the principle of the P2P file sharing systems (with TCP protocol). There are three roles, index servers, media sources and P2P clients. A media source is a special client that provides the raw Infected rate R s =30, T l =0.1, P u =0.02 P r =0.4 R s =30, T l =0.1, P u =0 P r = Time (second) Figure 6. The effect of active behaviors Traffic (Mbps) 350 real traffic video data. When starting, it will play a virtual TV program. The video stream it provides can be treated as a file consisting of a sequence of video packets labeled with serial IDs. The media source always caches the latest N packets that have been played, and notifies the index server about their IDs. There is a local ID indicator in each client. When a client logs on with a program name, the index server randomly provides it a group of neighbors caching the video packets of the same program with serial IDs. The indicator then is initialized by the minimal ID. The number of neighbors depends on the size of video cache in the requesting client. The client will try to connect to and download such packets from such neighbors. Once a video packet is completely downloaded, the client will notify the index server about its share. A client always tries to play the received packets according to the indicator. If a part of the packet with the indicated ID is received, the client plays it. Otherwise, it waits. When a packet is played completely, the client updates its indicator by plus 1 and drops the packet. Such change will be notified to the index server. Meanwhile, it tries to find a new neighbor by querying either its other neighbors or the index server. We design two neighbor selection policies, global and local, and compare them by analyzing the traffic (see Figure 7). When a local policy is adopt, the traffic on bone links is reduced obviously. We judge a local neighbor by its IP address. Any approaches can be tested in this model rapidly, which is useful to develop a practical P2P system. We further analyze the average wait rate when a user (client) watches a TV program through this P2P system. The relations among wait rate, the number of online clients and the cache size are shown in Figure 8. We find that the cache size of a client is more important to improve the play quality while a large number of online clients improve that. 344

7 P2P traffic (Mbps) 600 global neighbor policy 500 local neighbor policy Time (second) Figure 7. The traffic on an Inter-Area link Wait rate k clients, 2-packet cache 2k client, 6-pakcet cache 10k client, 2-packet cache 10k client, 6-packet cache Play time (second) Figure 8. The average wait rate 6. Evaluation 6.1. Size and memory cost of virtual network Considering the memory cost of virtual programs running on the virtual hosts is linearly dependent on the number of virtual hosts. Here, we only test the cost to construct a virtual network and its routing tables. We also compare the result with the well-known NS2 simulator. Since the NS2 uses a flat shortest path first (SPF) routing, the size of a network can be simply denoted by the number of routers. However, our simulator uses hierarchical routing, and the size of a network is complex to be described. We use a triple < As, Area, Router > to denote the size of a network, where x denotes the total number of x in the network. For convenient comparison, we use the same OSPF area sample class, consisting of 50 routers, to construct an AS. To obtain a network consisting of 100 routers, we construct a single AS network consisting of two such areas. Furthermore, we use the same ASs consisting of 20 such areas to construct networks consisting of 1000, and routers respectively. The memory costs are shown in Figure 9. It is obvious the hierarchical routing makes our simulator capable of constructing larger-scale virtual networks even in a single computer Parallel performance We use a metric speedup to evaluate the parallel performance. We find that it is different when the simulation instances change. Figure 10 shows the results corresponding to the applications mentioned in Section 5. The worm simulation obtains a high speedup, about 1.8, when running on two processors, and the corresponding parallel efficiency is 90%. The other two cases obtain a speedup of about 1.2. The corresponding parallel efficiency is 60%. The case of worm simulation is reasonable. It is because there is an excessive amount of events in worm simulation, which causes the performance descent of the sequential LEL operations while in parallel, the length of LEL is reduced by half, which relatively speeds up the LEL operations. The speedup of P2P simulation is slightly Memory (MB) NSME NS2 Speedup 8 Ideal Background traffic Worm+background 6 P2P+background The number of routers Figure 9. The comparison of basic memory cost Processors Figure 10. Speedup 345

8 higher than the background traffic simulation. We believe that the local neighbor policy used in P2P simulation is helpful for improving the parallel performance. In addition, the speedup slightly descends with the increasing of processors. There are many factors causing it, such as the quality of task mapping and the I/O performance. 7. Conclusion and future work In this paper, we propose a visualized parallel network simulator NSME. Different from the other traditional network simulators, the NSME simulator is application layer oriented. It provides APIs for virtual network programming and a GUI tool for simulation management. All details of simulation are transparent to researchers. We use a hierarchical routing to enhance the fidelity of simulation and meanwhile to reduce the basic memory cost to construct virtual networks. We also implement a variation of CMB algorithm for parallel synchronization. We then show the NSME by applying three distributed applications, a self-similar traffic model, a Slammer worm spreading model, and a simple P2P live streaming system. These simulation instances show that the NSME is convenient and effective to simulate any distributed applications. The performance evaluation shows the memory cost of the NSME is distinctly lower than NS2. The parallel efficiency of these simulation instances can reach about 60%. However, there are still some aspects to improve. We will continuatively enhance the parallel efficiency. The formalization of performance evaluation is also valuable to study in the future work. 8. Acknowledgments We would like to thank the anonymous reviewers for their valuable comments. This research is supported by the 973 National Basic Research Program of China (2004CB318109& 2007CB311100). 9. References [1] L. Breslau, D. Estrin, and K. Fall, et al, Advances in Network Simulation, IEEE Computer, [2] H. Akhtar, An Overview of Some Network Modeling, Simulation, & Performance Analysis Tools, Processings of the 2nd IEEE Symposium on Computers and Communications, [3] R. M. Fujimoto, K. Perumalla, A. Park, H. Wu, M. H. Ammar, Large-Scale Network Simulation: How Big? How Fast?, Processings of the 11TH IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer Telecommunications Systems (MASCOTS 03), 2003 [4] S. Lin and X. Cheng, NSME: A framework for Network Worm Modeling and Simulation, Processings of the 5th IFIP International Conference on Network Control and Engineering for QoS, Security and Mobility (Netcon'2006), 2006 [5] G. F. Riley, M. H. Ammar, and R. Fujimoto, A Federated Approach to Distributed Network Simulation, ACM Transactions on Modeling and Computer Simulation, 2004, pp [6] K. M. Chandy, J. Misra, Asynchronous Distributed Simulation via a Sequence of Parallel Computations, Communications of the ACM, 1981, 24(11): [7] L. Bajaj, M. Takai, R. Ahuja, K. Tang, R. Bagrodia, and M. Gerla, GloMoSim: A Scalable Network Simulation Environment, Technical Report , UCLA Computer Science Department, 1999 [8] H. Casanova, Simgrid: A Toolkit for the Simulation of Application Scheduling, Processings of the 1st IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2001), Brisbane, Australia, May. IEEE Computer Society Press, [9] H. Song, X. Liu, D. Jakobsen, R. Bhagwan, X. Zhang, K. Taura, A. Chien, The MicroGrid: A Scientific Tool for modeling computational Grids, Processings of IEEE Supercomputing (SC 2000), Dallas, TX, 4 10 November [10] G. Karypis. METIS: Family of Multilevel Partitioning Algorithms. [11] B. K. Szymanski, Yu Liu, R Gupta, Parallel Network Simulation under Distributed Genesis, Processings of the 17th Workshop on Parallel and Distributed Simulation (PADS'03). IEEE, [12] L. Song and G. A. Marin, Simulating Application Level Self-similar Network Using Hybrid Heavy-tailed Distributions, Processings of the 43rd ACM Southeast Conference, March 18-20, 2005 [13] CAIDA. Cooperative Association for Internet Data Analysis. Available at 346