Building and evaluating network simulation systems

Transcription

1 S Postgraduate Course in Radiocommunications Fall 2000 Building and evaluating network simulation systems Shkumbin Hamiti Nokia Research Center HUT Page 1 (14)

2 1. Introduction Objectives of Network Simulations Developing Valid and Credible Simulation Models Statistical Issues in Network Simulation Modeling System Randomness Design and Analysis of Simulation Experiments Examples of Simulation Software Some aspects of simulations of mobile networks Spectrum efficiency evaluation Active Session Throughput Satisfied user System Load Spectrum Efficiency Parameter values Required results Notes on traffic models in simulation of communication networks Conclusion Reference Page 2 (14)

3 1. INTRODUCTION This seminar report deals with basic principles of building and evaluating network level simulation models. The main topics discussed here are: Issues that network simulations address Techniques to build valid and credible model Care for statistical issues Available simulation software Notes on simulating cellular networks This report is largely based on references [1] and [2]. It is very common the case that there is a need for studying and evaluating existing or proposed communication systems. The aims of those studies are usually evaluation of system performance and impact on the existing systems. Performing studies and evaluation is quite difficult in live networks and more often in many cases the network does not even exist. Therefore using simulations for studying and evaluating has become a necessity. If the relationships between the nodes and functions of the communication system are simple, there is no better way than using analytical models for evaluation. Analytical solutions have been widely used, especially in queuing systems, however as the number of nodes increases, and stochastic processes becomes more complicated, the analytical solutions are harder to obtain. In addition analytical solutions suffer from several drawbacks: Only steady-state solutions are possible Performance measures other then mean values are quite difficult to get Requires a strong mathematical sophistication from the analyst Today's communication networks are quite complicated and the complexity of the networks is not expected to decrease in the near future. Adding a new feature, or node in the communication networks could have a severe impact on the performance of the system, therefore making practically impossible to evaluate a stand-alone feature without taking into account other features or nodes. Therefore simulation models are basically used everywhere nowadays. Consequently there are many simulation software packages available. In a simulation a mathematical/logical model is numerically evaluated over a time period of interest, and performance measures are estimated from model-generated data. Simulation Page 3 (14)

4 analyses are applicable to systems of almost any level of complexity limited mainly by available simulation resources. Page 4 (14)

5 2. OBJECTIVES OF NETWORK SIMULATIONS It has been shown in the previous presentation [S Pukkila] on radio link level simulation main components of the simulation process: Simulation environment, software and hardware Building Simulation Models based on a real problem that need to be solved Error sources Validation These components are also part of network simulation process, however the objectives of the network simulation differ from those of radio link in particular with respect to the desired performance measures expected from the simulations. Usually for link level simulations the desired simulation results are BER, FER or BLER vs SNR or C/I for various radio level parameters. In case of network simulations the following are the goals of using simulations to design and analyze communications networks: Determination of the system-wide impact of making "local" changes to the network Improved system performance (delays, throughput, etc.) Reduced expenditures Insurance that performance objectives are met before equipment is bought or leased Identification of bottlenecks before system implementation Reduced system development time Etc The usual performance measures that are commonly used in simulation studies of networks are: Throughput End-to-end delay Delay from point A to point B in a network Number of "data units" in a queue or a buffer Utilization of nodes or links Probability of blocked call Probability of lost call Number of collisions and deferrals Satisfied users Network QoS Spectral efficiency of a mobile system Page 5 (14)

6 3. DEVELOPING VALID AND CREDIBLE SIMULATION MODELS A simulation model usually represents some real system that need to be evaluated. Clearly, building all aspects of the simulation system into a simulation model is time consuming and expensive, thus the simulation model must idealize the real system to that extent to be valid enough so that any conclusions drawn from the model would be similar to those derived from physical experiment with the system (if this were possible). It is important for a model to be credible otherwise its results may never be used in the decision-making process, even if the model is valid. The following are some important ideas/techniques for deciding the appropriate level of model detail, for validating a simulation model, and for developing a model with high credibility: State definitely the issues to be addressed and the performance measures at the beginning of the study Collect information on the network topology and protocols Create a clear "assumption document" Perform a structured walk-through of the conceptual model (the best way is to do a walk-through with other experts and colleagues) Use sensitivity analyses to determine important model factors Compare performance measures for the existing network to comparable performance measures for simulation model of the existing network (time consuming but very important to "calibrate" the model) Page 6 (14)

7 4. STATISTICAL ISSUES IN NETWORK SIMULATION Since random samples from input probability distributions are used to "drive" a simulation model through time, basic output data or estimated performance measures computed from them are also random. Therefore it is important to model the random process inputs to a simulation model correctly and to design and analyze simulation experiments in a proper manner. 4.1 Modeling System Randomness The most important source of randomness for network simulations is usually associated with message traffic. In general, one should model messages (or transactions) not packets. The messages are fragmented into packets by the network protocols employed in the simulation. Note also that messages may not be independent from each other (transaction nature must be taken into account). The following methods of generating traffic are often used: Message departure times and message sizes for a particular node are application based (HTTP, FTP, , etc ) Message interarrival times and message sizes for a particular node are each independent samples from respective probability distributions Traffic data are read into the simulation model from a network analyzer 4.2 Design and Analysis of Simulation Experiments Because of the random nature of simulation input, a simulation model produces a statistical estimate of the (true) performance measure not the measure itself. In order for a simulation estimate to be statistically precise and free of bias, it is recommended to specify for each network configuration appropriate choices for the following: Length of each simulation run Number of independent simulation Length of the warm-up period, if one is appropriate Page 7 (14)

8 5. EXAMPLES OF SIMULATION SOFTWARE One of the major tasks in building a simulation model of a communications network is that of converting a system description into a computer program. An analyst may use either a general-purpose programming language (C or C++) or simulation software for this purpose. There are three major types of software for simulating communications networks. A general-purpose simulation language is a simulation package that is general in nature but may have some special features for communications such as explicit modules for Ethernet. Examples are Arena, BONes Designer, GPSS/H, MODSIM II, SIMSCRIPT etc. A communciation oriented simulation language is a simulation language that is specifically oriented toward communications networks, for example OPNET Modeler. Advantages are possibly reduced programming time and modeling constructs oriented towards communication systems. A communications oriented simulator, in its most basic form, is a simulation package that allows one to simulate a network in a specific class of communication networks with no programming. Examples are BONes Plannet, COMNET III, NET, NETWORK, etc. A one example of the communication oriented simulation language software package is OPNET. OPNET is an event-driven simulator. The modeling in opnet is done in three stages (examples shown in the figures below) network, node and process level. The network editor is used to specify geographically the network topology consisting of nodes and fixed-position links. The node editor is used to describe graphically the data flow between modules in a node. The modules in the node can be protocol layers or hardware models. Available modules are processors, queues and traffic generators. The process editor uses state diagrams and C/C++ programming language, In addition there is an extensive library of primitives for most serious protocol modeling. Figure 1. Network Editor Page 8 (14)

9 Figure 2. Node Editor Figure 3. Process Editor Page 9 (14)

10 6. SOME ASPECTS OF SIMULATIONS OF MOBILE NETWORKS 6.1 Spectrum efficiency evaluation This section presents the definition of system capacity to be used for spectrum efficiency evaluation, and a methodology to derive these figures using system simulations as specified in UMTS v Active Session Throughput The active session throughput, S, is defined as the ratio of correctly received user bits during the entire session and the session length excluding the time where there is nothing to transmit (i.e. empty buffer) Satisfied user For circuit switched services, we define a satisfied user as a user that have all three of the following constraints fulfilled: 1. The user do not get blocked when arriving to the system. If blocking is applied, the proponent must specify used blocking criteria. 2. The user have sufficiently good quality more than a certain time (fraction) of the session, i.e., Probability(BER > BER_Threshold) < x 1 % 3. The user does not get dropped. A call is dropped if BER > BER_Threshold more than t dropp1 seconds. In order to get comparable results for good quality percentage, quality statistics have to be collected every t 1 seconds. For packet services, we define a satisfied user as a user that have all three of the following constraints fulfilled: 1. The user do not get blocked when arriving to the system 1. If blocking is applied, the proponent must specify used blocking criteria. 2. The active session throughput, S, of the session is equal to or greater than S threshold. 3. The user does not get dropped. If dropping is applied, the proponent must specify used dropping criteria System Load The system load, ν, is measured in [kb/s/cell/mhz]. For circuit switched users, the system load, ν cs, is derived as follows: 1 The most common way of treating packet users is not to block them but to queue them. However, if the proponent applies some kind of admission control for packet users, there will exist a ratio of blocked packet users. Thus blocking of packet users, means that they are not put in a queue but entirely blocked from the system. Page 10 (14)

11 ν cs = ω cs * user_bitrate * activity_factor / system_bandwidth [kb/s/cell/mhz], where ω cs is the average number of (simultaneous) circuit switched users per cell, i.e. the offered load (Erlangs). System load for packet users, ν pkt, is derived as follows: ν pkt = D/T/Cells/system_bandwidht [kb/s/cell/mhz], where D is total number of correctly received user bits within the cells from where the statistics are collected T is the simulation measuring time, defined as the time during the simulation when the statistics are collected Cells is the number of cells in the system from where the statistics are collected. The system load is calculated separately for uplink and downlink respectively. In the case of mixed services configurations, the system load is derived as: N N sc pkt cs, i pkt, i i= 1 i= 1 ν = ν + ν where N cs is the number of circuit switched services and N pkt is the number of packet services Spectrum Efficiency For single service scenarios, the spectrum efficiency, ν*, is defined as the system load where there are exactly x 2 % satisfied users. In the case of mixed services configurations, there must be at least x 2 % satisfied users for each service independently. The spectrum efficiency is defined as the system load where any of the services has exactly x 2 % satisfied users, whereas the rest of the services have at least x 2 % satisfied users each. This is exemplified in Figure 4. The spectrum efficiency should be given separately for uplink and downlink respectively. Page 11 (14)

12 % Satisfied Users Packet x 2 % limit Circuit Switched Spectrum Efficiency, ν* System Load, ν [kb/s/cell/mhz] Figure 4. Example of spectrum efficiency for a mixed service scenario with one circuit switched service and one packet service Parameter values In Table 1 the values of the parameters previously mentioned are presented. TABLE 1: Values of parameter in spectrum efficiency evaluation Paramet er x 1, t 1 t dropp1 S threshold Name/Description Bad quality probability threshold sampling time for quality statistics Dropping time-out, circuit switched Active Session Throughput Threshold Value 5 % 0.5 second Max (5, 10/(bit rate. BER_threshold) seconds 10% of the average bit rates in footnote of Table (i.e., S threshold = 0.8, 3.2, 6.4, 14.4, 38.4 and kbit/s for average bit rates of 8,32,64,144,384 and 2048 kbit/s respectively). x 2 Threshold for ratio of satisfied 98 % users BW Bandwidth 30 MHz duplex Required results Within the spectrum efficiency evaluation the following results should be provided for each test case: 1. Numerical value of the spectrum efficiency [kb/s/cell/mhz] 2. Numerical value for ratio of satisfied users for the case from where the spectrum efficiency value (in 1.) is obtained. Page 12 (14)

13 3. Average active session throughput [kb/s], mean(s), for the case from where the spectrum efficiency value (in 1.) is obtained. 4. A sample density function of the active session throughput values (per session) for the case from where the spectrum efficiency value (in 1.) is obtained. 7. NOTES ON TRAFFIC MODELS IN SIMULATION OF COMMUNICATION NETWORKS When simulating a communication network, especially mobile network there is one quite important issue to note, and that relates to the traffic models that are used in simulations. The usual traffic sources in toady's network are from traffic sources in the Internet, like WWW, , ftp etc. For example in WWW session, transmission times appear to exhibit heavy-tailed characteristics, file requests, file transfers, unique files and available files are heavy-tailed too. In case of applications like Telnet, FTP, NNTP, SMTP components of connection characteristics are modeled in general using lognormal distributions. However, the number of bytes in an FTP burst appears to fit well with Pareto distribution. When using heavy-tailed distributions in the simulations there are two characteristics: slow convergence to a steady-state high variability at steady state The reason is that the average-case behavior depends on the presence of many small observations as well as few large observations, so the convergence to the steady state could be slow and the presence of large observations could have a dominating effect. So in order to obtain stable results in simulations when using heavy-tailed workloads one should be careful that α is greater then about 1.7, otherwise one should carefully consider the stability of the results. 8. CONCLUSION This report gives an overview of issues that one should pay attention when simulating communication networks. Some examples of available communication software are also presented. Parameters and performance measures that are usually used when simulating cellular networks are shown based on UMTS v When using traffic models with heavy tailed load, one should pay close attention to the stability of results. 2 The values of the active session throughput thresholds are chosen to make it possible to perform re-transmission, queuing, etc. in order to make the packet services effective from a system point of view. Page 13 (14)

14 9. REFERENCE [1] Law A.M, McComas M.G, Simulation Software for Communications Networks: The State of the Art, IEEE Communications Magazine, March 1994 [2] Law A.M, Simulation of Communication Networks, In Proceedings of the 1996 Winter Simulation Conference (eds. Charnes J.M et. al.) [3] UMTS v Page 14 (14)

15 HOME WORK 1. When simulating cellular networks, radio link level behaviour should be taken into account. Provide example how. 2. When simulating celluar networks, it is often required to run dynamic simulations. Why?1