Comparative study of LTE simulations with the ns-3 and the Vienna simulators

Transcription

1 Comparative study of LTE simulations with the ns-3 and the Vienna simulators Thiago Abreu 1 Bruno Baynat 1 Marouen Gachaoui 2 Tania Jimenez 2 Narcisse Nya 1 1 University Pierre et Marie Curie - LIP6 2 LIA/CERI University of Avignon August /23 SimuTools Comparative study with ns-3 and Vienna

2 Acknowledgment This work has been carried out in the framework of IDEFIX project, funded by the ANR under the contract number ANR-13-INFR /23 SimuTools Comparative study with ns-3 and Vienna

3 Goals of this work Give a performance comparison of two open source simulators in the context of LTE networks Evaluate their accuracy and compare their efficiency by simulating four basic scenarios with well-known analytical results Understand the software architecture of the simulators and their capabilities to simulate more realistic scenarios List the implementation issues and propose possible solutions to improve the simulators 3/23 SimuTools Comparative study with ns-3 and Vienna

4 Outline Introduction to ns-3 and Vienna simulators Simulated scenarios Implementation Simulators limitations Simulation results Performance comparison Conclusions 4/23 SimuTools Comparative study with ns-3 and Vienna

5 Vienna simulator A bench of MATLAB classes, functions and scripts Time-step discrete-event simulator for LTE networks Advances the time by time slices of 1ms (the duration of one Transmission Time Interval (TTI) in the LTE standard) Very precise on the modeling of the physical layer Performs many calculations at the physical layer level every time-step Trace files for each user at the end of each simulation (received bits, SINR, throughput..) 5/23 SimuTools Comparative study with ns-3 and Vienna

6 ns-3 simulator ns-3 is a research-oriented, discrete-event network simulator Open source licensing (GNU GPLv2) and development model Python scripts or C++ programs Several modules : designed to simulate different technologies and scenarios LTE module : developed by the LENA project following the 3GPP LTE standard specifications 6/23 SimuTools Comparative study with ns-3 and Vienna

7 Simulators comparison Vienna: - Specialized LTE network simulator - Clock advanced by time-step of 1ms - Performs physical layer calculations every time-step - Traces are simple to work with - Not modular core ns-3: - General network simulator - Clock advanced by the date of the next event - Directly uses tables provided by the standard specifications - Processing traces can be cumbersome - Modular, documented core /23 SimuTools Comparative study with ns-3 and Vienna

8 Simulated scenarios One LTE cell = N s elementary resources per frame Static users using the same MCS over the whole cell 1 resource = m bits/frame, each frame transmitted every T f Cell capacity C = mns T f bits/s Four basic scenarios with well-known analytical results Differentiate themselves by the nature of the traffic (data or voice) and by the number of sources (infinite or finite): - Data traffic with infinite source (DI) - Data traffic with finite source (DF) - Voice traffic with infinite source (VI) - Voice traffic with finite source (VF) 8/23 SimuTools Comparative study with ns-3 and Vienna

9 DI : Data traffic with infinite source Model by a M/M/1/ / /Procesor Sharing (PS) queue Requests for data transmission arrive to the cell according to a Poisson process with rate λ Data transmission = mean amount of data x on to be transferred Cell capacity C equally shared among active users Service rate µ = C x on /23 SimuTools Comparative study with ns-3 and Vienna

10 DF : Data traffic with finite source Model by a M/M/1/N/N/PS queue Finite number N of users generating an ON/OFF traffic ON = average volume of data x on to be transferred OFF = idle period t off Cell capacity C equally shared among active users Arrival rate λ = 1 t off Service rate µ = C x on ( ) ( ) ( ) /23 SimuTools Comparative study with ns-3 and Vienna

11 VI : Voice traffic with infinite source Model by a M/M/S/S queue Requests for voice calls arrive to the cell according to a Poisson process with rate λ Voice call = mean active period t on, requires a throughput of v bits/s Maximum number of simultaneous voice calls can be served = S = C v Service rate µ = 1 t on /23 SimuTools Comparative study with ns-3 and Vienna

12 VF : Voice traffic with finite source Finite number N of users generating an ON/OFF traffic ON = voice call : mean active period t on OFF = idle period t off S simultaneous voice calls can be served by the cell Arrival rate λ = 1 t off Service rate µ = 1 t on Two cases : up: N S, down: N > S ( ) ( ) ( ) ( ) ( ) ( ) ( + ) /23 SimuTools Comparative study with ns-3 and Vienna

13 Vienna simulator The code is not well structured : not easy to add new modules without modifying many other files. Issues: - Clock advanced by TTI, physical layer calculations performed every time-step very high simulation run-time unable to simulate realistic scenarios with long simulation times and high loaded systems - Constant number of users along the simulation unable to simulate infinite source scenarios - Existing ON/OFF traffic modules must be modified - Scheduling policy with S servers not implemented unable to simulate voice cases 13/23 SimuTools Comparative study with ns-3 and Vienna

14 Vienna simulator Solutions: - Improve the computation load: deactivate and detach OFF users from the system perform the physical layer calculations only when a user perceives a channel condition changing (instead of every TTI) - Implement new modules making the simulator capable of adding and deleting users during the simulation - Modify the existing ON/OFF traffic modules and add new ones - Implement a new scheduling policy for the voice cases (with S servers) 4/23 SimuTools Comparative study with ns-3 and Vienna

15 ns-3 simulator Issues: - ON/OFF application for data traffic not implemented - Number of users is limited to 320 per cell unable to carry out simulations of infinite sources Solutions: - Implement a new ON/OFF application to model data transmissions - Reuse of served users to model new ones to overcome the limit of 320 users per cell 15/23 SimuTools Comparative study with ns-3 and Vienna

16 DI scenario Metrics of interest: - Average number of active users (Q) - Mean throughput a user obtains Both simulators are accurate with small relative errors and narrow confidence intervals 14 8 Q ns-3 Q Vienna Q ns-3 throughput Vienna throughput theoretical throughput (Mbits/s) System bandwidth 5MHz (25 RBs) Cell capacity 18.3Mbits/s (best MCS in the cell) Scheduler Round Robin Mean size xon 1.7Mbits Mean inter-arrival time toff from 100ms to 150ms Simulation time 300s Table: Simulation parameters t off (ms) 16/23 SimuTools Comparative study with ns-3 and Vienna

17 DF scenario Metrics of interest: - Average number of active users (Q) - Mean throughput a user obtains Good accuracy: narrow confidence intervals 8 9 Q ns-3 Q Vienna Q ns-3 throughput Vienna throughput theoretical throughput (Mbits/s) System bandwidth 5MHz (25 RBs) Cell capacity 18.3 Mbits/s (best MCS in the cell) Scheduler Round Robin number of users N 10 Mean size xon 1.7Mbits Mean idle time toff from 200ms to 1200ms Simulation time 300s Table: Simulation parameters t off (ms) 17/23 SimuTools Comparative study with ns-3 and Vienna

18 VI scenario Metrics of interest: - Average number of active users (Q) - Rejection probability of a call request (P r ) Both simulators are accurate with small relative errors and narrow confidence intervals Q ns-3 Q Vienna Q ns-3 Pr Vienna Pr theoretical P r Scheduler S simultaneous voice calls S 10 Mean call duration ton 1000ms Mean OFF period toff from 100ms to 150ms Simulation time 300s Table: Simulation parameters t off (ms) 18/23 SimuTools Comparative study with ns-3 and Vienna

19 VF scenario: first case (N > S) Metrics of interest: - Average number of active users (Q) - Rejection probability of a call request (P r ) Good accuracy: small relative errors and narrow confidence intervals 10 Q ns-3 Q Vienna Q ns-3 Pr Vienna Pr theoretical P r Scheduler S simultaneous voice calls S 10 Number of users N 15 Mean call duration ton 950ms Mean OFF period toff from 200ms to 1200ms Simulation time 300s Table: Simulation parameters t off (ms) 19/23 SimuTools Comparative study with ns-3 and Vienna

20 VF scenario: second case (N S) Metrics of interest: - Average number of active users (Q) Good accuracy 14 Q ns-3 Q Vienna Q theoretical Scheduler S simultaneous voice calls S 20 Number of users N 15 Mean call duration ton 950ms Mean OFF period toff from 200ms to 1200ms Simulation time 300s Table: Simulation parameters t off (ms) 20/23 SimuTools Comparative study with ns-3 and Vienna

21 Simulation run-time Finite source cases: - ns-3 much faster than Vienna - Vienna advances the time by TTI and performs more calculations than ns-3 Infinite source cases: - Vienna faster than ns-3 - ns-3 uses large number of users to model the infinite source (320) - Vienna creates and eliminates nodes during the simulation model VF DF VI DI ns-3 37min 27min 465min 608min Vienna 780min 390min 270min 300min Table: Run-time performance 21/23 SimuTools Comparative study with ns-3 and Vienna

22 Memory usage CPU occupation Memory: - Vienna uses 46 times more memory than ns-3 CPU: - ns-3 uses less resources than Vienna model VF DF VI DI ns-3 52MB 44MB 71MB 64MB Vienna 1.6GB 1.25GB 2.25GB 3GB Table: Memory usage model VF DF VI DI ns % 3.05% 3.1% 3.1% Vienna 8.1% 8% 7.5% 7.6% Table: CPU occupation 22/23 SimuTools Comparative study with ns-3 and Vienna

23 Conclusions In terms of accuracy, both simulators provide good results, for the four scenarios we tested. Regarding the execution time: - With finite number of sources ns-3 is clearly faster than Vienna. - Vienna becomes faster for infinite source scenarios. - In ns-3 we create at the beginning of the simulation a big number of UEs, whereas in Vienna, UEs can be created and deleted on-the-fly. In terms of CPU and memory usage, ns-3 is much better than Vienna. Concerning the ease of handling and use: - It depends on the skills of the programmer, in C++ or in MATLAB language. - We find that output traces are easier to manipulate in Vienna than in ns-3. - ns-3 is quite well documented which is not the case for Vienna. 23/23 SimuTools Comparative study with ns-3 and Vienna