Utilizing Network Emulation to Provide Cost Effective Large Scale System of Systems Test and Evaluation

Size: px

Start display at page:

Download "Utilizing Network Emulation to Provide Cost Effective Large Scale System of Systems Test and Evaluation"

Eustacia Strickland
5 years ago
Views:

17 CERDEC S&TCD MODESTA Team Data Collection, Analysis,

1 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. Utilizing Network Emulation to Provide Cost Effective Large Scale System of Systems Test and Evaluation 20 Jan 17 CERDEC S&TCD MODESTA Team Data Collection, Analysis, Modeling and Simulation Branch System Engineering, Architectures, Modeling and Simulation Division 1

Challenges to SoS Engineering, Integration, and

Communications (4) Computers (6) Electronic Warfare

Systems: Communications (2) Handheld Computers (1)

including Comms, Computers, and EW, the task of

complex Integration of desperate systems Interleaving

sources and sinks Accomplishing the required testing

2 Challenges to SoS Engineering, Integration, and Testing CO CO CP CO CDR Mounted Systems: Communications (4) Computers (6) Electronic Warfare (2) Weapons Optics PLT PLT LDR PLT SGT Dismounted Systems: Communications (2) Handheld Computers (1) Weapons As each system contains multiple sub-systems, including Comms, Computers, and EW, the task of engineering, integration and testing can be very complex Integration of desperate systems Interleaving systems engineering tasks Many voluminous data sources and sinks Accomplishing the required testing and evaluation of the SoS can be very challenging Note: notional architecture and systems displayed 2

High Degree of Difficulty COMPLEXITY User Actions / Interaction Data Collection / Metrics

Expertise needed in the areas of + Tactical Radios Networks Applications Integration of

experts Unique configuration requirements Non-trivial maintenance and operation For each

3 High Degree of Difficulty COMPLEXITY User Actions / Interaction Data Collection / Metrics System Under Test / Evaluation Network Architecture Traffic Profiles Operational Scenarios Expertise needed in the areas of + Tactical Radios Networks Applications Integration of complex data from multiple sources Complex tools, each needing their own subject matter experts Unique configuration requirements Non-trivial maintenance and operation For each experiment, a complex working group dynamic must be in place: Force Structure / Unit Roles Environment / Terrain RESOURCES 3

SoS Evaluation Trade Space SDR, Multifunction Waveforms, Embedded Security, QoS Mechanisms, Complex Routing Algorithms Large Tactical Networks, Soldier to

attempt to capture notional behavior -Detailed models require excessive time/cost -Notional traffic -Higher fidelity des models typically don t scale well

-True ethernet frame/ip packet processing -Locked to real-time execution -Highly scalable on commodity hardware Live Testing -Interfaces with system at IP

resources -Challenging data collection -Typically all-manual configuration The desired SoS Evaluation is high fidelity and scalable.

4 SoS Evaluation Trade Space SDR, Multifunction Waveforms, Embedded Security, QoS Mechanisms, Complex Routing Algorithms Large Tactical Networks, Soldier to Brigade, Thousands of Nodes, Enterprise to Tactical, Many to Many Interfaces Behavioral Discreet Event Simulation: -Highly abstracted system models -Models attempt to capture notional behavior -Detailed models require excessive time/cost -Notional traffic -Higher fidelity des models typically don t scale well -Difficult to interface with live systems Shared Code Model / Real-time Emulation -Actual code take from live systems -Cross-compiled into lightweight VMs -True ethernet frame/ip packet processing -Locked to real-time execution -Highly scalable on commodity hardware Live Testing -Interfaces with system at IP layer -Live systems provide true performance characteristics -Lives systems allow full interaction with other systems -High costs of personnel, ranges, resources -Challenging data collection -Typically all-manual configuration The desired SoS Evaluation is high fidelity and scalable. Traditionally takes a considerable amount of time and money. Time to setup the simulations (develop/configure inputs) and execute scenarios Traditionally, high fidelity models tend to run slower than real-time. 4

5 Experimentation Fidelity Simulation provides high scalability and moderate costs while typically sacrificing realism due to a high level of abstraction Live Testing provides true performance and usability characteristics, but does not scale well and can be very costly Real-time Emulation provides a middle ground where system software (waveform, router, and application shared code) is executed in virtual machines to allow for scalability and high fidelity Scalability Simulation Emulation Test (Live) Realism Realism Scalability Abstraction Cost Simulation Low / Med High High Low Emulation Med / High High Med Low Test (Live) High Low Low High 5

6 System of Systems Emulation: MODESTA The Modeling, Emulation, Simulation, Tool for Analysis (MODESTA) is a CERDEC developed government purpose rights framework for conducting systems of systems network analysis. Allows for disparate models, such as the shared code emulation models ported into EMANE and the behavior models developed in JNE, to be deployed in large scale for realistic network effects analysis. High Fidelity Representation of Multiple Layers Including Physical Connectivity, Network, Transport, and Application Integrates live hardware when available (e.g. HMS, WIN-T, MNVR systems). Allows stimulation of network with live C4ISR systems (e.g. JBC-P, CPOF) and/or generated traffic. Provides the ability to deploy low fidelity surrogate networks when high fidelity is not available or unnecessary. MODESTA allows high fidelity analysis within realistic timeframes Reduces data import times up to 50% through advanced data parsers Reduces model configuration up to 50% by automatically configuring models based on imported BOIs and standard waveform parameters across disparate emulation environments Provides an ability to quickly iterate through different courses of actions/network configurations MODESTA provides capability to construct large scale, high fidelity scenarios to provide detailed and scalable analysis. 6

7 Network Emulation Testbed Tactical Networks Links MAC PHY MAC PHY The Extendable Mobile Ad-hoc Network Emulator (EMANE) models the radio and SATCOM networks EMANE s PHY/MAC model Models path loss with antenna pointing Allocates timeslots to emitters Connects to preferred neighbors first Varies data rate and latency per neighbor EMANE s Comm Effects model imposes fixed data rate/delay for satellite EMANE exposes virtual L2 Tap network interfaces that behave as the Ethernet port on an radio or SATCOM modem 7

8 Network Emulation Testbed Linux Containers Linux containers Process groups isolated from the rest of the system to some degree (lightweight virtualization) Takes a different approach than system virtualization (VMWare) MAC PHY * MAC PHY A container (optionally) uses a network namespace an isolated network stack containing its own interfaces, routing table, neighbor table, netfilter rules, etc Containers in the emulation allow multiple ciphertext and plaintext routers to co-exist in the same operating system instance * Lightweight Virtual Linux Container 8

Tactical Network Emulation Deployment on

Supercomputer Resources at Army Research

Software on the Fly Execute Test in Real

9 Tactical Network Emulation Deployment on HPC Compute Nodes To achieve Brigade level scalability of live/virtual tactical nodes: Define Scenario Allocate DoD Supercomputer Resources at Army Research Lab partner Configure Hardware and Software on the Fly Execute Test in Real Time Reduce, Analyze Collected Data Tear Down and Sanitize Resources Making Available for Next Queued Test Allocate Resources Configure Software Reduce & Analyze Data Define Test / Scenario Configure Hardware Execute Test Sanitize Resources 9

MODESTA Experimentation Workflow MAC layer stats C4 Data Model IP Layer stats Harvest & Reduction Execution, Data Collection & Visualization Consolidate collected

real-time utilizing all available hardware and load balancing Sync execution of traffic, movement, pathloss Display real-time stats on specific nodes and multiple

Equipment & Networks Positions, Movement & Pathloss Import NBOI xls Manually modify nets, roles, equipment Select network emulation tools Cutsheet import or

10 MODESTA Experimentation Workflow MAC layer stats C4 Data Model IP Layer stats Harvest & Reduction Execution, Data Collection & Visualization Consolidate collected data Transfer to ARL HPC Execute correlative analysis scripts Deploy each emulated switches, radios, routers, traffic generators & collectors Monitor hardware real-time utilizing all available hardware and load balancing Sync execution of traffic, movement, pathloss Display real-time stats on specific nodes and multiple OSI layers Traffic Import ATM traffic Must be mapped to NBOI objectids Pre-processing required to assign IPs, MC membership, etc Analysis Reqs Force Structure, Equipment & Networks Positions, Movement & Pathloss Import NBOI xls Manually modify nets, roles, equipment Select network emulation tools Cutsheet import or auto-assign IPs Import pre-run OneSAF positions export (mapped to NBOI objectids) Select pathloss calculation method (eg Tirem) & pre-calculate pathloss utilizing all available hardware 10

Capability is Being Utilized for PEO C3T, ASA(ALT) SOSE&I, etc Summary Takeaway and COA Refinement MODESTA Enabled Rapid

11 MODESTA Enables SoS Rapid Experimentation in a Controlled, Repeatable Lab Environment Emulation Provides Scalable, Shared Code, Real-time Execution Allows Low Risk Test-Tweak- Test Cycle Utilizes High Speed Data Reduction and Correlation Process Capability is Being Utilized for PEO C3T, ASA(ALT) SOSE&I, etc Summary Takeaway and COA Refinement MODESTA Enabled Rapid Experimentatio n Cycle Experimentation Planning and COA Development Data Reduction and Analysis High Fidelity Real- Time Emulation 11

12 Backup 12

SCALABLE. Network modeling software for: Development Analysis Testing Cyber Assessment DATASHEET NETWORK TECHNOLOGIES. Virtual Network Model

SCALABLE. Network modeling software for: Development Analysis Testing Cyber Assessment DATASHEET NETWORK TECHNOLOGIES. Virtual Network Model SCALABLE NETWORK TECHNOLOGIES DATASHEET Network modeling software for: Development Analysis Testing Cyber Assessment EXata software (EXata) is a tool for scientists, engineers, IT technicians and communications