Bridging OPNFV and ETSI Yardstick and the methodology for pre-deployment validation of NFV Infrastructure

Transcription

1 Bridging OPNFV and ETSI Yardstick and the methodology for pre-deployment validation of NFV Infrastructure Ana Cunha (Ericsson)

2 Agenda The facts The questions The ETSI-NFV methodology The realization: OPNFV Yardstick

3 The facts NFV Use cases in ETSI GS NFV 001: include a large variety of applications each defining specific requirements and complex configuration on the NFVI and test tools

4 The questions NFVI + VNF? VNF requirements? NFVI characteristics? HW faults? Bottlenecks?

5 End-to-end Testing process End-to-end NFV Testing NFVI testing VNF application testing NFVI Testing Pre-deployment validation of NFVI VNF infrastructure validation Deploy VNF TST001 chapter 6

6 The methodology Pre-Deployment validation of NFVI 1. Define infrastructure ETSI GS NFV-TST001 Chapter 6 Pre-Deployment validation of NFV infrastructure 2. Identify VNF type 3. Select Test Cases 4. Execute Tests - Control plane/user plane - Main components - Real time constraints - Hardware - Metrics - Test cases 1 N - Test stimuli - Test environment 5. Collect Results 6. Integrate VNF application

7 Metrics Categories Compute Storage Sub-categories: Performance/speed Capacity/Scale Reliability/availability Networking

8 Test cases selection VNF application Metrics vector Compute - Metric A VNF Storage - Metric B Networking - Metric C - Metric D - Metric E Test cases Compute - Test case 1, SLA1 Storage - Test case2, SLA2 Networking - Test case3, SLA3 - Test case4, SLA4

9 The test execution Test Environment Configure - SUT - Deployment rules Deploy - Generate template or - External templates Test Validate - Execute benchmarks - Trigger External scripts for SUT disturbances - Collect results - Graphical results - Assertion

10 The realization: OPNFV Yardstick Vendor independent Infrastructure independent Aligned with TST001 Open Source Compute Storage Networking Reliability Yardstick Capacity Performance OPNFV Yardstick Project Wiki Yardstick test cases Project status

11 Get started with Yardstick! 1. Install Yardstick 3. Build & load image 4. Run tests Cloud Environment 2. Create Test.yaml Yardstick installation video

12 Compute Performance/Speed Capacity/Scale Reliability/Availability - Latency for random memory access - Latency for cache read/write operations - Processing speed (instructions per second) -Throughput for random memory access (bytes per second) Network - Throughput per NFVI node (frames/byte per second) - Throughput provided to a VM (frames/byte per second) - Latency per traffic flow - Latency between VMs - Latency between NFVI nodes - Packet delay variation (jitter) between VMs - Packet delay variation (jitter) between NFVI nodes - Number of cores and threads - Available memory size - Cache size - Processor utilization (max, average, standard - Memory utilization (max, average, standard - Cache utilization (max, average, standard - Number of connections - Number of frames sent/received - Maximum throughput between VMs (frames/byte per second) - Maximum throughput between NFVI nodes (frames/byte per second) - Network utilization (max, average, standard - Number of traffic flows - Processor availability (Error free processing time) - Memory availability (Error free memory time) - Processor mean-time-to-failure - Memory mean-time-to-failure - Number of processing faults per second - NIC availability (Error free connection time) - Link availability (Error free transmission time) - NIC mean-time-to-failure - Network timeout duration due to link failure - Frame loss rate Storage - Sequential read/write IOPS - Random read/write IOPS - Latency for storage read/write operations - Throughput for storage read/write operations - Storage/Disk size - Capacity allocation (block-based, object-based) - Block size - Maximum sequential read/write IOPS - Maximum random read/write IOPS - Disk utilization (max, average, standard - Disk availability (Error free disk access time) - Disk mean-time-to-failure - Number of failed storage read/write operations per second

13 Compute Performance/Speed Capacity/Scale Reliability/Availability - Latency for random memory access - Latency for cache read/write operations - Processing speed (instructions per second) -Throughput for random memory access (bytes per second) Network - Throughput per NFVI node (frames/byte per second) - Throughput provided to a VM (frames/byte per second) - Latency per traffic flow - Latency between VMs - Latency between NFVI nodes - Packet delay variation (jitter) between VMs - Packet delay variation (jitter) between NFVI nodes - Number of cores and threads - Available memory size - Cache size - Processor utilization (max, average, standard - Memory utilization (max, average, standard - Cache utilization (max, average, standard - Number of connections - Number of frames sent/received - Maximum throughput between VMs (frames/byte per second) - Maximum throughput between NFVI nodes (frames/byte per second) - Network utilization (max, average, standard - Number of traffic flows - Processor availability (Error free processing time) - Memory availability (Error free memory time) - Processor mean-time-to-failure - Memory mean-time-to-failure - Number of processing faults per second - NIC availability (Error free connection time) - Link availability (Error free transmission time) - NIC mean-time-to-failure - Network timeout duration due to link failure - Frame loss rate Storage - Sequential read/write IOPS - Random read/write IOPS - Latency for storage read/write operations - Throughput for storage read/write operations - Storage/Disk size - Capacity allocation (block-based, object-based) - Block size - Maximum sequential read/write IOPS - Maximum random read/write IOPS - Disk utilization (max, average, standard - Disk availability (Error free disk access time) - Disk mean-time-to-failure - Number of failed storage read/write operations per second

14 Compute Performance/Speed Capacity/Scale Reliability/Availability - Latency for random memory access - Latency for cache read/write operations - Processing speed (instructions per second) -Throughput for random memory access (bytes per second) Network - Throughput per NFVI node (frames/byte per second) - Throughput provided to a VM (frames/byte per second) - Latency per traffic flow - Latency between VMs - Latency between NFVI nodes - Packet delay variation (jitter) between VMs - Packet delay variation (jitter) between NFVI nodes - Number of cores and threads - Available memory size - Cache size - Processor utilization (max, average, standard - Memory utilization (max, average, standard - Cache utilization (max, average, standard - Number of connections - Number of frames sent/received - Maximum throughput between VMs (frames/byte per second) - Maximum throughput between NFVI nodes (frames/byte per second) - Network utilization (max, average, standard - Number of traffic flows - Processor availability (Error free processing time) - Memory availability (Error free memory time) - Processor mean-time-to-failure - Memory mean-time-to-failure - Number of processing faults per second R3 R3 R3 - NIC availability (Error free connection time) - Link availability (Error free transmission time) - NIC mean-time-to-failure - Network timeout duration due to link failure - Frame loss rate R3 R3 R3 Storage - Sequential read/write IOPS - Random read/write R3 IOPS - Latency for storage read/write operations - Throughput for storage read/write operations - Storage/Disk size - Capacity allocation (block-based, R3 object-based) - Block size - Maximum sequential read/write IOPS - Maximum random read/write IOPS - Disk utilization (max, average, standard - Disk availability (Error free disk access time) R3 - Disk mean-time-to-failure - Number of failed storage read/write operations per second

15 Compute Performance/Speed Capacity/Scale Reliability/Availability - Latency for random memory access - Latency for cache read/write operations - Processing speed (instructions per second) -Throughput for random memory access (bytes per second) Network - Throughput per NFVI node (frames/byte per second) - Throughput provided to a VM (frames/byte per second) - Latency per traffic flow - Latency between VMs - Latency between NFVI nodes - Packet delay variation (jitter) between VMs - Packet delay variation (jitter) between NFVI nodes - Number of cores and threads - Available memory size - Cache size - Processor utilization (max, average, standard - Memory utilization (max, average, standard - Cache utilization (max, average, standard - Number of connections - Number of frames sent/received - Maximum throughput between VMs (frames/byte per second) - Maximum throughput between NFVI nodes (frames/byte per second) - Network utilization (max, average, standard - Number of traffic flows - Processor availability (Error free processing time) - Memory availability (Error free memory time) - Processor mean-time-to-failure - Memory mean-time-to-failure - Number of processing faults per second Rx Rx Rx R3 R3 R3 - NIC availability (Error free connection time) - Link availability (Error free transmission time) - NIC mean-time-to-failure - Network timeout duration due to link failure - Frame loss rate Rx Rx Rx R3 R3 R3 Storage - Sequential read/write IOPS - Random read/write IOPS Rx - Latency for storage read/write operations - Throughput for storage R3 read/write operations - Storage/Disk size - Capacity allocation (block-based, object-based) Rx - Block size - Maximum sequential R3 read/write IOPS - Maximum random read/write IOPS - Disk utilization (max, average, standard - Disk availability (Error free disk access time) Rx - Disk mean-time-to-failure - Number of failed R3 storage read/write operations per second

16 Yardstick Evolution - Framework Capabilities Results SUT Test Cases Result API, SLA Analytics Cloud, Node (Baremetal, Compute, Controller) NFVI Test Cases (Compute, Storage, Networking) R3 RX

17 Challenges NFVI + VNF? VNF requirements? NFVI characteristics? HW faults? Bottlenecks? Telco KPIs References What Reliability Limits

18 Want to help? Yardstick Thank you