PowerVM Single Root I/O Virtualization: Fundamentals, Configuration, and Advanced Topics

Transcription

1 PowerVM Single Root I/O Virtualization: Fundamentals, Configuration, and Advanced Topics Allyn Walsh Consulting IT Specialist Many contributions from Chuck Graham STSM, Lead Architect Alexander Paul Senior Systems and Network Engineer 2016 IBM Corporation

2 Topics Overview Performance vnic and Live Partition Mobility Fault-Tolerant Configurations vnic Failover Performance Monitor & Topology Maintenance Additional information 1

3 Network Technologies on POWER Systems Dedicated s Best possible performance exclusively bound to particular partition; no resource sharing Virtual Ethernet Hypervisor internal switching VIOS Shared Ethernet Hypervisor Switch uplink through VIOS Options for high availability SEA failover, SEA failover w. load sharing, NIB Single Root I/O Virtualization () and vnic is PCIe standard for hardware resource sharing vnic is new virtual adapter type vnic Announced 5 th October 2015 Host Ethernet (HEA) virtualization technology Not available for P7+ and P8 servers 2

4 Power Systems Solutions PowerVM Single Root I/O Virtualization () technology Sharing by up to 64 partitions per PCIe adapter. Direct Access I/O Performance - provides CPU utilization and latency characteristics similar to a dedicated adapter Function - partition access to advanced adapter features (e.g. RSS, LSO, etc.) port (VF) resource provisioning (e.g. desired bandwidth) Flexible deployment models Single partition Multi-partition without VIOS using Direct Access I/O Multi-partition thru VIOS Multi-partition mix of thru VIOS and Direct Access I/O PowerVM virtual Network Interface Controller (vnic) technology Leverages technology Advanced virtualization (e.g. LPM) capable Sharing by up to 64 partitions per adapter port (VF) resource provisioning (e.g. desired bandwidth) Requires VIOS 3

5 HW/SW minimums for POWER8 - June 2015 GA IBM Power System E870 (9119-MME), IBM Power System E880 (9119-MHE), IBM Power System E850 (8408-E8E) IBM Power System S824 ( A), IBM Power System S814( A), IBM Power System S822( A), IBM Power System S824L( L), IBM Power System S822L ( L), IBM Power System S812L( L) support for PCIe Gen3 I/O expansion drawer HMC is required for Server firmware 830 PowerVM is required - standard or enterprise edition PowerVM express edition allows only one partition to use the logical ports per adapter Minimum client operation systems: AIX 6.1 TL9 SP5 and APAR IV68443, or later AIX 7.1 TL3 SP5 and APAR IV68444, or later IBM i 7.1 TR10, or later IBM i 7.2 TR2, or later Red Hat Enterprise Linux 6.5, or later Red Hat Enterprise Linux 7, or later SUSE Linux Enterprise Server 11 SP3, or later SUSE Linux Enterprise Server 12, or later Ubuntu 15.04, or later logical ports assigned to VIOS requires VIOS , or later 4

6 Systems with Support 4/2014 GA 9117-MMD (IBM Power 770 ), 9179-MHD (IBM Power 780), 8412-EAD (IBM Power ESE ) System node PCIe slots 3/2015 GA 9119-MME (IBM Power System E870 ), 9119-MHE (IBM Power System E880 ) System node PCIe slots 6/2015 GA POWER8 scale-out servers, expanded options for Power E870 and E880, and Power E850 PCIe Gen3 I/O expansion drawer. The following POWER8 PCIe slots are capable: All Power E870/E880 and Power E850 system node slots. Slots C6, C7, C10, and C12 of a Power S814 (1S 4U) or S812L (1S 2U) server. Slots C2, C3, C4, C5, C6, C7, C10, and C12 of a S824 or S824L server (2-socket, 4U) with both sockets populated. If only one socket is populated, then C6, C7, C10, and C12. Slots C2, C3, C5, C6, C7, C10, and C12 of a S822 or S822L server (2-socket, 2U) with both sockets populated. If only one socket is populated, then C6, C7, C10, and C12. Slots C1 and C4 of the 6-slot Fan-out Module in a PCIe Gen3 I/O drawer. If system memory is <128GB only slot C1 of a Fan-out Module is capable. 12/2015 GA vnic support for POWER8 scale-out servers, Power E870 and E880, and PCIe Gen3 I/O drawer 5

7 Capable Slots in P8 Scale-out models such as the S824 placement in scale-out models will vary depending on 1 core or 2 core and may have other considerations such as installed memory adapter placement will vary - 2U vs 4U - example below S824 - URL to Knowledge Center 6 For platforms with less than 64GB of total system memory, should not be configured in slots: C2, C4, C10 and C12 as performance may be severely impacted. Two slot positions per Fan-out module are capable See adapter placement rules example below placement for in MEX drawer The capability varies in slots P1-C4 and P2-C4 based on the amount of system memory. If the EMX0 PCIe3 expansion drawer is connected to a system with a total amount of physical memory greater than or equal to 128 GB, slots P1-C4 and P2-C4 are capable. 6

8 2-port 10GbE CNA & 2-port 1GbE FC #EN0J: 10GbE Optical SR, Low Profile FC #EN0H: 10GbE Optical SR, Full High FC #EN0L: 10GbE Active Copper Twinax, Low Profile FC #EN0K: 10GbE Active Copper Twinax, Full High FC #EN0N: 10GbE Optical LR, Low Profile FC #EN0M: 10GbE Optical LR, Full High 20 VFs per port + 4 VFs per port = 48 VFs per adapter Dual 10 GbE SR Dual 1 GbE 1GBASE-T Announced GA EN0H,EN0K support for Power 770/780/ESE system node. April 8, 2014 April 2014 EN0J,EN0L,EN0N support for Power E870/E880 system node. Feb. 24, 2015 March 2015 EN0H,EN0K,EN0M support for other POWER8 systems and PCIe Gen3 I/O expansion drawer April 28, 2015 June

9 New PCIe Gen3 s PCIe3 4-port 10Gb Optical SR or Active Copper twinax #EN15: 10GbE Optical SR, Full Height #EN16: 10GbE Optical SR, Low Profile #EN17: 10GbE Active Copper Twinax, Full Height #EN18: 10GbE Active Copper Twinax, Low Profile 16 VFs per port 4 ports 10GbE CNA = 64 VFs per adapter Announced GA Supported in PCIe Gen3 I/O drawer and the 4U scale-out system units and in the Power E850/E870/E880 system node slots. April 28, 2015 June

10 s with Support s # of logical ports per adapter and per port Low profile - multi OS Full high - multi OS Low profile - Linux only - PowerVM Full high - Linux only - PowerVM PCIe3 4-port (2x10GbE+2x1GbE) SR Optical fiber and RJ45 PCIe3 4-port (2x10GbE+2x1GbE) copper twinax and RJ45 PCIe3 4-port (2x10GbE+2x1GbE) LR Optical fiber and RJ45 PCIe3 4-port 10GbE SR optical fiber PCIe3 4-port 10GbE copper twinax 48 20/20/4/ /20/4/ /20/4/ /16/16/ /16/16/16 #EN0J 1 #EN0H 3 #EL38 #EL56 #EN0L 1 #EN0K 3 #EL3C #EL57 #EN0N #EN0M n/a n/a #EN16 2 #EN15 n/a n/a #EN18 2 #EN17 n/a n/a Notes: 1. announced February 2015 for Power E870/E880 system node. Now available in other POWER8 servers. 2. is only available in Power E870/E880 system node, not 2U server. 3. announced April 2014 for Power 770/780/ESE system node. With April 2015 announce, available in POWER8 servers. 9

11 Power Systems Capable PCIe Slots IBM Power 770 (9117-MMD), IBM Power 780 (9179-MHD), or Power ESE (8412-EAD) Power Systems servers: all PCIe slot within the system units are capable. PCIe slots in the I/O expansion drawers are not capable. POWER8 Systems: consult IBM Knowledge Center for the specific system of interest. In some cases total system memory may determine if a PCIe slots is capable. Power System or I/O Expansion Drawer L, L, or A IBM Knowledge Center PCIe adapter placement rules L A and A E8E 9119-MHE or 9119-MME PCIe Gen3 I/O expansion drawer E8E/p8eab/p8eab_85x_slot_details.htm?cp=8408-E8E%2F MME/p8eab/p8eab_87x_88x_slot_details.htm 10

12 Architecture Internal Switching in conjunction with SEA Partition A Partition B Virtual I/O Server Partition C SEA Virtual Eth Virtual Eth 10 % Virtual hypervisor switch 2x 10 GbE SR 2x 1 GbE copper 10 Gbps - 20 Virtual Functions (VF) 10 Gbps - 20 Virtual Functions (VF) 1 Gbps - 4 Virtual Functions (VF) 1 Gbps - 4 Virtual Functions (VF) 4-port 10GbE CNA & 1GbE 11

13 Flexible Deployment Single partition All adapter resources available to a single partition Device Driver PF LPAR A (Dedicated mode) Device vfunc Driver DevDrv PF OR VF VF LPAR A Virtual Fabric vfunc DevDrv VF VF Multi-partition without VIOS Direct access to adapter features Capacity per logical port Fewer adapters for redundant adapter configurations. VF LPAR B VF VF VF VF VF Virtual Fabric Virtual Fabric SRIOV LPAR A VF VF VF Virtual Function PF Physical Function 12

14 Flexible Deployment Multi-partition thru VIOS s shared by VIOS partitions Fewer adapters for redundancy VIOS client partitions eligible for Live Partition Mobility VIOS LPAR 1 Virtual Virtual VF Virtual LPAR A Virtual VF VF VF VF VF Virtual Fabric Virtual Fabric SRIOV VF Virtual VF VIOS LPAR 2 Virtual Multi-partition mix of VIOS and nonvios For VIOS partition behavior is the same as Multi-partition thru VIOS above Direct access partitions Path length & latency comparable to dedicated adapter Direct access to adapter features Entitled capacity per logical port VIOS LPAR Virtual Virtual VF LPAR C Virtual VF Virtual Fabric VF Virtual LPAR B VF VF = Virtual Function LPAR A 13

15 PowerVM Single Root I/O Virtualization: Fundamentals, Configuration, and Advanced Topics Performance 14

16 Throughput [Gbps] Traditional Virtual Ethernet Performance 2.9 Throughput Virtual Ethernet MTU 1500 Maximum out-of-the-box throughput ~ 2.9 Gbit/s TP 9117-MMB default "TP 8202-E4C default" TP 9117-MMD default TP POWER MME CPU units 15

17 Internal Switching Setup on POWER8 S824 Benchmark with 8 parallel TCP sessions Client LPAR: Power S A AIX 7.1 TL3 SP 3 Capped 4 VPs MTU size: 1500 bytes Server LPAR: Power S A (Same as client) AIX 7.1 TL3 SP 3 EC=3.0 Units, uncapped 4 VPs MTU size: 1500 bytes 16

18 Thoughput [Gbit/s] POWER8 Internal Switching on POWER8 S824 POWER8 provides access to adapter line-speed with less CPU units compared to POWER Throughput internal vs. Virtual Ethernet VENT MTU 1500 VENT MTU 9000 P7+ SRIOV MTU 1500 P8 SRIOV MTU Proc Units 17

19 POWER8 External Switching on POWER8 S824 Throughput internal and external switching MTU 1500 byte 12,00 Thoughput [Gbit/s 10,00 8,00 6,00 4,00 2,00 P8 SRIOV internal P8 SRIOV external 0,00 0,40 0,50 0,60 0,70 0,80 0,90 1,00 1,10 1,20 1,30 1,40 Proc Units 18

20 CPU units consumption: SEA / Throughput: 2.5 Gbit/s 5 Gbit/s 5 Gbit/s 10 Gbit/s CPU Consumption 5.8 VIOS Rx VIOS Tx 1.65 Server 1.25 VIOS Rx 1.0 VIOS Tx 0.85 Server 0.81 Client 1.23 Server 0.59 Client 0.37 Server 0.43 Client 0.40 Client 0.80 E870 SEA default E870 SEA + LSO (large send offload) S824 S824 19

21 Case Study: Transaction Rate Performance Customer migrated SAP from POWER7 to POWER8. Expectation for SAP ERP was high network transaction rate performance. Issue: Benchmarks on POWER7 with Virtual Ethernet s showed limited TPS performance for small packets. Better results were expected from new POWER8 systems. Customer additionally evaluated for transaction rate tuning. Systems: Power (old) / Power8 E870 (new) 20

22 Transaction Rate Performance SAP ERP sizing for high network transaction rates. Packet size of 700 bytes assumed. Systems: Power8 E870 / Power

23 & vnic Capacity Capacity Controls adapter and system resource levels, including desired minimum bandwidth Desired minimum percent of physical port resources Unallocated or unused bandwidth is available for logical ports with more demand than their minimum Actual consumable bandwidth is an approximation Configured logical ports reserve a small amount of bandwidth even when they don t have demand LSO traffic on a logical port with a small Capacity value may result in overshoot of the minimum bandwidth This is especially visible for 1Gbs links 23

24 & vnic Capacity Capacity value can not be changed dynamically To change a capacity value for a logical port: dynamic remove the logical port, then dynamic add a logical port with new capacity value update profile with new value and activate profile It may be desirable to leave some capacity for new logical ports. Capacity setting must be a multiple of the default (2%). 24

25 & vnic Desired Bandwidth ports is using physical port exclusively 10 seconds competitive situation with other ports port with 2% capacity 26

26 PowerVM Single Root I/O Virtualization: Fundamentals, Configuration, and Advanced Topics vnic and Live Partition Mobility 27

27 Live Partition Mobility Options with Virtual Network Interface Controller (vnic) vnic is new virtual adapter type vnic leverages to provide a performance optimized virtual NIC solution vnic enables advanced virtualization features such as live partition mobility with adapter sharing Leverages Capacity value resource provisioning (e.g. minimum bandwidth) December 2015 GA for AIX & IBM i Linux in progress Each Distro will certify Power E850 support GA on 3/4/2016 with firmware SP Pre-req AIX 7.1 TL4 or later or AIX 7.2 or later IBM i 7.1 TR10 or later or 7.2 TR3 or later VIOS 2.2.4, or later Firmware 840 or later HMC V8R8.4.0 or later 28

28 vnic Architecture Virtual I/O Server Client Partition Data Buffer vnic Server vnic Client 30 % Hypervisor 2x 10 GbE SR 2x 1 GbE copper 4-port 10GbE CNA/FCoE & 1GbE 29

29 vnic Architecture Virtual I/O Server Client Partition Data Buffer Partition Mobility vnic Server vnic Client 30 % Hypervisor 2x 10 GbE SR 2x 1 GbE copper 4-port 10GbE CNA/FCoE & 1GbE 30

30 Comparison of Virtual Enet & vnic VIOS LPAR 1 VIOS LPAR 2 SEA Virtual LPAR A Data Virtual LPAR B Data Virtual Control flow Data flow vnic Server vnic Server LPAR A Data vnic Client LPAR B Data vnic Client vswitch Power Hypervisor Power Hypervisor VF VF VF Virtual Ethernet/SEA (current) Multiple copies of data VF Many-to-one relationship between virtual adapters and physical adapter QoS based on VLAN tag PCP bits (i.e. 8 traffic classes) VF VF VF VF vnic with advanced virtualization features (e.g. LPM) Improved performance Eliminates data copies Optimized control flow, no overhead from the vswitch or SEA Multiple queue support Efficient Lower CPU and Memory usage (no data copy) Leverages adapter offload for LPAR to LPAR communication Deterministic QoS One-to-one relationship between vnic client adapter and logical port Extends logical port QoS 31

31 Live Partition Mobilty/Remote Restart with vnic Target system: Must support vnic Must have at least one running adapter in shared mode Must to have at least one running VIOS that supports vnic Target physical port User can select any physical port on the target for each vnic on the LPAR to be migrated If physical port not selected, the HMC will map physical port by port label and port switch mode (VEB/VEPA). Empty string is a valid label. Target physical port must have sufficient available capacity and available logical port count 32

32 Miscellaneous vnic Notes vnic backing device logical ports owned by VIOSs will not be captured in the system templates vnic backing device logical ports will not be in the VIOS profile even when sync last activated profile with current configuration is on. vnic backing device logical ports can not be modified. Deleting a vnic backing device logical port is blocked by HMC unless its associated vnic is already gone Activating a client partition profile with vnics requires the specified hosting VIOSs to be in running state with RMC connection 33

33 THROUGHPUT [GBPS] vnic / : Throughput / CPU Comparison VNIC / NATIVE THROUGHPUT EXTERNAL SWITCHING vnic 4% difference in max. TP CPU UNITS 34

34 Total CPU Consumption: SEA / Throughput: 2.5 Gbit/s 5 Gbit/s 5 Gbit/s 10 Gbit/s 5 Gbit/s 10 Gbit/s CPU Consumption 5.8 VIOS Rx VIOS Tx 1.65 Server 1.25 Client 1.23 E870 SEA default VIOS Rx 1.0 VIOS Tx 0.85 Server 0.59 Client 0.37 E870 SEA + LSO (large segment offload) Server 0.43 Client 0.40 S824 Server 0.81 Client 0.80 S824 VIOS Rx 0.89 VIOS Tx 0.48 Server 0.7 Client 0.37 E870 vnic VIOS Rx 1.47 VIOS Tx 1.00 Server 1.54 Client 0.92 E870 vnic 36

35 vnic / : TPS Maximum w. Small Packets VNIC / NATIVE TPS EXTERNAL SWITCHING SEA 1 Gigabit SEA 10 Gigabit vnic 10 Gigabit 10 Gigabit 38

36 Live Partition Mobility Options with Virtual Ethernet configuration VIOS LPAR 1 VIOS LPAR 2 Use current Virtual Ethernet support with logical ports as Shared Ethernet (SEA) physical connections to the network SEA Virtual LPAR A Virtual LPAR B Virtual Virtual SEA Does not receive performance benefits provided with Direct Access vswitch Power Hypervisor No client partition resource allocation (e.g. desired bandwidth) Benefits: LPM Capability /port sharing to reduce number of adapters VF VF VF VF VF VF Virtual Fabric Virtual Fabric SRIOV VF VF 39

37 Live Partition Mobility Options with Active-backup configuration Configure logical port as Active connection and Virtual Ethernet adapter or vnic client virtual adapter as backup Prior to migration, use dynamic LPAR operation to remove SR- IOV logical port VIOS LPAR Virtual Virtual LPAR Virtual Ethernet becomes Active connection Backup Active Migrate the partition On target system, configure logical port as Active connection Option for AIX and Linux Physical I/O can not be assigned (even temporarily) to an IBM i LPM capable partitions VF VF Virtual Fabric VF VF Normal Active-Backup configuration 40

38 PowerVM Single Root I/O Virtualization: Fundamentals, Configuration, and Advanced Topics Fault-Tolerant Configurations 41

39 Link Aggregation AIX EtherChannel or Static Link Aggregation IEEE 802.3ad/802.1ax Link Aggregation Control Protocol (LACP) Network Interface Backup (NIB) IBM i EtherChannel or Static Link Aggregation IEEE 802.3ad/802.1ax Link Aggregation Control Protocol (LACP) Virtual IP Address (VIPA) Linux Several Bonding/port trunking modes including LACP and Active-Backup 42

40 Link Aggregation Using LACP Issue LPAR A LPAR B Link Aggregation (LACP) will not function properly with multiple logical ports using the same physical port Switch expects a single partner (MAC physical layer) on a link Multiple logical ports on the same physical port creates multiple partners on the link VF VF VF VF VF VF Virtual Fabric Virtual Fabric Switch SRIOV VF VF Invalid Link Aggregation configuration. Two logical ports assigned to one physical port. VF = Virtual Function 43

41 EtherChannel or Static Link Aggregation Issue LPAR A LPAR B logical ports may go down while the physical link remains up Switch port failover occurs when physical link goes down Switch does not recognize a logical port going down and will continue to send traffic on the physical port Etherchannel is not recommended for an configuration Failed logical port EtherChannel VF VF VF VF VF VF Virtual Fabric Virtual Fabric SRIOV Switch VF VF Not recommended Switch will not detect if logical link fails 44

42 Link Aggregation Recommendations If you require bandwidth greater than a single link s bandwidth with link failover? Use Link Aggregation (LACP) with one logical port per physical port. Provides greater bandwidth than a single link with failover Other adapter ports may be shared or used in a LACP configuration by other partitions Best Practice Assign 100% capacity to each logical port in the Link Aggregation Group to prevent accidental assignment of another SR- IOV logical port to the same physical port VF LPAR A Virtual Fabric VF VF VF VF VF Virtual Fabric SRIOV LPAR B VF VF Link aggregation with one logical port assigned to each physical port. Link Agg. (LACP) 45

43 Link Aggregation Recommendations If you require bandwidth less than a single link s bandwidth and failover? Use Active-Backup approach (e.g. AIX NIB, IBM i VIPA, or Linux bonding driver active-backup) For Linux the fail_over_mac parameter must be set to active or 1 or follow or 2. Allows sharing of the physical port by multiple partitions When configured in an active-backup configuration, you should configure the capability to detect when to failover On AIX, configure a backup adapter and an IP address to ping. On IBM i with VIPA, options for detecting network failures besides link failures include Routing Information Protocol (RIP), Open Shortest Path First (OSPF) or customer monitor script. On Linux, use the bonding support to configure monitoring to detect network failures. VF LPAR A Virtual Fabric VF VF VF VF VF Virtual Fabric SRIOV LPAR B VF VF Active backup configuration (no switch configuration required) allows sharing of physical port 46

44 PowerVM Single Root I/O Virtualization: Fundamentals, Configuration, and Advanced Topics vnic Failover 53

45 vnic Failover vnic with vnic server side redundancy (analogous to SEA failover) Multiple backing devices (up to 6) for per vnic client One active, others as inactive standby Backing device configuration includes selection of VIOS physical port Failover priority Capacity Provides flexible deployment and load balancing options Health check of active and inactive backing devices Hypervisor manages failover based on operational state and failover priority Dynamic add and remove of backing devices 54

46 vnic Failover Architecture Virtual I/O Server Virtual I/O Server Virtual I/O Server Client Partition Standby Failover Priority 100 Standby Failover Priority 50 Active Failover Priority 1 Data Buffers vnic Server vnic Server vnic Server vnic Client Hypervisor vnic failover configuration Up to 6 backing devices per vnic client Select VIOS & adapter physical port for each backing device Set Failover priority for each backing device Auto Priority Failover: Enabled or Disabled 56

47 vnic Failover Architecture Virtual I/O Server Virtual I/O Server Virtual I/O Server Standby Active Not Operational Failover Priority 100 Failover Priority 50 Failover Priority 1 Client Partition Data Buffers vnic Server vnic Server vnic Server vnic Client Hypervisor vnic failover configuration Up to 6 backing devices per vnic client Select VIOS & adapter physical port for each backing device Set Failover priority for each backing device Auto Priority Failover: Enabled or Disabled Failover triggers No VIOS heartbeat 57

48 vnic Failover Architecture Virtual I/O Server Active Failover Priority 100 Virtual I/O Server Link Down Failover Priority 50 Virtual I/O Server Not Operational Failover Priority 1 Client Partition Data Buffers vnic Server vnic Server vnic Server vnic Client Hypervisor vnic failover configuration Up to 6 backing devices per vnic client Select VIOS & adapter physical port for each backing device Set Failover priority for each backing device Auto Priority Failover: Enabled or Disabled Failover triggers No VIOS heartbeat or adapter link failures 58

49 vnic Failover Architecture Virtual I/O Server Standby Failover Priority 100 Virtual I/O Server Link Down Failover Priority 50 Virtual I/O Server Active Failover Priority 1 Client Partition Data Buffers vnic Server vnic Server vnic Server vnic Client Hypervisor vnic failover configuration Up to 6 backing devices per vnic client Select VIOS & adapter physical port for each backing device Set Failover priority for each backing device Auto Priority Failover: Enabled or Disabled Failover triggers No VIOS heartbeat or adapter link failures Auto Priority Failover if Enabled 59

50 vnic Failover Architecture Virtual I/O Server Virtual I/O Server Virtual I/O Server Active Link Down Standby Failover Priority 100 Failover Priority 50 Failover Priority 1 Client Partition Data Buffers vnic Server vnic Server vnic Server vnic Client Hypervisor vnic failover configuration Up to 6 backing devices per vnic client Select VIOS & adapter physical port for each backing device Set Failover priority for each backing device Auto Priority Failover: Enabled or Disabled Failover triggers No VIOS heartbeat or adapter link failures Auto Priority Failover if Enabled HMC user initiated (Sets APF to disabled) 60

51 vnic Failover Configuration Partition Properties->Virtual NICS New Virtual NICs interface, includes support for multiple backing devices To Add vnic Client click on Add Virtual NIC 61

52 vnic Failover Configuration Add vnic Select the desired physical port 62

53 vnic Failover Configuration Add vnic Physical port information updated Select Hosting Partition (VIOS) Optionally set Capacity % Optionally set Failover Priority lower # is more favored 63

54 vnic Failover Configuration Add vnic Click on the Advanced Virtual NIC Settings for additional options 64

55 vnic Failover Configuration Additional backing devices Create additional backing devices per vnic client Up to 6 backing devices total Click the Add Entry button to add new backing device 65

56 vnic Failover Configuration Add Entry New row for new backing device information 66

57 vnic Failover Configuration Add Entry Select Physical, Hosting Partition (VIOS), Capacity value, Failover Priority Physical port selection list limited to previously unselected physical ports Capacity % may be different for each backing device 67

58 vnic Failover Configuration Add Entry Select vnic Allow Auto Priority Failover option Enabled: Always failover to more favored (lower Failover Priority number) backing device Disabled: Only failover when current backing device is not operational Click OK to create vnic with backing devices 68

59 vnic Failover Configuration vnic list Device Name indicates device name in client partition vnic Auto Prioirty Failover state selected Other columns associated with backing devices Backing Device State indicates operational state of backing devices and which one is the Active backing device 69

60 vnic Failover Configuration Modify Backing Devices Click on the Action button for a list of Actions you can perform on the vnic Click on Modify Backing Device 70

61 vnic Failover Configuration Modify Backing Device Modify the vnic Auto Priority Failover options Click on Add Backing Devices button to configure additional backing devices 71

62 vnic Failover Configuration Add vnic Backing Devices Select a physical port, VIOS, Capacity %, and Failover Priority 72

63 vnic Failover Configuration Add Backing Device Click Add entry to configure additional backing devices Click OK when done 73

64 vnic Failover Configuration Live Partition Migration Partition Migration wizard provides a proposed mapping from Source Backing Device to Destination Backing Device. The Destination Backing Device for a Source Backing Device can be modified. 74

65 vnic Failover Configuration Live Partition Migration The modify Destination Backing Device allows for the following Change the physical adapter Change the physical port Change the Host VIOS Change the Capacity(%) 75

66 vnic Failover Target 4Q2016 GA (except E850 (8408-E8E)) E850 (8408-E8E) GA 1H2017 Pre-req (current targets) AIX 7.1 TL4 or later or AIX 7.2 or later IBM i 7.1 TR11 or later or 7.2 TR3 or later Linux not supported at GA. Phased implementation via Linux community in 2017 VIOS , or later Firmware FW or later HMC V8R8.6.0 with mandatory PTF or later 76

67 PowerVM Network Virtualization Comparison Technology Live Partition Mobility Quality of service (QoS) Direct access perf. Link Aggregation Server Side Failover Requires VIOS No 1 Yes Yes Yes 2 No No vnic Yes Yes No 3 Yes 2 vnic failover 4 Yes SEA/vEth Yes No No Yes SEA failover Yes Notes: 1. can optionally be combined with VIOS and virtual Ethernet to use higher-level virtualization functions like Live Partition Mobility (LPM); however, client partition will not receive the performance or QoS benefit. 2. Some limitations apply. For and vnic see link aggregation support 3. Generally better performance and requires fewer system resources when compared to SEA/virtual Ethernet 4. Available 2H

68 PowerVM Single Root I/O Virtualization: Fundamentals, Configuration, and Advanced Topics Performance Monitor & Topology 78

69 Performance Monitor for From the performance monitor screen click Network Utilization Trend -> More Graphs -> adapters The breakdown by physical ports shows how heavily utilized a physical port is and can be used to determine whether or not there is additional bandwidth available 79

70 Performance Monitor for The breakdown by partitions shows each logical port individually and which LPAR owns it This can be used to determine which logical ports are using the physical ports bandwidth 80

71 Physical and Counters Physical and logical port counters available via the HMC GUI or CLI 81

72 New HMC GUI: and vnic Diagram 82

73 New HMC GUI: and vnic Diagram 83

74 New HMC GUI: and vnic Diagram 88

75 New HMC GUI: and vnic Diagram 89

76 PowerVM Single Root I/O Virtualization: Fundamentals, Configuration, and Advanced Topics Maintenance 90

77 Firmware Update There are 2 pieces of firmware for that are built into system firmware driver firmware The driver code that configures the adapter and logical ports firmware The firmware that runs on the adapter Both levels of firmware are automatically updated to the levels included in the active system firmware in the following cases System boot/reboot transitioned into mode level concurrent maintenance The level description will indicate if a new version of adapter firmware is included 91

78 Firmware Update When system firmware is updated concurrently, the levels on currently configured adapters are not automatically updated Updating the levels will cause a temporary network outage on the logical ports on the affected adapter Starting with system firmware level FW830, the firmware levels can be viewed and updated using the HMC GUI On the HMC enhanced+ GUI select the Server -> Actions -> Firmware Update 92

79 Firmware Update Each adapter in mode will display the active adapter driver and adapter levels. If an update is available the update available column will indicate Yes Select the set of adapters to update, then right click to launch the context menu Update Driver Firmware - will update only the adapter driver code and will not update the adapter firmware. This type of update results in a shorter network outage on logical ports If only the adapter driver firmware is updated then another update may still be available for the adapter firmware Update Driver and Firmware - will update both levels s are updated serially to ensure that both devices in a multipath setup are not affected at the same time. This means that the totally time to update can be significant if a lot of adapters are selected. 93

80 Physical Replacement adapters can be added, removed, and replaced without disrupting the system or shutting down the partitions. For adapter replacement, all the logical ports must be de-configured. The HMC provides a GUI for adapter concurrent maintenance operations. (Serviceability Hardware MES Tasks Exchange FRU) New adapter must have the same capabilities (same type/feature code). When new adapter is plugged into the same slot as the original adapter, the hypervisor will automatically associate the old adapter s configuration with the new adapter. If the new adapter is plugged in to a different slot, the chhwres command is needed to associate the original adapter configuration with the new adapter. $ chhwres -m Server1 -r sriov rsubtype adapter -o m -a \ slot_id= b,target_slot_id=

81 Known / vnic Issues AIX NIB configuration issue AIX APARs: IV77944, IV80034, IV80127, IV82254, IV82479 IBM i logical port / vnic VLAN restrictions issue V7R1 Resolution is not required as OS generated VLAN tags are not supported in V7R1 V7R2 For apply PTFs MF62338, MF62348, MF62349; For vnic PTF MF62676 V7R3 For apply PTFs MF62340, MF62350, MF62351; For vnic PTF MF62703 PVID issue FW or later FW830.xx (target availability 12/2016) Transmit hang issue FW or later FW or later FW or later 95

82 Links and Additional Information IBM Power Systems : Technical Overview and Introduction Redpaper Linkedin PowerVM group FAQs blog and wiki troduction%20to%20%20faqs R-IOV%20Frequently%20Asked%20Questions vnic FAQs blog and wiki troduction%20to%20vnic%20faqs NIC%20Frequently%20Asked%20Questions Introducing New PowerVM Virtual Networking Technology Knowledge center links for firmware update on prior releases 01.ibm.com/support/knowledgecenter/POWER7/p7hb1/p7hb1_updating_sriov_firmware.htm?cp=POWER7 %2F Our contact info Allyn Walsh awalsh@us.ibm.com or Chuck Graham csg@us.ibm.com 96