Port Channel for Cisco Cloud Services Platform (CSP)

Transcription

1 Page 1 of 10

2 Port Channel for Cisco Cloud Services Platform (CSP) 2100 view online To improve available network bandwidth and fault tolerance, multiple virtual function (VF) network interfaces assigned to Avi Service Engines can be aggregated to form port channels or bonded interfaces. ## Topology ## Prerequisites * No port channel specific configuration is needed on the switch. * To achieve the intended benefit, port channel member interfaces should be associated with different physical function (PF) devices or pnics. * CSP configuration SRIOV should be enabled on CSP pnics being used. The following CSP command could be used for this. avinet-1(config)# pnic enp7s0f0 sr-iov numvfs 32 switchmode vep Configuration The member (slave) interfaces to be bonded together to form a port channel (bond) can be configured using Avi SE metadata (yml) file as follows. Copyright 2018 Avi Networks, Inc. Page 2 of 10

3 avi.bond-ifs.se: '1,2' The above configuration directs the Service Engine to bond together interfaces 1 and 2 to form a port channel. If multiple port channels are required, the indices of member interfaces for different port channels should be specified with space in-between as shown below. avi.bond-ifs.se: '1,2 3,4' The above configuration makes the SE create two port channels - one with interfaces 1 and 2, and the other with interfaces 3 and 4. Once the SE is up, the Edit Service Engine page in the Avi UI shows information about bonded and slave interfaces. Run the CLI command show serviceengine <SE_IP_Address> interfaceto view the interface configuration. [admin: ]: > show serviceengine interface Field Value se_uuid :se-bb2c9806-fe17-4c66-9fb0-dd5b89a456f8 proc_id PROC_Aggregate Copyright 2018 Avi Networks, Inc. Page 3 of 10

4 vnics[1] vnic_name eth2 vnic_id 4 mac_address 02:5a:80:02:11:0c interface_stats ipackets ibytes ierrors 0 rx_nombuf 0 opackets 0 obytes 0 oerrors 0 rx_kni 0 tx_kni 2175 tx_kni_errs 0 ip_checksum_drops 0 l4_checksum_drops 0 tx_queue_full_retries 0 rx_pkt_iterations rx_queue_full 0 rx_max_single_burst 22 rx_mim_etype_p2s 0 tx_mim_etype_p2s 0 tx_mim_frags_etype_p2s 0 rx_mim_etype_s2p 0 tx_mim_etype_s2p 0 tx_mim_frags_etype_s2p 0 tx_frags_p2s 0 local_flow_probes_req_sent 0 local_flow_probes_req_received 0 flow_probes_ignored_same_vnic 0 flow_probes_ignored_in_tw 0 vnic_parent 4 vnic_weight -1 vlan_id 0 num_vs_delete_drops 0 vrf_id 1 vrf_ref global vnic_mtu 1500 interface_disabled False linux_intf_name eth2 vnic_owner 0 vnics[2] vnic_name eth1 vnic_id 5 mac_address 02:4a:80:02:11:0c interface_stats ipackets ibytes ierrors 0 rx_nombuf 0 Copyright 2018 Avi Networks, Inc. Page 4 of 10

5 opackets obytes oerrors 0 rx_kni 0 tx_kni 2175 tx_kni_errs 0 ip_checksum_drops 0 l4_checksum_drops 0 tx_queue_full_retries 0 rx_pkt_iterations rx_queue_full 0 rx_max_single_burst 22 rx_mim_etype_p2s 0 tx_mim_etype_p2s 0 tx_mim_frags_etype_p2s 0 rx_mim_etype_s2p 0 tx_mim_etype_s2p 0 tx_mim_frags_etype_s2p 0 tx_frags_p2s 0 local_flow_probes_req_sent 0 local_flow_probes_req_received 0 flow_probes_ignored_same_vnic 0 flow_probes_ignored_in_tw 0 vnic_parent 5 vnic_weight -1 vlan_id 0 num_vs_delete_drops 0 vrf_id 1 vrf_ref global vnic_mtu 1500 interface_disabled False linux_intf_name eth1 vnic_owner 0 vnics[3] vnic_name bond0 vnic_id 32 mac_address 02:5a:80:02:11:0c ip_info[1] ip_addr net_mask ip_info[2] ip_addr net_mask ip_info[3] ip_addr net_mask interface_stats ipackets ibytes ierrors 0 rx_nombuf 0 Copyright 2018 Avi Networks, Inc. Page 5 of 10

6 opackets obytes oerrors 0 rx_kni 0 tx_kni 0 tx_kni_errs 0 ip_checksum_drops 0 l4_checksum_drops 0 tx_queue_full_retries 0 rx_pkt_iterations 0 rx_queue_full 0 rx_max_single_burst 0 rx_mim_etype_p2s 0 tx_mim_etype_p2s 0 tx_mim_frags_etype_p2s 0 rx_mim_etype_s2p 0 tx_mim_etype_s2p 0 tx_mim_frags_etype_s2p 0 tx_frags_p2s 0 local_flow_probes_req_sent 0 local_flow_probes_req_received 0 flow_probes_ignored_same_vnic 0 flow_probes_ignored_in_tw 0 vnic_parent 32 vnic_weight -1 vlan_id 0 num_vs_delete_drops 0 vrf_id 1 vrf_ref global vnic_mtu 1500 mbr_intfs[1] if_name eth2 linux_name eth2 active True mbr_intfs[2] if_name eth1 linux_name eth1 active True interface_disabled False linux_intf_name bond0 vnic_owner Troubleshooting 1. Linux interface configuration After port channel has been successfully configured, the Linux network interface configuration file (/etc/network /interfaces) includes information about bonded and slave interfaces as shown in the example below. Copyright 2018 Avi Networks, Inc. Page 6 of 10

7 When the Avi SE metadata file has avi.bond-ifs.se: '1,2', the SE makes Linux form a bonding interface (bond0) with interfaces eth1 and eth2 as slaves. The following configuration features in the interface configuration file ( /etc/network /interfaces) in the default namespace. auto eth1 iface eth1 inet manual bond-master bond0 auto eth2 iface eth2 inet manual bond-master bond0 auto bond0 iface bond0 inet manual bond-mode balance-xor bond-miimon 100 bond-slaves eth1 eth2 2. Interface configuration inside Avi Vantage namespace Inside the namespace (avi_ns1), ip addr gives the following information with the bonded and slave interfaces using the same MAC address. avi_bond0: <broadcast,multicast,master,up,lower_up> mtu 1500 qdisc noqueue state UNKNOWN group default link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff inet YY.YY.YY.YY/ <broadcast,multicast,slave,up,lower_up> mtu 1500 qdisc pfifo_fast master avi_bond0 state UP group default qlen 1000 link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff: <broadcast,multicast,slave,up,lower_up> mtu 1500 qdisc pfifo_fast master avi_bond0 state UP group default qlen 1000 link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff </broadcast,multicast,slave,up,lower_up> </broadcast,multicast,slave,up,lower_up> </broadcast,multicast,master,up,lower_up> 3. Bond configuration inside Avi Vantage namespace Inside the namespace (avi_ns1), /proc/net/bonding/avi_bond0 gives the information about bond configuration. Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: load balancing (xor) Transmit Hash Policy: layer3+4 (1) MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Copyright 2018 Avi Networks, Inc. Page 7 of 10

8 Slave Interface: avi_eth2 MII Status: up Speed: Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 02:5a:80:02:11:0c Slave queue ID: 0 Slave Interface: avi_eth1 MII Status: up Speed: Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 02:4a:80:02:11:0c Slave queue ID: 0 MII status shows the physical link status of the interfaces. Starting with release , bond mode can be balance-rr, active-backup, or balance-xor. Previously, only xor was supported. Example: avi.bond-mode.se: 'active-backup' 4. Statistics This would set the port-channel mode to active-backup. Notes: 1. balance-xor is the default port-channel mode. 2. If active-backup is used as the port-channel mode, then the first member vnic is chosen as the primary link. Permanent hardware addresses of the slaves are the original MAC addresses and should be different for each of the slaves. Statistics of interfaces in the output of show serviceengine <SE_IP_Address> interface can be used to see if traffic is being distributed fairly among the slave interfaces. A balanced distribution results in comparable statistics for all slave interfaces if all of them were active together during their lifetime. 5. CSP pnic and SRIOV Configuration The following commands in CSP shell help identify errors in pnic and SRIOV configuration in CSP. SRIOV status on pnics helps make sure that SRIOV is on for pnics of interest. avinet-1# show running-config pnic sr-iov pnic enp7s0f0 sr-iov numvfs 32 sr-iov switchmode veb! pnic enp7s0f1 Copyright 2018 Avi Networks, Inc. Page 8 of 10

9 sr-iov numvfs 32 sr-iov switchmode veb! avinet-1# show pnic enp7s0f0 passthrough passthrough sriov avinet-1# show pnic enp7s0f0 sriov_intf sriov_intf sriov_pf View the virtual function usage on a pnic as shown below. avinet-1# show pnic enp7s0f0 vfnics VF INUSE SERVICE IDX NAME FLAG OWNER enp7s16 in-use none 1 enp7s16f2 in-use none 2 enp7s16f4 in-use none 3 enp7s16f6 in-use none 4 enp7s17 in-use none 5 enp7s17f2 in-use none 6 enp7s17f4 in-use none 7 enp7s17f6 not-used none 8 enp7s18 not-used none 9 enp7s18f2 not-used none 10 enp7s18f4 not-used none 11 enp7s18f6 not-used none 12 enp7s19 not-used none 13 enp7s19f2 not-used none 14 enp7s19f4 not-used none 15 enp7s19f6 not-used none 16 enp7s20 not-used none 17 enp7s20f2 not-used none 18 enp7s20f4 not-used none 19 enp7s20f6 not-used none 20 enp7s21 not-used none 21 enp7s21f2 not-used none 22 enp7s21f4 not-used none 23 enp7s21f6 not-used none 24 enp7s22 not-used none 25 enp7s22f2 not-used none 26 enp7s22f4 not-used none 27 enp7s22f6 not-used none 28 enp7s23 not-used none 29 enp7s23f2 not-used none 30 enp7s23f4 not-used none 31 enp7s23f6 not-used none Copyright 2018 Avi Networks, Inc. Page 9 of 10

10 Reference and Related Aricles Port Channeling on Linux Server Hosts Installing Avi Vantage for CSP 2100 Understanding NIC Bonding with Linux Ubuntu Bonding Copyright 2018 Avi Networks, Inc. Page 10 of 10