Port Channel for Cisco Cloud Services Platform (CSP)

Transcription

1 Page 1 of 8

2 Port Channel for Cisco Cloud Services Platform (CSP) 2100 view online To improve available network bandwidth and fault tolerance, multiple virtual function (VF) network interfaces assigned to Avi Service Engines can be aggregated to form port channels or bonded interfaces. ## Topology ## Prerequisites * No port channel specific configuration is needed on the switch. * To get the intended benefit, port channel member interfaces should be associated with different physical function (PF) devices or pnics. * CSP configuration SRIOV should be enabled on CSP pnics being used. The following CSP command could be used for this. avinet-1(config)# pnic enp7s0f0 sr-iov numvfs 32 switchmode vep Configuration The member (slave) interfaces to be bonded together to form a port channel (bond) could be configured using Avi SE metadata (yml) file as follows. Copyright 2018 Avi Networks, Inc. Page 2 of 8

3 avi.bond-ifs.se: '1,2' The above configuration directs the Service Engine to bond together interfaces 1 and 2 to form a port channel. If multiple port channels are required, the indices of member interfaces for different port channels should be specified with space in-between as shown below. avi.bond-ifs.se: '1,2 3,4' The above configuration makes the SE create two port channels - one with interfaces 1 and 2, and the other with interfaces 3 and 4. Once the SE is up, the?edit Service Engine? page in the Controller shows information about bonded and slave interfaces. Run the CLI command show serviceengine <se_ip_address> interface </se_ip_address> to view the interface configuration. [admin: ]: > show serviceengine interface Field Value se_uuid :se-bb2c9806-fe17-4c66-9fb0-dd5b89a456f8 proc_id PROC_Aggregate Copyright 2018 Avi Networks, Inc. Page 3 of 8

4 vnics[1] vnic_name eth2 vnic_id 4 mac_address 02:5a:80:02:11:0c interface_stats ipackets ibytes ierrors 0 rx_nombuf 0 opackets 0 obytes 0 oerrors 0 rx_kni 0 tx_kni 2175 tx_kni_errs 0 ip_checksum_drops 0 l4_checksum_drops 0 tx_queue_full_retries 0 rx_pkt_iterations rx_queue_full 0 rx_max_single_burst 22 rx_mim_etype_p2s 0 tx_mim_etype_p2s 0 tx_mim_frags_etype_p2s 0 rx_mim_etype_s2p 0 tx_mim_etype_s2p 0 tx_mim_frags_etype_s2p 0 tx_frags_p2s 0 local_flow_probes_req_sent 0 local_flow_probes_req_received 0 flow_probes_ignored_same_vnic 0 flow_probes_ignored_in_tw 0 vnic_parent 4 vnic_weight -1 vlan_id 0 num_vs_delete_drops 0 vrf_id 1 vrf_ref global vnic_mtu 1500 interface_disabled False linux_intf_name eth2 vnic_owner 0 vnics[2] vnic_name eth1 vnic_id 5 mac_address 02:4a:80:02:11:0c interface_stats ipackets ibytes ierrors 0 rx_nombuf 0 Copyright 2018 Avi Networks, Inc. Page 4 of 8

5 opackets obytes oerrors 0 rx_kni 0 tx_kni 2175 tx_kni_errs 0 ip_checksum_drops 0 l4_checksum_drops 0 tx_queue_full_retries 0 rx_pkt_iterations rx_queue_full 0 rx_max_single_burst 22 rx_mim_etype_p2s 0 tx_mim_etype_p2s 0 tx_mim_frags_etype_p2s 0 rx_mim_etype_s2p 0 tx_mim_etype_s2p 0 tx_mim_frags_etype_s2p 0 tx_frags_p2s 0 local_flow_probes_req_sent 0 local_flow_probes_req_received 0 flow_probes_ignored_same_vnic 0 flow_probes_ignored_in_tw 0 vnic_parent 5 vnic_weight -1 vlan_id 0 num_vs_delete_drops 0 vrf_id 1 vrf_ref global vnic_mtu 1500 interface_disabled False linux_intf_name eth1 vnic_owner 0 vnics[3] vnic_name bond0 vnic_id 32 mac_address 02:5a:80:02:11:0c ip_info[1] ip_addr net_mask ip_info[2] ip_addr net_mask ip_info[3] ip_addr net_mask interface_stats ipackets ibytes ierrors 0 rx_nombuf 0 Copyright 2018 Avi Networks, Inc. Page 5 of 8

6 opackets obytes oerrors 0 rx_kni 0 tx_kni 0 tx_kni_errs 0 ip_checksum_drops 0 l4_checksum_drops 0 tx_queue_full_retries 0 rx_pkt_iterations 0 rx_queue_full 0 rx_max_single_burst 0 rx_mim_etype_p2s 0 tx_mim_etype_p2s 0 tx_mim_frags_etype_p2s 0 rx_mim_etype_s2p 0 tx_mim_etype_s2p 0 tx_mim_frags_etype_s2p 0 tx_frags_p2s 0 local_flow_probes_req_sent 0 local_flow_probes_req_received 0 flow_probes_ignored_same_vnic 0 flow_probes_ignored_in_tw 0 vnic_parent 32 vnic_weight -1 vlan_id 0 num_vs_delete_drops 0 vrf_id 1 vrf_ref global vnic_mtu 1500 mbr_intfs[1] if_name eth2 linux_name eth2 active True mbr_intfs[2] if_name eth1 linux_name eth1 active True interface_disabled False linux_intf_name bond0 vnic_owner <code></code> <code> <h2 id="troubleshooting">troubleshooting</h2> <h3 id="linux-interface-configuration" >1. Linux interface configuration</h3> <p>after port channel has been successfully configured, the Linux network interface configuration file (<code>/etc/network/interfaces< /code>) includes information about bonded and slave interfaces as shown in the example below.</p> When the Avi SE metadata file has <code>avi.bond-ifs.se:?1,2?</code>, the SE makes Linux form a bonding interface (bond0) with interfaces eth1 and eth2 as slaves. The Copyright 2018 Avi Networks, Inc. Page 6 of 8

7 following configuration features in the interface configuration file (<code>/etc/network /interfaces</code>) in the default namespace. <pre> auto eth1 iface eth1 inet manual bondmaster bond0 auto eth2 iface eth2 inet manual bond-master bond0 auto bond0 iface bond0 inet manual bond-mode balance-xor bond-miimon 100 bond-slaves eth1 eth2 </pre> <h3 id=" interface-configuration-inside-avi-vantage-namespace">2. Interface configuration inside Avi Vantage namespace</h3> Inside the namespace (avi_ns1), <code>ip addr</code> gives the following information with the bonded and slave interfaces using the same MAC address. <pre> avi_bond0: <broadcast,multicast,master,up,lower_up> mtu 1500 qdisc noqueue state UNKNOWN group default link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff inet YY.YY.YY.YY /ZZ scope global avi_bond0 valid_lft forever preferred_lft forever avi_eth2: <broadcast, multicast,slave,up,lower_up> mtu 1500 qdisc pfifo_fast master avi_bond0 state UP group default qlen 1000 link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff avi_eth1: <broadcast, multicast,slave,up,lower_up> mtu 1500 qdisc pfifo_fast master avi_bond0 state UP group default qlen 1000 link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff </broadcast,multicast, slave,up,lower_up> </broadcast,multicast,slave,up,lower_up> </broadcast,multicast,master, up,lower_up></pre> <h3 id="bond-configuration-inside-ave-vantage-namespace">3. Bond configuration inside Ave Vantage namespace</h3> Inside the namespace (avi_ns1), <code>/proc /net/bonding/avi_bond0</code> gives the information about bond configuration. <pre> Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: load balancing (xor) Transmit Hash Policy: layer3+4 (1) MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: avi_eth2 MII Status: up Speed: Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 02:5a:80:02:11:0c Slave queue ID: 0 Slave Interface: avi_eth1 MII Status: up Speed: Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 02:4a:80:02:11:0c Slave queue ID: 0 </pre> <ul> <li>mii status shows the physical link status of the interfaces.</li> <li>bond mode should be <code>xor< /code>.</li> <li>permanent HW addresses of the slaves are the original MAC addresses and should be different for each of the slaves. ### 4. Statistics Statistics of interfaces in the output of <code>show serviceengine <se_ip_address> interface </se_ip_address></code> can be used to see if traffic is being distributed fairly among the slave interfaces. A balanced distribution results in comparable statistics for all slave interfaces if all of them were active together during their lifetime.</li> </ul> <h3 id="csp-pnic-and-sriovconfiguration">5. CSP pnic and SRIOV Configuration</h3> The following commands in CSP shell help identify errors in pnic and SRIOV configuration in CSP. * SRIOV status on pnics helps make sure that SRIOV is on for pnics of interest. <pre><code> avinet-1# show runningconfig pnic sr-iov pnic enp7s0f0 sr-iov numvfs 32 sr-iov switchmode veb! pnic enp7s0f1 sriov numvfs 32 sr-iov switchmode veb! avinet-1# show pnic enp7s0f0 passthrough passthrough sriov avinet-1# show pnic enp7s0f0 sriov_intf sriov_intf sriov_pf </code></pre> <ul> <li>view the virtual function usage on a pnic as shown below. <pre> avinet-1# show pnic enp7s0f0 vfnics VF INUSE SERVICE IDX NAME FLAG OWNER enp7s16 in-use none 1 enp7s16f2 in-use none 2 enp7s16f4 in-use none 3 enp7s16f6 in-use none 4 enp7s17 in-use none 5 enp7s17f2 in-use none 6 enp7s17f4 in-use none 7 enp7s17f6 notused none 8 enp7s18 not-used none 9 enp7s18f2 not-used none 10 enp7s18f4 not-used none 11 enp7s18f6 not-used none 12 enp7s19 not-used none 13 enp7s19f2 not-used none 14 enp7s19f4 not-used none 15 enp7s19f6 not-used none 16 enp7s20 not-used none 17 enp7s20f2 not-used none 18 enp7s20f4 not-used none 19 enp7s20f6 not-used none 20 enp7s21 not-used none 21 enp7s21f2 not-used none 22 enp7s21f4 not-used none 23 enp7s21f6 not-used none 24 enp7s22 not-used none 25 enp7s22f2 not-used none 26 enp7s22f4 not-used none 27 enp7s22f6 not-used Copyright 2018 Avi Networks, Inc. Page 7 of 8

8 none 28 enp7s23 not-used none 29 enp7s23f2 not-used none 30 enp7s23f4 not-used none 31 enp7s23f6 not-used none </pre></li> </ul> <h2 id="reference-and-related-aricles">reference and Related Aricles</h2> <ul> <li><a href=" Channeling on Linux Server Hosts</a></li> <li><a href=" Avi Vantage for CSP 2100</a></li> <li><a href="/ /linux_unix/article.php/ /understanding-nic-bonding-with-linux.htm/">understanding NIC Bonding with Linux</a></li> <li><a href="/ /UbuntuBonding/">Ubuntu Bonding</a></li> </ul> </code> Copyright 2018 Avi Networks, Inc. Page 8 of 8