BRKDCT The Comprehensive Guide to Securing NX-OS Devices Derek Huckaby Technical Marketing Engineer (TME)

Transcription

1

2 BRKDCT The Comprehensive Guide to Securing NX-OS Devices Derek Huckaby Technical Marketing Engineer (TME)

3 Session Goals Learn about the solutions Nexus switches have for securing themselves and the network Control Plane Learn how CoPP and Hardware Rate Limiters work and how to customize them Learn from others who have experienced issues and worked with us to over come them Nexus 9000 = 9300 / 9500 Nexus 7000 = 7000 / 7700 Nexus 5000 = 5500 / 5600 / 6000

4 Session Non-Goals Data Plane Security ACLs PVLANs etc. MDS Product Family Cisco TrustSec Security Group Tags Security Group ACLs SGT Exchange Protocol

5 Agenda Introduction Safety Controls within Nexus Packet Sanity urpf Storm Control Hardware Rate Limiters CoPP Overview CoPP Deep Dive What we see in the Real World Wrap Up

6 Introduction Data Centers are growing and becoming more concentrated with DC consolidation Network issues have wider impacts More devices increase chances of having failures ACI Fabric APIC STP Based Tiered Design VPC Based Tiered Design FabricPath Based Fabric Design Stand Alone Fabric Design Spine Leaf

7 Why Protect the Control Plane CPU? Nexus Series Switches are deployed as Data Center and Campus switches The Nexus Control Plane CPU is the brain of the network and handles the maximum load of the network, which includes frequent bursts of control plane requests Some of those requests include: OSPF HSRP 3 OTV STP 2 EIGRP DHCP 1 BFD 50 ms MPLS ARP 1 BGP LISP Glean 1 Control Plane CPU attacks are DoS attaches which can be perpetrated either inadvertently or maliciously resulting excessive CPU utilization which can lead to business-impacting network outages

8 Nexus Solutions for Managing Control Plane Attacks Hardware Based Ingress Checks IDS Packet Sanity Hardware level checks to prevent corrupted frames from ever being processed. Unicast Reverse Path Forwarding Checks done within the linecards to prevent the Supervisor CPU from wasting cycles with lookups Storm Control Protection from host or edge packet flooding Hardware Based Egress Checks CoPP and HWRL are a hardware-based feature that protects the Control Plane CPU/Supervisor from DoS attacks by working in conjunction to control the rate at which packets are allowed to reach the Supervisor.

9 Packet Paths to the Nexus 7000/9000 Supervisors Traffic hitting the CPU on the Supervisor module can come in through three paths Switched Ethernet Out Band Channel (EOBC) to control the linecards from the Supervisor module and exchange status messages. Inband interfaces (front panel port) for traffic sent by linecards Management Interface (mgmt0) used for management traffic Only traffic sent through an Inband interface is subject to CoPP or HWRL because it is the only traffic that reaches the Supervisor via forwarding engines (FEs)

10 Where is Control Plane Protection Being Done? Fabric Interconnect to I/O Ports Control Plane CPU int mgmt 0 Dedicated Hardware Rate Limited Control Plane Policing (CoPP) Forwarding Engines Storm Control IDS Packet Sanity (Nexus 7000) and urpf (Nexus 5000 & 7000) Hardware Rate Limiters

11 Where are CoPP Policies Applied Nexus 9000 One Stage 2 NFE is selected at boot ore reload time to process all traffic destined to the Control Plane CPU ECP Switch Supervisor CPU CoPP Policies are applied to the input service interface on the control plane Fabric 1 Fabric 2 Fabric 3 Fabric 4 Fabric 5 Fabric 6 NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE CoPP Policies are pushed to Queues on each NFE 12 x 40 Gbps 12 x 40 Gbps 12 x 40 Gbps 12 x 40 Gbps 12 x 40 Gbps 12 x 40 Gbps HWRL are pushed to each NFE 36-port 40 Gbps QSFP+ 36-port 40 Gbps QSFP+

12 Where are CoPP Policies Applied Nexus 9000 ECP Switch Supervisor CPU CoPP Policies are applied to the input service interface on the control plane Stage 2 Shaping is aggregated at the designated NFE handling CP traffic Fabric 1 Fabric 2 Fabric 3 Fabric 4 Fabric 5 Fabric 6 NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE NFE 12 x 40 Gbps 12 x 40 Gbps 12 x 40 Gbps 12 x 40 Gbps 12 x 40 Gbps Stage 1 Shaping at each NFE on linecard 36-port 40 Gbps QSFP+ 36-port 40 Gbps QSFP+ 12 x 40 Gbps CoPP Policies are pushed to egress TCAM on each NFE HWRL are enforced on NFE egress

13 Where are CoPP Policies Applied Nexus 7000/7700 mgmt0 10/100/1000 Console Mgmt Enet usb expansion usb expansion CoPP Policies are applied to the input service interface on the control plane Internal eusb bootflash Internal eusb logflash DRAM Main CPU Main CPU NVRAM I / O C o n t r o l l e r Supervisor 2e Switched EOBC Fabric ASIC VOQs Central Arbiter Fabric 1 Fabric 2 Fabric 3 Fabric 4 Fabric 5 Fabric 6 LC CPU Fabric 2 ASIC Fabric 2 ASIC Arbitration Aggregator CoPP Policies are pushed to egress TCAM on each SoC/Earl 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 4 X 10G SoC 48-port 10G Module F2e HWRL are enforced on the egress SoC or Earl Port ASICs are used to enforce port level stormcontrol settings

14 Detailed Look Within the Forwarding Engine FE Daughter Card Egress NetFlow collection Ingress NetFlow collection Egress ACL/QoS classification includes CoPP classification Ingress ACL/QoS classification CL TCAM MAC Table Ingress MAC table lookups Port-channel hash result Ingress IGMP snooping lookups L3 Engine L2 Engine NetFlow Classification (ACL/QoS) Ingress lookup pipeline L2 Lookup (pre-l3) Ingress Parser PKT HDR From I/O Module Replication Engines Egress lookup pipeline Layer 3 FIB Policing L2 Lookup (post-l3) Final Results To I/O Module Replication Engines FIB TCAM/ ADJ FIB TCAM and adjacency table lookups for Layer 3 forwarding ECMP hashing Multicast RPF check Ingress policing Egress policing includes CoPP and Hardware Rate limiting Egress MAC lookups Egress IGMP snooping lookups

15 Where are CoPP Policies Applied Nexus 5672 CoPP Policies are pushed to egress TCAM on the Supervisor s UPC

16 Safety Controls within Nexus: Packet Sanity

17 IDS Packet Sanity Checks Nexus 7000/7700 The Intrusion Detection System (IDS) check performs sanity checks on the IP headers to protect the network and the system NX-OS generates SYSLOGs on IDS drops (max is one every 30 min) Nexus 7000/7700 Forwarding Engines perform sanity checks on the header fields of IPv4 and IPv6 packets Packet Sanity checks protect the network and the system from illegal packets.

18 Default Sanity Checks Nexus 7000/7700 n77k1# show hardware ip verify IPv4 IDS Checks Status Packets Failed address source broadcast Enabled 0 address source multicast Enabled 0 address destination zero Enabled 0 address identical Disabled -- address reserved Disabled -- address class-e Disabled -- checksum Enabled 0 protocol Enabled 0 fragment Disabled -- length minimum Enabled 0 length consistent Enabled 0 length maximum max-frag Enabled 0 length maximum udp Disabled -- length maximum max-tcp Enabled 0 tcp flags Disabled -- tcp tiny-frag Enabled 0 version Enabled The IP sanity checks are enabled by default and can be individually disabled Packets failing a sanity check will be dropped and a counter will be kept Certain IDS checks are disabled to allow known protocols to pass by default: Fragment IDS check is disabled since some applications sends IP packets with DF bit and fragment offset IP Address Identical check is disabled, which allows BFD Echo to function FEX uses Reserved Addresses IPv6 IDS Checks Status Packets Failed length consistent Enabled 0 length maximum max-frag Enabled 0 length maximum udp Disabled -- length maximum max-tcp Enabled 0 tcp tiny-frag Enabled 0 version Enabled 0

19 Nexus CLI for Configuring IP Sanity Checking n7k1(config)# hardware ip verify? address IPv4 Source and destination address validation checksum Verify IPv4 and IPv6 packet checksum fragment Check IPv4 and IPv6 fragment with non-zero offset and DF bit active length Validate IPv4 packet header and payload length protocol Verify IP procotol syslog Syslog Messages logging configuration for IDS check drops tcp Validate TCP packet header version Must be 4 for an ethertype of IPv4 (0x0800) n7k1(config)# hardware ip verify address? class-e Class E IDS check destination Check destination address identical Same IP SA and DA reserved Source address is 127.x.x.x source Check source address n7k1(config)# hardware ip verify length? consistent Actual frame size is equal to or more than IPv4 length plus ethernet header maximum Check max fragment offset and payload length minimum Minimum IPv4 header length n7k1(config)# hardware ip verify tcp? flags Check TCP flags tiny-frag Check TCP tiny fragment n7k1(config)# hardware ipv6 verify? length Validate IPv6 packet header and payload length tcp Validate TCP packet header version Must be 6 for an ethertype of IPv6 (0x86DD) n7k1(config)# hardware ipv6 verify length? consistent Actual frame size is equal to or more than IPv6 length plus ethernet header maximum Check max fragment offset and payload length n7k1(config)# hardware ipv6 verify tcp? tiny-frag Check TCP tiny fragment

20 IP Sanity checks in detail source: Drops packets if the IP source address is Drops packets if the source address is in the 224.x.x.x or FF00::/8 range Drops packets if the source address matches a user defined IP destination: Drops packets if the destination IPv4 address is or :: The version field must be 4 for an IPv4 Ethernet type or 6 for an IPv6 Ethernet type Drops IPv4 or IPv6 packets if the packet under or over runs the length field or if the fragment exceeds bytes. If the DF bit is set, the offset must be zero Verify IPv4 and IPv6 packet checksum identical: Drops packets if the source address is identical to the destination address reserved: Drops packets if the IPv4 address is in the 127.x.x.x or ::1 range Drops IPv4 or IPv6 packets if the packet fragment has an invalid IP protocol number class-e: Drops packets if source or destination address are in the range:

21 TCP Sanity checks in detail FIN = 1 and ACK = 0 PUSH = 1 and ACK = 0 URG = 1 and ACK = 0 SYN = 1 and RST = 1 SYN = 0 and RST = 0 and ACK = 0 TCP tiny fragment If the IPv4 Fragment Offset is 0 and the IPv4 Protocol is TCP, the IPv4 payload length should be greater than or equal to a programmable minimum value. The programmable minimum value default is 16.

22 Benefits of IDS Packet Protection IDS packet sanity checks protects the Nexus control plane and other devices on the network IDS drops bad and malformed traffic before CoPP, helping ensure bad traffic does not hit the Supervisor With IDS Corrupt Packets: Are less likely to destabilize operating systems causing host or network device outage Optimizes CPU cycles by avoiding filtering and discarding malformed traffic

23 Safety Controls within Nexus: urpf

24 Unicast Reverse Path Forwarding (urpf) RPF does a reverse path lookup for packets to ensure their source IP addresses are known and installed in the local forwarding table Packets received on a L3 interface with unknown IP source addresses are dropped Nexus 5000/7000 perform multi-path urpf checks in hardware, no impact to performance Nexus 5000/7000 supports strict & loose modes together with the allow-default option The strict & loose modes can be configured per interface /24 E 2/1 FIB Table Note: urfp must be disabled for BFD to work IP packet with spoofed address Source IP Address = Prefixes Next Hop Interfaces / E 8/ / E 2/1 No info about this prefix! Drop the packet

25 Strict urpf mode in action The strict urpf check verifies both: 1. Source prefix exists in the routing table 2. The source prefix is reachable via the interface on which it was received n7k1(config)# int e 2/1 n7k1(config-if)# ip verify unicast source reachable-via rx Symmetric Routing /24 E 2/1 FIB Table E 4/ /24 Asymmetric Routing 1 IP packet with spoofed address Source IP Address = No info about this prefix! Drop the packet Prefixes Next Hop Interfaces / E 8/ / E 2/ / E 4/16 IP packet from a valid source address arrives on different interface Unexpected Ingress Interface! Drop the packet

26 Loose urpf mode in action The loose urpf check verifies only: 1. Source prefix exists in the routing table 2. The source prefix is reachable via the interface on which it was received n7k1(config)# int e 2/1 n7k1(config-if)# ip verify unicast source reachable-via any Symmetric Routing /24 E 2/1 FIB Table E 4/ /24 Asymmetric Routing 1 IP packet with spoofed address Source IP Address = No info about this prefix! Drop the packet Prefixes Next Hop Interfaces / E 8/ / E 2/ / E 4/16 IP packet from a valid source address arrives on different interface Unexpected Ingress Interface Allow the packet

27 Use urpf to Protect Against DDoS DoS attacks (e.g.smurf) present rapidly changing source IP addresses to the network urpf can help prevent these attacks in hardware Saves the Supervisor CPU from participating or being a victim

28 Safety Controls within Nexus: Storm Control

29 Storm Control - Broadcast Port Suppression High volumes of broadcast traffic can impact bandwidth availability and impact network performance so a way to limit this traffic type is required Traffic Storm Control allows controlled amount of storm traffic to be forwarded out a target port as a percentage of the total bandwidth of the port Nexus hardware monitors ingress storm traffic at intervals comparing the volume of storm traffic with the bandwidth capacity of the ingress interface Nexus ms Nexus ms Nexus µs Traffic in excess of the configured limit is dropped The suppression mechanism is the same on all Nexus switches except for unicast traffic Double digit granularity Recommended best practice is to apply storm control on host & edge ports, it is not recommended on infra links

30 Storm Control Nuances Nexus 7000 provides a single level n7k1-dc1agg1(config-if)# storm-control multicast level 10 n7k1-dc1agg1(config-if)# sho run int e8/22 interface Ethernet8/22 storm-control multicast level n7k1-dc1agg1(config-if)# storm-control broadcast level 30 n7k1-dc1agg1(config-if)# sho run int e8/22 interface Ethernet8/22 storm-control broadcast level storm-control multicast level n7k1-dc1agg1(config-if)# storm-control unicast level 80 n7k1-dc1agg1(config-if)# sho run int e8/22 interface Ethernet8/22 storm-control broadcast level storm-control multicast level storm-control unicast level HEX BIN HEX BIN Unicast storm control Nexus 9000 & 5000 is applicable to unknown unicast ingress traffic Nexus 7000 is for all unicast traffic, essentially a port level ingress rate limiter. A Broadcast packet contains all 1 s in the DMAC FF FF FF FF FF FF A Multicast packet I/G bit of the DMAC is set to E XX XX XX Nexus 9000/5000 support FEX NIF ports FEX HIF ports do not support storm control

31 The Benefits of using Storm Control Unicast Broadcast Multicast Total BW 10G 8G 6G 4G 2G Broadcast Storm on default Interface Interface Throughput 10G 8G 6G 4G 2G Broadcast Storm using Storm Control t 0 t 1 t 2 t 3 t 0 t 1 t 2 t 3 n7k1(config)# int e 2/24 n7k1(config-if)# storm-control broadcast level 30 Rate Sampling t 0 t 1 t 2 t 3 t 0 t 1 t 2 t 3

32 Differences Between Storm Controls Interface level storm control is designed to protect other hosts and network devices from a host creating a storm n7k1(config)# int e 8/4 n7k1(config-if)# storm-control broadcast level 30 The hardware rate-limiter for storm control is specifically for protecting the control plane CPU from aggregated broadcast traffic n7k1(config)# hardware rate-limiter layer-2 storm-control? < > Value in packets per sec disable Disable the rate-limiter

33 Safety Controls within Nexus: Hardware Rate Limiters

34 What are Hardware Rate Limiters (HWRL)? On the Nexus 7000/9000 platforms Hardware Rate Limiters (HWRL) complement CoPP to protect the CPU The HWRL are enabled by default to regulate traffic destined to the CP CPU On Nexus HWRL can be applied to both packets that are redirected to the CP CPU, and received packets

35 Traffic Shaping is used for HWRL on Nexus 9000 The Nexus 9000 supports Hardware Rate Limiters using 7 queues BFD and FEX have the highest priority HWRL Queues are assigned at boot time and cannot be re-prioritized N9372(config-pmap)# show system internal access-list copp < snip > Queue Name Rate(pps) Burst(pkts) class-default-proxy Multicast (*,G) ACL LOG Exception Default LC OTHER Glean SPAN < snip > 38 FEX Internal DIAG(GOLD) LC BFD

36 What is Regulated by the Hardware Rate Limiters? We recommended to use default hardware rate-limiter setting and modify based on the Data Center application requirements n7k1# show hardware rate-limiter module 8 < snip > Module: 8 R-L Class Config Allowed Dropped Total L3 mtu L3 ttl L3 control L3 glean n9k1# show hardware rate-limiter L3 mcast dirconn L3 mcast loc-grp Units for Config: packets per secon L3 mcast rpf-leak Allowed, Dropped & Total: aggregate L2 storm-ctrl Disable access-list-log copy Module: 1 receive R-L Class Config L2 port-sec L2 mcast-snoop L3 glean 100 L2 vpc-low L3 mcast loc-grp 3000 L2 l2pt access-list-log 100 L2 vpc-peer-gw bfd L2 lisp-map-cache exception 50 L2 dpss fex 3000 L3 glean-fast span 50 L2 otv L2 netflow

37 Applying and Modifying HWRL on Nexus 7000/9000 Hardware Rate-Limiters can be applied at the module level On the Nexus 7000 F2 and F3 modules HWRL can be applied at the port-group level Users can modify the PPS values of any HWRL New rate-limiters cannot be defined 7k1(config)# show hardware rate-limiter layer-3 glean begin "Module: 8" Module: 8 R-L Class Config Allowed Dropped Total L3 glean Port group Eth8/1-4 R-L Class Config Allowed Dropped Total L3 glean Port group with configuration same as default configuration Eth8/5-8 Eth8/9-12 Eth8/13-16 Eth8/17-20 Eth8/21-24 Eth8/25-28 Eth8/29-32 Eth8/33-36 Eth8/37-40 Eth8/41-44 Eth8/45-48

38 Safety Controls within Nexus: Control Plane Policing

39 What is CoPP? Control Plane Policing is a security feature that leverages Nexus hardware to regulate traffic destined to the control plane CPU CoPP provides a distributed policing mechanism that is synchronized across individual forwarding engines The CoPP feature is modeled as an input Quality of Service (QoS) policy attached to the special interface called the control-plane. CoPP is applied on the control-plane interface and provides: Exception Logic: Separate data plane and control plane packets Classification: Identify DoS attack packets Service Policies: Mark, drop or police

40 CoPP Allows Classification for These Packet Types Receive packets: Packets that have the destination address as a router address. (Packets such as router update, or keep-alive messages come under this category). Multicast packets: Multicast packets can be classified into the following types of packets. Directly connected sources Multi-cast control packets Multicast partially switched packets. Copy packets: For supporting some features like acl-log, copy of the original packet is made and sent to the supervisor. The different types of packets that get copied to the sup are. ACL-log copy User can specify logging option on an accesslist and a copy of the packet gets send to the supervisor. FIB unicast copy Multicast copy Exception packets: Some packets need special handling and the hardware may not be able to handle them, therefore they get sent to the supervisor. For example, if a destination address is not present in the FIB and results in a miss, then an ICMP unreachable packet needs to be sent back to the sender. Redirected Packets: Features like DHCP snooping or Dynamic ARP inspection redirects some packets to the supervisor.. Glean packets: If a L2 MAC address for a destination ipaddress or next hop is not present in the FIB, the packet gets sent to the supervisor, so that an ARP request can be generated to the host or next-hop. Broadcast, non-ip packets: Packets with broadcast MAC + non-ip, broadcast MAC + IP unicast, Multicast MAC + IP unicast all under this category.

41 Closer Look at a CoPP Profile n7k1(config-cp)# show copp profile strict permit any 0180.c ip access-list copp-system-p-acl-pim-mdt-join ip access-list copp-system-p-acl-tftp match access-group name copp-system-p-acl-mac-lldp match access-group name copp-system-p-acl-dhcp6 mac access-list copp-system-p-acl-mac-fcoe permit udp any /32 permit udp any any eq tftp match access-group name copp-system-p-acl-mac-mvrp match redirect dhcp-snoop ip access-list copp-system-p-acl-bgp Nexus permit any 7000/9000 any 0x8906 ip access-list Share copp-system-p-acl-pim-reg CoPP permit udp any any eq Interface 1758 match access-group Look name copp-system-p-acl-mac-flow-control and class-map Feel type control-plane match-any copp-system-p-classnormal-dhcp-relay-res permit tcp any gt 1024 any eq bgp permit any any 0x8914 permit pim any any permit udp any eq tftp any class-map type control-plane match-any copp-system-p-class-l2- permit tcp any eq bgp any gt 1024 mac access-list copp-system-p-acl-mac-flow-control ipv6 access-list copp-system-p-acl-pim6 permit udp any eq 1758 any default ponse ipv6 access-list copp-system-p-acl-bgp6 permit any 0180.c x8808 permit pim any ff02::d/128 ipv6 access-list copp-system-p-acl-tftp6 match access-group name copp-system-p-acl-mac-undesirable match access-group name copp-system-p-acl-dhcp-relayresponse permit tcp any gt 1024 any eq bgp mac access-list copp-system-p-acl-mac-l2-tunnel permit udp any any eq 496 permit udp any any eq tftp match protocol mpls permit tcp any eq bgp any gt 1024 permit any any 0x8840 ipv6 access-list copp-system-p-acl-pim6-reg permit udp any any eq 1758 class-map type control-plane match-any copp-system-p-class-l2- match access-group name copp-system-p-acl-dhcp6-relayresponse ip access-list copp-system-p-acl-cts mac access-list copp-system-p-acl-mac-l2pt permit pim any any permit udp any eq tftp any unpoliced permit tcp any any eq permit any ccd.cdd ip access-list copp-system-p-acl-radius permit udp any eq 1758 any match access-group name copp-system-p-acl-mac-stp class-map type control-plane match-any copp-system-p-classredirect permit tcp any eq any mac access-list copp-system-p-acl-mac-l3-isis permit udp any any eq 1812 ip access-list copp-system-p-acl-traceroute match access-group name copp-system-p-acl-mac-lacp ip access-list copp-system-p-acl-dhcp A CoPP permit any 0180.c Profile Consists permit udp any any eq 1813 of Three permit icmp any any ttl-exceeded Components: match access-group name copp-system-p-acl-mac-cfsoe match redirect arp-inspect permit udp any eq bootpc any permit any 0180.c permit udp any any eq 1645 permit icmp any any port-unreachable match access-group name copp-system-p-acl-mac-sdp-srp class-map type control-plane match-any copp-system-p-classundesirable permit udp any neq bootps any eq bootps permit any b permit udp any any eq 1646 permit udp any any range match access-group name copp-system-p-acl-mac-l2-tunnel ip access-list copp-system-p-acl-dhcp-relay-response mac access-list copp-system-p-acl-mac-lacp permit udp any eq 1812 any ip access-list copp-system-p-acl-undesirable match access-group name copp-system-p-acl-mac-cdp-udld-vtp match access-group name copp-system-p-acl-undesirable permit udp any eq bootps any permit any 0180.c x8809 permit udp any eq 1813 any permit udp any any eq 1434 class-map type control-plane match-any copp-system-p-classmanagement match exception fcoe-fib-miss permit udp any any eq bootpc mac access-list copp-system-p-acl-mac-lldp permit udp any eq 1645 any ip access-list copp-system-p-acl-vpc ipv6 access-list copp-system-p-acl-dhcp6 permit any 0180.c e x88cc permit udp any eq 1646 any permit udp any any eq 3200 match access-group name copp-system-p-acl-ftp policy-map type control-plane copp-system-p-policy-strict permit udp any eq 546 any mac access-list copp-system-p-acl-mac-mvrp ipv6 access-list copp-system-p-acl-radius6 ip access-list copp-system-p-acl-vrrp match access-group name copp-system-p-acl-ntp class copp-system-p-class-critical permit udp any neq 547 any eq 547 permit any 0180.c x88f5 permit udp any any eq 1812 permit ip any /32 match access-group name copp-system-p-acl-ssh set cos 7 ipv6 access-list copp-system-p-acl-dhcp6-relay-response mac access-list copp-system-p-acl-mac-otv-isis permit udp any any eq 1813 ipv6 access-list copp-system-p-acl-vrrp6 match access-group name copp-system-p-acl-ntp6 police cir kbps bc 250 ms conform transmit violate drop permit udp any eq 547 any permit any cdf.dfdf permit udp any any eq 1645 permit ipv6 any ff02::12/128 match access-group name copp-system-p-acl-sftp class copp-system-p-class-important permit udp any any eq 546 mac access-list copp-system-p-acl-mac-sdp-srp permit udp any any eq 1646 ip access-list copp-system-p-acl-wccp match access-group name copp-system-p-acl-snmp set cos 6 ip access-list copp-system-p-acl-eigrp permit any 0180.c e x3401 permit udp any eq 1812 any permit udp any eq 2048 any eq 2048 match access-group name copp-system-p-acl-ssh6 police cir 1400 kbps bc 1500 ms conform transmit violate drop permit eigrp any any mac access-list with copp-system-p-acl-mac-stp rate policies permit udp any eq 1813 any match access-group name copp-system-p-acl-tftp class copp-system-p-class-multicast-router ipv6 access-list copp-system-p-acl-eigrp6 permit any ccc.cccd permit udp any eq 1645 any class-map type control-plane match-any copp-system-p-classcritical match access-group name copp-system-p-acl-radius police cir 2600 kbps bc 1000 ms conform transmit violate drop match access-group name copp-system-p-acl-tftp6 set cos 6 permit eigrp any any permit any 0180.c permit udp any eq 1646 any ip access-list copp-system-p-acl-ftp mac access-list copp-system-p-acl-mac-undesirable ip access-list copp-system-p-acl-rip match access-group name copp-system-p-acl-bgp match access-group name copp-system-p-acl-tacacs class copp-system-p-class-management permit tcp any any eq ftp-data permit any any permit udp any /24 eq 520 match access-group name copp-system-p-acl-rip match access-group name copp-system-p-acl-telnet set cos 2 permit tcp any any eq ftp ipv6 access-list copp-system-p-acl-mld ipv6 access-list copp-system-p-acl-rip6 match access-group name copp-system-p-acl-vpc match access-group name copp-system-p-acl-radius6 police cir kbps bc 250 ms conform transmit violate drop permit tcp any eq ftp-data any permit icmp any any mld-query permit udp any ff02::9/64 eq 521 match access-group name copp-system-p-acl-bgp6 match access-group name copp-system-p-acl-tacacs6 class copp-system-p-class-multicast-host permit tcp any eq ftp any permit icmp any any mld-report ip access-list copp-system-p-acl-rise match access-group name copp-system-p-acl-lisp match access-group name copp-system-p-acl-telnet6 set cos 1 ip access-list copp-system-p-acl-glbp permit icmp any any mld-reduction permit tcp any range any match access-group name copp-system-p-acl-ospf class-map type control-plane match-any copp-system-p-classmonitoring class copp-system-p-class-redirect police cir 1000 kbps bc 1000 ms conform transmit violate drop permit udp any eq /24 eq 3222 permit icmp any any 143 ipv6 access-list copp-system-p-acl-rise6 match access-group name copp-system-p-acl-rip6 ip access-list copp-system-p-acl-hsrp ip access-list copp-system-p-acl-mpls-ldp permit tcp any range any match access-group name copp-system-p-acl-rise match access-group name copp-system-p-acl-icmp set cos 1 permit udp any /32 eq 1985 permit udp any eq 646 any eq 646 ip access-list copp-system-p-acl-sftp match access-group name copp-system-p-acl-eigrp match access-group name copp-system-p-acl-icmp6 police cir 280 kbps bc 250 ms conform transmit violate drop permit udp any /32 eq 1985 permit tcp destined any any eq 646 to the permit tcp any any Supervisor/ eq 115 match access-group CP name copp-system-p-acl-lisp6 CPU match access-group name copp-system-p-acl-mpls-oam class copp-system-p-class-normal ipv6 access-list copp-system-p-acl-hsrp6 permit tcp any eq 646 any permit tcp any eq 115 any match access-group name copp-system-p-acl-ospf6 match access-group name copp-system-p-acl-traceroute set cos 1 permit udp any ff02::66/128 eq 2029 ip access-list copp-system-p-acl-mpls-oam ip access-list copp-system-p-acl-smtp-response match access-group name copp-system-p-acl-rise6 match access-group name copp-system-p-acl-http-response police cir 680 kbps bc 250 ms conform transmit violate drop ip access-list copp-system-p-acl-http-response permit udp any any eq 3503 permit tcp any eq 25 any gt 1024 match access-group name copp-system-p-acl-eigrp6 match access-group name copp-system-p-acl-smtp-response class copp-system-p-class-ndp permit tcp any eq 80 any gt 1024 ip access-list copp-system-p-acl-mpls-rsvp ipv6 access-list copp-system-p-acl-smtp6-response match access-group name copp-system-p-acl-otv-as match access-group name copp-system-p-acl-http6-response set cos 6 permit tcp any eq 443 any gt 1024 permit 46 any any permit tcp any eq 25 any gt 1024 match access-group name copp-system-p-acl-mac-l2pt match access-group name copp-system-p-acl-smtp6-response police cir 680 kbps bc 250 ms conform transmit violate drop ipv6 access-list copp-system-p-acl-http6-response ip access-list copp-system-p-acl-msdp ip access-list copp-system-p-acl-snmp match access-group name copp-system-p-acl-mpls-ldp class-map type control-plane match-any copp-system-p-classmulticast-host set cos 1 class copp-system-p-class-normal-dhcp permit tcp any eq 80 any gt 1024 permit tcp any gt 1024 any eq 639 permit udp any any eq snmp match access-group name copp-system-p-acl-mpls-rsvp permit tcp any eq 443 any gt 1024 permit tcp any eq 639 any gt 1024 permit udp any any eq snmptrap match access-group name copp-system-p-acl-mac-l3-isis match access-group name copp-system-p-acl-mld police cir 1500 kbps bc 250 ms conform transmit violate drop ip access-list copp-system-p-acl-icmp ipv6 access-list copp-system-p-acl-ndp ip access-list copp-system-p-acl-ssh match access-group name copp-system-p-acl-mac-otv-isis match access-group name copp-system-p-acl-igmp class copp-system-p-class-normal-dhcp-relay-response permit icmp any any echo permit icmp any any router-solicitation permit tcp any any eq ssh match access-group name copp-system-p-acl-mac-fabricpathisimulticast-router police cir 1800 kbps bc 500 ms conform transmit violate drop class-map type control-plane match-any copp-system-p-class- set cos 1 permit icmp any any echo-reply permit icmp any any router-advertisement permit tcp any eq ssh any ipv6 access-list copp-system-p-acl-icmp6 permit icmp to any any take 137 on traffic ipv6 access-list copp-system-p-acl-ssh6 match protocol mpls router-alert match access-group name copp-system-p-acl-pim class copp-system-p-class-exception permit icmp any any echo-request permit icmp any any nd-ns permit tcp any any eq ssh class-map type control-plane match-any copp-system-p-classexception match access-group name copp-system-p-acl-pim6 police cir 360 kbps bc 250 ms conform transmit violate drop match access-group name copp-system-p-acl-msdp set cos 1 permit icmp any any echo-reply permit icmp any any nd-na permit tcp any eq ssh any ip access-list copp-system-p-acl-igmp ip access-list copp-system-p-acl-ntp ip access-list copp-system-p-acl-tacacs match exception ip option match access-group name copp-system-p-acl-pim-reg class copp-system-p-class-monitoring permit igmp any /3 permit udp any any eq ntp permit tcp any any eq tacacs match exception ip icmp unreachable match access-group name copp-system-p-acl-pim6-reg set cos 1 ip access-list copp-system-p-acl-lisp permit udp any eq ntp any permit tcp any eq tacacs any match exception ipv6 option match access-group name copp-system-p-acl-pim-mdt-join police cir 130 kbps bc 1000 ms conform transmit violate drop permit udp any any eq 4342 ipv6 access-list copp-system-p-acl-ntp6 ipv6 access-list copp-system-p-acl-tacacs6 match exception ipv6 icmp unreachable match protocol mpls exp 6 class copp-system-p-class-l2-unpoliced permit udp any eq 4342 any permit udp any any eq ntp permit tcp any any eq tacacs class-map type control-plane match-any copp-system-p-classfcondp class copp-system-p-class-undesirable class-map type control-plane match-any copp-system-p-class- police cir 8 gbps bc 5 mbytes conform transmit violate transmit ipv6 access-list copp-system-p-acl-lisp6 permit udp any eq ntp any permit tcp any eq tacacs any permit udp any any eq 4342 ip access-list copp-system-p-acl-ospf ip access-list copp-system-p-acl-telnet match access-group name copp-system-p-acl-mac-fcoe match access-group name copp-system-p-acl-ndp set cos 0 permit udp any eq 4342 any permit ospf any any permit tcp any any eq telnet class-map type control-plane match-any copp-system-p-classimportannormal class copp-system-p-class-fcoe class-map type control-plane match-any copp-system-p-class- police cir 32 kbps bc 250 ms conform drop violate drop mac access-list copp-system-p-acl-mac-cdp-udld-vtp ipv6 access-list copp-system-p-acl-ospf6 permit tcp any any eq 107 permit any ccc.cccc permit ospf any any permit tcp any eq telnet any match access-group name copp-system-p-acl-cts match access-group name copp-system-p-acl-mac-dot1x set cos 6 mac access-list copp-system-p-acl-mac-cfsoe ip access-list copp-system-p-acl-otv-as permit tcp any eq 107 any match access-group name copp-system-p-acl-glbp match exception ip multicast directly-connected-sources police cir 1060 kbps bc 1000 ms conform transmit violate drop permit any 0180.c e x8843 permit udp any any eq 8472 ipv6 access-list copp-system-p-acl-telnet6 match access-group name copp-system-p-acl-hsrp match exception ipv6 multicast directly-connected-sources class copp-system-p-class-l2-default permit any 0180.c e ip access-list copp-system-p-acl-pim permit tcp any any eq telnet match access-group name copp-system-p-acl-vrrp match protocol arp police cir 100 kbps bc 250 ms conform transmit violate drop mac access-list copp-system-p-acl-mac-dot1x permit pim any /24 permit tcp any any eq 107 match access-group name copp-system-p-acl-wccp class-map type control-plane match-any copp-system-p-classnormal-dhcp set cos 0 class class-default permit any 0180.c x888e permit udp any any eq 496 permit tcp any eq telnet any match access-group name copp-system-p-acl-hsrp6 mac access-list copp-system-p-acl-mac-fabricpath-isis permit ip any /32 permit tcp any eq 107 any match access-group name copp-system-p-acl-vrrp6 match access-group name copp-system-p-acl-dhcp police cir 100 kbps bc 250 ms conform transmit violate drop 1. Class-maps are used to associate traffic classifications 2. Traffic classifications use ACLs to classify types of traffic 3. Rate policy to specify the allowed traffic rates and actions

42 Closer Look at a CoPP Profile n7k1(config-cp)# show copp profile strict ip access-list copp-system-p-acl-bgp permit tcp any gt 1024 any eq bgp permit tcp any eq bgp any gt 1024 ipv6 access-list copp-system-p-acl-bgp6 permit tcp any gt 1024 any eq bgp permit tcp any eq bgp any gt 1024 ip access-list copp-system-p-acl-cts permit tcp any any eq permit tcp any eq any ip access-list copp-system-p-acl-dhcp permit udp any eq bootpc any permit udp any neq bootps any eq bootps ip access-list copp-system-p-acl-dhcp-relay-response permit udp any eq bootps any permit udp any any eq bootpc ipv6 access-list copp-system-p-acl-dhcp6 permit udp any eq 546 any permit udp any neq 547 any eq 547 ipv6 access-list copp-system-p-acl-dhcp6-relay-response permit udp any eq 547 any permit udp any any eq 546 ip access-list copp-system-p-acl-eigrp permit eigrp any any ipv6 access-list copp-system-p-acl-eigrp6 permit eigrp any any ip access-list copp-system-p-acl-ftp permit tcp any any eq ftp-data permit tcp any any eq ftp permit tcp any eq ftp-data any permit tcp any eq ftp any ip access-list copp-system-p-acl-glbp permit udp any eq /24 eq 3222 ip access-list copp-system-p-acl-hsrp permit udp any /32 eq 1985 permit udp any /32 eq 1985 ipv6 access-list copp-system-p-acl-hsrp6 permit udp any ff02::66/128 eq 2029 ip access-list copp-system-p-acl-http-response permit tcp any eq 80 any gt 1024 permit tcp any eq 443 any gt 1024 ipv6 access-list copp-system-p-acl-http6-response permit tcp any eq 80 any gt 1024 permit tcp any eq 443 any gt 1024 ip access-list copp-system-p-acl-icmp permit icmp any any echo permit icmp any any echo-reply ipv6 access-list copp-system-p-acl-icmp6 permit icmp any any echo-request permit icmp any any echo-reply ip access-list copp-system-p-acl-igmp permit igmp any /3 ip access-list copp-system-p-acl-lisp permit udp any any eq 4342 permit udp any eq 4342 any ipv6 access-list copp-system-p-acl-lisp6 permit udp any any eq 4342 permit udp any eq 4342 any mac access-list copp-system-p-acl-mac-cdp-udld-vtp set cos 1 permit any ccc.cccc mac access-list copp-system-p-acl-mac-cfsoe permit any 0180.c e x8843 permit any 0180.c e mac access-list copp-system-p-acl-mac-dot1x permit any 0180.c x888e mac access-list copp-system-p-acl-mac-fabricpath-isis permit any 0180.c mac access-list copp-system-p-acl-mac-fcoe permit any any 0x8906 permit any any 0x8914 mac access-list copp-system-p-acl-mac-flow-control permit any 0180.c x8808 mac access-list copp-system-p-acl-mac-l2-tunnel permit any any 0x8840 mac access-list copp-system-p-acl-mac-l2pt permit any ccd.cdd mac access-list copp-system-p-acl-mac-l3-isis permit any 0180.c permit any 0180.c permit any b mac access-list copp-system-p-acl-mac-lacp permit any 0180.c x8809 mac access-list copp-system-p-acl-mac-lldp permit any 0180.c e x88cc mac access-list copp-system-p-acl-mac-mvrp permit any 0180.c x88f5 mac access-list copp-system-p-acl-mac-otv-isis permit any cdf.dfdf mac access-list copp-system-p-acl-mac-sdp-srp permit any 0180.c e x3401 mac access-list copp-system-p-acl-mac-stp permit any ccc.cccd permit any 0180.c mac access-list copp-system-p-acl-mac-undesirable permit any any ipv6 access-list copp-system-p-acl-mld permit icmp any any mld-query permit icmp any any mld-report permit icmp any any mld-reduction permit icmp any any 143 ip access-list copp-system-p-acl-mpls-ldp permit udp any eq 646 any eq 646 permit tcp any any eq 646 permit tcp any eq 646 any ip access-list copp-system-p-acl-mpls-oam permit udp any any eq 3503 ip access-list copp-system-p-acl-mpls-rsvp permit 46 any any ip access-list copp-system-p-acl-msdp permit tcp any gt 1024 any eq 639 permit tcp any eq 639 any gt 1024 ipv6 access-list copp-system-p-acl-ndp permit icmp any any router-solicitation permit icmp any any router-advertisement permit icmp any any 137 permit icmp any any nd-ns permit icmp any any nd-na ip access-list copp-system-p-acl-ntp permit udp any any eq ntp permit udp any eq ntp any ipv6 access-list copp-system-p-acl-ntp6 permit udp any any eq ntp permit udp any eq ntp any ip access-list copp-system-p-acl-ospf permit ospf any any ipv6 access-list copp-system-p-acl-ospf6 permit ospf any any ip access-list copp-system-p-acl-otv-as permit udp any any eq 8472 ip access-list copp-system-p-acl-pim permit pim any /24 permit udp any any eq 496 permit ip any /32 ip access-list copp-system-p-acl-pim-mdt-join permit udp any /32 ip access-list copp-system-p-acl-pim-reg permit pim any any ipv6 access-list copp-system-p-acl-pim6 permit pim any ff02::d/128 permit udp any any eq 496 ipv6 access-list copp-system-p-acl-pim6-reg permit pim any any ip access-list copp-system-p-acl-radius permit udp any any eq 1812 permit udp any any eq 1813 permit udp any any eq 1645 permit udp any any eq 1646 permit udp any eq 1812 any permit udp any eq 1813 any permit udp any eq 1645 any permit udp any eq 1646 any ipv6 access-list copp-system-p-acl-radius6 permit udp any any eq 1812 permit udp any any eq 1813 permit udp any any eq 1645 permit udp any any eq 1646 permit udp any eq 1812 any permit udp any eq 1813 any permit udp any eq 1645 any permit udp any eq 1646 any ip access-list copp-system-p-acl-rip permit udp any /24 eq 520 ipv6 access-list copp-system-p-acl-rip6 permit udp any ff02::9/64 eq 521 ip access-list copp-system-p-acl-rise permit tcp any range any ipv6 access-list copp-system-p-acl-rise6 permit tcp any range any ip access-list copp-system-p-acl-sftp permit tcp any any eq 115 permit tcp any eq 115 any ip access-list copp-system-p-acl-smtp-response permit tcp any eq 25 any gt 1024 ipv6 access-list copp-system-p-acl-smtp6-response permit tcp any eq 25 any gt 1024 ip access-list copp-system-p-acl-snmp permit udp any any eq snmp permit udp any any eq snmptrap ip access-list copp-system-p-acl-ssh permit tcp any any eq ssh permit tcp any eq ssh any ipv6 access-list copp-system-p-acl-ssh6 permit tcp any any eq ssh permit tcp any eq ssh any ip access-list copp-system-p-acl-tacacs permit tcp any any eq tacacs permit tcp any eq tacacs any ipv6 access-list copp-system-p-acl-tacacs6 permit tcp any any eq tacacs permit tcp any eq tacacs any ip access-list copp-system-p-acl-telnet permit tcp any any eq telnet permit tcp any any eq 107 permit tcp any eq telnet any permit tcp any eq 107 any ipv6 access-list copp-system-p-acl-telnet6 permit tcp any any eq telnet permit tcp any any eq 107 permit tcp any eq telnet any permit tcp any eq 107 any ip access-list copp-system-p-acl-tftp permit udp any any eq tftp permit udp any any eq 1758 permit udp any eq tftp any permit udp any eq 1758 any ipv6 access-list copp-system-p-acl-tftp6 permit udp any any eq tftp permit udp any any eq 1758 permit udp any eq tftp any permit udp any eq 1758 any ip access-list copp-system-p-acl-traceroute permit icmp any any ttl-exceeded permit icmp any any port-unreachable permit udp any any range ip access-list copp-system-p-acl-undesirable permit udp any any eq 1434 ip access-list copp-system-p-acl-vpc permit udp any any eq 3200 ip access-list copp-system-p-acl-vrrp permit ip any /32 ipv6 access-list copp-system-p-acl-vrrp6 permit ipv6 any ff02::12/128 ip access-list copp-system-p-acl-wccp permit udp any eq 2048 any eq 2048 class-map type control-plane match-any copp-system-p-classcritical match access-group name copp-system-p-acl-bgp match access-group name copp-system-p-acl-rip match access-group name copp-system-p-acl-vpc match access-group name copp-system-p-acl-bgp6 match access-group name copp-system-p-acl-lisp match access-group name copp-system-p-acl-ospf match access-group name copp-system-p-acl-rip6 match access-group name copp-system-p-acl-rise match access-group name copp-system-p-acl-eigrp match access-group name copp-system-p-acl-lisp6 match access-group name copp-system-p-acl-ospf6 match access-group name copp-system-p-acl-rise6 match access-group name copp-system-p-acl-eigrp6 match access-group name copp-system-p-acl-otv-as match access-group name copp-system-p-acl-mac-l2pt match access-group name copp-system-p-acl-mpls-ldp match access-group name copp-system-p-acl-mpls-rsvp match access-group name copp-system-p-acl-mac-l3-isis match access-group name copp-system-p-acl-mac-otv-isis match access-group name copp-system-p-acl-mac-fabricpathisis match protocol mpls router-alert class-map type control-plane match-any copp-system-p-classexception match exception ip option match exception ip icmp unreachable match exception ipv6 option match exception ipv6 icmp unreachable class-map type control-plane match-any copp-system-p-classfcoe match access-group name copp-system-p-acl-mac-fcoe class-map type control-plane match-any copp-system-p-classimportant match access-group name copp-system-p-acl-cts match access-group name copp-system-p-acl-glbp match access-group name copp-system-p-acl-hsrp match access-group name copp-system-p-acl-vrrp match access-group name copp-system-p-acl-wccp match access-group name copp-system-p-acl-hsrp6 match access-group name copp-system-p-acl-vrrp6 match access-group name copp-system-p-acl-mac-lldp match access-group name copp-system-p-acl-mac-mvrp match access-group name copp-system-p-acl-mac-flow-control class-map type control-plane match-any copp-system-p-class-l2- default match access-group name copp-system-p-acl-mac-undesirable match protocol mpls class-map type control-plane match-any copp-system-p-class-l2- unpoliced match access-group name copp-system-p-acl-mac-stp match access-group name copp-system-p-acl-mac-lacp match access-group name copp-system-p-acl-mac-cfsoe match access-group name copp-system-p-acl-mac-sdp-srp match access-group name copp-system-p-acl-mac-l2-tunnel match access-group name copp-system-p-acl-mac-cdp-udld-vtp class-map type control-plane match-any copp-system-p-classmanagement match access-group name copp-system-p-acl-ftp match access-group name copp-system-p-acl-ntp match access-group name copp-system-p-acl-ssh match access-group name copp-system-p-acl-ntp6 match access-group name copp-system-p-acl-sftp match access-group name copp-system-p-acl-snmp match access-group name copp-system-p-acl-ssh6 match access-group name copp-system-p-acl-tftp match access-group name copp-system-p-acl-tftp6 match access-group name copp-system-p-acl-radius match access-group name copp-system-p-acl-tacacs match access-group name copp-system-p-acl-telnet match access-group name copp-system-p-acl-radius6 match access-group name copp-system-p-acl-tacacs6 match access-group name copp-system-p-acl-telnet6 class-map type control-plane match-any copp-system-p-classmonitoring match access-group name copp-system-p-acl-icmp match access-group name copp-system-p-acl-icmp6 match access-group name copp-system-p-acl-mpls-oam match access-group name copp-system-p-acl-traceroute match access-group name copp-system-p-acl-http-response match access-group name copp-system-p-acl-smtp-response match access-group name copp-system-p-acl-http6-response match access-group name copp-system-p-acl-smtp6-response class-map type control-plane match-any copp-system-p-classmulticast-host match access-group name copp-system-p-acl-mld match access-group name copp-system-p-acl-igmp class-map type control-plane match-any copp-system-p-classmulticast-router match access-group name copp-system-p-acl-pim match access-group name copp-system-p-acl-msdp match access-group name copp-system-p-acl-pim6 match access-group name copp-system-p-acl-pim-reg match access-group name copp-system-p-acl-pim6-reg match access-group name copp-system-p-acl-pim-mdt-join match protocol mpls exp 6 class-map type control-plane match-any copp-system-p-classndp match access-group name copp-system-p-acl-ndp class-map type control-plane match-any copp-system-p-classnormal match access-group name copp-system-p-acl-mac-dot1x match exception ip multicast directly-connected-sources match exception ipv6 multicast directly-connected-sources match protocol arp class-map type control-plane match-any copp-system-p-classnormal-dhcp match access-group name copp-system-p-acl-dhcp n7k1(config-cp)# show copp profile strict section normal head lines 5 class-map type control-plane match-any copp-system-p-class-normal match access-group name copp-system-p-acl-mac-dot1x match exception ip multicast directly-connected-sources match exception ipv6 multicast directly-connected-sources match protocol arp n7k1(config-cp)# show copp profile strict section dot1x head lines 2 mac access-list copp-system-p-acl-mac-dot1x permit any 0180.c x888e match access-group name copp-system-p-acl-dhcp6 match redirect dhcp-snoop class-map type control-plane match-any copp-system-p-classnormal-dhcp-relay-res ponse match access-group name copp-system-p-acl-dhcp-relayresponse match access-group name copp-system-p-acl-dhcp6-relayresponse class-map type control-plane match-any copp-system-p-classredirect match redirect arp-inspect class-map type control-plane match-any copp-system-p-classundesirable match access-group name copp-system-p-acl-undesirable match exception fcoe-fib-miss policy-map type control-plane copp-system-p-policy-strict class copp-system-p-class-critical set cos 7 police cir kbps bc 250 ms conform transmit violate drop class copp-system-p-class-important set cos 6 police cir 1400 kbps bc 1500 ms conform transmit violate drop class copp-system-p-class-multicast-router set cos 6 police cir 2600 kbps bc 1000 ms conform transmit violate drop class copp-system-p-class-management set cos 2 police cir kbps bc 250 ms conform transmit violate drop class copp-system-p-class-multicast-host set cos 1 police cir 1000 kbps bc 1000 ms conform transmit violate drop class copp-system-p-class-redirect set cos 1 police cir 280 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-normal set cos 1 police cir 680 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-ndp set cos 6 police cir 680 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-normal-dhcp set cos 1 police cir 1500 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-normal-dhcp-relay-response set cos 1 police cir 1800 kbps bc 500 ms conform transmit violate drop class copp-system-p-class-exception set cos 1 police cir 360 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-monitoring set cos 1 police cir 130 kbps bc 1000 ms conform transmit violate drop class copp-system-p-class-l2-unpoliced police cir 8 gbps bc 5 mbytes conform transmit violate transmit class copp-system-p-class-undesirable set cos 0 police cir 32 kbps bc 250 ms conform drop violate drop class copp-system-p-class-fcoe set cos 6 police cir 1060 kbps bc 1000 ms conform transmit violate drop class copp-system-p-class-l2-default police cir 100 kbps bc 250 ms conform transmit violate drop class class-default set cos 0 police cir 100 kbps bc 250 ms conform transmit violate drop n7k1(config-cp)# show copp profile strict section "class copp-system-p-class-normal" head lines 3 class copp-system-p-class-normal police cir 680 kbps bc 250 ms conform transmit violate drop The normal class-map matches on ACLs, exception and protocol Nexus 7000: bps, pps, time Nexus 5000: kbps and bytes

43 Nexus 7000/9000 Share CoPP Interface Look and Feel n7k1(config-cp)# show copp profile strict ip access-list copp-system-p-acl-bgp permit tcp any gt 1024 any eq bgp permit tcp any eq bgp any gt 1024 ipv6 access-list copp-system-p-acl-bgp6 permit tcp any gt 1024 any eq bgp permit tcp any eq bgp any gt 1024 ip access-list copp-system-p-acl-cts permit tcp any any eq permit tcp any eq any ip access-list copp-system-p-acl-dhcp permit udp any eq bootpc any permit udp any neq bootps any eq bootps ip access-list copp-system-p-acl-dhcp-relay-response permit udp any eq bootps any permit udp any any eq bootpc ipv6 access-list copp-system-p-acl-dhcp6 permit udp any eq 546 any permit udp any neq 547 any eq 547 ipv6 access-list copp-system-p-acl-dhcp6-relay-response permit udp any eq 547 any permit udp any any eq 546 ip access-list copp-system-p-acl-eigrp permit eigrp any any ipv6 access-list copp-system-p-acl-eigrp6 permit eigrp any any ip access-list copp-system-p-acl-ftp permit tcp any any eq ftp-data permit tcp any any eq ftp permit tcp any eq ftp-data any permit tcp any eq ftp any ip access-list copp-system-p-acl-glbp permit udp any eq /24 eq 3222 ip access-list copp-system-p-acl-hsrp permit udp any /32 eq 1985 permit udp any /32 eq 1985 ipv6 access-list copp-system-p-acl-hsrp6 permit udp any ff02::66/128 eq 2029 ip access-list copp-system-p-acl-http-response permit tcp any eq 80 any gt 1024 permit tcp any eq 443 any gt 1024 ipv6 access-list copp-system-p-acl-http6-response permit tcp any eq 80 any gt 1024 permit tcp any eq 443 any gt 1024 ip access-list copp-system-p-acl-icmp permit icmp any any echo permit icmp any any echo-reply ipv6 access-list copp-system-p-acl-icmp6 permit icmp any any echo-request permit icmp any any echo-reply ip access-list copp-system-p-acl-igmp permit igmp any /3 ip access-list copp-system-p-acl-lisp permit udp any any eq 4342 permit udp any eq 4342 any ipv6 access-list copp-system-p-acl-lisp6 permit udp any any eq 4342 permit udp any eq 4342 any mac access-list copp-system-p-acl-mac-cdp-udld-vtp set cos 1 permit any ccc.cccc mac access-list copp-system-p-acl-mac-cfsoe permit any 0180.c e x8843 permit any 0180.c e mac access-list copp-system-p-acl-mac-dot1x permit any 0180.c x888e mac access-list copp-system-p-acl-mac-fabricpath-isis permit any 0180.c mac access-list copp-system-p-acl-mac-fcoe permit any any 0x8906 permit any any 0x8914 mac access-list copp-system-p-acl-mac-flow-control permit any 0180.c x8808 mac access-list copp-system-p-acl-mac-l2-tunnel permit any any 0x8840 mac access-list copp-system-p-acl-mac-l2pt permit any ccd.cdd mac access-list copp-system-p-acl-mac-l3-isis permit any 0180.c permit any 0180.c permit any b mac access-list copp-system-p-acl-mac-lacp permit any 0180.c x8809 mac access-list copp-system-p-acl-mac-lldp permit any 0180.c e x88cc mac access-list copp-system-p-acl-mac-mvrp permit any 0180.c x88f5 mac access-list copp-system-p-acl-mac-otv-isis permit any cdf.dfdf mac access-list copp-system-p-acl-mac-sdp-srp permit any 0180.c e x3401 mac access-list copp-system-p-acl-mac-stp permit any ccc.cccd permit any 0180.c mac access-list copp-system-p-acl-mac-undesirable permit any any ipv6 access-list copp-system-p-acl-mld permit icmp any any mld-query permit icmp any any mld-report permit icmp any any mld-reduction permit icmp any any 143 ip access-list copp-system-p-acl-mpls-ldp permit udp any eq 646 any eq 646 permit tcp any any eq 646 permit tcp any eq 646 any ip access-list copp-system-p-acl-mpls-oam permit udp any any eq 3503 ip access-list copp-system-p-acl-mpls-rsvp permit 46 any any ip access-list copp-system-p-acl-msdp permit tcp any gt 1024 any eq 639 permit tcp any eq 639 any gt 1024 ipv6 access-list copp-system-p-acl-ndp permit icmp any any router-solicitation permit icmp any any router-advertisement permit icmp any any 137 permit icmp any any nd-ns permit icmp any any nd-na ip access-list copp-system-p-acl-ntp permit udp any any eq ntp permit udp any eq ntp any ipv6 access-list copp-system-p-acl-ntp6 permit udp any any eq ntp permit udp any eq ntp any ip access-list copp-system-p-acl-ospf permit ospf any any ipv6 access-list copp-system-p-acl-ospf6 permit ospf any any ip access-list copp-system-p-acl-otv-as permit udp any any eq 8472 ip access-list copp-system-p-acl-pim permit pim any /24 permit udp any any eq 496 permit ip any /32 ip access-list copp-system-p-acl-pim-mdt-join permit udp any /32 ip access-list copp-system-p-acl-pim-reg permit pim any any ipv6 access-list copp-system-p-acl-pim6 permit pim any ff02::d/128 permit udp any any eq 496 ipv6 access-list copp-system-p-acl-pim6-reg permit pim any any ip access-list copp-system-p-acl-radius permit udp any any eq 1812 permit udp any any eq 1813 permit udp any any eq 1645 permit udp any any eq 1646 permit udp any eq 1812 any permit udp any eq 1813 any permit udp any eq 1645 any permit udp any eq 1646 any ipv6 access-list copp-system-p-acl-radius6 permit udp any any eq 1812 permit udp any any eq 1813 permit udp any any eq 1645 permit udp any any eq 1646 permit udp any eq 1812 any permit udp any eq 1813 any permit udp any eq 1645 any permit udp any eq 1646 any ip access-list copp-system-p-acl-rip permit udp any /24 eq 520 ipv6 access-list copp-system-p-acl-rip6 permit udp any ff02::9/64 eq 521 ip access-list copp-system-p-acl-rise permit tcp any range any ipv6 access-list copp-system-p-acl-rise6 permit tcp any range any ip access-list copp-system-p-acl-sftp permit tcp any any eq 115 permit tcp any eq 115 any ip access-list copp-system-p-acl-smtp-response permit tcp any eq 25 any gt 1024 ipv6 access-list copp-system-p-acl-smtp6-response permit tcp any eq 25 any gt 1024 ip access-list copp-system-p-acl-snmp permit udp any any eq snmp permit udp any any eq snmptrap ip access-list copp-system-p-acl-ssh permit tcp any any eq ssh permit tcp any eq ssh any ipv6 access-list copp-system-p-acl-ssh6 permit tcp any any eq ssh permit tcp any eq ssh any ip access-list copp-system-p-acl-tacacs permit tcp any any eq tacacs permit tcp any eq tacacs any ipv6 access-list copp-system-p-acl-tacacs6 permit tcp any any eq tacacs permit tcp any eq tacacs any ip access-list copp-system-p-acl-telnet permit tcp any any eq telnet permit tcp any any eq 107 permit tcp any eq telnet any permit tcp any eq 107 any ipv6 access-list copp-system-p-acl-telnet6 permit tcp any any eq telnet permit tcp any any eq 107 permit tcp any eq telnet any permit tcp any eq 107 any ip access-list copp-system-p-acl-tftp permit udp any any eq tftp permit udp any any eq 1758 permit udp any eq tftp any permit udp any eq 1758 any ipv6 access-list copp-system-p-acl-tftp6 permit udp any any eq tftp permit udp any any eq 1758 permit udp any eq tftp any permit udp any eq 1758 any ip access-list copp-system-p-acl-traceroute permit icmp any any ttl-exceeded permit icmp any any port-unreachable permit udp any any range ip access-list copp-system-p-acl-undesirable permit udp any any eq 1434 ip access-list copp-system-p-acl-vpc permit udp any any eq 3200 ip access-list copp-system-p-acl-vrrp permit ip any /32 ipv6 access-list copp-system-p-acl-vrrp6 permit ipv6 any ff02::12/128 ip access-list copp-system-p-acl-wccp permit udp any eq 2048 any eq 2048 class-map type control-plane match-any copp-system-p-classcritical match access-group name copp-system-p-acl-bgp match access-group name copp-system-p-acl-rip match access-group name copp-system-p-acl-vpc match access-group name copp-system-p-acl-bgp6 match access-group name copp-system-p-acl-lisp match access-group name copp-system-p-acl-ospf match access-group name copp-system-p-acl-rip6 match access-group name copp-system-p-acl-rise match access-group name copp-system-p-acl-eigrp match access-group name copp-system-p-acl-lisp6 match access-group name copp-system-p-acl-ospf6 match access-group name copp-system-p-acl-rise6 match access-group name copp-system-p-acl-eigrp6 match access-group name copp-system-p-acl-otv-as match access-group name copp-system-p-acl-mac-l2pt match access-group name copp-system-p-acl-mpls-ldp match access-group name copp-system-p-acl-mpls-rsvp match access-group name copp-system-p-acl-mac-l3-isis match access-group name copp-system-p-acl-mac-otv-isis match access-group name copp-system-p-acl-mac-fabricpathisis match protocol mpls router-alert class-map type control-plane match-any copp-system-p-classexception match exception ip option match exception ip icmp unreachable match exception ipv6 option match exception ipv6 icmp unreachable class-map type control-plane match-any copp-system-p-classfcoe match access-group name copp-system-p-acl-mac-fcoe class-map type control-plane match-any copp-system-p-classimportant match access-group name copp-system-p-acl-cts match access-group name copp-system-p-acl-glbp match access-group name copp-system-p-acl-hsrp match access-group name copp-system-p-acl-vrrp match access-group name copp-system-p-acl-wccp match access-group name copp-system-p-acl-hsrp6 match access-group name copp-system-p-acl-vrrp6 match access-group name copp-system-p-acl-mac-lldp match access-group name copp-system-p-acl-mac-mvrp match access-group name copp-system-p-acl-mac-flow-control class-map type control-plane match-any copp-system-p-class-l2- default match access-group name copp-system-p-acl-mac-undesirable match protocol mpls class-map type control-plane match-any copp-system-p-class-l2- unpoliced match access-group name copp-system-p-acl-mac-stp match access-group name copp-system-p-acl-mac-lacp match access-group name copp-system-p-acl-mac-cfsoe match access-group name copp-system-p-acl-mac-sdp-srp match access-group name copp-system-p-acl-mac-l2-tunnel match access-group name copp-system-p-acl-mac-cdp-udld-vtp class-map type control-plane match-any copp-system-p-classmanagement match access-group name copp-system-p-acl-ftp match access-group name copp-system-p-acl-ntp match access-group name copp-system-p-acl-ssh match access-group name copp-system-p-acl-ntp6 match access-group name copp-system-p-acl-sftp match access-group name copp-system-p-acl-snmp match access-group name copp-system-p-acl-ssh6 match access-group name copp-system-p-acl-tftp match access-group name copp-system-p-acl-tftp6 match access-group name copp-system-p-acl-radius match access-group name copp-system-p-acl-tacacs match access-group name copp-system-p-acl-telnet match access-group name copp-system-p-acl-radius6 match access-group name copp-system-p-acl-tacacs6 match access-group name copp-system-p-acl-telnet6 class-map type control-plane match-any copp-system-p-classmonitoring match access-group name copp-system-p-acl-icmp match access-group name copp-system-p-acl-icmp6 match access-group name copp-system-p-acl-mpls-oam match access-group name copp-system-p-acl-traceroute match access-group name copp-system-p-acl-http-response match access-group name copp-system-p-acl-smtp-response match access-group name copp-system-p-acl-http6-response match access-group name copp-system-p-acl-smtp6-response class-map type control-plane match-any copp-system-p-classmulticast-host match access-group name copp-system-p-acl-mld match access-group name copp-system-p-acl-igmp class-map type control-plane match-any copp-system-p-classmulticast-router match access-group name copp-system-p-acl-pim match access-group name copp-system-p-acl-msdp match access-group name copp-system-p-acl-pim6 match access-group name copp-system-p-acl-pim-reg match access-group name copp-system-p-acl-pim6-reg match access-group name copp-system-p-acl-pim-mdt-join match protocol mpls exp 6 class-map type control-plane match-any copp-system-p-classndp match access-group name copp-system-p-acl-ndp class-map type control-plane match-any copp-system-p-classnormal match access-group name copp-system-p-acl-mac-dot1x match exception ip multicast directly-connected-sources match exception ipv6 multicast directly-connected-sources match protocol arp class-map type control-plane match-any copp-system-p-classnormal-dhcp match access-group name copp-system-p-acl-dhcp n7k1(config-cp)# show copp profile strict section normal head lines 5 class-map n9k1(config)# type show control-plane copp profile match-any strict copp-system-p-class-normal section head lines 3 class-map match access-group type control-plane name copp-system-p-acl-mac-dot1x match-any copp-system-p-class-normal match exception access-group ip name multicast copp-system-p-acl-mac-dot1x directly-connected-sources match exception protocol arp ipv6 multicast directly-connected-sources match protocol arp n7k1(config-cp)# n9k1(config-cp)# show copp profile strict section dot1x head lines 2 mac access-list copp-system-p-acl-mac-dot1x permit any 0180.c x888e match access-group name copp-system-p-acl-dhcp6 match redirect dhcp-snoop class-map type control-plane match-any copp-system-p-classnormal-dhcp-relay-res ponse match access-group name copp-system-p-acl-dhcp-relayresponse match access-group name copp-system-p-acl-dhcp6-relayresponse class-map type control-plane match-any copp-system-p-classredirect match use redirect arp-inspect similar class-map type control-plane match-any copp-system-p-classundesirable match access-group name copp-system-p-acl-undesirable match exception fcoe-fib-miss policy-map type control-plane copp-system-p-policy-strict class copp-system-p-class-critical set cos 7 police cir kbps bc 250 ms conform transmit violate drop class copp-system-p-class-important set cos matches 6 police cir 1400 kbps bc 1500 ms conform transmit violate drop class copp-system-p-class-multicast-router set cos 6 police cir 2600 kbps bc 1000 ms conform transmit violate drop class copp-system-p-class-management set cos 2 police cir kbps bc 250 ms conform transmit violate drop class copp-system-p-class-multicast-host set cos 1 police cir 1000 kbps bc 1000 ms conform transmit violate drop class copp-system-p-class-redirect set cos 1 police cir 280 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-normal set cos 1 police cir 680 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-ndp set cos 6 police cir 680 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-normal-dhcp set cos 1 police cir 1500 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-normal-dhcp-relay-response set cos 1 police cir 1800 kbps bc 500 ms conform transmit violate drop class copp-system-p-class-exception set cos 1 police cir 360 kbps bc 250 ms conform transmit violate drop class copp-system-p-class-monitoring set cos 1 police cir 130 kbps bc 1000 ms conform transmit violate drop class copp-system-p-class-l2-unpoliced police cir 8 gbps bc 5 mbytes conform transmit violate transmit class copp-system-p-class-undesirable set cos 0 police cir 32 kbps bc 250 ms conform drop violate drop class copp-system-p-class-fcoe set cos 6 police cir 1060 kbps bc 1000 ms conform transmit violate drop class copp-system-p-class-l2-default police cir 100 kbps bc 250 ms conform transmit violate drop class class-default set cos 0 police cir 100 kbps bc 250 ms conform transmit violate drop n7k1(config-cp)# n9k1(config-cp)# show copp profile strict section "class copp-system-p-class-normal" head lines 3 class copp-system-p-class-normal police cir kbps pps bc packets ms conform conform transmit transmit violate violate drop drop Nexus 7000/9000 class-maps and similar ACL Nexus 9000: pps or packets

44 Safety Controls within Nexus: Customizing CoPP

45 Understanding Default CoPP Profiles Default CoPP profiles have been optimized for scale so policy values are suitable for device operations within a Data Center On the Nexus 7000/9000 it is recommended to use the strict profile For Nexus 7000 deployment where there are a majority of F-series modules it is recommended to use the dense profile On the Nexus 5000 platform it is recommended to use the default profile It is not recommended to disable CoPP or HWRLs!

46 Nexus 9000 CoPP uses Rate Shaping on Queues Nexus 9000 applies CoPP policy in a distributed and centralized manner to provide a flexible solution for regulating aggregated packet flow to the CP CPU CoPP Class-maps are assigned to queues. Class-order dictates the priority COS dictates the queue-length/buffer-capacity CoPP takes action on traffic based on policer configuration: A packet is marked green (conforms) if it is within the Committed Information Rate (CIR). A packet is marked red (violate) if it exceeds the Committed Information Rate (CIR). Traffic bursts are regulated using the Committed Burst (BC) setting. Committed Information Rate (CIR) Green - Conform Red - Violate

47 Nexus 7000 CoPP uses Two Rate Three Color Marker (trtcm) CoPP uses an RFC complaint trtcm to provide a flexible solution for regulating packet flow to the CP CPU CoPP takes action (transmit or drop) on traffic based on policer configuration: A packet is marked green (conforms) if it is within the Committed Information Rate (CIR) A packet will be marked yellow (exceeds) if it is in excess of the Committed Information Rate (CIR) but within the Peak Information Rate (PIR). A packet is marked red (violate) if it exceeds the Peak Information Rate (PIR). Traffic bursts are regulated using Committed Burst (BC) and Extended Burst (BE) settings. Note: The Nexus 7000 uses milliseconds to denote BC size, while the Nexus 5000 uses bytes. Committed Information Rate (CIR) Peak Information Rate (PIR) Green - Conform Yellow - Exceed Red - Violate

48 What does the Default CoPP Setting Check? Nexus 7000/9000 class-maps group related protocols arp bgp cdp dhcp eigrp icmp igmp ip isis lacp msdp nd ntp ospf pim rip rise rise6 snmp ssh stp telnet udld vrrp Nexus & 9000 critical exception important default unpolicied management monitoring normal redirect bgp6 cfsoe dhcp6 dot1x eigrp6 ftp hsrp6 icmp6 ipv6 l2-tunnel ntp6 ospf6 pim6 radius radius6 rip6 sftp ssh6 tacacs tacacs6 telnet6 tftp tftp6 traceroute vpc vrrp6 vtp Nexus 7000 & Nexus 9000 fcoe http glean mcast lldp wccp rpf ttl Nexus 5000 & Nexus 7000 Nexus 5000 & Nexus 9000 Nexus 5000 uses protocol specific class-maps glean mtu ptp arpinspect cts dhcpsnoop glbp fabricpath l2pt flowcontrol lisp lisp6 mld mpls otv smtp bfd ecp exception lldp Nexus Platform Specific Nexus 9000 Nexus 7000 Nexus 5000

49 Comparing the Nexus 9000 CoPP Profiles show copp profile < dense lenient moderate strict> Moderate allows ~50% more traffic to the Supervisor Lenient allows double the traffic to the Supervisor The Dense profile provides parity with the Nexus 7000 NX-OS version 7.0(3)I1(2) strict moderate lenient dense cir bc cir bc cir bc cir bc class copp-system-p-class (pps) (pkts) (pps) (pkts) (pps) (pkts) (pps) (pkts) critical cos important cos multicast-router cos % % management normal cos 2cos l3mc-data cos l3uc-data cos normal cos normal-dhcp cos normal-dhcp-relayresponse cos normal-igmp cos redirect cos exception cos exception-diag cos monitoring cos l2-unpoliced cos undesirable cos l2-default cos class class-default cos

50 Comparing the Nexus 7000 CoPP Profiles show copp profile < dense lenient moderate strict> Moderate allows ~25% more traffic to the Supervisor Lenient allows ~50% more traffic to the Supervisor The Dense profile is optimized for port dense chassis where the majority of linecards are F2e modules NX-OS version 6.2(10) strict moderate lenient dense class copp-system-p-class cir (kbps) bc (ms) cir (kbps) bc (ms) cir (kbps) bc (ms) cir (kbps) bc (ms) critical cos important cos multicast-router cos % % management normal cos 2cos multicast-host cos redirect cos normal cos ndp cos dhcp cos dhcp-relay-response cos exception cos monitoring cos undesirable cos fcoe cos l2-default default cos cir (gbps) bc (mbytes) cir (gbps) bc (mbytes) cir (gbps) bc (mbytes) cir (gbps) bc (mbytes) l2-unpoliced

51 Viewing CoPP Profiles on the Nexus 7000/9000 CoPP CLI commands allow you to view entire or specific sections of the CoPP Profiles When viewing CoPP profile, leverage section n7k1(config)# show copp profile strict ip access-list copp-system-p-acl-bgp permit tcp any gt 1024 any eq bgp permit tcp any eq bgp any gt 1024 ipv6 access-list copp-system-p-acl-bgp6 permit tcp any gt 1024 any eq bgp permit tcp any eq bgp any gt 1024 ip access-list copp-system-p-acl-cts permit tcp any any eq permit tcp any eq any ip access-list copp-system-p-acl-dhcp Show CoPP profile allows tab completion of the profile names n7k1(config)# show copp profile strict section management class-map type control-plane match-any copp-system-p-class-management match access-group name copp-system-p-acl-ftp match access-group name copp-system-p-acl-ntp match access-group name copp-system-p-acl-ssh match access-group name copp-system-p-acl-ntp6 match access-group name copp-system-p-acl-sftp match access-group name copp-system-p-acl-snmp match access-group name copp-system-p-acl-ssh6 match access-group name copp-system-p-acl-tftp match access-group n7k1(config)# name show copp-system-p-acl-tftp6 profile strict section snmp match access-group ip access-list name copp-system-p-acl-radius copp-system-p-acl-snmp match access-group permit udp name any copp-system-p-acl-tacacs any eq snmp match access-group permit udp name any copp-system-p-acl-telnet any eq snmp match access-group permit udp name any copp-system-p-acl-radius6 any eq snmptrap match access-group match access-group name copp-system-p-acl-tacacs6 name copp-system-p-acl-snmp match access-group name copp-system-p-acl-telnet6 class copp-system-p-class-management set cos 2 police cir kbps bc 250 ms conform transmit violate drop

52 Viewing CoPP Profiles on the Nexus 7000/9000 CoPP CLI commands also allow you to compare any two of the default profiles n7k1(config)# show copp diff profile moderate profile lenient '+' Line appears only in profile moderate, version 6.2(10E3) '-' Line appears only in profile lenient, version 6.2(10E3) -policy-map type control-plane copp-system-p-policy-lenient - class copp-system-p-class-critical - set cos 7 - police cir kbps bc 375 ms conform transmit violate drop - class copp-system-p-class-important - set cos 6 - police cir 1400 kbps bc 1500 ms conform transmit violate drop - class copp-system-p-class-multicast-router - set cos 6 - police cir 2600 kbps bc 1000 ms conform transmit violate drop - class copp-system-p-class-management - set cos 2 - police cir kbps bc 375 ms conform transmit violate drop - class copp-system-p-class-multicast-host - set cos 1 - police cir 1000 kbps bc 1000 ms conform transmit violate drop +policy-map type control-plane copp-system-p-policy-moderate + class copp-system-p-class-critical + set cos 7 + police cir kbps bc 310 ms conform transmit violate drop + class copp-system-p-class-important + set cos 6 + police cir 1400 kbps bc 1500 ms conform transmit violate drop + class copp-system-p-class-multicast-router + set cos 6 + police cir 2600 kbps bc 1000 ms conform transmit violate drop + class copp-system-p-class-management + set cos 2 + police cir kbps bc 310 ms conform transmit violate drop + class copp-system-p-class-multicast-host + set cos 1 + police cir 1000 kbps bc 1000 ms conform transmit violate drop

53 Changing CoPP Profiles on the Nexus 7000/9000 CoPP CLI allows you to easily apply default CoPP profiles Only one control-plane policy can be applied at a time Removing a policy will disable CoPP and is not recommended n7k1(config)# show copp status Last Config Operation: copp profile lenient Last Config Operation Timestamp: 11:28:17 EDT May Last Config Operation Status: Success Policy-map attached to the control-plane: copp-system-p-policy-lenient n7k1(config)# copp profile strict Use show copp status to view the current profile n7k1(config)# show copp status Last Config Operation: copp profile strict Last Config Operation Timestamp: 11:28:51 EDT May Last Config Operation Status: Success Policy-map attached to the control-plane: copp-system-p-policy-strict

54 Viewing CoPP Statistics on Nexus 7000/9000 n7k1(config)# n9k1(config-pmap-c)# show policy-map show policy-map interface interface control-plane control-plane module 8 module class 1 class-default class copp-system-p-class-normal Control Control Plane Plane service-policy input copp-system-p-policy-strict Service-policy input: copp-system-p-policy-strict class-map class-default (match-any) class-map set cos 0 copp-system-p-class-normal (match-any) police match access-group cir 100 kbps name bc 250 copp-system-p-acl-mac-dot1x ms match conform protocol action: arp transmit set violate cos 1 action: drop module police 8: cir 1500 pps, bc 32 packets module conformed 1 : bytes, transmitted 5-min offered 614 packets; rate 16 bytes/sec dropped peak rate 0 packets; 31 bytes/sec at Tue May 26 11:35: violated 0 bytes, 5-min violate rate 0 bytes/sec peak rate 0 bytes/sec Dropped packets due to CoPP are counted as violated bytes n7k2# show policy-map interface control-plane class violated Control Plane scale-factor 0.10 module 1 scale-factor 0.50 module 2 service-policy input copp-system-p-policy-strict class-map class-default (match-any) set cos 0 police cir 100 kbps bc 250 ms conform action: transmit violate action: drop module 10: conformed bytes, 5-min offered rate 2 bytes/sec peak rate bytes/sec at Wed May 20 23:00: violated bytes, 5-min violate rate 0 bytes/sec peak rate 836 bytes/sec at Wed May 20 23:00:

55 Viewing CoPP Statistics on Nexus 9000 n9k1(config-pmap-c)# show policy-map interface control-plane section critical class-map copp-system-p-class-critical (match-any) match access-group name copp-system-p-acl-bgp match access-group name copp-system-p-acl-rip match access-group name copp-system-p-acl-vpc match access-group name copp-system-p-acl-bgp6 match access-group name copp-system-p-acl-ospf match access-group name copp-system-p-acl-rip6 match access-group name copp-system-p-acl-eigrp n9k1(config-pmap-c)# show policy-map interface control-plane match access-group module 1 class name copp-system-p-class-normal copp-system-p-acl-ospf6 n9k1(config)# show policy-map interface control-plane Control Plane Service-policy input: copp-system-p-policy-strict class-map copp-system-p-class-critical (match-any) match access-group name copp-system-p-acl-bgp match access-group name copp-system-p-acl-rip < snip > match access-group name copp-system-p-acl-eigrp match access-group Control Plane name copp-system-p-acl-ospf6 match access-group name copp-system-p-acl-eigrp6 match access-group Service-policy name copp-system-p-acl-auto-rp input: copp-system-p-policy-strict match access-group name copp-system-p-acl-mac-l3-isis set cos 7 class-map copp-system-p-class-normal (match-any) police cir match pps access-group, bc 128 packets name copp-system-p-acl-mac-dot1x module 1 : match protocol arp transmitted set 0 packets; cos 1 dropped 0 packets; police cir 1500 pps, bc 32 packets module 1 : transmitted 614 packets; dropped 0 packets; match access-group name copp-system-p-acl-eigrp6 match access-group name copp-system-p-acl-auto-rp match access-group name copp-system-p-acl-mac-l3-isis set cos 7 police cir pps, bc 128 packets module 1 : transmitted 0 packets; dropped 0 packets; Dropped packets due to CoPP enforcement

56 Viewing CoPP Statistics on Nexus 7000 n7k1(config)# show policy-map interface control-plane Control Plane service-policy input copp-system-p-policy-strict class-map copp-system-p-class-critical (match-any) match access-group name copp-system-p-acl-bgp match access-group name copp-system-p-acl-rip < snip > match protocol Control mpls Planerouter-alert set cos 7 service-policy input copp-system-p-policy-strict police cir kbps bc 250 ms conform action: class-map transmit class-default (match-any) violate action: set cos drop0 module 1: police cir 100 kbps bc 250 ms conformed 0 bytes, conform action: transmit 5-min offered violate rate 0 action: bytes/sec drop peak rate module 0 bytes/sec 8: violated 0 bytes, conformed bytes, 5-min violate 5-min rate 0 offered bytes/sec rate 16 bytes/sec peak rate 0 bytes/sec < snip > n7k2# show policy-map interface control-plane class violated Control Plane scale-factor 0.10 module 1 scale-factor 0.50 module 2 service-policy input copp-system-p-policy-strict n7k1(config)# show policy-map interface control-plane class-map module class-default 8 class-default (match-any) set cos 0 police cir 100 kbps bc 250 ms conform action: transmit violate action: drop module 10: conformed bytes, 5-min offered rate 2 bytes/sec peak rate bytes/sec at Wed May 20 23:00: violated bytes, 5-min violate rate 0 bytes/sec peak rate 836 bytes/sec at Wed May 20 23:00: peak rate 31 bytes/sec at Tue May 26 11:35: violated 0 bytes, 5-min violate rate 0 bytes/sec peak rate 0 bytes/sec Dropped packets due to CoPP are counted as violated bytes

57 Monitoring CoPP Statistics SNMP and XML can be used to monitor CoPP Statistics CISCO-CLASS-BASED-QOS-MIB cbqoscmname STRING: "class-default cbqospoliceviolatedbyte Counter64: n7k2# show policy-map interface control-plane class violated class-map class-default (match-any) set cos 0 police cir 100 kbps bc 250 ms conform action: transmit violate action: drop module 10: conformed bytes, 5-min offered rate 2 bytes/sec peak rate bytes/sec at Wed May 20 23:00: violated bytes, 5-min violate rate 0 bytes/sec peak rate 836 bytes/sec at Wed May 20 23:00:

58 Modifying CoPP Policer Rates on Nexus 7000/9000 n7k1# n9k1# copp copy profile strict prefix CLSD2015 CoPP profiles must be copied to be modified n7k1# n9k1# show run begin policy policy-map type control-plane CLSD2015-copp-policy-strict class CLSD2015-copp-class-critical set cos 7 police cir kbps pps bc bc packets ms conform conform transmit transmit violate violate drop drop class CLSD2015-copp-class-important set cos 6 police cir kbps pps bc bc packets ms conform conform transmit violate violate drop drop The copp copy command allows you to prefix or suffix string to the profile name Copied Profiles are known as custom Profiles and appear in the running configuration n7k1(config)# n9k1(config)# policy-map type control-plane CLSD2015-copp-policy-strict n7k1(config-pmap-c)# n9k1(config-pmap-c)# class CLSD2015-copp-class-critical n7k1(config-pmap-c)# n9k1(config-pmap-c)# police cir kbps bc 256 bc 500 ms conform transmit violate drop Any part of the custom CoPP profile can be modified n7k1(config-pmap-c)# n9k1(config-pmap-c)# show run section class-critical tail lines 3 class CLSD2015-copp-class-critical set cos 7 police cir kbps pps bc bc packets ms conform conform transmit transmit violate violate drop drop Show run can be used to verify changes

59 Modifying CoPP Policer Rates on Nexus 9000 n9k1# copp copy profile strict prefix CLSD2015 CoPP profiles must be copied to be modified n9k1# show run begin policy policy-map type control-plane CLSD2015-copp-policy-strict class CLSD2015-copp-class-critical set cos 7 police cir pps bc 128 packets conform transmit violate drop class CLSD2015-copp-class-important set cos 6 police cir 3000 pps bc 128 packets conform transmit violate drop The copp copy command allows you to prefix or suffix string to the profile name Copied Profiles are known as custom Profiles and appear in the running configuration n9k1(config)# policy-map type control-plane CLSD2015-copp-policy-strict n9k1(config-pmap-c)# class CLSD2015-copp-class-critical n9k1(config-pmap-c)# police cir bc 256 No units are required Any part of the custom CoPP profile can be modified n9k1(config-pmap-c)# show run section class-critical tail lines 3 class CLSD2015-copp-class-critical set cos 7 police cir pps bc 256 packets conform transmit violate drop Show run can be used to verify changes

60 Modifying CoPP Policer Rates on Nexus 7000 n7k1# copp copy profile strict prefix CLSD2015 CoPP profiles must be copied to be modified n7k1# show run begin policy policy-map type control-plane CLSD2015-copp-policy-strict class CLSD2015-copp-class-critical set cos 7 police cir kbps bc 250 ms conform transmit violate drop class CLSD2015-copp-class-important set cos 6 police cir 1400 kbps bc 1500 ms conform transmit violate drop The copp copy command allows you to prefix or suffix string to the profile name Copied Profiles are known as custom Profiles and appear in the running configuration n7k1(config)# policy-map type control-plane CLSD2015-copp-policy-strict n7k1(config-pmap-c)# class CLSD2015-copp-class-critical n7k1(config-pmap-c)# police cir kbps bc 500 ms conform transmit violate drop Any part of the custom CoPP profile can be modified n7k1(config-pmap-c)# show run section class-critical tail lines 3 class CLSD2015-copp-class-critical set cos 7 police cir kbps bc 500 ms conform transmit violate drop Show run can be used to verify changes

61 Logging Dropped CoPP Traffic on Nexus 7000/9000 Syslog messages are not generated by default when control plane traffic is dropped. Logging can be enabled within custom profiles per class-map n7k1(config)# policy-map type control-plane CLSD2015-copp-policy-strict n7k1(config-pmap)# class CLSD2015-copp-class-normal n7k1(config-pmap-c)# logging drop threshold level 5 n7k1(config-pmap-c)# sho run section "class CLSD2015-copp-class-normal" head lines 4 class CLSD2015-copp-class-normal set cos 1 logging drop threshold level 5 police cir 680 kbps bc 250 ms conform transmit violate drop %COPP-5-COPP_DROPS5: CoPP drops exceed threshold in class: copp-system-class-normal, check show policy-map interface control-plane for more info Syslog messages will be generated if drops within a traffic class exceeds user configured threshold The logging threshold and logging level should be based on monitoring policy

62 Applying a Custom CoPP Profile Unlike default CoPP profile, custom CoPP profiles must be manually applied to the control plane interface Applying a default CoPP Profile n7k1(config)# copp profile strict Applying a custom CoPP Profile n7k1(config)# control-plane n7k1(config-cp)# service-policy input CLSD2015-copp-policy-strict Note custom CoPP policies TAB complete! n7k1(config-cp)# show copp status Last Config Operation: service-policy input CLSD2015-copp-policy-strict Last Config Operation Timestamp: 15:27:31 EDT May Last Config Operation Status: Success Policy-map attached to the control-plane: CLSD2015-copp-policy-strict

63 CoPP Class-map to Queue Association Nexus 9000 The Nexus 9000 supports up to 24 queues for handling CoPP traffic Queue are used for CoPP classes in a 1 to 1 mapping The highest priority class is the first in the policy & applied to Queue 37 Use insert-before to define class priority Queues are assigned dynamically when classes are defined N9372(config-pmap)# show system internal access-list copp < snip > Queue Name Rate(pps) Burst(pkts) < snip > 28 CLSD2015-copp-class-normal-igmp CLSD2015-copp-class-normal-dhcp-relay-response CLSD2015-copp-class-normal-dhcp CLSD2015-copp-class-normal CLSD2015-copp-class-l3uc-data CLSD2015-copp-class-l3mc-data CLSD2015-copp-class-management CLSD2015-copp-class-multicast-router CLSD2015-copp-class-important CLSD2015-copp-class-critical

64 CoPP is Preformed at the Forwarding Engine CoPP functionality (policing) is implemented on each FE independently When customizing rates for Nexus 7000 CoPP policy-map, consideration must be taken in regard to the type and number of line cards in the system. Since CoPP is applied per forwarding engine (FE) the maximum traffic received by the supervisor on the Nexus 7000 will be the total number of FE multiplied by the rate allowed Recall the Nexus 9000 Aggregates CoPP Enforcement

65 Using scale-factor with Multiple Module Types One solution could be to equally distribute CoPP policy to each FE strict profile scaled Mod Type Scale Rate (bps) Burst (bytes) 1 M1 1 35,979,264 1,124,352 2 F2e.1 3,600,000 11,248 3 M2.5 17,989, ,088 4 F3.16 5,760,000 28,800 n7k2(config-cp)# show system internal qos copp <...snip...> Copp Scale-factor is 1 Copp Scale-factor is 0.5 PL Profile # used: 2 inst: 0 PL Profile # used: 2 inst: 0 Commited rate = bps Commited rate = bps Commited burst= bytes Commited burst= bytes Peak rate = bps Peak rate = bps Peak burst= bytes Peak burst= bytes Violated bytes are going to be Dropped Commited rate / (1/BC) = Committed burst Module 1 Commited rate = 35,979,264 bps / 8 = 4,497,408 Bps / (1/.250) = 1,124,352 Bytes (Committed burst) Module 3 Commited rate = 17,989,632 bps / 8 = 2,248,704 Bps / (1/.250) = 562,176 Bytes (Committed burst) 50% of Mod1 CIR Module 3 Commited rate = 17,989,632 bps / 8 = 2,248,704 Bps / (1/.125) = Bytes (Committed burst) Also 50% of Mod1 BC

66 Using scale-factor with Multiple Module Types Main CPU Sup2e DRAM NVRAM Main CPU M1 Earl SoC SoC SoC F2e SoC SoC SoC SoC SoC SoC I/O Controller SoC SoC SoC Fabric Modules Another solution could be to scale up the dense CoPP profile Mod Type Scale Rate (bps) Burst (bytes) Earl Earl SoC SoC SoC SoC SoC SoC 1 M1 2 8,994, ,176 2 F2e 1 4,497, ,544 3 M2 2 8,994, ,176 4 F ,750, ,416 M2 F3

67 CoPP Differences Between Nexus 7000 & 5000 Nexus 7000 Nexus 5000 Full Range of Classification and Policer capabilities Four default profiles. Any can be copied and fully customized Default profiles use classes to group ACLs, exception, and protocol matches Allows new classifications and matches to be defined by the user Policer supports CIR, BC, PIR, BE settings by bandwidth, size and time CoPP and HWRLs can be disable, however it is not recommended Subset of the Nexus 7000 CoPP functionality Four default profiles, one allows user to customize policer rates Default profiles match and rate limit protocols All classification is predefined Policer supports CIR and BC settings by bandwidth and size CoPP cannot be disabled

68 Comparing the Nexus 5000 CoPP Profiles show policy-map type control-plane NX-OS version 7.0(2)N1(1) default scaled-l2 scaled-l3 customized Scaled L2 & Scaled L3 policies quadruple the IGMP rate & burst Customized is based on the Default policy copp-system-policy (page 1 of 2) cir (kbps) bc (bytes) cir (kbps) bc (bytes) cir (kbps) bc (bytes) cir (kbps) bc (bytes) igmp , , , ,535 pim-hello ,800, ,800, ,800, ,800,000 bridging ,800, ,800, ,800, ,800,000 arp ,600, ,600, ,600, ,600,000 dhcp ,800, ,800, ,800, ,800,000 wccp ,800, ,800, ,800, ,800,000 mgmt ,800, ,800, ,800, ,800,000 ecp ,200, ,200, ,200, ,200,000 lacp ,800, ,800, ,800, ,800,000 lldp ,800, ,800, ,800, ,800,000 udld ,800, ,800, ,800, ,800,000 isis ,800, ,800, ,800, ,800,000 msdp ,800, ,800, ,800, ,800,000 cdp ,800, ,800, ,800, ,800,000 fip ,800, ,800, ,800, ,800,000 bgp ,800, ,800, ,800, ,800,000 eigrp ,800, ,800, ,800, ,800,000

69 Comparing the Nexus 5000 CoPP Profiles show policy-map type control-plane (cont d) NX-OS version 7.0(2)N1(1) default scaled-l2 scaled-l3 customized Scaled L3 policies allow for significantly increased traffic rates for glean and icmp-echo traffic copp-system-policy (page 2 of 2) cir (kbps) bc (bytes) cir (kbps) bc (bytes) cir (kbps) bc (bytes) cir (kbps) bc (bytes) exception 64 4,800, ,800, ,800, ,800,000 glean ,800, ,800, ,800, ,800,000 hsrp-vrrp , , , ,000 icmp-echo 64 3,600, ,600, ,600, ,600,000 ospf ,800, ,800, ,800, ,800,000 bfd ,800, ,800, ,800, ,800,000 pim-register ,800, ,800, ,800, ,800,000 rip ,800, ,800, ,800, ,800,000 l3dest-miss 64 16, , , ,000 mcast-miss 256 3,200, ,200, ,200, ,200,000 excp-ip-frag 64 3,200, ,200, ,200, ,200,000 excp-same-if 64 3,200, ,200, ,200, ,200,000 excp-ttl 64 3,200, ,200, ,200, ,200,000 default 512 6,400, ,400, ,400, ,400,000 rpf-fail 512 3,200, ,200, ,200, ,200,000 mcast-last-hop 512 3,200, ,200, ,200, ,200,000 onep-dpss 625 3,200, ,200, ,200, ,200,000

70 Changing CoPP Profiles on the Nexus 5000 Only one control-plane policy can be applied at a time Removing any policy will apply the default policy n5k1(config)# show copp status Last Config Operation: class copp-system-class-glean Last Config Operation Timestamp: 16:18:10 EST Mon May Last Config Operation Status: Success Policy-map attached to the control-plane: copp-system-policy-default atl-tme-n5k1(config)# control-plane atl-tme-n5k1(config-cp)# service-policy input copp-system-policy-customized Use show copp status to view the current profile CoPP profiles must be applied at the control-plane n5k1(config-cp)# show copp status Last Config Operation: service-policy input copp-system-policy-customized Last Config Operation Timestamp: 16:19:41 EST Mon May Last Config Operation Status: Success Policy-map attached to the control-plane: copp-system-policy-customized

71 Viewing CoPP statistics show policy-map interface control-plane will show all classes, rates, and statistics n5k1# show policy-map type control-plane inc class sort uniq class copp-system-class-arp class copp-system-class-bfd class copp-system-class-bgp class copp-system-class-bridging class copp-system-class-cdp class copp-system-class-default class copp-system-class-dhcp class copp-system-class-ecp class copp-system-class-eigrp class copp-system-class-exception class copp-system-class-excp-ip-frag class copp-system-class-excp-same-if class copp-system-class-excp-ttl class copp-system-class-fip class copp-system-class-glean < snip > n5k1# show policy-map interface control-plane class copp-system-class-glean Control Plane service-policy input: copp-system-policy-customized class-map copp-system-class-glean (match-any) match protocol glean police cir 1024 kbps, bc bytes conformed 0 bytes; action: transmit violated 0 bytes; Dropped packets due to CoPP are counted as violated bytes

72 Modifying CoPP Policer Rates CoPP policer rates can be changed only on the customized CoPP profile n5k1(config)# policy-map type control-plane copp-system-policy-customized n5k1(config-pmap)# class copp-system-class-lacp n5k1(config-pmap-c)# police cir 2048 kbps bc bytes n5k1(config-pmap-c)# show policy-map interface control-plane class copp-system-class-lacp Control Plane service-policy input: copp-system-policy-customized class-map copp-system-class-glean (match-any) match protocol glean police cir 2048 kbps, bc bytes conformed 0 bytes; action: transmit violated 0 bytes;

73 Increasing CIR Rates Beyond 20Mbps Default Policer behaviour ( without ingress-copp ) n5600(config)# policy-map type control-plane copp-system-policy-customized n5600(config-pmap)# class copp-system-class-lacp n5600(config-pmap-c)# police cir kbps bytes ERROR: Rate value should be between 1 and kbps Increase Policer CIR rate above the 20Mbps limit using ingress-copp n5600(config)# control-plane n5600(config-cp)# ingress-copp n5600(config-cp)# policy-map type control-plane copp-system-policy-customized n5600(config-pmap)# class copp-system-class-lacp n5600(config-pmap-c)# police cir kbps bytes n5600(config-pmap-c)# show policy-map interface control-plane section lacp class-map copp-system-class-lacp (match-any) match protocol lacp police cir kbps, bc bytes conformed 0 bytes; action: transmit violated 0 bytes;

74 What Can Happen in Real World Data Centers

75 Protect Against CP Failure Financial Scenario Real World Failure Scenario: Leaf Switch Loses Control Plane Core L3 L2 Spines Leafs are using VPC+ to Servers FabricPath

76 Protect Against CP Failure Financial Scenario Real World Failure Scenario: Leaf Switch Loses Control Plane Data Plane Traffic Floods Network FabricPath If the Control Plane Fails, the Data Plane could continue to Forward Date Traffic LACP Suspend Individual is not enabled by default on the Nexus 5000 or Nexus 6000 It is enabled by default on the Nexus 7000

77 Protect Against CP Failure Financial Scenario Real World Failure Scenario: Leaf Switch Loses Control Plane Data Plane Traffic Floods Network Data Center Fabric Propagates Flood Core Control Plane is Overwhelmed and Becomes Unstable L3 L2 Core Spines Leafs are using VPC+ to Servers FabricPath

78 Protect Against CP Failure Real World Failure Scenario: Leaf Switch Loses Control Plane Data Plane Traffic Floods Network Data Center Fabric Propagates Flood Core Control Plane is Overwhelmed and Becomes Unstable The Business Impact L3 Over 50,000 Employees L2 across the region could not work for hours Core Spines Financial Scenario Leafs are using VPC+ to Servers FabricPath

79 Adding Safeguards Against CP Failure Financial Scenario Configure lacp suspend-individual on all Nexus 5000 and Nexus 6000 port-channels facing other switches N7K has this as default, so no need on the N7K Note: This command can only be configure when the port-channel is in shutdown mode Implement Storm control. Baseline to determine a rate which works on steady state traffic. This will not entirely prevent scenarios like this, but it will minimized impact to the rest of the network. LACP and STP Bridge Assurance were in use so UDLD could be removed to protect against false positives These recommendations are in the area of basic switching and would be effective if FabricPath was not in use

80 Impact of Fast Timers Enterprise Scenario There are two reasons for LACP storms: 1. Intentional malicious storm created by an impersonating device in the network trying intrusion or DoS attacks 2. Fast LACP, where some networks employ these in their networks by tuning up the timers so that the frequency of transmitting the LACP is very high, and in presence of many ports increases the load on CPU of switch receiving it and effectively is similar to a storm. During an LACP storm LACP port-channels flap causing network disruptions.

81 Modify CoPP Rate to Manage Fast Timers Enterprise Scenario In order to achieve stability, the Nexus 5600 needs to weather the storm. Ingress CoPP was added to extend CIR rates above the default limit of 20Mbps n5600(config)# control-plane n5600(config-cp)# ingress-copp n5600(config-cp)# policy-map type control-plane copp-system-policy-customized n5600(config-pmap)# class copp-system-class-lacp n5600(config-pmap-c)# police cir kbps bytes The Nexus 5600 has been tested to support up to 60Mbps of LACP messages

82 Local Issues Have Wide Impact Enterprise Scenario Nexus 7000 Core interconnected to Nexus 5000 Aggregation layer. In the event of an LACP Storm at any Nexus 5000 it will likely impact all LACP port channel s link The LACP storm could lead to flapping due to LACP packets being dropped by CoPP on the Nexus OSPF detects that the link has flapped or is down, it declares the OSPF session down now all adjacencies are lost and the site is down Solution: Increase CoPP rate limiting for LACP on the Nexus 5000 In large scale scenarios it may require ingress-copp x x xx LACP Storm at access layer Core OSPF over L3 LACP PCs Aggregation

83 Security Tools Scanning for Risk Exposure Enterprise Scenario Security teams scan network devices to determine risk and exposure Some of the new tools are probing and querying all interfaces In certain conditions this becomes an attack on the control plane One such tool created an ARP storm due to the manner in which it was probing. (SNMP)

84 Isolate and Rate Limit Scanning Devices Enterprise Scenario Best practice is to add a unique class for any network scanners that may poll the switch (SNMP) ip access list copp-acl-valid-scanners permit ip /32 any permit ip /32 any class-map type control-plane match-any copp-class-valid-security-tools match access-group name copp-acl-valid-scanners class copp-class-valid-security-tools insert-before copp-class-critical set cos 2 police cir 1000 kbps bc 250ms conform transmit violate drop

85 Protect Against OSPF Storms Service Provider Scenario Typical network running CoPP Strict OSPF Flood occurs in the Access Aggregation Nexus 7000 Rate Limits OSPF Side-effect it Drops OSPF Adjacencies L3 L2 Core OSPF Peers Aggregation

86 Service Provider Scenario Protect Against OSPF Storms Using CoPP Correct by Customizing CoPP to add a new policer for only valid OSPF Peers ip access list copp-acl-valid-ospf-peers permit ip /32 any permit ip /32 any permit ip /32 any permit ip /32 any class-map type control-plane match-all copp-class-valid-ospf-peers match access-group name copp-acl-valid-ospf-peers match protocol ospf class copp-class-valid-ospf-peers insert-before copp-class-critical set cos 6 police cir kbps bc 250ms conform transmit violate drop L3 L2 Core Aggregation OSPF Peers

87 Scaling Older But Proven Designs Enterprise Scenario Deployed a every VLAN everywhere solution using RSTP VMs are automatically spun up and auto configure requiring access to all VLANs at every host Thus they cannot prune VLANs on interfaces They are running UDLD normal mode on the VPC Peer and Keepalive links They support frequent vmotions of applications and services UDLD

88 Risk of Over Scaling Older Designs Enterprise Scenario Due to their scale they are seeing Supervisor utilization rise: 85% of traffic seen on the Sup is CP traffic 92% CPU utilization Numerous processes 5-10% CPU utilization This combination led to UDLD false positives EMPTY ECHO condition detected VPC Peer Link and KeepAlive Links Error Disabled 2015 Jan 15 11:35:57 n7k1 %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Ethernet8/11, empty echo detected UDLD B 2 B V P C This is why we recommend to not run UDLD on VPC Peer and KAL links Becomes very busy during vmotion MAC moves and ARP gleans caused UDLD messages to not be sent

89 ARP: Scaling Servers and Link Detection Financial Scenario Slow growth of new servers everything is working as expected A switch reload causes servers to use standby links causing ARP Storm CoPP drops ARPs resulting in other serves swapping interfaces creates more ARPs Result cascading failure at the access layer Prolonged ARP storms cause larger network outage 30 min Lose Adjacencies Back to Back VPCs Core Aggregation ARP Storm across access layer OSPF Peers

90 Customize CoPP as You Grow Financial Scenario Use CoPP to classify well known network devices to ensure they have priority above other devices Classify and rate limit service devices known to use ARP for availability Isolate devices known to ARP frequently and rate limit Reduce the normal class accordingly to ensure CPU is not over consumed with additional classes Back to Back VPCs Add CoPP logging to be notified when ARP storms are seen Test during network failover and upgrade scenarios Core Aggregation OSPF Peers

91 Clustering Across Data Center Interconnects DCI Scenario VPC peer was reloaded, during reboot traffic is failing over, back to primary nexus There are small packet drop. Drop last 3 seconds multiple times (flows are getting redistributed as modules booting) this cause the cluster to go Active/Active This leads to a high rate of GARP for same IP and different MAC from 2 different parts of network Result: ARPs are dropped in general by CoPP. This leads to other servers and clusters failing and even more ARPs being sent Within 1 minute network is down until half of network is shut and clusters are manually recovered from problem state. DCI / OTV ARP Storm Data Center 1 Data Center 2

92 Clustering Across Data Center Interconnects DCI Scenario Use CoPP to isolate service cluster devices and ARP protocol Reduce the rate for this class to as little as possible during a failover event Subtract this traffic from the policy for the normal class of traffic ip access list copp-clustered-services permit ip /32 any permit ip /32 any class-map type control-plane match-all copp-class-clusters match access-group name copp-clustered-services match protocol arp class copp-class-clusters set cos 1 police cir 50 kbps bc 20 ms conform transmit violate drop class copp-system-p-class-normal set cos 1 police cir 630 kbps bc 230 ms conform transmit violate drop 680 kbps 250 ms by default

93 Wrap Up

94 Summary Do not turn off CoPP! Use Storm Control on host & edge ports Monitor Hardware Rate Limiters and CoPP enforcement Use CoPP to isolate and protect traffic from known and valid sources Nexus switches have the right tools to protect the control plane

95 Recommended Sessions BRKARC-2222 BRKARC-3470 BRKARC-3452 BRKDCT-3101 BRKDCT-3234 BRKDCT-3100 Cisco Nexus 9000 Architecture Cisco Nexus 7000/7700 Switch Architecture Cisco Nexus 5600/6000 Switch Architecture Nexus 9000 (Standalone) Architecture Brief & Troubleshooting Advanced Troubleshooting Cisco 7000 Series Troubleshooting Nexus 5600/6000 Series switches

96 Complete Your Online Session Evaluation Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card. Complete your session surveys though the Cisco Live mobile app or your computer on Cisco Live Connect. Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online