LON2 is Live LINX May

Transcription

1 LON2 21/05/18

2 LON2 is Live LINX May

3 The New Network LINX 101 3

4 The Live Network LINX May

5 Remaining ExtremeLAN LINX May

6 New Technologies > IP Fabric leaf/spine architecture > EVPN control plane > Proxy-ARP > MAC hold-down > Faster Reconvergence LINX May

7 MAC Hold-down > Local Flooding on old LON2 Interface down -> delete fdb entry Flooding as BUM is on ingress switch Need to wait for traffic to stop, or FDB to age out at ingress LINX May

8 New Behaviour > New LON2 behavior on Interface down Next hop replaced by temporary route to /dev/null BGP update to other peers BUM rate limits kick in Temporary route deleted > Looking at keeping route indefinitely so it continues to sponge ARP/ND LINX May

9 Last Hurdles LINX 101 9

10 Last minute challenges FEC recommended for 100G SR4 & AOC Tested clean for few months Got issue, and Finisar flagged running FEC recommended Not planned feature No FEC corrected counters in CLI or SNMP Uncorrected error rate falls off cliff Adding QoS to LAGs dependent on provisioning order Few operational niggles picked up during network commissioning Again configuration order dependent LINX May

11 Bringing up the NNI Low level drops picked up by Flowmon, (SLA measurement device) That included inside the non-migrated half Rolled back NNI activation, planned follow up Had potential root causes, with mitigations for each Turned out to be MAC Learning issue on BDX8 - feature we do not usually use. So migration continued without learning on Old LON2 side of NNI. It did make the migration quite a bit more fiddly. Not a new LON2 bug LINX May

12 Initial Site Migration Migration itself was successful Following day, one switch flapped all BFD sessions at once recovered fast Repeat following day on NNI facing switch, and more than one bounce Found issue that looked like targeted RIP sent to CPU for inspection High priority CPU Queue, which got full, and triggered ingress queuing BFD packets got delayed in queue CPU uplift rule edited by hand, including other similar traffic types LINX May

13 Collector Migration Cisco router did not install IPv6 Address Looked like Duplicate Address Issue Not seen on any previous member migration Suspect mis-configured member somewhere IPv6 Logging on 7200 not very helpful Possible to totally turn off Proxy ND. No Difference. Fixed itself after a few hours Not a new LON2 bug LINX May

14 One more release Fixes the FEC issues, including counters Fixes LAG QoS configuration sequence issue Makes CPU uplift rule permanent Root Cause Fix of putting BFD in higher priority queue than all other traffic Also fixing a few operator niggles Will include testing for related Bugs July/August LINX May

15 Still Work in Progress LINX

16 GigE Members not physically migrated yet > Still physically connected to X460 > But now acting as a ConneXions switch Each member in their own VLAN All traffic goes via local edge1 switch (AS5812) Fully part of EVPN > GigE auto-negotiation sometimes still can give surprises > Possibility of buffering issues > Did not want to run into those in the middle of the Migration LINX May

17 Buffers > GigE members: For now exact same buffers as before (same memory gates) Will migrate on a managed basis > All analysis points to buffers sufficient > Same family ASICs as within existing 1u Extreme switches > External buffers expensive including on power & complexity > If not needed, don t want to use them LINX

18 Not running Proxy-ND yet > ND has more ways to go wrong > Less strict at allocating addresses to members > Don t have MOU mandatory sessions to validate success > Will revisit after migration, should not be service affecting LINX May

19 Still relaxed on BUM rate limit > Are aggressive on Unknown > But Broadcast and Multicast relaxed for now Proxy ND not live yet If was an issue with Proxy-ARP, it can be disabled, so did not want rate-limit to kick-in Will get better counters on the new network to set to correct levels LINX May

20 Next Steps LINX

21 Software upgrade for Snags GigE Members and Proxy-ND Merge project code with IP Infusion Mainline code Full Netconf based automation EVPN Multihoming LINX May

22 Thank Yous LINX

23 Edge Core Believing in project helping kicking it off Investing development harness Hours and hours of testing often not in correct timezone Championed out cause with Broadcom Real support when things got tough LINX May

24 IP Infusion Great attitude Continuous prototyping Continuous learning/improvement Very Transparent Put a great team on the project I dread to think how many hours of development and testing Including at times thought weekend LINX May

25 LINX Staff Not killing me when I said Sorry we need another round of testing Long, often anti-social hours Software team integrated a new evolving vendor with little prior art Still being enthusiastic after 2 years LINX May

26 Our Members Giving us confidence to try something ambitious Backing when first announced (some a bit skeptical) Sticking with us through delays and setbacks Keeping your traffic on LON2 LINX May

27 Questions? 27