Forwarding Plane Correctness. Nick McKeown Stanford University

Transcription

1 Plane Correctness Nick McKeown Stanford University

2 App App App App App App App App App App App Specialized Applications Specialized Operating System Specialized Hardware Windows (OS) Open Interface or Linux or Open Interface Microprocessor Mac OS Vertically integrated Closed, proprietary Slow innovation Small industry Horizontal Open interfaces Rapid innovation Huge industry

3 App App App App App App App App App App App Specialized Features Specialized Control Plane Specialized Hardware Vertically integrated Closed, proprietary Slow innovation Open Interface Control or Plane Control or Plane Open Interface Merchant Switching Chips Horizontal Open interfaces Rapid innovation Control Plane

4 What is SDN? (when we clear away all the hype)

5 A network in which the control plane is physically separate from the forwarding plane. and A single control plane controls several forwarding devices. (That s it)

6 How do other industries check for correctness?

7 Making ASICs Work SpecificaTon FuncTonal DescripTon (RTL) FuncTonal VerificaTon Logical Synthesis StaTc Timing Place & Route Design Rule Checking (DRC) Layout vs SchemaTc (LVS) Layout ParasiTc ExtracTon (LPE) Testbench & Vectors Manufacture & Validate $10B tool business supports a $250B chip industry 100s of Books >10,000 Papers 10s of Classes

8 Making So\ware Work FuncTonal DescripTon (Code) SpecificaTon Testbench $10B tool business supports a $300B S/W industry StaTc Code Analysis Run- Tme Checker Invariant Checker InteracTve Debugger Model Checking 100s of Books >100,000 Papers 10s of Classes

9 Making Networks Work (Today) traceroute, ping, tcpdump, SNMP, Netflow. er, that s about it.

10 Networks are kept working by Masters of Complexity A handful of books Almost no papers No classes

11 Philosophy of Making Networks Work YoYo You re On Your Own

12 Even simple questons are hard 1. Can host A talk to host B? 2. What are all the packet headers from A that can reach B? 3. Are there any loops in the network? 4. Is Group X provably isolated from Group Y? 5. What happens if I remove a line in the config file? 12

13 A Bug Story Tue, Oct 2, 2012 at 7:54 PM: Between 18:20-19:00 tonight we experienced a complete network outage in the building when a loop was accidentally created by CSD- CF staff. We're investgatng the exact circumstances to understand why this caused a problem, since automatc protectons are supposed to be in place to prevent loops from disabling the network. 13

14 Why TroubleshooTng is Hard 14

15 Why TroubleshooTng is Hard L3 L2 Firewall Router Switch Match If dst If dst IP address MAC port address is 22 Y is X Decrement Send Drop out packet port TTL, 1 Update AcTon checksum, Modify dst MAC, Send out port 2 15

16 Why TroubleshooTng is Hard Control Plane Control Plane Control Plane Control Plane Control Plane 16

17 Why TroubleshooTng is Hard 17

18 TroubleshooTng Today ping traceroute tcpdump SNMP Tedious and ad hoc Requires skill and experience Not guaranteed to provide helpful answers 18

19 Neither observe or control Complex interacton Between multple protocols on a switch/router. Between state on different switches/routers. MulTple uncoordinated writers of state. Operators can t Observe all state. Control all state.

20 How SDN helps

21 Scoo Shenker at 1 st ONS in 2011 The Future of Networking and the Past of Protocols

22 So\ware Defined Network (SDN) f ( View) Control Programs f ( View) Control Programs f ( View) Control Programs Abstract Network View Network VirtualizaTon Global Network View Network OS Abstract Model (e.g. OpenFlow) Packet Packet Packet Packet Packet

23 The Technical Benefits of SDN 1. Well- defined control abstracton 2. Well- defined forwarding abstracton 3. Well- defined forwarding behavior

24 The Technical Benefits (1) Well- defined control abstracton Control plane can run on modern servers Can adopt so\ware engineering best- practces Easier to add new control programs or customize locally Solve distributed systems problem once, rather than for every protocol

25 OSPF 5% Dijkstra Network 95% Map OSPF Dijkstra Global Network Map Network OS OS Specialized Hardware Packet Packet Packet Packet

26 The Technical Benefits of SDN 1. Well- defined control abstracton 2. Well- defined forwarding abstracton 3. Well- defined forwarding behavior

27 The Technical Benefits (2) Well- defined forwarding abstracton e.g. OpenFlow Vendor- agnostc interface to forwarding plane Simpler, lower- cost, lower- power hardware

28 Match- AcTon AbstracTon L3 L2 Firewall Router Switch Match If dst If dst IP address MAC port address is 22 Y is X Decrement Send Drop out packet port TTL, 1 Update AcTon checksum, Modify dst MAC, Send out port 2 28

29 Match- AcTon AbstracTon Ac<on Primi<ves Plumbing primitves 1. Forward to ports 4 & 5 2. Push header Y a\er bit Pop header bits Decrement bits Drop packet 6. H H Match Action F Action(F) G H Action(G) Action(H)

30 MulTple Table Match- AcTon H n H 1 H Match Action Match Action F 1 Action(F) F n Action(F) G 1 Action(G) G n Action(G) H 1 Action(H) H n Action(H)

31 OpenFlow Philosophy Long- term, forwarding looking Match: Very general, not protocol specific. Ac<on: Small instructon set, not protocol specific. Make it easy to add new headers and actons. Any network (packet, circuit, radio). Short- term, backward looking Match: include well- known header fields. Ac<on: necessary set for existng protocols. Support existng protocols on existng switch chips.

32 The Technical Benefits of SDN 1. Well- defined control abstracton 2. Well- defined forwarding abstracton 3. Well- defined forwarding behavior

33 The Technical Benefits (3) Well- defined forwarding behavior The forwarding tables capture the entre forwarding behavior. Control plane writes the forwarding state. Therefore, we can verify its correctness.

34 So\ware Defined Network (SDN) firewall.c Control Program if( TCP_port == SMTP ) Control droppacket(); Program Control Program Match Action F Packet G Action(F) Action(G) H Action(H) Global Network Map Match Action Packet A Action(A) B Action(B) C Action(C) Match Action A Network OS Action(A) Packet G Action(G) Match Action X Action(X) Packet Y Action(Y) Z Action(Z) Match Action A Action(A) Packet G Action(G) H Action(H) D Action(D)

35 So\ware Defined Network (SDN) firewall.c Control Program if( TCP_port == SMTP) Control droppacket(); Program Control Program Match Action F Packet G Action(F) Action(G) H Action(H) Global Network Map Match Action Packet A Action(A) B Action(B) C Action(C) Match Action A Network OS Action(A) Packet G Action(G) Match Action X Action(X) Packet Y Action(Y) Z Action(Z) Match Action A Action(A) Packet G Action(G) H Action(H) D Action(D)

36 So\ware Defined Network (SDN) Match Control Program Action F Packet G H Action(F) Action(G) Action(H) Global Network Map Match Action Packet A Action(A) B C Match A Packet G Control Program Network OS Action(B) Action(C) Action Action(A) Action(G) Match X Action Action(X) Packet Y Z Action(Y) Action(Z) Control Program Match A Action Action(A) Packet G H Action(G) Action(H) Policy A can talk to B Guests can t reach PaTentRecords No loops Behavior D Action(D)

37 Methods for Correctness 1. StaTc Checking (e.g. HSA, Anteater) Independently checking correctness 2. Dynamic Checking (e.g. NetPlumber, Veriflow) Checking new state before it is added 3. AutomaTc TesTng (e.g. ATPG) Is the datapath behaving correctly? 4. InteracTve Debugging (e.g. NetSight) Finding bugs, and their root cause, in an operatonal network

38 Header Space Analysis [NSDI 12] Header L Data Header Data

39 Header Space Analysis 1 Match + AcTon 2

40 The set of packets from A that can reach B T 1 (X, P in ) T 2 (T 1 (X, P in )) T 3 (T 2 (T 1 (X, P in ))) [ T 3 (T 4 (T 1 (X, P in ))) T 2 (h, p) T 1 (h, p) T 1 (X, P in ) T 3 (h, p) T 4 (T 1 (X, P in )) T 4 (h, p)

41 All packets from A that can reach B T 2 (h, p) T 1 (h, p) T 3 (h, p) T 4 (h, p)

42 ImplicaTons Short term: 1. Finds all packets from A that can reach B 2. Find loops, regardless of protocol or layer 3. Can prove that two groups are isolated 4. for (mostly) stateless graph mappings Longer term: 1. Abstract forwarding model; protocol independent 2. Analagy to Boolean algebra for digital logic design 3. Can be a foundaton for a variety of tools

43 Methods for Correctness 1. StaTc Checking (e.g. HSA, Anteater) Independently checking correctness 2. Dynamic Checking (e.g. NetPlumber, Veriflow) Checking new state before it is added 3. AutomaTc TesTng (e.g. ATPG) Is the datapath behaving correctly? 4. InteracTve Debugging (e.g. NetSight) Finding bugs, and their root cause, in an operatonal network

44 Packet Histories Basic idea: Capture forwarding plane events Look for errant behavior (real Tme or offline) Enormous amounts of data: Scalability is essental Another example of Big Data

45 NetSight A playorm to capture and filter packet histories 45

46 A TroubleshooTng Construct A B Symptom: Errant packet behavior in the dataplane + Cause: Packet History 46

47 Packet History Packet history = Path taken by a packet + Header modificatons + Switch state encountered 47

48 TroubleshooTng Workflow A B 1. Ask for the packet history of errant packets 2. Use the packet history to diagnose the root cause 48

49 Packet History How to specify packet histories of interest? 49

50 Querying Packet Histories Switch 1: { header: H1, inport: p0, outports: [p1] mods: [...] state version: 3 } Switch 2: { header: H2, inport: p0, outports: [p3] mods: [...] state version: 5 } Switch N: { header: HN, inport: p3, outports: [p2] mods: [...] state version: 13 } Packet History Filter: A regular- expression- like language to specify packet histories of interest 50

51 Postcard Filter Switch 1: { header: H1, inport: p0, outports: [p1] mods: [...] state version: 3 } Postcard Filter --bpf [not] <BPF> --dpid [not] <switch ID> --inport [not] <input port> --outport [not] <output port> --version [not] <state version> Example: --bpf "ip src " - dpid 5 --inport not 1 51

52 Building PHFs using Postcard Filters PHF to match packets that: start at switch X, and end at switch Y: {{--dpid X}}.*{{--dpid Y}}$ are of type UDP, start at switch X, never reach Y: {{--bpf "udp" -dpid X}}[^{{--dpid Y}}]*$ 52

53 Control Plane Flow Table Recorder Postcard Collector 53

54 Match Control ACT Plane Flow Match Table ACT Recorder 1. Stamp Postcard InformaTon 2. Add postcard ID tag Postcard Collector 3. Send copy out of specific port 54

55 Control Plane Flow Table Recorder Postcard Collector 55

56 Control Plane Flow Table Recorder Packet Header Switch ID Output port Version Postcard Collector Version - > Flow Table 56

57 1. <Match, AcTon> 2. <Match, AcTon> 3. <Match, AcTon> 4. <Match, AcTon> 5. <Match, AcTon> Control Plane Flow Table Recorder 1. <Match, AcTon> 2. <Match, AcTon> 3. <Match, AcTon> 4. <Match, AcTon> 5. <Match, AcTon> <Match, AcTon> 2. <Match, AcTon> 3. <Match, AcTon> 4. <Match, AcTon> 5. <Match, AcTon> <Match, AcTon> 2. <Match, AcTon> 3. <Match, AcTon> 4. <Match, AcTon> 5. <Match, AcTon> Postcard Collector 57

58 Methods for Correctness 1. StaTc Checking (e.g. HSA, Anteater) Independently checking correctness 2. Dynamic Checking (e.g. NetPlumber, Veriflow) Checking new state before it is added 3. AutomaTc TesTng (e.g. ATPG) Is the datapath behaving correctly? 4. InteracTve Debugging (e.g. NetSight) Finding bugs, and their root cause, in an operatonal network

59 <end>