SDN Workshop. Contact: TSDN01_v0.1. [xx] Revision:

Transcription

1 SDN Workshop Contact: Issue Date: [Date] TSDN01_v0.1 Revision: [xx]

2 Routers Two key roles: Determining network paths Packet forwarding 2

3 Today s router Management High Availability FCAPS CLI SNMP Resiliency Protocols Hardware Redundancy Network Layer Services Layer Routing Protocols (unicast/multicast) RIB IP L2 L3 Application Layer (DPI etc) QoS Traffic Managers Packet Memory Queue Management Scheduling Algorithms Accounting Other Hardware Security CPUs Control Memory Switch Fabric (T)CAM AAA Network Interfaces ASICs NPUs FIB CPU Protection 3

4 How did we get here? Distribution of complexity Backwards compatibility Unanticipated applications Need for higher performance End-to-end principle Better scaling Survivability;; spreading of risk Flag days not realistic Short-term, incremental evolution of technology;; no major overhaul in last 20 years Networking is a victim of its own success New applications have been delivered on top of existing capabilities Tight coupling between different planes seen as critical for delivering higher performance 4

5 Clean Slate Project (1) Two research questions: With what we know today, if we were to start again with a clean slate, how would we design a global communications infrastructure How should the Internet look in 15 years? Mission: Re-invent the Internet 5

6 Clean Slate Project (2) One of the flagship projects was Internet Infrastructure: OpenFlow and Software Defined Networking Seminal paper on OpenFlow...kicked off the SDN movement and the data communications world would never be the same again 6

7 OpenFlow: The Problem Initial Problem: A mechanism was required for researchers to run experimental network protocols. Open software platforms did not provide the required performance and commercial solutions were too closed and inflexible. Closed systems;; only functionality exposed by vendors is available Software Hardware Tight coupling Challenge: how do we influence packet switching/forwarding behaviour? 7

8 OpenFlow: The Solution (1) Control Plane Protocols and algorithms to calculate forwarding paths OpenFlow Controller Control Plane Data Plane Forwarding frames/packets based on paths calculated by control plane OpenFlow Protocol Control Plane Routing/Bridging Protocols, RIBs, routing policy and logic Secure Channel Data Plane Forwarding Tables Abstracted Flow Table Data Plane FROM TO 8

9 OpenFlow: The Solution (2) OpenFlow Controller OpenFlow Protocol Secure Channel Control Plane The Solution: OpenFlow provided a compromise that provided a means of influencing switching/routing decisions without opening up network software. Abstracted Flow Table Data Plane The control software would run on a controller;; the outcomes of the calculations would be pushed down to the data plane running on the network element 9

10 OpenFlow: How it works (1) OpenFlow Controller OpenFlow Protocol Secure Channel Control Plane Adds, deletes and modifies flow table entries Header Fields* Actions Counters Flow 1 Forward to port 1/1 Flow 2 Flow n Drop Send to controller Abstracted Flow Table Switch forwards traffic by matching against header fields and taking corresponding actions * Ingress Port, Ethernet SA, Ethernet DA, VLAN ID, VLAN PCP, IP SA, IP DA, IP Proto, IP ToS, Source L4 Port, Dest L2 Port etc. 10

11 OpenFlow: How it works (2) One controller manages many switches OpenFlow Controller OpenFlow Protocol Control Plane OpenFlow Protocol Secure Channel Secure Channel Secure Channel Abstracted Flow Table Abstracted Flow Table... Abstracted Flow Table Switch 1 Switch 2 Switch n 11

12 OpenFlow: Today Initially synonymous with SDN Today, OpenFlow is relegated to being just a part of the greater SDN architecture, with other protocols competing in the same space It is, however: The spark that lit the fuse that started the fire (SDN) 12

13 OpenFlow: Implications Two primary implications: The control plane (processes to determine how traffic is handled) is physically decoupled from the data plane (forwards traffic according to decisions passed down by the control plane). The control plane is consolidated and centralised: a single software control plane controls multiple data planes (previously a 1:1 correspondence). 13

14 The Birth of SDN The separation of control and data plane was not an objective in itself but was a consequence of the compromise approach taken by OpenFlow It heralded a new era of programmability that has been vastly enhanced with new architectures and capabilities The term SDN itself was coined in an article about the OpenFlow project at Stanford ( 14

15 Emergence and evolution of SDN OpenFlow was a starting point Ushered in an era of programmability But a complete decoupling of the control plane and data plane was not practical: We would have had to solve all the problems the industry had spent decades solving and refining: resiliency, scalability, convergence, redundancy etc SDN architecture today Hybrid approach where some elements of the control plane remain distributed while others are centralised. Many different architectural models All of them aspire to achieve the goals of agility and network programmability 15

16 Defining SDN ONF: The physical separation of the network control plane from the forwarding plane, and where a control plane controls several devices. This definition is too narrow SDN is A new approach to networking that provides greater network agility and flexibility by: Automation through enhanced programmability and open interfaces Dis-aggregation and abstraction Centralisation of network control with real-time network visibiity As much a marketing term as a technical one 16

17 Objectives and benefits of SDN Quicker introduction of new services for faster time to revenue Agility Automation Service provisioning Network provisioning Service automation Reduction in hardware and network operations costs CAPEX/OPEX reduction Programmability Abstraction via simplified, open interfaces End-to-end optimisation Centralised Control End-to-end service and network management 17

18 SDN SDOs 18

19 SDN architectural framework (1) SDN Applications ITU-T Y.3300 SDN Controllers Network Resources 19

20 SDN architectural framework (2) Application Plane Application Service RFC 7426 Control Plane Network Services Abstraction Layer Service Interface Management Plane Service App App Control Abstraction Layer (CAL) Mgmt Abstraction Layer (MAL) CP Southbound Interface MP Southbound Interface Device & Resource Abstraction Layer (DAL) Forwarding Plane App Operational Plane Network Device 20

21 SDN architectural framework (3) Application Plane Northbound Interfaces Application Service REST/RESTCONF/NETCONF/XMPP Network Services Abstraction Layer Control Plane (controller) Topology Discovery & Management Route selection & failover Traffic Engineering Resource Management Configuration East/Westbound interfaces BGP Southbound Interfaces BGP-LS i2rs PCE-P ForCES IPFIX SNMP Open Flow Netconf BGP RIBs PCC SNMP MIBs OpenFlow YANG Data Plane Segment Routing RSVP-TE Device & Resource Abstraction Layer (DAL) Network Devices IP/MPLS/Transport Note: designations of north-bound and south-bound are relative to the control plane ( controller ) 21

22 Elements of SDN architecture (1) Application Plane Northbound Interfaces Application Service REST/RESTCONF/NETCONF/XMPP Network Services Abstraction Layer Application Plane Consumers of the network Traffic optimisation applications OSS systems End-customer self-service portals Etc. Northbound interfaces Abstraction of network services towards applications and services Network Services Abstraction Layer: Normalises network and service constructs via an open API or interfaces - YANG models, NETCONF, RESTCONF 22

23 Elements of SDN architecture (2) Northbound Interfaces Network Services Abstraction Layer Control Plane (controller) Topology Discovery & Management Route selection & failover Traffic Engineering Resource Management Configuration East/Westbound interfaces BGP Southbound Interfaces Control Plane layer The Controller ;; the brains of the operation Translates high-level instructions from north-bound interfaces and converts them to instructions for the resource layer Collection of key functions: Topology discovery Traffic engineering Resource management Route selection and failover Service configuration Mediation Southbound interfaces 23

24 Elements of SDN architecture (3) Southbound Interfaces BGP-LS i2rs PCE-P ForCES IPFIX SNMP Open Flow Netconf BGP RIBs PCC SNMP MIBs OpenFlow YANG Data Plane Segment Routing RSVP-TE Device & Resource Abstraction Layer (DAL) Network Devices IP/MPLS/Transport Southbound interfaces Myriad interfaces, plug-ins, and protocols, including OpenFlow Device-specific details abstracted from higher layers of the controller Data Plane Traditional and newer generation dataplanes, physical and virtual Augmented by SDN-friendly protocols such as Segment Routing 24

25 Key SDN use cases Data Centre network automation Most widely-deployed and mature solution Automation of network connectivity via overlay networks Multi-tenancy SD-WAN Extension of DC automation concepts Site connectivity via overlay networking Service Automation & provisioning Direct customer access via portals Bandwidth on demand Bandwidth calendaring Network optimisation Link and path optimisation based on real-time network state Running networks "hotter" 25

26 Open source projects 26

27 Comparing and contrasting with NFV SDN: decouples control plane and data plane NFV: decouples network software from closed, proprietary hardware systems Software Tightly coupled Purposebuilt hardware Virtualised Software COTS hardware FROM TO 27

28 OpenFlow SDN Workshop Issue Date: [Date] TSDN01_v0.1 Revision: [xx]

29 OpenFlow versions From v1.0.0 in 2009 to v1.5.2 in 2015 Developed by the ONF since its foundation in March 2011 We shall start with v1.0.0 to get a basic understanding of how it operates. Note that there are significant changes in newer versions that will be pointed out. 29

30 OpenFlow revision timeline Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

31 OpenFlow revision: features <<Summarise features introduced in each release>> 31

32 OpenFlow v

33 OpenFlow v1.0.0 Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

34 OpenFlow v1.0.0 First non-experimental release version Wire protocol 0x01 December 31, 2009 Specified two types of OpenFlow-compliant switches*: OpenFlow-only: perform forwarding based purely on OpenFlow flow tables OpenFlow-enabled: support traditional Ethernet switching and routing functions in addition to OpenFlow packet forwarding * Definition was modified in later revisions of OpenFlow 34

35 OpenFlow components OpenFlow Controller OpenFlow Protocol Secure Channel An OpenFlow switch communicates with a controller over a secure connection using the OpenFlow protocol. Flow Table OpenFlow Switch 35

36 OpenFlow Switch components Flow table (single): Performs packet lookup and forwarding Secure channel: To an external controller which manages the switch using the OpenFlow protocol 36

37 Flow table Contains: A set of flow entries (e.g. header fields to match against packets) Zero or more actions to apply to matching packets Activity counters that are updated for matching packets Header Fields Actions Counters Flow entry 1 Forward to port 1/1 Flow entry 2 Flow entry n Drop Send to controller 37

38 Match fields Ingress port Ethernet source address Ethernet destination address Ethertype VLAN ID VLAN priority IP source address IP destination address IP protocol IP ToS bits TCP/UDP source port TCP/UDP destination port Match can be an exact value or ANY which matches any value (wildcard) Some fields may have dependencies e.g. IP protocol field can only be used if there is a corresponding match for the IPv4 EtherType. 38

39 Actions (1) Each flow has zero or more actions that determine how the switch handles matching packets If no forward actions are specified, the packet is dropped Forward (output) action [required]: Forwarding of packet to physical or virtual ports Enqueue action [optional]: Forward a packet through a specified queue attached to a port 39

40 Supported Actions Output Action Output to switch port Description Set VLAN VID Set VLAN PCP Strip VLAN Set Ethernet source address Set Ethernet destination address Set IP source address Set IP destination address Set IP ToS Set TCP/UDP source port Set TCP /UDP destination port Enqueue Set the IEEE802.1q VLAN ID Set IEEE802.1q priority Strip the IEEE802.1q header Set Ethernet source address Set Ethernet destination address Set IP source address Set IP destination address Set IP Type of Service (ToS) bits Set TCP/UDP source port Set TCP /UDP destination port Output packet to a queue 40

41 Actions (2) Drop action [required]: Implicit action associated with a flow-entry that has no specified action Modify-field action [optional]: Specify modification of packet header fields 41

42 OpenFlow ports (1) Physical ports: Correspond to physical hardware ports For an OpenFlow-enabled switch, these are ports that have been explicitly configured to be OpenFlow ports 42

43 OpenFlow ports (2) Virtual ports: ALL : all OpenFlow interfaces except the incoming interface CONTROLLER : logical interface to the OpenFlow controller LOCAL : local networking stack of the switch TABLE : sends packet for processing through the flow table (only for Packet-Out messages) IN_PORT : ingress port of packet NORMAL : processes packets via the traditional forwarding path supported by the switch FLOOD : flood along the minimum spanning tree 43

44 Modify-field actions Set VLAN ID Set VLAN priority Strip VLAN header Modify Ethernet source address Modify Ethernet destination address Modify IPv4 source address Modify IPv4 destination address Modify IPv4 ToS bits Modify TCP/UDP source port Modify TCP/UDP destination port 44

45 Counters Per-table, per-flow, per-port and per-queue Per-table Active Entries Packet lookups Packet matches Per-flow Received packets Received bytes Duration (seconds) Duration (nanoseconds) Per-port Received packets Transmitted packets Received bytes Transmitted bytes Receive drops Transmit drops Receive errors Transmit errors Receive frame alignment errors Received overrun errors Receive CRC errors Collisions Per-queue Transmit packets Transmit bytes Transmit overrun errors 45

46 Flow table example <Give an example of a flow table with realistic entries> Example for ARP packet NAT function VID addition 46

47 Basic packet processing All packets ingressing the switch via an OpenFlow port are compared against the flow table. If a matching entry is found, any actions for that entry are performed on the packet e.g. forward to a specified port If no match is found, the packet is forwarded to the controller over the secure channel 47

48 Matching Packet ingressing into switch Parse header fields (see next slide) Match flow table? Yes Perform actions No Send to controller via Secure channel 48

49 Header field parsing Initialise headers Set input port, Ethernet source & destination, Ethertype Eth type = 0x8100? Yes Set VLAN ID and PCP No Eth type = 0x0806? Yes Set IP source/dest from within ARP packet IP Proto = 6 or 17? Yes Use UDP/TCP source and dest for L4 fields No No Eth type = 0x0800? Yes Set IP source/dest. protocol and ToS fields Not IP Fragment? Yes IP Proto = 1? Yes Use ICMP type and code for L4 fields No No Packet Lookup No 49

50 OpenFlow Secure Channel This is the logical interface that connects each OpenFlow switch to an OpenFlow controller. The OpenFlow controller uses this interface to: Configure and manage the switch Add, delete and modify flow entries Receive events from the switch Send packets out the switch OpenFlow version only supported use of a single controller. 50

51 Switch bootstrapping 51

52 Connection setup A TLS connection is established by the switch to a configured IP address and TCP port 6633 (changed in later releases to an IANA-assigned port number) Traffic to/from the secure channel is not processed by the flow table 52

53 Connection interruption If connectivity with the controller is lost, the switch enters emergency mode. In emergency mode : Matching process is based only on flow table entries marked with the emergency bit (indicated via the Flags field of the Flow-Mod message) All non-emergency entries are deleted when entering emergency mode On initial startup, all switches are in emergency mode Emergency mode was removed from later versions. CHANGED BEHAVIOUR IN LATER VERSIONS 53

54 OpenFlow protocol messages Protocol defines three types of messages. Controller-to-switch: Are initiated by the control and used to configure the switch or query its state Asynchronous: Are initiated by the switch and used to notify the controller about network events or changes to the switch state Symmetric: Can be initiated by either the controller or the switch and sent without solicitation 54

55 Controller-to-Switch messages Initiated by the controller and may or may not require a response from the switch. Messages include: Features: used by the controller to discover the capabilities supported by the switch Configuration: used to set and query configuration parameters Modify-state: sent by the controller to manage state on the switch. Main purpose is to add/delete/modify flows Read-state: used by the controller to query stats from the switch Packet-out: used by the controller to send a packet out of a specified port of the switch Barrier: used to ensure message dependencies 55

56 Asynchronous messages Initiated by the switch without solicitation from the controller. Messages include: Packet-in: sent to the controller for all packets that: do not have a matching flow entry OR are explicitly sent to the controller Flow-removed: sent when flows are removed from the flow-table. May be due to expiration or explicit deletion. Port-status: sent by the switch on port configuration or state changes. Errors: sent when errors are detected 56

57 Symmetric messages Can be initiated by either the controller or the switch and sent without solicitation Messages include: Hello: sent between the controller and switch upon connection establishment Echo: echo request/reply messages can be sent from either the switch or the controller;; request messages must be responded to with a reply. Vendor: vendor-specific messages 57

58 OpenFlow protocol Common OpenFlow packet header All OpenFlow messages start with this header 32 bits 8 bits 8 bits 16 bits version type length xid Version: Type: Length: xid: - version of OpenFlow protocol - type of OpenFlow protocol message - total length of message in octets - transaction ID used to match responses with requests 58

59 OpenFlow version numbers Version number has incremented with every major release of the OpenFlow specification Version of specification 1.0.x 1.1.x 1.2.x 1.3.x 1.4.x 1.5.x OpenFlow protocol version 0x01 0x02 0x03 0x04 0x05 0x06 59

60 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Vendor Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Port Mod 16 Stats Request 17 Stats Reply 18 Barrier Request 19 Barrier Reply 20 Queue Get Config Request 21 Queue Get Config Reply Features Configuration Packet-out Modify-state Read-state Barrier Configuration 60

61 Version negotiation On connection establishment: Each side sends a Hello message with the version set to the highest OpenFlow version supported by the sender. The Hello message does not have a body, just the OpenFlow header. The OpenFlow protocol negotiated independently by each side is the lower of (version number that was sent, version number that was received) This only works if the side with the higher version number also supports the lower version number. If not, an error occurs. 61

62 Understanding switch capabilities Due to the large number of required and optional OpenFlow capabilities, it is important for the controller to understand the features supported by the switch it is managing. A features/capabilities discovery is done via a handshake to acquire this information. 62

63 Handshake Once TLS session is established, the controller sends a Features Request message. The switch responds with a Features Reply message: #tables 32 bits datapath id #buffers Padding Capabilities actions port descriptors Datapath ID: - Uniquely identifies a datapath. Lower 48-bits are the switch MAC address. Capabilities: - Types of stats supported etc. Actions: - Action types supported by switch Ports: - Array of OpenFlowenabled physical ports 63

64 Flow table modification messages 5 possible operations: Add: instantiates a new flow entry in the flow table Modify: modifies elements of all matching (existing) flow entries Modify-Strict: modifies elements of flow entries that exactly match all fields including wildcards and priority Delete: deletes all matching (existing) flow entries Delete-Strict: deletes flow entries that exactly match all fields including wildcards and priority 64

65 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits flow match descriptor Cookie: - Opaque value set by the controller command hard timeout output port cookie buffer id idle timeout priority flags Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority flow action descriptor 65

66 Flow match descriptor Flow match descriptor structure Structure used to describe flow match requirements 32 bits wildcard fields ingress port ethernet source address ethernet destination address pcp ip tos padding ip protocol tcp/udp source port ipv4 source address ipv4 destination address vid ethertype padding tcp/udp dest port Wildcard fields: - Bitmap indicating which fields are wildcards CHANGED BEHAVIOUR IN LATER VERSIONS 66

67 Flow action descriptor Flow action descriptor structure Structures used to describe flow actions 32 bits 32 bits Type= OUTPUT Length Type= SET VLAN ID Length Output port Max Length VID Padding 32 bits 32 bits Type= ENQUEUE Length Type= SET NETWORK SRC/DST Length Port IP address Padding Queue ID CHANGED BEHAVIOUR IN LATER VERSIONS 67

68 Queue structures Limited QoS support is provided through a simple queuing mechanism Flows can be mapped to queues which attach to a port and can be used to schedule the packets exiting the datapath on that output port Each queue is identified by a port number and a queue ID The only queues configuration available are: min-rate: minimum guaranteed data-rate max-rate: maximum data-rate 68

69 Proactive vs reactive flow entries <Explain the two approaches> 69

70 Flow removal All flow entries have two timers associated with them: idle_timeout: maximum time that can elapse without a flow matching the flow entry hard_timeout: maximum time that a flow entry can remain in the flow table A Flow-removed message is sent by the switch to the controller when a flow entry is removed from the flow table 70

71 Send Packet Message Packet-Out message structure For packets sent from the controller to the switch 32 bits input port buffer id size of actions array Buffer ID: - Same as the buffer ID in the original Packet-In message flow action descriptor Packet data: - Initial portion of packet packet data 71

72 Packet-In Message Packet-Out message structure Structure used to add/delete/modify flow entries buffer id total length input port reason padding data (ethernet frame) Buffer ID: - Identifies where packet is buffered Reason: - Either due to no match or explicit action Data: - Initial portion of packet 72

73 Echo Request/Reply Messages Echo Request may be initiated by either the controller or the switch May be used for a number of reasons: To determine latency of connection between controller and switch To verify liveness of the connection between controller and switch 73

74 Message flow example Controller Switch Hello Hello Features Request Features Reply 74

75 Topology discovery <proactive rules for dealing with LLDP packets> If ethertype=lldp, output to CONTROLLER 75

76 Exclusions What OpenFlow does not do (or specify): Communication between controllers when using multiple controllers (v1.2.0+) 76

77 Versions 1.0.1/1.0.2 Both releases were errata and/or clarifications for the specification Clarifications: Packets that do not match any flow should be forwarded to the controller using a Packet-In message Changes: In addition to emergency mode, a new fail-secure mode was defined. In fail-secure mode, all packets and messages destined to the controller are dropped. Flow entries continue to be used and expire based on their timeouts. IANA allocated port 6653 for OpenFlow communications and was required to be used as the default port (for both TLS or plain TCP) 77

78 OpenFlow v

79 OpenFlow v1.1.0 Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

80 OpenFlow v1.1.0 Second major release version Wire protocol 0x02 February 28, 2011 This section highlights deltas from the previous release v

81 New features Multiple flow tables, pipeline and pipeline processing Group table Actions in flow table changed to instructions which describe a set or list of actions Outcome for packets without a match in a flow table are configurable Per-packet metadata for communication between tables Vendor message changed to Experimenter message 81

82 OpenFlow switch types Specifies two types of OpenFlow-compliant switches: OpenFlow-only: perform forwarding based purely on OpenFlow flow tables OpenFlow-hybrid: support traditional Ethernet switching and routing functions in addition to OpenFlow packet forwarding (was referred to as OpenFlow-enabled in v1.0.0). 82

83 OpenFlow components Controller OpenFlow Protocol Secure Channel Group Table Flow Table Flow Table OpenFlow Switch 83

84 Flow tables Each flow table contains a set of flow entries Each flow entry consists of: Match fields Counters Set of instructions to apply to matching packets Match Fields Counters Instructions Ingress port Packet headers Metadata Modify action set Apply actions Modify pipeline processing 84

85 New flow table elements Metadata: A maskable register value that is used in order to carry information from one flow table to another Instructions: either a set of actions to add to the action set, a list of actions to apply immediately to the packet or a modification to pipeline processing. Action Set: a set of actions associated with a packet that are accumulated while the packet is processed by each table. The action set is executed when the instruction set instructs the packet to exit the processing pipeline. 85

86 Match fields Ingress port Metadata Ethernet source address Ethernet destination address Ethertype VLAN ID VLAN priority MPLS label MPLS traffic class IPv4 source address IPv4 destination address IPv4 protocol IPv4 ToS bits TCP/UDP source port TCP/UDP destination port Fields new to v1.1.0 are in bold. Match can be an exact value or ANY which matches any value (wildcard) Some fields may have dependencies e.g. IP protocol field can only be used if there is a corresponding match for the IPv4 EtherType. 86

87 Header field parsing Initialise headers Set input port, Ethernet source & destination, Ethertype VLAN Tag Eth type = 0x8100? Yes Set VLAN ID & PCP Skip any remaining VLAN tags No MPLS label ARP Switch supports MPLS? No Supports ARP Yes No Yes Eth type = 0x8847/8? Eth type = 0x0806? Use MPLS label and TC Set IP source/dest from within ARP packet Skip and remaining MPLS shim headers IP Proto = 6 or 17? Yes Use UDP/TCP/SCTP source and dest for L4 fields No No No IPv4 Eth type = 0x0800? Yes Set IP source/dest. protocol and ToS fields Not IP Fragment? Yes IP Proto = 1? Yes Use ICMP type and code for L4 fields No No Packet Lookup No

88 Supported Actions Action Description Action Description Output Output to switch port Set MPLS label Set value of MPLS label Set VLAN VID Set the IEEE802.1q VLAN ID Set MPLS TC Set MPLS TC bits Set VLAN PCP Set IEEE802.1q priority Set MPLS TTL Set value of MPLS TTL Set Ethernet src addr Set Ethernet dest addr Set IP src addr Set IP dest addr Set IP ToS Set IP ECN Set TCP/UDP src port Set TCP /UDP dest port Set Ethernet source address Set Ethernet destination address Set IP source address Set IP destination address Set IP Type of Service (ToS) bits Set IP ECN bits Set TCP/UDP source port Set TCP /UDP destination port Decrement MPLS TTL Push VLAN Pop VLAN Push MPLS Pop MPLS Set queue Set group Set IP TTL Decrement value of MPLS TTL Push a new VLAN header Pop the outermost VLAN header Push a new MPLS label Pop the outermost MPLS label Set ID of queue to output packet to Set group ID Set value of IP header TTL Copy TTL out Copy TTL from next-tooutermost to outermost header Decrement IP TTL Decrement value of IP TTL Copy TTL in Copy TTL from outermost header to next-to-outermost Actions new to v1.1.0 are in bold. 88

89 Actions (1) Output (forward) action [required]: Forwarding of packet to physical or virtual ports Set-Queue action [optional]: Forward a packet through a specified queue attached to a port Drop action [required]: Implicit action associated with a flow-entry that has no specified action 89

90 Actions (2) Group action [required]: processes the packet through the specified group Push-Tag/Pop-Tag action [optional]: Push/pop of VLAN and MPLS headers Set-Field action [optional]: Set packet header fields, manipulate TTL 90

91 Action Set (1) An Action Set is associated with each packet and is empty by default As the packet passes through the pipeline the Action Set is modified by instructions (Write-Actions, Clear-Actions) of matching flow entries The Action Set is carried between flow tables as the packet progresses through the pipeline There is a maximum of one action of each type in the Action Set. 91

92 Action Set (2) The Action Set is executed when an instruction set does not include a Goto-Table action If no output action or group action are specified in an action set, the packet is dropped 92

93 Actions Order of application of actions in the Action Set Order Action 1 Copy TTL inwards 2 Pop 3 Push 4 Copy TTL outwards 5 Decrement TTL 6 Set 7 QoS 8 Group 9 Output If no output action or group action are specified in an action set the packet is dropped. 93

94 Action List Associated with the Apply-Actions instruction and the Packet-Out message. Actions in the Action List are immediately executed in the order specified in the list. Multiple actions of the same type may appear in the same Action List and have a cumulative effect. 94

95 Group table Flow entries may point to a group in the group table. Group provide sets of actions for flooding, multipath, fast reroute, link aggregation and indirection. The group table contains group entries. Each group entry has a list of action buckets with semantics depending on group type. The group type determines which of the buckets are applied to each packet. 95

96 Group table The group table contains group entries. Each group entry contains: Group Identifier Group Type Counters Action Buckets Group Type Description all Executes all buckets in the group Multicast/broadcast forwarding Packet is replicated for each bucket select Executes one bucket in the group Packets are sent to a single bucket, based on a hash algorithm indirect Executes the one defined bucket in the group For example, BGP next-hop indirection fast-failover Executes the first live bucket Bucket liveness tied to port(s) or group 96

97 Counters Per-table Active Entries Packet lookups Packet matches Per-flow Received packets Received bytes Duration (seconds) Duration (nanoseconds) Per-port Received packets Transmitted packets Received bytes Transmitted bytes Receive drops Transmit drops Receive errors Transmit errors Receive frame alignment errors Received overrun errors Receive CRC errors Collisions Per-queue Transmit packets Transmit bytes Transmit overrun errors Per-group # flow entries Transmit bytes Transmit overrun errors Per-bucket Packet count Byte count Fields new to v

98 Instructions Executed when a packet matches a flow entry. Supported instructions include: Apply-Actions: immediately applies the specified actions. The Action Set is not modified. Clear-Actions: clears all actions in the Action Set Write-Actions: merges the specified actions into the current Action Set. Write-Metadata: writes to the metadata field Goto-Table: indicates that the packet should next be processed through the specified table Each instruction type may only appear once in the instruction set. 98

99 Matching (1) Packet In Start at table 0 Yes Match in table n? Yes Update counters Execute instructions: update action set update packet/match set fields update metadata Goto table n? No No Do one of following, depending on table configuration: send to controller drop continue to next table Execute action set 99

100 Matching (2) Every flow entry has a 16-bit priority value associated with it It is possible that a packet may match more than one flow entry. Only the highest-priority flow entry matching the packet is used as the matching flow entry for the packet. 100

101 Pipeline processing (1) Definition: the set of linked tables that provide matching, forwarding and packet modifications in an OpenFlow switch Matching starts at the first table and may continue to other tables If a matching entry is found, the instructions associated with the flow entry are executed. 101

102 Pipeline processing (2) Pipeline processing stops when the instruction set associated with a matching flow entry does not specify a next table. The packet s action set is processed and it is forwarded at this point. If no match is found (called a table miss), the behaviour depends on switch configuration;; the packet may: Be forwarded to the controller (default option) Continue to the next table Be dropped 102

103 Pipeline processing (3) Packet In Ingress port Action set = {} Flow Table 0 Packet+ Ingress port + metadata Action set Flow Table 1 Flow Table n Packet Action set Execute Action Set Packet Out OpenFlow Switch 103

104 Pipeline processing (4) Per-table packet processing: A Find highest-priority matching flow entry Match fields: Ingress port + metadata + packet headers Action set Flow Table A B Match fields: Ingress port + metadata + packet headers Action set C B Apply instructions: i. Modify packet and update match fields (APPLY- ACTIONS) ii. Update action set (CLEAR- ACTIONS, WRITE-ACTIONS) iii. Update metadata C Send match data and action set to next table 104

105 Connection interruption If connectivity with the controller is lost, the switch enters either fail-secure or fail-standalone mode The concept of emergency mode was deprecated. Fail-secure mode: In all packets and messages destined to the controller are dropped. Flow entries continue to be used and expire based on their timeouts. Fail-standalone mode: All packets are processed via the NORMAL port i.e. the switch acts as a traditional Ethernet switch or router Applies only to OpenFlow-hybrid switches 105

106 OpenFlow message types Symmetric Controller-to-Switch ID Type ID Type 0 Hello 5 Features Request 1 Error 6 Features Reply 2 Echo Request 7 Get Config Request 3 Echo Reply 8 Get Config Reply 4 Experimenter 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Stats Request ID Type 19 Stats Reply 10 Packet-In 20 Barrier Request 11 Flow-removed 21 Barrier Reply 12 Port status 22 Queue Get Config Request 23 Queue Get Config Reply Messages new to v1.1.0 are in bold. Asynchronous Features Configuration Packet-out Modify-state Read-state Barrier Configuration 106

107 Flow match descriptor Flow match descriptor structure Structure used to describe flow match requirements CHANGED BEHAVIOUR IN LATER VERSIONS type ethernet source address 32 bits length ingress port wildcard fields ethernet source address mask ethernet destination address ethernet destination address mask vid pcp padding ethertype ip tos ip protocol ipv4 source address ipv4 source address mask ipv4 destination address ipv4 destination address mask tcp/udp source port tcp/udp dest port mpls label mpls tc padding metadata metadata mask Fields new to v1.1.0 are in bold. 107

108 Table Mod Message Table-Mod message structure Table ID 32 bits Config Padding Config: - Bitmap of flags to describe the behaviour of the table for unmatched packets Flag Description TABLE_MISS_CONTROLLER Send packet to controller (Packet-In) TABLE_MISS_CONTINUE Continue to the next table in the pipeline TABLE_MISS_DROP Drop the packet 108

109 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits cookie cookie mask table id command idle timeout hard timeout priority buffer id output port output group flags padding Cookie: - Opaque value set by the controller Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority flow match descriptor instructions descriptor Fields new to v1.1.0 are in bold. 109

110 OpenFlow v

111 OpenFlow v1.2.0 Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

112 OpenFlow v1.2.0 Third major release version Wire protocol 0x03 December 5, 2011 This section highlights deltas from the previous release v

113 New features New OpenFlow Extensible Match (OXM) instead of the previous static, fixed-length structures. Use of OXM for writing to packet header fields Support of IPv6 Packet parsing specification is removed Support for multiple controllers 113

114 OpenFlow ports OpenFlow Ports Physical Ports Logical Ports Reserved Ports Correspond to hardware interfaces of the switch Abstracted interfaces that do not directly correspond to hardware interfaces of the switch For example: LAGs, tunnels, loopback interfaces Specify generic forwarding actions: ALL CONTROLLER TABLE IN_PORT ANY LOCAL NORMAL* FLOOD* * Only supported by OpenFlow-hybrid switches 114

115 Supported Actions Action Description Action Description Output Copy TTL out Copy TTL in Set MPLS TTL Output to switch port Copy TTL from next-tooutermost to outermost header Copy TTL from outermost header to next-to-outermost Set value of the MPLS TTL Set Network TTL Decrement Network TTL Set Field Set value of the IP TTL Decrement IP TTL Set a header field using OXM TLV format Decrement MPLS TTL Decrement MPLS TTL Push VLAN Push a new VLAN tag Pop VLAN Pop the outer VLAN tag Push MPLS Push a new MPLS label Pop MPLS Pop the outer MPLS label Set Queue ID Set queue ID when outputting to a port Set Group Apply group 115

116 Multiple controllers Multiple controllers supported to improve reliability Communication between controllers is not specified by the OpenFlow specification Controller roles: EQUAL: controller has complete access to the switch and is equal to all other controllers in the same role SLAVE: controller only has read-only access to the switch MASTER: controller has complete access to the switch;; there can only be one controller with this role A switchmay be simultaneously connected to multiple controllers in Equal state, multiple controllers in Slave state and at most a single controller in Master state. 116

117 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Stats Request 19 Stats Reply ID Type 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply 24 Role Request 25 Role Reply Messages new to v1.2.0 are in bold. 117

118 Flow match descriptor Flow match descriptor structure Payload is a set of OXM (OpenFlow Extensible Match) flow match fields 32 bits type padding OXM TLVs length oxm_class: - Specifies a set of related match types - OFPXMC_OPENFLOW_BASIC: contains the basic set of OpenFlow match fields oxm_class oxm_field length OXM TLV header oxm_field: - Match field within the oxm_class payload oxm_type: - combination of oxm_class and oxm_type 118

119 Flow match field prerequisites The matching of header fields of a protocol can only be done if the OpenFlow match also explicitly matches the corresponding protocol For example, a match for the TCP source port is only allowed if it is preceded by: A match for an IP Ethertype (either 0x0800 or ox86dd) AND A match for IP protocol = 6 (TCP) In other words, matching on the TCP port is only allowed if the EtherType is IP and the IP protocol is TCP 119

120 Basic OpenFlow match fields OXM_field types for the OXM_class: OFPXMC_OPENFLOW_BASIC Ingress port Ingress physical port Metadata Ethernet destination address Ethernet source address Ethertype VLAN ID VLAN priority IP DSCP IP ECN IP protocol IPv4 source address IPv4 destination address TCP source port TCP destination port UDP source port UDP destination port SCTP source port SCTP destination port ICMPv4 type ICMPv4 code ARP OP ARP SPA ARP TPA ARP SHA ARP THA IPv6 source address IPv6 destination address IPv6 Flow Label ICMPv6 type ICMPv6 code IPv6 ND Target IPv6 ND SLL IPv6 ND TLL MPLS Label MPLS TC Fields new to v1.2.0 are in bold. 120

121 OpenFlow v

122 OpenFlow v1.3.x Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

123 OpenFlow v1.3.x Fourth major release version Wire protocol 0x04 April 13, 2012 Updated in v1.3.1, v1.3.2, v1.3.3, v1.3.4, v1.3.5 This section provides a ground-up description of v1.3.5 March 26,

124 New features Stats framework renamed to multipart framework Introduction of table-miss flow entry Support for per-flow meters Support for PBB Auxiliary connections Improved version negotiation via version bitmap 124

125 OpenFlow switch types Specifies two types of OpenFlow-compliant switches: OpenFlow-only: perform forwarding based purely on OpenFlow flow tables OpenFlow-hybrid: support traditional Ethernet switching and routing functions in addition to OpenFlow packet forwarding (was referred to as OpenFlow-enabled in v1.0.0). As with prior versions of the protocol, v1.3.x only supports Ethernet packets 125

126 OpenFlow components Controller OpenFlow Protocol OpenFlow Channel Group Table Flow Table Flow Table OpenFlow Switch 126

127 OpenFlow components OpenFlow controller: An entity that interacts with the OpenFlow switch using the OpenFlow switch protocol. Typically, a single controller manages multiple OpenFlow Logical Switches OpenFlow Logical Switch: A set of OpenFlow resources that can be managed as a single entity Includes a datapath and control channel 127

128 OpenFlow Logical Switch One or more flow tables: Performs packet lookup and forwarding A Group Table Datapaths: components of the switch that are directly involved in traffic processing and forwarding. Includes the pipeline of flow tables, the group table and the ports. OpenFlow channel: To an external controller which manages the switch using the OpenFlow protocol 128

129 Flow tables Each flow table contains a set of flow entries Each flow entry consists of: Match Fields Priority Counters Instructions Timeouts Cookie Flags Header fields Pipeline fields Precedence of flow entry Modify action set Apply actions Modify pipeline processing The match fields and priority take together identify a unique flow entry in a specific flow table 129

130 Match Fields Two types of match fields: Header match fields: match values extracted from the packet header Pipeline match fields: match fields matching values attached to the packet for pipeline processing and not associated with packet headers e.g. IN_PORT IN_PHY_PORT METADATA TUNNEL_ID 130

131 Basic OpenFlow match fields OXM_field types for the OXM_class: OFPXMC_OPENFLOW_BASIC Ingress port Ingress physical port Metadata Ethernet destination address Ethernet source address Ethertype VLAN ID VLAN priority IP DSCP IP ECN IP protocol IPv4 source address IPv4 destination address TCP source port TCP destination port UDP source port UDP destination port SCTP source port SCTP destination port ICMPv4 type ICMPv4 code ARP OP ARP SPA ARP TPA ARP SHA ARP THA IPv6 source address IPv6 destination address IPv6 Flow Label ICMPv6 type ICMPv6 code IPv6 ND Target IPv6 ND SLL IPv6 ND TLL MPLS label MPLS TC MPLS BoS PBB ISID Tunnel ID IPv4 Ext Header Fields new to v1.3.x are in bold. 131

132 Counters Per-table Active Entries Packet lookups Packet matches Per-flow Received packets Received bytes Duration (seconds) Duration (nanoseconds) Per-queue Transmit packets Transmit bytes Transmit overrun errors Duration (seconds) Duration (nanoseconds) Counters new to v1.3.x are in bold Per-port Received packets Transmitted packets Received bytes Transmitted bytes Receive drops Transmit drops Receive errors Transmit errors Receive frame alignment errors Received overrun errors Receive CRC errors Collisions Duration (seconds) Duration (nanoseconds) Per-meter band In band packet count In Band byte count Per-group # flow entries Transmit bytes Transmit overrun errors Duration (seconds) Duration (nanoseconds) Per-bucket Packet count Byte count Per-meter Flow count Input packet count Input byte count Duration (seconds) Duration (nanoseconds) 132

133 Instructions (1) DEFINITION: attached to a flow entry as part of an Instruction Set and describe the OpenFlow processing that takes place when a packet matches the flow entry. Each instruction either: Modifies pipeline processing e.g. directing the packet to another flow table OR Contains a set of actions to add to the Action Set OR Contains a list of actions to apply immediately to the packet 133

134 Instructions (2) Supported instructions include: Meter: direct packet to the specified meter Apply-Actions: immediately applies the specified actions. The Action Set is not modified. Clear-Actions: immediately clears all actions in the Action Set Write-Actions: merges the specified actions into the current Action Set. Write-Metadata: writes to the metadata field Goto-Table: indicates that the packet should next be processed through the specified table Each instruction type may only appear once in the Instruction Set. Instructions new to v1.3.x are in bold 134

135 Actions (1) DEFINITION: an operation that acts on a packet Output (forward) action [required]: Forwarding of packet to physical or virtual ports Set-Queue action [optional]: Forward a packet through a specified queue attached to a port Drop action [required]: Implicit action associated with a flow-entry that has no specified action 135

136 Actions (2) Group action [required]: processes the packet through the specified group Push-Tag/Pop-Tag action [optional]: Push/pop of VLAN and MPLS headers Set-Field action [optional]: Set packet header fields, manipulate TTL 136

137 Supported Actions Action Description Action Description Output Copy TTL out Copy TTL in Set MPLS TTL Decrement MPLS TTL Push VLAN Output to switch port Copy TTL from next-tooutermost to outermost header Copy TTL from outermost header to next-to-outermost Set value of the MPLS TTL Decrement MPLS TTL Push a new VLAN tag Set Network TTL Decrement Network TTL Set Field Push PBB Pop PBB Set value of the IP TTL Decrement IP TTL Set a header field using OXM TLV format Push a new PBB service tag (I-tag) Pop the outer PBB service tag (I-tag) Pop VLAN Pop the outer VLAN tag Push MPLS Push a new MPLS label Pop MPLS Pop the outer MPLS label Set Queue ID Set queue ID when outputting to a port Set Group Apply group Actions new to v1.3.0 are in bold. 137

138 Action Set (1) Definition: a set of actions associated with the packet that are accumulated while the packet is processed by each table and that are executed when pipeline processing terminates An Action Set is associated with each packet and is empty by default As the packet passes through the pipeline the Action Set is modified by instructions (Write-Actions, Clear-Actions) of matching flow entries 138

139 Action Set (2) The Action Set is carried between flow tables as the packet progresses through the pipeline There is a maximum of one action of each type in the Action Set. The Action Set is executed when an instruction set does not include a Goto-Table action If no output action or group action are specified in an action set, the packet is dropped 139

140 Actions Order of application of actions in the Action Set Order Action 1 Copy TTL inwards 2 Pop 3 Push-MPLS 4 Push-PBB 5 Push-VLAN 6 Copy TTL outwards 7 Decrement TTL 8 Set 9 QoS 10 Group 11 Output If no output action or group action are specified in an action set the packet is dropped. 140

141 Action List DEFINITION: ordered list of actions included in a flow entry in the Apply-Actions instruction or a Packet-Out message Actions in the Action List are immediately executed in the order specified in the list. Multiple actions of the same type may appear in the same Action List and have a cumulative effect. 141

142 Group table Flow entries may point to a group in the group table. Group provide sets of actions for flooding, multipath, fast reroute, link aggregation and indirection. The group table contains group entries. Each group entry has an ordered list of action buckets with semantics depending on group type. The group type determines which of the buckets are applied to each packet. 142

143 Group table The group table contains group entries. Each group entry contains: Group Identifier Group Type Counters Action Buckets Group Type Description all Executes all buckets in the group Multicast/broadcast forwarding Packet is replicated for each bucket select Executes one bucket in the group Packets are sent to a single bucket, based on a hash algorithm indirect Executes the one defined bucket in the group For example, BGP next-hop indirection fast-failover Executes the first live bucket Bucket liveness tied to port(s) or group 143

144 Table-miss flow entry Specifies how to process packets unmatched by other flow entries in the flow table Identified by its match and priority: Wildcards all match fields Has the lowest priority (zero) Has similar behaviour to other flow entries: does not exist by default can be added or removed by the controller at any time it may expire If no table-miss flow entry exists, unmatched packets are dropped 144

145 Meter table Consists of meter entries, defining per-flow meters A meter measures the rate of packets assigned to it and enables controlling the rate of those packets Meter Identifier Meter Bands Counters Band Type Rate Burst Counters Type specific arguments Defines the lowest rate at which the band can apply The meter applies the band with the highest configured rate that is lower than the current measured rate. 145

146 OpenFlow ports OpenFlow Ports Physical Ports Logical Ports Reserved Ports Correspond to hardware interfaces of the switch Abstracted interfaces that do not directly correspond to hardware interfaces of the switch For example: LAGs, tunnels, loopback interfaces Specify generic forwarding actions: ALL CONTROLLER TABLE IN_PORT ANY LOCAL NORMAL* FLOOD* * Only supported by OpenFlow-hybrid switches 146

147 Matching (1) Packet In Start at table 0 Yes Match in table n? Yes Update counters Execute instructions: update action set update packet/match set fields update metadata Goto table n? No No Table-miss flow entry exists? Yes Execute action set No Drop packet 147

148 Matching (2) Every flow entry has a 16-bit priority value associated with it It is possible that a packet may match more than one flow entry. Only the highest-priority flow entry matching the packet is used as the matching flow entry for the packet. 148

149 Pipeline processing (1) Definition: the set of linked tables that provide matching, forwarding and packet modifications in an OpenFlow switch Matching starts at the first table and may continue to other tables If a matching entry is found, the instructions associated with the flow entry are executed. The instructions may explicitly direct the packet to another flow table. 149

150 Pipeline processing (2) Pipeline processing stops when the instruction set associated with a matching flow entry does not specify a next table. The packet s action set is processed and it is forwarded at this point. If no match is found (called a table miss), the behaviour depends on the table-miss flow entry in the table. The actions may include: forwarding to the controller continue to the next table being dropped 150

151 Pipeline processing (3) Packet In Ingress port Action set = {} Flow Table 0 Packet+ Ingress port + metadata Action set Flow Table 1 Flow Table n Packet Action set Execute Action Set Packet Out OpenFlow Switch 151

152 Pipeline processing (4) Per-table packet processing: A Find highest-priority matching flow entry Match fields: Ingress port + metadata + packet headers Action set Flow Table A B Match fields: Ingress port + metadata + packet headers Action set C B Apply instructions: i. Modify packet and update match fields (APPLY- ACTIONS) ii. Update action set (CLEAR- ACTIONS, WRITE-ACTIONS) iii. Update metadata C Send match data and action set to next table 152

153 Flow table example <Give an example of a flow table with realistic entries> Example for ARP packet NAT function VID addition 153

154 OpenFlow Channel This is the logical interface that connects each OpenFlow switch to an OpenFlow controller. The OpenFlow controller uses this interface to: Configure and manage the switch Add, delete and modify flow entries Receive events from the switch Send packets out the switch There is one OpenFlow channel per OpenFlow controller 154

155 OpenFlow Connection A TLS or TCP network connection that is used by the OpenFlow channel to carry OpenFlow messages between a switch and a controller. An OpenFlow channel has a main connection (tcp or tls) and optionally, a number of auxiliary connections (tcp, tls, dtls or udp), in order to exploit parallelism Auxiliary connections on non-reliable transport, such as dtls or udp, can only support a small subset of the OpenFlow protocol e.g. they can be used to read stats 155

156 Multiple controllers Multiple controllers supported to improve reliability Communication between controllers is not specified by the OpenFlow specification Controller roles: EQUAL: controller has complete access to the switch and is equal to all other controllers in the same role SLAVE: controller only has read-only access to the switch MASTER: controller has complete access to the switch;; there can only be one controller with this role 156

157 Switch bootstrapping 157

158 Connection setup A TLS connection is established by the switch to a configured IP address and TCP port 6633 (changed in later releases to an IANA-assigned port number) Traffic to/from the secure channel is not processed by the flow table 158

159 Connection interruption If connectivity with the controller is lost, the switch enters either fail-secure or fail-standalone mode The concept of emergency mode was deprecated in v1.1.0 Fail-secure mode: In all packets and messages destined to the controller are dropped. Flow entries continue to be used and expire based on their timeouts. Fail-standalone mode: All packets are processed via the NORMAL port i.e. the switch acts as a traditional Ethernet switch or route Applies only to OpenFlow-hybrid switches 159

160 OpenFlow protocol messages Protocol defines three types of messages. Controller-to-switch: Are initiated by the control and used to configure the switch or query its state Asynchronous: Are initiated by the switch and used to notify the controller about network events or changes to the switch state Symmetric: Can be initiated by either the controller or the switch and sent without solicitation 160

161 Controller-to-Switch messages Initiated by the controller and may or may not require a response from the switch. Messages include: Features: used by the controller to discover the capabilities supported by the switch Configuration: used to set and query configuration parameters Modify-state: sent by the controller to manage state on the switch. Main purpose is to add/delete/modify flows Read-state: used by the controller to query stats from the switch Packet-out: used by the controller to send a packet out of a specified port of the switch Barrier: used to ensure message dependencies Role-Request: used by the controller to set or query the role of its OpenFlow channel Asynchronous-Configuration: used by the controller to filter asynchronous messages it receives Messages new to v1.3.x are in bold 161

162 Asynchronous messages Initiated by the switch without solicitation from the controller. Messages include: Packet-in: sent to the controller for all packets that: do not have a matching flow entry OR are explicitly sent to the controller Flow-removed: sent when flows are removed from the flow-table. May be due to expiration or explicit deletion. Port-status: sent by the switch on port configuration or state changes. Errors: sent when errors are detected 162

163 Symmetric messages Can be initiated by either the controller or the switch and sent without solicitation Messages include: Hello: sent between the controller and switch upon connection establishment Echo: echo request/reply messages can be sent from either the switch or the controller;; request messages must be responded to with a reply. Experimenter: vendor-specific messages 163

164 OpenFlow protocol Common OpenFlow packet header All OpenFlow messages start with this header 32 bits 8 bits 8 bits 16 bits version type length xid Version: Type: Length: xid: - version of OpenFlow protocol - type of OpenFlow protocol message - total length of message in octets - transaction ID used to match responses with requests 164

165 OpenFlow version numbers Version number has incremented with every major release of the OpenFlow specification Version of specification 1.0.x 1.1.x 1.2.x 1.3.x 1.4.x 1.5.x OpenFlow protocol version 0x01 0x02 0x03 0x04 0x05 0x06 165

166 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Multipart Request 19 Multipart Reply ID Type 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply 24 Role Request 25 Role Reply 26 Get Async Request 27 Get Async Reply 28 Set Async 29 Meter Mod Messages new to v1.3.x are in bold. 166

167 Version negotiation On connection establishment: Each side sends a Hello message with the version set to the highest OpenFlow version supported by the sender. The Hello message can optionally include a version bitmap that specifies all the versions supported by the sender. If (the version bitmap is supported by both sides) AND (the two bitmaps have some common bits set) negotiated version = highest version set in both bitmaps Else negotiated version = minimum (version number that was sent, version number that was received) 167

168 Understanding switch capabilities Due to the large number of required and optional OpenFlow capabilities, it is important for the controller to understand the features supported by the switch it is managing. A features/capabilities discovery is done via a handshake to acquire this information. 168

169 Handshake Once TLS session is established, the controller sends a Features Request message. The switch responds with a Features Reply message: #tables 32 bits datapath id #buffers auxiliary ID Padding Capabilities Reserved Datapath ID: - Uniquely identifies a datapath. Lower 48-bits are the switch MAC address. Capabilities: - Types of stats supported etc. Ports: - Array of OpenFlowenabled physical ports 169

170 Flow table modification messages 5 possible operations: Add: instantiates a new flow entry in the flow table Modify: modifies elements of all matching (existing) flow entries Modify-Strict: modifies elements of flow entries that exactly match all fields including wildcards and priority Delete: deletes all matching (existing) flow entries Delete-Strict: deletes flow entries that exactly match all fields including wildcards and priority 170

171 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits cookie cookie mask table id command idle timeout hard timeout priority buffer id output port output group flags padding Cookie: - Opaque value set by the controller Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority flow match descriptor instructions descriptor 171

172 Flow match descriptor Flow match descriptor structure Payload is a set of OXM (OpenFlow Extensible Match) flow match fields 32 bits type padding OXM TLVs length oxm_class: - Specifies a set of related match types - OFPXMC_OPENFLOW_BASIC: contains the basic set of OpenFlow match fields oxm_class oxm_field length OXM TLV header oxm_field: - Match field within the oxm_class payload oxm_type: - combination of oxm_class and oxm_type 172

173 Flow match field prerequisites The matching of header fields of a protocol can only be done if the OpenFlow match also explicitly matches the corresponding protocol For example, a match for the TCP source port is only allowed if it is preceded by: A match for an IP Ethertype (either 0x0800 or ox86dd) AND A match for IP protocol = 6 (TCP) In other words, matching on the TCP port is only allowed if the EtherType is IP and the IP protocol is TCP 173

174 Flow action descriptor Flow action descriptor structure Structures used to describe flow actions 32 bits 32 bits Type= OUTPUT Length Output port Max Length Padding Type= SET MPLS TTL TTL Padding Length 32 bits 32 bits Type= SET QUEUE Length Type= SET FIELD Length Queue ID OXM TLV 174

175 Proactive vs reactive flow entries <Explain the two approaches> 175

176 Flow removal All flow entries have two timers associated with them: idle_timeout: maximum time that can elapse without a flow matching the flow entry hard_timeout: maximum time that a flow entry can remain in the flow table A Flow-Removed message is sent by the switch to the controller when a flow entry is removed from the flow table 176

177 Send Packet Message Packet-Out message structure For packets sent from the controller to the switch 32 bits buffer id input port length of actions array padding action list Buffer ID: - Same as the buffer ID in the original Packet-In message Packet data: - Initial portion of packet packet data 177

178 Packet-In Message Packet-Out message structure Structure used to add/delete/modify flow entries 32 bits buffer id total length reason cookie match fields (OXM TLVs) data (ethernet frame) table ID Buffer ID: - Identifies where packet is buffered Reason: - One of: - no match - explicit action - TTL expired Match fields: - Pipeline fields associated with the packet Data: - Initial portion of packet 178

179 Multipart Request/Reply Messages Replace Stats-Request and Stats-Reply messages in earlier versions Used to encode requests or replies that may carry a large amount of data which may not be able to fit within a single OpenFlow message (max length of 64KB) A request or reply can span multiple messages and must use the same xid (transaction ID) for all messages in the message sequence 179

180 Multipart message types Type DESC FLOW AGGREGATE TABLE PORT_STATS QUEUE GROUP GROUP_DESC GROUP_FEATURES METER METER_CONFIG METER_FEATURES TABLE_FEATURES PORT_DESC EXPERIMENTER Description Information about the switch manufacturer, hardware revision, software revision, serial number etc Individual flow statistics Statistics about multiple flow entries Table statistics Port statistics Queue statistics Group statistics Lists the set of groups in the switch together with their bucket actions Capabilities of groups on a switch Meter statistics Configuration for one more more meters Set of features of the metering system Capabilities of the currently configured tables e.g. supported actions, instructions, match fields etc. Description of all the standard ports of the OpenFlow switch Experimenter-defined behaviour 180

181 Echo Request/Reply Messages Echo Request may be initiated by either the controller or the switch May be used for a number of reasons: To determine latency of connection between controller and switch To verify liveness of the connection between controller and switch 181

182 QoS structures: queues Limited QoS support is provided through a simple queuing mechanism Flows can be mapped to queues which attach to a port The only queue configuration available is: min-rate: minimum guaranteed data-rate 182

183 QoS structures: meters DEFINITION: switch elements that can measure and control the rate of packets The meter triggers a meter band if the rate passing through the meter exceeds a predefined threshold 183

184 Message flow example Controller Switch Hello Hello Features Request Features Reply 184

185 Topology discovery <proactive rules for dealing with LLDP packets> If ethertype=lldp, output to CONTROLLER 185

186 Exclusions What OpenFlow does NOT do (or specify): Communication between controllers when using multiple controllers (v1.2.0+) 186

187 OpenFlow v1.4.x 187

188 OpenFlow v1.4.x Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

189 OpenFlow v1.4.x Fifth major release version Wire protocol 0x05 October 14, 2013 This section highlights deltas from the previous release v1.3.x 189

190 New features More extensible wire protocol (TLV-based structures) Optical port properties Flow monitoring to allow better co-ordination between multiple controllers Eviction and vacancy events to deal with finite table capacity Message bundling Table synchronisation 190

191 Flow eviction Mechanism used to reclaim switch resources Has to be explicitly enabled Flow entries are selected for eviction based on: A new flow entry field importance. Flow entries with lower importance will always be evicted before entries with higher importance. The remaining lifetime of the flow entry. Flow entries with shorter remaining lifetimes will be evicted before entries with longer remaining lifetimes. 191

192 Table vacancy events Generates events (TABLE_STATUS) messages based on occupancy of flow tables vacancy_down: generated when the remaining space in the flow table falls to less than the configured threshold vacancy_updown: generated when the remaining space in the flow table increases to more than the configured threshold The specification does not define the behaviour of controllers on receiving these messages. 192

193 Flow monitoring (1) Flow monitoring allows a controller to keep track of changes to flow tables Useful for multi-controller environments where controllers can be made aware of changes made to the flow table by other controllers Flow monitors can be created to match a subset of flow entries in selected flow tables. Events are generated for any changes to matching flow entries. Flow monitoring requests are done via multipart messages. 193

194 Flow monitoring (2) Types of flow monitors: Initial: all flow entries matching the flow monitor at the time of the request Add: new additions of flow entries matching the flow monitor Removed: removal of flow entries matching the flow monitor Modification: modification of flow entries matching the flow monitor 194

195 Flow table synchronisation (1) Allows flow entries in a table to be synchronised with another table: Flow entries in the synchronised table are automatically updated to reflect changes in the table it is synchronised with Enables multiple matches on different views of the same data at different points of the OpenFlow pipeline. Synchronisation can be uni-directal or bi-directional. When a flow entry is added, modified or removed in the source table, a corresponding flow entry is automatically added, modified or removed in the synchronised table 195

196 Flow table synchronisation (2) Entries in the synchronised table may not be identical to the corresponding entry in the source flow table e.g. transposed source/destination matches, different instruction sets etc. Recommended to be created as permanent flow entries (expiry timers set to zero) so that the lifetime of the correspond ing flow entries is also synchronised Flow entry sychronisation can be unidirectional or bidirectional. 196

197 Bundle messages Bundle sequence of OpenFlow modification messages that are applied as a single OpenFlow operation Provides a degree of atomicity (either all changes are applied or none at all) Example: 1. OFPBCT_OPEN_REQUEST bundle_id 2. OFPT_BUNDLE_ADD_MESSAGE bundle_id modication 1 3. OFPT_BUNDLE_ADD_MESSAGE bundle_id OFPT_BUNDLE_ADD_MESSAGE bundle_id modication n 5. OFPBCT_CLOSE_REQUEST bundle_id 6. OFPBCT_COMMIT_REQUEST bundle_id 197

198 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status 30 Role status 31 Table status 32 Request Forward Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Multipart Request 19 Multipart Reply ID Type 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply 24 Role Request 25 Role Reply 26 Get Async Request 27 Get Async Reply 28 Set Async 29 Meter Mod 33 Bundle Control 34 Bundle Add Messages new to v1.4.0 are in bold. 198

199 Basic OpenFlow match fields OXM_field types for the OXM_class: OFPXMC_OPENFLOW_BASIC Ingress port Ingress physical port Metadata Ethernet destination address Ethernet source address Ethertype VLAN ID VLAN priority IP DSCP IP ECN IP protocol IPv4 source address IPv4 destination address Fields new to v1.4.0 are in bold. TCP source port TCP destination port UDP source port UDP destination port SCTP source port SCTP destination port ICMPv4 type ICMPv4 code ARP OP ARP SPA ARP TPA ARP SHA ARP THA IPv6 source address IPv6 destination address IPv6 Flow Label ICMPv6 type ICMPv6 code IPv6 ND Target IPv6 ND SLL IPv6 ND TLL MPLS label MPLS TC MPLS BoS PBB ISID Tunnel ID IPv4 Ext Header PBB UCA 199

200 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits cookie cookie mask table id command idle timeout hard timeout priority buffer id output port output group flags importance flow match descriptor instructions descriptor Cookie: - Opaque value set by the controller Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority importance: - Used for flow eviction purposes Fields new to v1.4.0 are in bold. 200

201 OpenFlow v1.5.x 201

202 OpenFlow v1.5.x Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

203 OpenFlow v1.5.x Sixth major release version Wire protocol 0x06 December 19, 2014 This section provides a ground-up description of v1.5.1 March 26,

204 New features Egress tables Packet Type-aware pipeline Extensible flow entry statistics: OpenFlow extensible Statistics (OXS) Flow entry statistics trigger Copy-Field action to copy between two OXM fields Packet Register pipeline fields Scheduled Bundles Meter action 204

205 OpenFlow switch types Specifies two types of OpenFlow-compliant switches: OpenFlow-only: perform forwarding based purely on OpenFlow flow tables OpenFlow-hybrid: support traditional Ethernet switching and routing functions in addition to OpenFlow packet forwarding (was referred to as OpenFlow-enabled in v1.0.0). Unlike prior versions of the protocol, v1.5.x also supports non-ethernet packet types such as IPv4 and IPv6 for support of IP routing functionality. 205

206 OpenFlow components Controller Controller OpenFlow Protocol OpenFlow Protocol OpenFlow Channel Control Channel OpenFlow Channel Group Table Datapath Meter Table Port Port Flow Table Flow Table Flow Table Port Port Pipeline OpenFlow Switch 206

207 OpenFlow components OpenFlow controller: An entity that interacts with the OpenFlow switch using the OpenFlow switch protocol. Typically, a single controller manages multiple OpenFlow Logical Switches OpenFlow Logical Switch: A set of OpenFlow resources that can be managed as a single entity Includes a datapath and control channel 207

208 OpenFlow Logical Switch One or more flow tables: Performs packet lookup and forwarding A Group Table Datapaths: components of the switch that are directly involved in traffic processing and forwarding. Includes the pipeline of flow tables, the group table and the ports. One or more OpenFlow channel: To externals controllers which manage the switch using the OpenFlow protocol 208

209 Flow tables Each flow table contains a set of flow entries Each flow entry consists of: Match Fields Priority Counters Instructions Timeouts Cookie Flags Header fields Pipeline fields Precedence of flow entry Modify action set Apply actions Modify pipeline processing The match fields and priority take together identify a unique flow entry in a specific flow table 209

210 Match Fields Two types of match fields: Header match fields: match values extracted from the packet header Pipeline match fields: match fields matching values attached to the packet for pipeline processing and not associated with packet headers e.g. IN_PORT IN_PHY_PORT METADATA TUNNEL_ID ACTION_SET_OUTPUT PACKET_TYPE 210

211 Basic OpenFlow match fields OXM_field types for the OXM_class: OFPXMC_OPENFLOW_BASIC Ingress port Ingress physical port Metadata Ethernet destination address Ethernet source address Ethertype VLAN ID VLAN priority IP DSCP IP ECN IP protocol IPv4 source address IPv4 destination address TCP source port Fields new to v1.5.0 are in bold. TCP destination port UDP source port UDP destination port SCTP source port SCTP destination port ICMPv4 type ICMPv4 code ARP OP ARP SPA ARP TPA ARP SHA ARP THA IPv6 source address IPv6 destination address IPv6 Flow Label ICMPv6 type ICMPv6 code IPv6 ND Target IPv6 ND SLL IPv6 ND TLL MPLS label MPLS TC MPLS BoS PBB ISID Tunnel ID IPv4 Ext Header PBB UCA TCP Flags Action Set Output port Packet Type 211

212 Counters Per-table Active Entries Packet lookups Packet matches Per-flow Received packets Received bytes Duration (seconds) Duration (nanoseconds) Per-queue Transmit packets Transmit bytes Transmit overrun errors Duration (seconds) Duration (nanoseconds) Counters new to v1.5.x are in bold Per-port Received packets Transmitted packets Received bytes Transmitted bytes Receive drops Transmit drops Receive errors Transmit errors Receive frame alignment errors Received overrun errors Receive CRC errors Collisions Duration (seconds) Duration (nanoseconds) Per-meter band In band packet count In Band byte count Per-group # flow entries Transmit bytes Transmit overrun errors Duration (seconds) Duration (nanoseconds) Per-bucket Packet count Byte count Per-meter Flow count Input packet count Input byte count Duration (seconds) Duration (nanoseconds) 212

213 Instructions (1) DEFINITION: attached to a flow entry as part of an Instruction Set and describe the OpenFlow processing that takes place when a packet matches the flow entry. Each instruction either: Modifies pipeline processing e.g. directing the packet to another flow table OR Contains a set of actions to add to the Action Set OR Contains a list of actions to apply immediately to the packet 213

214 Instructions (2) Supported instructions include: Apply-Actions: immediately applies the specified actions. The Action Set is not modified. Clear-Actions: immediately clears all actions in the Action Set Write-Actions: merges the specified actions into the current Action Set. Write-Metadata: writes to the metadata field Stat-Trigger: generate events based on stats thresholds Goto-Table: indicates that the packet should next be processed through the specified table Each instruction type may only appear once in the Instruction Set. Instructions new to v1.5.x are in bold 214

215 Instructions (3) Find highestpriority matching flow entry Match flow entry flow entry flow entry flow entry Flow Table Apply instructions Action Set Pipeline fields Packet Extract header fields flow entry table-miss flow entry Apply-actions {list of actions} modify packet update match fields update pipeline fields if output or group -> clone packet Clear-actions empty action set Write-actions {set of actions} merge in action set Goto-table {table-id} Flow Table Execute Action Set Packet clones Egress 215

216 Actions (1) DEFINITION: an operation that acts on a packet Output (forward) action [required]: Forwarding of packet to physical or virtual ports Set-Queue action [optional]: Forward a packet through a specified queue attached to a port Drop action [required]: Implicit action associated with a flow-entry that has no specified action Actions new to v1.5.x are in bold. 216

217 Actions (2) Group action [required]: processes the packet through the specified group Push-Tag/Pop-Tag action [optional]: Push/pop of VLAN and MPLS headers Set-Field action [optional]: Set packet header fields, manipulate TTL Actions new to v1.5.x are in bold. 217

218 Actions (3) Meter [optional]: Directs the packet to the specified header Copy-field [optional]: Copies data between pipeline or header fields Actions new to v1.5.x are in bold. 218

219 Supported Actions Action Description Action Description Output Copy TTL out Copy TTL in Set MPLS TTL Decrement MPLS TTL Push VLAN Output to switch port Copy TTL from next-tooutermost to outermost header Copy TTL from outermost header to next-to-outermost Set value of the MPLS TTL Decrement MPLS TTL Push a new VLAN tag Set Network TTL Decrement Network TTL Set Field Push PBB Pop PBB Set value of the IP TTL Decrement IP TTL Set a header field using OXM TLV format Push a new PBB service tag (Itag) Pop the outer PBB service tag (I-tag) Pop VLAN Push MPLS Pop MPLS Pop the outer VLAN tag Push a new MPLS label Pop the outer MPLS label Copy Field Meter Copy value between header and register Apply meter (rate limiter) Set Queue ID Set queue ID when outputting to a port Set Group Apply group Actions new to v1.5.0 are in bold. 219

220 Action Set (1) Definition: a set of actions associated with the packet that are accumulated while the packet is processed by each table and that are executed when pipeline processing terminates An Action Set is associated with each packet and: is empty by default for ingress processing is initialised with the output action for the current output port As the packet passes through the pipeline the Action Set is modified by instructions (Write-Actions, Clear-Actions) of matching flow entries 220

221 Action Set (2) The Action Set is carried between flow tables as the packet progresses through the pipeline There is a maximum of one action of each type in the Action Set. The Action Set is executed when an instruction set does not include a Goto-Table action If no output action or group action are specified in an action set, the packet is dropped 221

222 Actions Order of application of actions in the Action Set Order Action 1 Copy TTL inwards 2 Pop 3 Push-MPLS 4 Push-PBB 5 Push-VLAN 6 Copy TTL outwards 7 Decrement TTL 8 Set 9 QoS 10 Group 11 Output If no output action or group action are specified in an action set the packet is dropped. 222

223 Action List DEFINITION: ordered list of actions included in a flow entry in the Apply-Actions instruction or a Packet-Out message Actions in the Action List are immediately executed in the order specified in the list. Multiple actions of the same type may appear in the same Action List and have a cumulative effect. 223

224 Group table Flow entries may point to a group in the group table. Group provide sets of actions for flooding, multipath, fast reroute, link aggregation and indirection. The group table contains group entries. Each group entry has an ordered list of action buckets with semantics depending on group type. The group type determines which of the buckets are applied to each packet. 224

225 Group table The group table contains group entries. Each group entry contains: Group Identifier Group Type Counters Action Buckets Group Type Description all Executes all buckets in the group Multicast/broadcast forwarding Packet is replicated for each bucket select Executes one bucket in the group Packets are sent to a single bucket, based on a hash algorithm indirect Executes the one defined bucket in the group For example, BGP next-hop indirection fast-failover Executes the first live bucket Bucket liveness tied to port(s) or group 225

226 Egress tables In older versions, all processing was done in the context of the input port Egress tables allow processing to be done in the context of the output port 226

227 Table-miss flow entry Specifies how to process packets unmatched by other flow entries in the flow table Identified by its match and priority: Wildcards all match fields Has the lowest priority (zero) Has similar behaviour to other flow entries: does not exist by default can be added or removed by the controller at any time it may expire If no table-miss flow entry exists, unmatched packets are dropped 227

228 Meter table Consists of meter entries, defining per-flow meters A meter measures the rate of packets assigned to it and enables controlling the rate of those packets Meter Identifier Meter Bands Counters Band Type Rate Burst Counters Type specific arguments Defines the lowest rate at which the band can apply The meter applies the band with the highest configured rate that is lower than the current measured rate. 228

229 OpenFlow ports OpenFlow Ports Physical Ports Logical Ports Reserved Ports Correspond to hardware interfaces of the switch Abstracted interfaces that do not directly correspond to hardware interfaces of the switch For example: LAGs, tunnels, loopback interfaces Specify generic forwarding actions: ALL CONTROLLER TABLE IN_PORT ANY LOCAL NORMAL* FLOOD* * Only supported by OpenFlow-hybrid switches 229

230 Matching (1) Packet In Clear action set Initialise pipeline fields Start at table 0 Match in table n? No Yes Update counters Execute instruction set: update action set update packet headers update match set fields update pipeline fields as needed, clone packet to egress Yes Goto table n? No Execute action set: update packet headers update match set fields update pipeline fields Table-miss flow entry exists? Yes Group Action? No Yes No Drop packet Drop packet No Output Action? Ingress Processing Yes Egress Processing (next slide) 230

231 Matching (2) Ingress Processing (previous slide) Egress Processing Yes Egress tables exist? No Start egress processing action set = {output port} start at first egress table Match in table n? No Yes Update counters Execute instruction set: update action set update packet headers update match set fields update pipeline fields as needed, clone packet to egress Yes Goto table n? No Execute action set: update packet headers update match set fields update pipeline fields Table-miss flow entry exists? Yes Drop packet No Output Action? Yes No Packet Out Drop packet 231

232 Matching (3) Every flow entry has a 16-bit priority value associated with it It is possible that a packet may match more than one flow entry. Only the highest-priority flow entry matching the packet is used as the matching flow entry for the packet. 232

233 Packet Type-aware pipeline First release to support non-ethernet packets: IPv4 and IPv6 New OXM pipeline field identifies the packet type. Namespace ns_type Match description Packet-in and packet-out format 0 0 Ethernet packet (default) Ethernet header and Ethernet payload 1 0x0800 IPv4 packet (no preceding header) 1 0x86dd IPv6 packet (no preceding header) IPv4 header and IPv4 payload IPv6 header and IPv6 payload 0 1 No packet Empty 0 0xFFFF Experimenter-defined Experimenter-defined 233

234 Pipeline processing (1) Definition: the set of linked tables that provide matching, forwarding and packet modifications in an OpenFlow switch Pipeline processing happens in two stages: ingress processing and egress processing Separation between the two stages is indicated by the first egress table Ingress tables: table IDs < first egress table Egress tables: table IDs first egress table 234

235 Pipeline processing (2) Pipeline processing starts with ingress processing at at the first table and may continue to other tables If a matching entry is found, the instructions associated with the flow entry are executed. The instructions may explicitly direct the packet to another flow table. Pipeline processing stops when the instruction set associated with a matching flow entry does not specify a next table. The packet s action set is processed at this point. 235

236 Pipeline processing (3) If the outcome of ingress processing is to forward the packet to an output port, (optional) egress processing may be performed in the context of that output port. If no match is found (called a table miss), the behaviour depends on the table-miss flow entry in the table. The actions may include: forwarding to the controller continue to the next table being dropped 236

237 Pipeline processing (4) Packet In Ingress Port Ingress processing Set Ingress port Action set = {} Flow Table 0 Flow Table 1 Packet + pipeline fields (Ingress port, metadata etc.) Action set Flow Table n Action set Execute Action Set Group Table Egress processing Set output port Action set = {output} Flow Table e Flow Table e+1 e = first egress table ID Packet + pipeline fields (output port, metadata etc.) Action set Flow Table e+m Action set Execute Action Set Packet Out Output Port 237

238 Pipeline processing (5) Per-table packet processing: A Find highest-priority matching flow entry Match fields: Ingress port + metadata + packet headers Action set Flow Table A B Match fields: Ingress port + metadata + packet headers Action set C B Apply instructions: i. Modify packet and update match fields (APPLY- ACTIONS) ii. Update action set (CLEAR- ACTIONS, WRITE-ACTIONS) iii. Update metadata C Send match data and action set to next table 238

239 Ingress and egress processing Similarities in behaviour: Flow table matching Execution of instructions Table-miss processing Differences: At the beginning of ingress processing, the Action Set is empty At the beginning of egress processing, the Action Set is initialised to contain only the Output action for the current output port 239

240 Flow eviction Mechanism used to reclaim switch resources Has to be explicitly enabled Flow entries are selected for eviction based on: A new flow entry field importance. Flow entries with lower importance will always be evicted before entries with higher importance. The remaining lifetime of the flow entry. Flow entries with shorter remaining lifetimes will be evicted before entries with longer remaining lifetimes. 240

241 Table vacancy events Generates events (TABLE_STATUS) messages based on occupancy of flow tables vacancy_down: generated when the remaining space in the flow table falls to less than the configured threshold vacancy_updown: generated when the remaining space in the flow table increases to more than the configured threshold The specification does not define the behaviour of controllers on receiving these messages. 241

242 Flow monitoring (1) Flow monitoring allows a controller to keep track of changes to flow tables Useful for multi-controller environments where controllers can be made aware of changes made to the flow table by other controllers Flow monitors can be created to match a subset of flow entries in selected flow tables. Events are generated for any changes to matching flow entries. Flow monitoring requests are done via multipart messages. 242

243 Flow monitoring (2) Types of flow monitors: Initial: all flow entries matching the flow monitor at the time of the request Add: new additions of flow entries matching the flow monitor Removed: removal of flow entries matching the flow monitor Modification: modification of flow entries matching the flow monitor 243

244 Flow table synchronisation (1) Allows flow entries in a table to be synchronised with another table: Flow entries in the synchronised table are automatically updated to reflect changes in the table it is synchronised with Enables multiple matches on different views of the same data at different points of the OpenFlow pipeline. Synchronisation can be uni-directal or bi-directional. When a flow entry is added, modified or removed in the source table, a corresponding flow entry is automatically added, modified or removed in the synchronised table 244

245 Flow table synchronisation (2) Entries in the synchronised table may not be identical to the corresponding entry in the source flow table e.g. transposed source/destination matches, different instruction sets etc. Recommended to be created as permanent flow entries (expiry timers set to zero) so that the lifetime of the correspond ing flow entries is also synchronised Flow entry sychronisation can be unidirectional or bidirectional. 245

246 Flow table example <Give an example of a flow table with realistic entries> Example for ARP packet NAT function VID addition 246

247 OpenFlow Channel This is the logical interface that connects each OpenFlow switch to an OpenFlow controller. The OpenFlow controller uses this interface to: Configure and manage the switch Add, delete and modify flow entries Receive events from the switch Send packets out the switch There is one OpenFlow channel per OpenFlow controller 247

248 OpenFlow Connection A TLS or TCP network connection that is used by the OpenFlow channel to carry OpenFlow messages between a switch and a controller. An OpenFlow channel has a main connection (tcp or tls) and optionally, a number of auxiliary connections (tcp, tls, dtls or udp), in order to exploit parallelism Auxiliary connections on non-reliable transport, such as dtls or udp, can only support a small subset of the OpenFlow protocol e.g. they can be used to read stats 248

249 Multiple controllers Multiple controllers supported to improve reliability Communication between controllers is not specified by the OpenFlow specification Controller roles: EQUAL: controller has complete access to the switch and is equal to all other controllers in the same role SLAVE: controller only has read-only access to the switch MASTER: controller has complete access to the switch;; there can only be one controller with this role 249

250 Switch bootstrapping 250

251 Connection setup A TLS connection is established by the switch to a configured IP address and TCP port 6633 (changed in later releases to an IANA-assigned port number) Traffic to/from the secure channel is not processed by the flow table 251

252 Connection interruption If connectivity with the controller is lost, the switch enters either fail-secure or fail-standalone mode The concept of emergency mode was deprecated in v1.1.0 Fail-secure mode: In all packets and messages destined to the controller are dropped. Flow entries continue to be used and expire based on their timeouts. Fail-standalone mode: All packets are processed via the NORMAL port i.e. the switch acts as a traditional Ethernet switch or route Applies only to OpenFlow-hybrid switches 252

253 OpenFlow protocol messages Protocol defines three types of messages. Controller-to-switch: Are initiated by the control and used to configure the switch or query its state Asynchronous: Are initiated by the switch and used to notify the controller about network events or changes to the switch state Symmetric: Can be initiated by either the controller or the switch and sent without solicitation 253

254 Controller-to-Switch messages Initiated by the controller and may or may not require a response from the switch. Messages include: Features: used by the controller to discover the capabilities supported by the switch Configuration: used to set and query configuration parameters Modify-state: sent by the controller to manage state on the switch. Main purpose is to add/delete/modify flows Read-state: used by the controller to query stats from the switch Packet-out: used by the controller to send a packet out of a specified port of the switch Barrier: used to ensure message dependencies Role-Request: used by the controller to set or query the role of its OpenFlow channel Asynchronous-Configuration: used by the controller to filter asynchronous messages it receives Messages new to v1.5.x are in bold 254

255 Asynchronous messages Initiated by the switch without solicitation from the controller. Messages include: Packet-in: sent to the controller for all packets that: do not have a matching flow entry OR are explicitly sent to the controller Flow-removed: sent when flows are removed from the flow-table. May be due to expiration or explicit deletion. Port-status: sent by the switch on port configuration or state changes. Role-status: informs the controller of a change in its role Controller-status: informs the controller when the status of an OpenFlow channel changes Flow-monitor: informs the controller of a change in a flow Errors: sent when errors are detected Messages new to v1.5.x are in bold 255

256 Symmetric messages Can be initiated by either the controller or the switch and sent without solicitation Messages include: Hello: sent between the controller and switch upon connection establishment Echo: echo request/reply messages can be sent from either the switch or the controller;; request messages must be responded to with a reply. Experimenter: vendor-specific messages 256

257 OpenFlow protocol Common OpenFlow packet header All OpenFlow messages start with this header 32 bits 8 bits 8 bits 16 bits version type length xid Version: Type: Length: xid: - version of OpenFlow protocol - type of OpenFlow protocol message - total length of message in octets - transaction ID used to match responses with requests 257

258 OpenFlow version numbers Version number has incremented with every major release of the OpenFlow specification Version of specification 1.0.x 1.1.x 1.2.x 1.3.x 1.4.x 1.5.x OpenFlow protocol version 0x01 0x02 0x03 0x04 0x05 0x06 258

259 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status 30 Role status 31 Table status 32 Request Forward 35 Controller Status Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Multipart Request 19 Multipart Reply ID Type 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply 24 Role Request 25 Role Reply 26 Get Async Request 27 Get Async Reply 28 Set Async 29 Meter Mod 33 Bundle Control 34 Bundle Add Messages new to v1.5.0 are in bold. 259

260 Version negotiation On connection establishment: Each side sends a Hello message with the version set to the highest OpenFlow version supported by the sender. The Hello message can optionally include a version bitmap that specifies all the versions supported by the sender. If (the version bitmap is supported by both sides) AND (the two bitmaps have some common bits set) negotiated version = highest version set in both bitmaps Else negotiated version = minimum (version number that was sent, version number that was received) 260

261 Understanding switch capabilities Due to the large number of required and optional OpenFlow capabilities, it is important for the controller to understand the features supported by the switch it is managing. A features/capabilities discovery is done via a handshake to acquire this information. 261

262 Handshake Once TLS session is established, the controller sends a Features Request message. The switch responds with a Features Reply message: #tables 32 bits datapath id #buffers auxiliary ID Padding Capabilities Reserved Datapath ID: - Uniquely identifies a datapath. Lower 48-bits are the switch MAC address. Capabilities: - Types of stats supported etc. Ports: - Array of OpenFlowenabled physical ports 262

263 Flow table modification messages 5 possible operations: Add: instantiates a new flow entry in the flow table Modify: modifies elements of all matching (existing) flow entries Modify-Strict: modifies elements of flow entries that exactly match all fields including wildcards and priority Delete: deletes all matching (existing) flow entries Delete-Strict: deletes flow entries that exactly match all fields including wildcards and priority 263

264 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits cookie cookie mask table id command idle timeout hard timeout priority buffer id output port output group flags importance flow match descriptor instructions descriptor Cookie: - Opaque value set by the controller Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority importance: - Used for flow eviction purposes Fields new to v1.5.0 are in bold. 264

265 Flow match descriptor Flow match descriptor structure Payload is a set of OXM (OpenFlow Extensible Match) flow match fields 32 bits type padding OXM TLVs length oxm_class: - Specifies a set of related match types - OFPXMC_OPENFLOW_BASIC: contains the basic set of OpenFlow match fields oxm_class oxm_field length OXM TLV header oxm_field: - Match field within the oxm_class payload oxm_type: - combination of oxm_class and oxm_type 265

266 Flow match field prerequisites The matching of header fields of a protocol can only be done if the OpenFlow match also explicitly matches the corresponding protocol For example, a match for the TCP source port is only allowed if it is preceded by: A match for an IP Ethertype (either 0x0800 or ox86dd) AND A match for IP protocol = 6 (TCP) In other words, matching on the TCP port is only allowed if the EtherType is IP and the IP protocol is TCP 266

267 Flow action descriptor Flow action descriptor structure Structures used to describe flow actions 32 bits 32 bits Type= OUTPUT Length Output port Max Length Padding Type= SET MPLS TTL TTL Padding Length 32 bits 32 bits Type= SET QUEUE Length Type= SET FIELD Length Queue ID OXM TLV 267

268 Proactive vs reactive flow entries <Explain the two approaches> 268