SDN Workshop. Contact: WSDN01_v0.1

Transcription

1 SDN Workshop Contact: WSDN01_v0.1 Issue Date: [Date] Revision: [xx]

2 OpenFlow SDN Workshop WSDN01_v0.1 Issue Date: [Date] Revision: [xx]

3 SDN architectural framework Application Plane Northbound Interfaces Application Service REST/RESTCONF/NETCONF/XMPP Network Services Abstraction Layer Control Plane (controller) Topology Discovery & Management Route selection & failover Traffic Engineering Resource Management Configuration East/Westbound interfaces BGP Southbound Interfaces BGP-LS i2rs PCE-P ForCES IPFIX SNMP Open Flow Netconf BGP RIBs PCC SNMP MIBs OpenFlow YANG Data Plane Segment Routing RSVP- TE Device & Resource Abstraction Layer (DAL) Network Devices IP/MPLS/Transport Note: designations of north-bound and south-bound are relative to the control plane ( controller ) 3

4 OpenFlow versions From v1.0.0 in 2009 to v1.5.2 in 2015 Developed by the Open Networking Foundation (ONF) since its foundation in March 2011 We shall start with v1.0.0 to get a basic understanding of how it operates. Note that there are significant changes in newer versions that will be pointed out. 4

5 OpenFlow revision timeline Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

6 OpenFlow revisions: features First nonexperimental version Single table Hard-coded 12-tuple match Multiple flow tables Group tables Instructions Metadata MPLS support OpenFlow extensible Match IPv6 Multiple controllers Egress tables Packet-aware pipeline OpenFlow extensible Statistics Stats trigger Non-ethernet packets Protocol extensibility Flow monitoring Eviction and vacancy events Message bundling Table synchronisation Multipart framework Table-miss flow entry Per-flow meters

7 OpenFlow v

8 OpenFlow v1.0.0 Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

9 OpenFlow v1.0.0 First non-experimental release version Wire protocol 0x01 December 31, 2009 Specified two types of OpenFlow-compliant switches*: OpenFlow-only: perform forwarding based purely on OpenFlow flow tables OpenFlow-enabled: support traditional Ethernet switching and routing functions in addition to OpenFlow packet forwarding * Definition was modified in later revisions of OpenFlow 9

10 OpenFlow components OpenFlow Controller OpenFlow Protocol Secure Channel An OpenFlow switch communicates with a controller over a secure connection using the OpenFlow protocol. Flow Table OpenFlow Switch 10

11 OpenFlow Switch components Flow table (single): Set of flow entries which specify packet match conditions and resulting actions Secure channel: Channel to an external controller which manages the switch using the OpenFlow protocol 11

12 Flow table Contains a set of flow entries with: Packet match criteria (e.g. header fields to match against packets) Zero or more actions to apply to matching packets Activity counters that are updated for matching packets Header Fields Actions Counters Flow entry 1 Forward to port 1/1 Flow entry 2 Drop Flow entry n Send to controller 12

13 Match fields Ingress port Ethernet source address Ethernet destination address Ethertype VLAN ID VLAN priority IP source address IP destination address IP protocol IP ToS bits TCP/UDP source port TCP/UDP destination port Match can be an exact value or ANY which matches any value (wildcard). Bitmasks can also be used for partial matches. Some fields may have dependencies e.g. IP protocol field can only be used if there is a corresponding match for the IPv4 EtherType. 13

14 Actions Each flow entry has zero or more actions that determine how the switch handles matching packets If no forward actions are specified, the packet is dropped Forward (output) Mandatory Forwarding of packet to physical or virtual ports Enqueue Optional Forward a packet through a specified queue attached to a port Drop Mandatory Implicit action associated with a flow-entry that has no specified action Modify-field Optional Specify modification of packet header fields 14

15 Supported Actions Output Action Description Output to switch port Set VLAN VID Set VLAN PCP Strip VLAN Set Ethernet source address Set Ethernet destination address Set IP source address Set IP destination address Set IP ToS Set TCP/UDP source port Set TCP /UDP destination port Enqueue Set the IEEE802.1q VLAN ID Set IEEE802.1q priority Strip the IEEE802.1q header Set Ethernet source address Set Ethernet destination address Set IP source address Set IP destination address Set IP Type of Service (ToS) bits Set TCP/UDP source port Set TCP /UDP destination port Output packet to a queue 15

16 Counters Per-table, per-flow, per-port and per-queue Per-table Active Entries Packet lookups Packet matches Per-flow Received packets Received bytes Duration (seconds) Duration (nanoseconds) Per-port Received packets Transmitted packets Received bytes Transmitted bytes Receive drops Transmit drops Receive errors Transmit errors Receive frame alignment errors Received overrun errors Receive CRC errors Collisions Per-queue Transmit packets Transmit bytes Transmit overrun errors 16

17 Flow table example Ethernet learning switch Header Fields Actions Input port Eth Src Eth Dest Ether Type VID PCP IP Src * * 12:34:56:AB:CD:EF * * * * * * * * * Output to port 1/2 * * 11:22:33:44:55:66 * * * * * * * * * Output to port 3/8 IP Dest IP Proto IP ToS L4 Src Port L4 Dest Port Firewall Header Fields Actions Input port Eth Src Eth Dest Ether Type VID PCP IP Src IP Dest IP Prot o * * * 0x0800 * * / /32 6 * * 80 Forward * * * 0x0800 * * / 16 IP ToS L4 Src Port L4 Dest Port /32 6 * * 80 Forward * * * * * * * * * * * * Drop 17

18 OpenFlow ports ALL : all OpenFlow interfaces except the incoming interface Physical Ports Physical hardware ports* Virtual Ports *For an OpenFlow-enabled switch, these are ports that have been explicitly configured to be OpenFlow ports CONTROLLER : logical interface to the OpenFlow controller LOCAL : local networking stack of the switch TABLE : sends packet for processing through the flow table (only for Packet-Out messages) IN_PORT : ingress port of packet NORMAL : processes packets via the traditional forwarding path supported by the switch FLOOD : flood along the minimum spanning tree 18

19 Basic packet processing All packets ingressing the switch via an OpenFlow port are compared against the flow table. If a matching entry is found, any actions for that entry are performed on the packet e.g. forward to a specified port If no match is found, the packet is forwarded to the controller over the secure channel 19

20 Matching Packet ingressing into switch Parse header fields (see next slide) Match flow table? Yes Perform actions No Send to controller via Secure channel 20

21 Header field parsing Initialise headers Set input port, Ethernet source & destination, Ethertype Eth type = 0x8100? Yes Set VLAN ID and PCP No Eth type = 0x0806? Yes Set IP source/dest from within ARP packet IP Proto = 6 or 17? Yes Use UDP/TCP source and dest for L4 fields No No Eth type = 0x0800? Yes Set IP source/dest. protocol and ToS fields Not IP Fragment? Yes IP Proto = 1? Yes Use ICMP type and code for L4 fields No No Packet Lookup No 21

22 OpenFlow Secure Channel This is the logical interface that connects each OpenFlow switch to an OpenFlow controller. The OpenFlow controller uses this interface to: Configure and manage the switch Add, delete and modify flow entries Receive events from the switch Send packets out the switch OpenFlow version only supported use of a single controller. 22

23 Connection setup A TLS connection is established by the switch to a configured IP address and TCP port 6633 (changed in later releases to the IANA-assigned port number of 6653) Traffic to/from the secure channel is not processed by the flow table 23

24 Connection interruption If connectivity with the controller is lost, the switch enters emergency mode. In emergency mode : Matching process is based only on flow table entries marked with the emergency bit (indicated via the Flags field of the Flow-Mod message) All non-emergency entries are deleted when entering emergency mode On initial startup, all switches are in emergency mode Emergency mode was removed from later versions. CHANGED BEHAVIOUR IN LATER VERSIONS 24

25 OpenFlow protocol messages Protocol defines three types of messages. Controller-to-switch: Are initiated by the controller and used to configure the switch or query its state Asynchronous: Are initiated by the switch and used to notify the controller about network events or changes to the switch state Symmetric: Can be initiated by either the controller or the switch and sent without solicitation 25

26 Controller-to-Switch messages Initiated by the controller and may or may not require a response from the switch. Messages include: Features: used by the controller to discover the capabilities supported by the switch Configuration: used to set and query configuration parameters Modify-state: sent by the controller to manage state on the switch. Main purpose is to add/delete/modify flows Read-state: used by the controller to query stats from the switch Packet-out: used by the controller to send a packet out of a specified port of the switch Barrier: used to ensure message dependencies 26

27 Asynchronous messages Initiated by the switch without solicitation from the controller. Messages include: Packet-in: sent to the controller for all packets that: do not have a matching flow entry OR are explicitly sent to the controller Flow-removed: sent when flows are removed from the flow-table. May be due to expiration or explicit deletion. Port-status: sent by the switch on port configuration or state changes. Errors: sent when errors are detected 27

28 Symmetric messages Can be initiated by either the controller or the switch and sent without solicitation Messages include: Hello: sent between the controller and switch upon connection establishment Echo: echo request/reply messages can be sent from either the switch or the controller; request messages must be responded to with a reply. Vendor: vendor-specific messages 28

29 OpenFlow protocol Common OpenFlow packet header All OpenFlow messages start with this header 32 bits 8 bits 8 bits 16 bits version type length xid Version: Type: Length: xid: - version of OpenFlow protocol - type of OpenFlow protocol message - total length of message in octets - transaction ID used to match responses with requests 29

30 OpenFlow version numbers Version number has incremented with every major release of the OpenFlow specification Version of specification 1.0.x 1.1.x 1.2.x 1.3.x 1.4.x 1.5.x OpenFlow protocol version 0x01 0x02 0x03 0x04 0x05 0x06 OpenFlow versions are NOT backwards-compatible. For example, a device running version 0x03 will not fall back to 0x01 to interwork with a device that only supports 0x01. 30

31 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Vendor Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Port Mod 16 Stats Request 17 Stats Reply 18 Barrier Request 19 Barrier Reply 20 Queue Get Config Request 21 Queue Get Config Reply Features Configuration Packet-out Modify-state Read-state Barrier Configuration 31

32 Version negotiation On connection establishment: Each side sends a Hello message with the version set to the highest OpenFlow version supported by the sender. The Hello message does not have a body, just the OpenFlow header. The OpenFlow protocol version negotiated independently by each side is: minimum (version number that was sent, version number that was received) This only works if the side with the higher version number also supports the lower version number. If not, an error occurs. 32

33 Understanding switch capabilities Due to the large number of required and optional OpenFlow capabilities, it is important for the controller to understand the features supported by the switch it is managing. A features/capabilities discovery is done via a handshake to acquire this information. 33

34 Handshake Once the TLS session is established, the controller sends a Features Request message. The switch responds with a Features Reply message: #tables 32 bits datapath id #buffers Padding Capabilities actions port descriptors Datapath ID: - Uniquely identifies a datapath. Lower 48-bits are the switch MAC address. Capabilities: - Types of stats supported etc. Actions: - Action types supported by switch Ports: - Array of OpenFlowenabled physical ports 34

35 Flow table modification messages 5 possible operations: Add: instantiates a new flow entry in the flow table Modify: modifies elements of all (existing) matching flow entries Modify-Strict: modifies elements of flow entries that exactly match all fields including wildcards and priority Delete: deletes all (existing) matching flow entries Delete-Strict: deletes flow entries that exactly match all fields including wildcards and priority 35

36 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits flow match descriptor Cookie: - Opaque value set by the controller command hard timeout output port cookie buffer id idle timeout priority flags Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority flow action descriptor 36

37 Flow match descriptor Flow match descriptor structure Structure used to describe flow match requirements 32 bits wildcard fields ingress port ethernet destination address pcp ip tos padding ip protocol tcp/udp source port ipv4 source address ipv4 destination address ethernet source address vid ethertype padding tcp/udp dest port Wildcard fields: - Bitmap indicating which fields are wildcards CHANGED FORMAT IN LATER VERSIONS 37

38 Flow action descriptor Flow action descriptor structure Structures used to describe flow actions 32 bits 32 bits Type= OUTPUT Length Type= SET VLAN ID Length Output port Max Length VID Padding 32 bits 32 bits Type= ENQUEUE Length Type= SET NETWORK SRC/DST Length Port IP address Padding Queue ID CHANGED FORMAT IN LATER VERSIONS 38

39 Proactive vs reactive flow entries Entries in the flow table can be installed either a priori (proactive) or on demand (reactive): Proactive Applicable when flow patterns are known ahead of time More suitable for aggregate traffic flows May require larger tables to allow a complete set of flow entries No delays with flow installation Reactive May be more applicable to dynamic flow patterns Optimises flow table usage as inactive flows may be timed out Delays may be experienced with flow installation as first packet needs to be sent to controller Uninterrupted connection to controller is essential It is also possible to have a combination of proactive and reactive flow entries 39

40 Flow removal All flow entries have two timers associated with them: idle_timeout: maximum time that can elapse without a flow matching the flow entry hard_timeout: maximum time that a flow entry can remain in the flow table A Flow-removed message is sent by the switch to the controller when a flow entry is removed from the flow table 40

41 Packet-In Message Packet-In message structure For packets sent from the switch to the controller buffer id total length input port reason padding data (ethernet frame) Buffer ID: - Identifies where packet is buffered Reason: - Either due to no match or explicit action Data: - Initial portion of packet 41

42 Packet-Out Message Packet-Out message structure For packets sent from the controller to the switch 32 bits input port buffer id size of actions array Buffer ID: - Same as the buffer ID in the original Packet-In message flow action descriptor packet data Packet data: - original packet (useful only when original packet was not buffered) 42

43 Queue structures Limited QoS support is provided through a simple queuing mechanism Flows can be mapped to queues which attach to a port and can be used to schedule the packets exiting the datapath on that output port Each queue is identified by a port number and a queue ID The only queue configuration options available are: min-rate: minimum guaranteed data-rate max-rate: maximum data-rate 43

44 Echo Request/Reply Messages Echo Request may be initiated by either the controller or the switch May be used for a number of reasons: To determine latency of connection between controller and switch As a liveness detection mechanism to verify liveness of the connection between controller and switch 44

45 Switch bootstrapping OpenFlow switches need to be configured with: URI or <IP address>:<port> of OpenFlow controllers Can be accomplished via OF-CONFIG For OpenFlow-enabled switches: OpenFlow-capable ports need to be identified and configured A mechanism must exist to channel flows to either OpenFlow processing or normal processing Ports and queues need to be configured For topology discovery via LLDP (de-facto mechanism): A flow entry to direct all received LLDP packets to the controller should be installed 45

46 Message flow example Controller Switch Initial exchange of Hellos with version negotiation Hello Hello Features Request Features Reply Discovery of switch features Reaction to unknown packet flow Packet- Out Packet- In Flow- Mod Installation of new flow entry 46

47 Topology discovery (1) The challenge: How can an OpenFlow controller discover the topology of a network comprising of OpenFlow switches in the absence of a distributed control plane? OpenFlow Discovery Protocol (OFDP): Not a formally specified protocol (topology discovery is not specified in any OpenFlow specification documents) The concept was inherited from the first implementation of an OpenFlow controller (the NOX implementation) 47

48 Topology discovery (2) Switches need to be bootstrapped as follows: URI or <IP address>:<port> of OpenFlow controllers A proactive rule is instantiated on all switches to allow dealing with LLDP packets: If ethertype=lldp, output to CONTROLLER In other words, if a packet is received with an ethertype of 0x88cc, it must be encapsulated within a Packet-In frame and sent to the controller 48

49 Aside: LLDP Standardised by IEEE 802.1ab Single-hop neighbour discovery protocol Operates at Layer 2 (Ethernet layer) Allows nodes to advertise their identities and capabilities and learn the identities and capabilities of directlyconnected neighbours Uses an Ethertype of 0x88cc and a destination multicast address of C E 49

50 Aside: LLDP (2) LLDP Ethertype LLDPDU Chassis ID TLV Port ID TLV TTL TLV Optional TLV End of LLDPDU TLV Chassis ID TLV Port ID TLV TTL TLV End of LLDPDU Description Identifier of the switch that sends the LLDP packet Identifier of the port through which the packet is sent Time validity of the information in the LLDP frame Indicates end of the payload in the LLDP frame 50

51 Topology discovery process (1) 1 Switches establish OpenFlow channel with the controller 1 Controller 1 1 OpenFlow Switch 1 (OFS1) p1 p1 OpenFlow Switch 3 (OFS3) p2 p2 p1 OpenFlow Switch 2 (OFS2) p2 51

52 Topology discovery process (2) 2 Controller learns of all active ports on all switches via Features Reply message Controller 2 2 OpenFlow Switch 1 (OFS1) p1 p1 OpenFlow Switch 3 (OFS3) p2 2 p2 p1 OpenFlow Switch 2 (OFS2) p2 52

53 Topology discovery process (3) 3 Input port Flow entry installed to forward all LLDP packets to controller Eth Src Eth Dest Ether Type VID PCP IP Src OpenFlow Switch 1 (OFS1) p2 IP Dest 3 IP Proto p1 IP ToS L4 Src Port 3 Controller L4 Dest Port * * * 0x88cc * * * * * * * * Send to controller Action p1 3 OpenFlow Switch 3 (OFS3) p2 p1 OpenFlow Switch 2 (OFS2) p2 53

54 Topology discovery process (4) 4 Controller generates a Packet-Out message (with an encapsulated LLDP packet) for each active port on each switch (only switch OFS1 shown) Packet-Out Output port: p1 Encapsulated packet: LLDP Chassis ID: OFS 1 Port ID: p1 OpenFlow Switch 1 (OFS1) 4 p1 Controller Packet-Out Output port: p2 Encapsulated packet: LLDP Chassis ID: OFS 1 Port ID: p2 p2 54

55 Topology discovery process (5) 5 Switch sends encapsulated LLDP packet out of each active port (only switch OFS1 shown) Controller OpenFlow Switch 1 (OFS1) p2 5 p1 5 LLDP Chassis ID: OFS 1 Port ID: p1 p1 OpenFlow Switch 3 (OFS3) p2 LLDP Chassis ID: OFS 1 Port ID: p2 p1 OpenFlow Switch 2 (OFS2) p2 55

56 Topology discovery process (6) 6 Switches OFS1 and OFS2 forward received LLDP packets to controller via Packet-In message Controller Packet-In Input port: p1 Encapsulated packet: LLDP Chassis ID: OFS 1 Port ID: p1 6 OpenFlow Switch 1 (OFS1) p2 p1 6 Packet-In Input port: p1 Encapsulated packet: LLDP Chassis ID: OFS 1 Port ID: p2 p1 OpenFlow Switch 3 (OFS3) p2 p1 OpenFlow Switch 2 (OFS2) p2 56

57 Topology discovery process (7) Controller learns that: Port p1 of OFS1 is directly connected to port p1 of OFS3 Port p2 of OFS1 is directly connected to port p1 of OFS2 Port p2 of OFS2 is directly connected to port p2 of OFS3 Port p1 of OFS3 is directly connected to port p1 of OFS1 Port p1 of OFS2 is directly connected to port p2 of OFS1 Port p2 of OFS3 is directly connected to port p2 of OFS2 57

58 Exclusions What OpenFlow does not do (or specify): Communication between controllers when using multiple controllers (v1.2.0+) How OpenFlow is used by northbound applications Topology discovery How to bootstrap the network Construction of paths that traverse multiple OpenFlow switches Configuration of OpenFlow switches (some of this is enabled by OF- CONFIG) 58

59 Versions 1.0.1/1.0.2 Both releases were errata and/or clarifications for the specification Clarifications: Packets that do not match any flow should be forwarded to the controller using a Packet-In message Changes: In addition to emergency mode, a new fail-secure mode was defined. In fail-secure mode, all packets and messages destined to the controller are dropped. Flow entries continue to be used and expire based on their timeouts. IANA allocated port 6653 for OpenFlow communications and was required to be used as the default port (for both TLS or plain TCP) 59

60 OpenFlow v

61 OpenFlow v1.1.0 Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

62 OpenFlow v1.1.0 Second major release version Wire protocol 0x02 February 28, 2011 This section highlights deltas from the previous release v

63 New features Multiple flow tables, pipeline and pipeline processing Group table Actions in flow table changed to instructions which describe a set or list of actions MPLS header fields supported Outcome for packets without a match in a flow table are configurable Per-packet metadata for communication between tables Vendor message changed to Experimenter message 63

64 OpenFlow switch types Specifies two types of OpenFlow-compliant switches: OpenFlow-only: perform forwarding based purely on OpenFlow flow tables OpenFlow-hybrid: support traditional Ethernet switching and routing functions in addition to OpenFlow packet forwarding (was referred to as OpenFlow-enabled in v1.0.0). 64

65 OpenFlow components Controller OpenFlow Protocol Secure Channel Group Table Flow Table Flow Table OpenFlow Switch 65

66 Flow tables Each flow table contains a set of flow entries Each flow entry consists of: Match fields Counters Set of instructions to apply to matching packets Match Fields Counters Instructions Ingress port Packet headers Metadata Modify action set Apply actions Modify pipeline processing 66

67 New flow table elements Metadata: A maskable register value that is used in order to carry information from one flow table to another Instructions: either a set of actions to add to the action set, a list of actions to apply immediately to the packet or a modification to pipeline processing. Action Set: a set of actions associated with a packet that are accumulated while the packet is processed by each table. The 'Action Set' is executed when the instruction set instructs the packet to exit the processing pipeline. 67

68 Match fields Ingress port Metadata Ethernet source address Ethernet destination address Ethertype VLAN ID VLAN priority MPLS label MPLS traffic class IPv4 source address IPv4 destination address IPv4 protocol IPv4 ToS bits TCP/UDP source port TCP/UDP destination port Fields new to v1.1.0 are in bold. Match can be an exact value or ANY which matches any value (wildcard). Bitmasks can also be used for partial matches Some fields may have dependencies e.g. IP protocol field can only be used if there is a corresponding match for the IPv4 EtherType. 68

69 Header field parsing Initialise headers Set input port, Ethernet source & destination, Ethertype VLAN Tag Eth type = 0x8100? Yes Set VLAN ID & PCP Skip any remaining VLAN tags No MPLS label ARP Switch supports MPLS? No Supports ARP Yes No Yes Eth type = 0x8847/8? Eth type = 0x0806? Use MPLS label and TC Set IP source/dest from within ARP packet Skip any remaining MPLS shim headers IP Proto = 6 or 17? Yes Use UDP/TCP/SCTP source and dest for L4 fields No No No IPv4 Eth type = 0x0800? Yes Set IP source/dest. protocol and ToS fields Not IP Fragment? Yes IP Proto = 1? Yes Use ICMP type and code for L4 fields No No No Packet Lookup

70 Instructions (1) Definition: attached to a flow entry as part of an Instruction Set and describe the OpenFlow processing that takes place when a packet matches the flow entry. Each instruction either: Modifies pipeline processing e.g. directing the packet to another flow table OR Contains a set of actions to add to the Action Set OR Contains a list of actions to apply immediately to the packet 70

71 Instructions (2) Supported instructions include: Apply-Actions: immediately applies the specified actions. The Action Set is not modified. Clear-Actions: clears all actions in the Action Set Write-Actions: merges the specified actions into the current Action Set. Write-Metadata: writes to the metadata field Goto-Table: indicates that the packet should next be processed through the specified table Each instruction type may only appear once in the instruction set. 71

72 Supported Actions Action Description Action Description Output Output to switch port Set MPLS label Set value of MPLS label Set VLAN VID Set the IEEE802.1q VLAN ID Set MPLS TC Set MPLS TC bits Set VLAN PCP Set IEEE802.1q priority Set MPLS TTL Set value of MPLS TTL Set Ethernet src addr Set Ethernet dest addr Set IP src addr Set IP dest addr Set IP ToS Set IP ECN Set TCP/UDP src port Set TCP /UDP dest port Set Ethernet source address Set Ethernet destination address Set IP source address Set IP destination address Set IP Type of Service (ToS) bits Set IP ECN bits Set TCP/UDP source port Set TCP /UDP destination port Decrement MPLS TTL Push VLAN Pop VLAN Push MPLS Pop MPLS Set queue Set group Set IP TTL Decrement value of MPLS TTL Push a new VLAN header Pop the outermost VLAN header Push a new MPLS label Pop the outermost MPLS label Set ID of queue to output packet to Set group ID Set value of IP header TTL Copy TTL out Copy TTL from next-tooutermost to outermost header Decrement IP TTL Decrement value of IP TTL Copy TTL in Copy TTL from outermost header to next-to-outermost Actions new to v1.1.0 are in bold. 72

73 Actions Definition: an operation that acts on a packet Forward (output) Mandatory Forwarding of packet to physical or virtual ports Set-Queue Optional Forward a packet through a specified queue attached to a port Drop Mandatory Implicit action associated with a flow-entry that has no specified action Group Mandatory Processes the packet through the specified group Push/Pop-Tag Optional Push/pop of VLAN and MPLS headers Set-field Optional Set packet header fields, manipulate TTL etc. 73

74 Action Set (1) An Action Set is associated with each packet and is empty by default As the packet passes through the pipeline the Action Set is modified by instructions (Write-Actions, Clear-Actions) of matching flow entries The Action Set is carried between flow tables as the packet progresses through the pipeline There is a maximum of one action of each type in the Action Set. 74

75 Action Set (2) The Action Set is executed when an instruction set does not include a Goto-Table action i.e pipeline processing terminates If no output action or group action are specified in an action set, the packet is dropped 75

76 Actions in Action Set Order of application of actions in the Action Set Order Action 1 Copy TTL inwards 2 Pop 3 Push 4 Copy TTL outwards 5 Decrement TTL 6 Set 7 QoS 8 Group 9 Output If no output action or group action are specified in an action set the packet is dropped. 76

77 Action List Associated with the Apply-Actions instruction and the Packet-Out message. Actions in the Action List are immediately executed in the order specified in the list. Multiple actions of the same type may appear in the same Action List and have a cumulative effect. 77

78 Group table (1) Flow entries may point to a group in the group table. The group table provides sets of actions for flooding, multipath, fast reroute, link aggregation and indirection. The group table contains group entries. Each group entry has a list of action buckets with semantics depending on group type. The group type determines which of the buckets are applied to each packet. 78

79 Group table (2) The group table contains group entries. Each group entry contains: Group Identifier Group Type Counters Action Buckets Group Type Description all Executes all buckets in the group Multicast/broadcast forwarding Packet is replicated for each bucket select Executes one bucket in the group Packets are sent to a single bucket, based on a hash algorithm indirect Executes the one defined bucket in the group For example, BGP next-hop indirection fast-failover Executes the first live bucket Bucket liveness tied to port(s) or group 79

80 Group table: all Group Table ID=1 Type = all Counters Bucket 1 Bucket 2 Actions Set output port 1/1 Actions Set output port 1/2 Replicates packet to all buckets and executes corresponding actions... Bucket n Actions Set output port m/n 80

81 Group table: select Group Table ID=2 Type = select Counters Bucket 1 Bucket 2 Weight = 1 Weight = 1 Actions Set output port 1/1 Actions Set output port 1/2 Hashes packet to one of the buckets in proportion to the configured weight Bucket 3 Weight = 10 Actions Set output port 2/1 81

82 Group table: indirect Group Table ID=3 Type = indirect Counters Bucket 1 Actions Set output port 1/1 All packets are directed to the single bucket 82

83 Group table: fast-failover ID=4 Bucket 1 Bucket 2 Group Table Type = fastfailover Watch port/group Watch port/group Counters Actions Set output port 1/1 Actions Set output port 1/2 Only a single bucket is used at a time. All packets are sent to the first active bucket. Bucket 3 Watch port/group Actions Set output port 2/1 Liveness of buckets depends on liveness of watched port or group 83

84 Counters Per-table Active Entries Packet lookups Packet matches Per-flow Received packets Received bytes Duration (seconds) Duration (nanoseconds) Per-port Received packets Transmitted packets Received bytes Transmitted bytes Receive drops Transmit drops Receive errors Transmit errors Receive frame alignment errors Received overrun errors Receive CRC errors Collisions Per-queue Transmit packets Transmit bytes Transmit overrun errors Per-group # flow entries Transmit bytes Transmit overrun errors Per-bucket Packet count Byte count Fields new to v

85 Matching (1) Packet In Start at table 0 Yes Match in table n? Yes Update counters Execute instructions: update action set update packet/match set fields update metadata Goto table n? No No Do one of following, depending on table configuration: send to controller drop continue to next table Execute action set 85

86 Matching (2) Every flow entry has a 16-bit priority value associated with it It is possible that a packet may match more than one flow entry. Only the highest-priority flow entry matching the packet is used as the matching flow entry for the packet. 86

87 Pipeline processing (1) Definition: the set of linked tables that provide matching, forwarding and packet modifications in an OpenFlow switch Matching starts at the first table and may continue to other tables If a matching entry is found, the instructions associated with the flow entry are executed. 87

88 Pipeline processing (2) Pipeline processing stops when the instruction set associated with a matching flow entry does not specify a next table. The packet s action set is processed and it is forwarded at this point. If no match is found (called a table miss), the behaviour depends on switch configuration; the packet may: Be forwarded to the controller (default option) Continuing to the next table Be dropped 88

89 Pipeline processing (3) Packet In Ingress port Action set = {} Flow Table 0 Packet+ Ingress port + metadata Action set Flow Table 1 Flow Table n Packet Action set Execute Action Set Packet Out OpenFlow Switch 89

90 Pipeline processing (4) Per-table packet processing: A Find highest-priority matching flow entry Match fields: Ingress port + metadata + packet headers Action set Flow Table A B Match fields: Ingress port + metadata + packet headers Action set C B Apply instructions: i. Modify packet and update match fields (APPLY- ACTIONS) ii. Update action set (CLEAR- ACTIONS, WRITE-ACTIONS) iii. Update metadata C Send match data and action set to next table 90

91 Connection interruption If connectivity with the controller is lost, the switch enters either fail-secure or fail-standalone mode The concept of emergency mode was deprecated. Fail-secure mode: In all packets and messages destined to the controller are dropped. Flow entries continue to be used and expire based on their timeouts. Fail-standalone mode: All packets are processed via the NORMAL port i.e. the switch acts as a traditional Ethernet switch or router Applies only to OpenFlow-hybrid switches 91

92 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status Controller-to-Switch Messages new to v1.1.0 are in bold. ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Stats Request 19 Stats Reply 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply Features Configuration Packet-out Modify-state Read-state Barrier Configuration 92

93 Flow match descriptor Flow match descriptor structure Structure used to describe flow match requirements CHANGED FORMAT IN LATER VERSIONS type ethernet source address 32 bits length ingress port wildcard fields ethernet source address mask ethernet destination address ethernet destination address mask vid pcp padding ethertype ip tos ip protocol ipv4 source address ipv4 source address mask ipv4 destination address ipv4 destination address mask tcp/udp source port tcp/udp dest port mpls label mpls tc padding metadata metadata mask Fields new to v1.1.0 are in bold. 93

94 Table Mod Message Table-Mod message structure Table ID 32 bits Config Padding Config: - Bitmap of flags to describe the behaviour of the table for unmatched packets Flag Description TABLE_MISS_CONTROLLER Send packet to controller (Packet-In) TABLE_MISS_CONTINUE Continue to the next table in the pipeline TABLE_MISS_DROP Drop the packet 94

95 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits cookie cookie mask table id command idle timeout hard timeout priority buffer id output port output group flags padding Cookie: - Opaque value set by the controller Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority flow match descriptor instructions descriptor Fields new to v1.1.0 are in bold. 95

96 OpenFlow v

97 OpenFlow v1.2.0 Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

98 OpenFlow v1.2.0 Third major release version Wire protocol 0x03 December 5, 2011 This section highlights deltas from the previous release v

99 New features New OpenFlow Extensible Match (OXM) instead of the previous static, fixed-length structures. Use of OXM for writing to packet header fields Support of IPv6 Packet parsing specification is removed Support for multiple controllers 99

100 OpenFlow ports OpenFlow Ports Physical Ports Logical Ports Reserved Ports Correspond to hardware interfaces of the switch Abstracted interfaces that do not directly correspond to hardware interfaces of the switch For example: LAGs, tunnels, loopback interfaces Specify generic forwarding actions: ALL CONTROLLER TABLE IN_PORT ANY LOCAL NORMAL* FLOOD* * Only supported by OpenFlow-hybrid switches 100

101 Supported Actions Action Description Action Description Output Copy TTL out Copy TTL in Set MPLS TTL Output to switch port Copy TTL from next-tooutermost to outermost header Copy TTL from outermost header to next-to-outermost Set value of the MPLS TTL Set Network TTL Decrement Network TTL Set Field Set value of the IP TTL Decrement IP TTL Set a header field using OXM TLV format Decrement MPLS TTL Decrement MPLS TTL Push VLAN Push a new VLAN tag Pop VLAN Pop the outer VLAN tag Push MPLS Push a new MPLS label Pop MPLS Pop the outer MPLS label Set Queue ID Set queue ID when outputting to a port Set Group Apply group 101

102 Multiple controllers Multiple controllers are supported to improve reliability Communication between controllers is not specified by the OpenFlow specification Controller roles: EQUAL: controller has complete access to the switch and is equal to all other controllers in the same role SLAVE: controller only has read-only access to the switch MASTER: controller has complete access to the switch; there can only be one controller with this role A switch may be simultaneously connected to multiple controllers in Equal state, multiple controllers in Slave state and at most a single controller in Master state. 102

103 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Stats Request 19 Stats Reply ID Type 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply 24 Role Request 25 Role Reply Messages new to v1.2.0 are in bold. 103

104 Flow match descriptor Flow match descriptor structure Payload is a set of OXM (OpenFlow Extensible Match) flow match fields 32 bits type padding OXM TLVs length oxm_class: - Specifies a set of related match types - OFPXMC_OPENFLOW_BASIC: contains the basic set of OpenFlow match fields oxm_class oxm_field length OXM TLV header oxm_field: - Match field within the oxm_class payload oxm_type: - combination of oxm_class and oxm_type 104

105 OXM example OXM TLV for TCP source port oxm_class=0x8000 (OFPXMC_OPENFLOW_BASIC) oxm_field=13 (TCP source port)_ H M length payload 105

106 Flow match field prerequisites The matching of header fields of a protocol can only be done if the OpenFlow match also explicitly matches the corresponding protocol For example, a match for the TCP source port is only allowed if it is preceded by: A match for an IP Ethertype (either 0x0800 or ox86dd) AND A match for IP protocol = 6 (TCP) In other words, matching on the TCP port is only allowed if the EtherType is IP and the IP protocol is TCP 106

107 Basic OpenFlow match fields OXM_field types for the OXM_class: OFPXMC_OPENFLOW_BASIC Ingress port Ingress physical port Metadata Ethernet destination address Ethernet source address Ethertype VLAN ID VLAN priority IP DSCP IP ECN IP protocol IPv4 source address IPv4 destination address TCP source port TCP destination port UDP source port UDP destination port SCTP source port SCTP destination port ICMPv4 type ICMPv4 code ARP OP ARP SPA ARP TPA ARP SHA ARP THA IPv6 source address IPv6 destination address IPv6 Flow Label ICMPv6 type ICMPv6 code IPv6 ND Target IPv6 ND SLL IPv6 ND TLL MPLS Label MPLS TC Fields new to v1.2.0 are in bold. 107

108 OpenFlow v

109 OpenFlow v1.3.x Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

110 OpenFlow v1.3.x Fourth major release version Wire protocol 0x04 April 13, 2012 Updated in v1.3.1, v1.3.2, v1.3.3, v1.3.4, v1.3.5 This section provides a ground-up description of v1.3.5 March 26,

111 New features Stats framework renamed to multipart framework Introduction of table-miss flow entry Support for per-flow meters Support for PBB Auxiliary connections Improved version negotiation via version bitmap 111

112 OpenFlow switch types Specifies two types of OpenFlow-compliant switches: OpenFlow-only: perform forwarding based purely on OpenFlow flow tables OpenFlow-hybrid: support traditional Ethernet switching and routing functions in addition to OpenFlow packet forwarding (was referred to as OpenFlow-enabled in v1.0.0). As with prior versions of the protocol, v1.3.x only supports Ethernet packets 112

113 OpenFlow components Controller OpenFlow Protocol OpenFlow Channel Group Table Flow Table Flow Table OpenFlow Logical Switch 113

114 OpenFlow components OpenFlow controller: An entity that interacts with the OpenFlow switch using the OpenFlow switch protocol. Typically, a single controller manages multiple OpenFlow Logical Switches OpenFlow Logical Switch: A set of OpenFlow resources that can be managed as a single entity Includes a datapath and control channel Was previously referred to simply as an OpenFlow Switch. The concept of an OpenFlow Logical Switch allows multiple such logical switches to be configured on a single physical switch. 114

115 OpenFlow Logical Switch One or more flow tables: Performs packet lookup and forwarding A Group Table Datapaths: components of the switch that are directly involved in traffic processing and forwarding. Includes the pipeline of flow tables, the group table and the ports. OpenFlow channel: Channel to an external controller which manages the switch using the OpenFlow protocol 115

116 Flow tables Each flow table contains a set of flow entries Each flow entry consists of: Match Fields Priority Counters Instructions Timeouts Cookie Flags Header fields Pipeline fields Precedence of flow entry Modify action set Apply actions Modify pipeline processing The match fields and priority taken together identify a unique flow entry in a specific flow table 116

117 Match Fields Two types of match fields: Header match fields: match values extracted from the packet header Pipeline match fields: match fields matching values attached to the packet for pipeline processing and not associated with packet headers e.g. IN_PORT IN_PHY_PORT METADATA TUNNEL_ID 117

118 Basic OpenFlow match fields OXM_field types for the OXM_class: OFPXMC_OPENFLOW_BASIC Ingress port Ingress physical port Metadata Ethernet destination address Ethernet source address Ethertype VLAN ID VLAN priority IP DSCP IP ECN IP protocol IPv4 source address IPv4 destination address TCP source port TCP destination port UDP source port UDP destination port SCTP source port SCTP destination port ICMPv4 type ICMPv4 code ARP OP ARP SPA ARP TPA ARP SHA ARP THA IPv6 source address IPv6 destination address IPv6 Flow Label ICMPv6 type ICMPv6 code IPv6 ND Target IPv6 ND SLL IPv6 ND TLL MPLS label MPLS TC MPLS BoS PBB ISID Tunnel ID IPv4 Ext Header Fields new to v1.3.x are in bold. 118

119 Counters Per-table Active Entries Packet lookups Packet matches Per-flow Received packets Received bytes Duration (seconds) Duration (nanoseconds) Per-queue Transmit packets Transmit bytes Transmit overrun errors Duration (seconds) Duration (nanoseconds) Counters new to v1.3.x are in bold Per-port Received packets Transmitted packets Received bytes Transmitted bytes Receive drops Transmit drops Receive errors Transmit errors Receive frame alignment errors Received overrun errors Receive CRC errors Collisions Duration (seconds) Duration (nanoseconds) Per-meter band In band packet count In Band byte count Per-group # flow entries Transmit bytes Transmit overrun errors Duration (seconds) Duration (nanoseconds) Per-bucket Packet count Byte count Per-meter Flow count Input packet count Input byte count Duration (seconds) Duration (nanoseconds) 119

120 Instructions (1) Definition: attached to a flow entry as part of an Instruction Set and describe the OpenFlow processing that takes place when a packet matches the flow entry. Each instruction either: Modifies pipeline processing e.g. directing the packet to another flow table OR Contains a set of actions to add to the Action Set OR Contains a list of actions to apply immediately to the packet 120

121 Instructions (2) Supported instructions include: Meter: direct packet to the specified meter Apply-Actions: immediately applies the specified actions. The Action Set is not modified. Clear-Actions: immediately clears all actions in the Action Set Write-Actions: merges the specified actions into the current Action Set. Write-Metadata: writes to the metadata field Goto-Table: indicates that the packet should next be processed through the specified table Each instruction type may only appear once in the Instruction Set. Instructions new to v1.3.x are in bold 121

122 Actions Definition: an operation that acts on a packet Forward (output) Mandatory Forwarding of packet to physical or virtual ports Set-Queue Optional Forward a packet through a specified queue attached to a port Drop Mandatory Implicit action associated with a flow-entry that has no specified action Group Mandatory Processes the packet through the specified group Push/Pop-Tag Optional Push/pop of VLAN and MPLS headers Set-field Optional Set packet header fields, manipulate TTL etc. 122

123 Supported Actions Action Description Action Description Output Copy TTL out Copy TTL in Set MPLS TTL Decrement MPLS TTL Push VLAN Output to switch port Copy TTL from next-tooutermost to outermost header Copy TTL from outermost header to next-to-outermost Set value of the MPLS TTL Decrement MPLS TTL Push a new VLAN tag Set Network TTL Decrement Network TTL Set Field Push PBB Pop PBB Set value of the IP TTL Decrement IP TTL Set a header field using OXM TLV format Push a new PBB service tag (I-tag) Pop the outer PBB service tag (I-tag) Pop VLAN Pop the outer VLAN tag Push MPLS Push a new MPLS label Pop MPLS Pop the outer MPLS label Set Queue ID Set queue ID when outputting to a port Set Group Apply group Actions new to v1.3.0 are in bold. 123

124 Action Set (1) Definition: a set of actions associated with the packet that are accumulated while the packet is processed by each table and that are executed when pipeline processing terminates An Action Set is associated with each packet and is empty by default As the packet passes through the pipeline the Action Set is modified by instructions (Write-Actions, Clear-Actions) of matching flow entries 124

125 Action Set (2) The Action Set is carried between flow tables as the packet progresses through the pipeline There is a maximum of one action of each type in the Action Set. The Action Set is executed when an instruction set does not include a Goto-Table action.pipeline processing terminates If no output action or group action are specified in an action set, the packet is dropped 125

126 Actions Order of application of actions in the Action Set Order Action 1 Copy TTL inwards 2 Pop 3 Push-MPLS 4 Push-PBB 5 Push-VLAN 6 Copy TTL outwards 7 Decrement TTL 8 Set 9 QoS 10 Group 11 Output If no output action or group action are specified in an action set the packet is dropped. 126

127 Action List Definition: ordered list of actions included in a flow entry in the Apply-Actions instruction or a Packet-Out message Actions in the Action List are immediately executed in the order specified in the list. Multiple actions of the same type may appear in the same Action List and have a cumulative effect. 127

128 Group table (1) Flow entries may point to a group in the group table. The group table provides sets of actions for flooding, multipath, fast reroute, link aggregation and indirection. The group table contains group entries. Each group entry has a list of action buckets with semantics depending on group type. The group type determines which of the buckets are applied to each packet. 128

129 Group table (2) The group table contains group entries. Each group entry contains: Group Identifier Group Type Counters Action Buckets Group Type Description all Executes all buckets in the group Multicast/broadcast forwarding Packet is replicated for each bucket select Executes one bucket in the group Packets are sent to a single bucket, based on a hash algorithm indirect Executes the one defined bucket in the group For example, BGP next-hop indirection fast-failover Executes the first live bucket Bucket liveness tied to port(s) or group 129

130 Group table: all Group Table ID=1 Type = all Counters Bucket 1 Bucket 2 Actions Set output port 1/1 Actions Set output port 1/2 Replicates packet to all buckets and executes corresponding actions... Bucket n Actions Set output port m/n 130

131 Group table: select Group Table ID=2 Type = select Counters Bucket 1 Bucket 2 Weight = 1 Weight = 1 Actions Set output port 1/1 Actions Set output port 1/2 Hashes packet to one of the buckets in proportion to the configured weight Bucket 3 Weight = 10 Actions Set output port 2/1 131

132 Group table: indirect Group Table ID=3 Type = indirect Counters Bucket 1 Actions Set output port 1/1 All packets are directed to the single bucket 132

133 Group table: fast-failover ID=4 Bucket 1 Bucket 2 Group Table Type = fastfailover Watch port/group Watch port/group Counters Actions Set output port 1/1 Actions Set output port 1/2 Only a single bucket is used at a time. All packets are sent to the first active bucket. Bucket 3 Watch port/group Actions Set output port 2/1 Liveness of buckets depends on liveness of watched port or group 133

134 Table-miss flow entry Specifies how to process packets unmatched by other flow entries in the flow table Identified by its match and priority: Wildcards all match fields Has the lowest priority (zero) Has similar behaviour to other flow entries: does not exist by default can be added or removed by the controller at any time it may expire If no table-miss flow entry exists, unmatched packets are dropped 134

135 Meter table Consists of meter entries, defining per-flow meters A meter measures the rate of packets assigned to it and enables controlling the rate of those packets Meter Identifier Meter Bands Counters Band Type Rate Burst Counters Type specific arguments Defines the lowest rate at which the band can apply The meter applies the band with the highest configured rate that is lower than the current measured rate. 135

136 OpenFlow ports OpenFlow Ports Physical Ports Logical Ports Reserved Ports Correspond to hardware interfaces of the switch Abstracted interfaces that do not directly correspond to hardware interfaces of the switch For example: LAGs, tunnels, loopback interfaces Specify generic forwarding actions: ALL CONTROLLER TABLE IN_PORT ANY LOCAL NORMAL* FLOOD* * Only supported by OpenFlow-hybrid switches 136

137 Reserved ports ALL : all OpenFlow interfaces except the incoming interface CONTROLLER : logical interface to the OpenFlow controller Reserved Ports LOCAL : local networking stack of the switch TABLE : sends packet for processing through the flow table (only for Packet-Out messages) IN_PORT : ingress port of packet NORMAL : processes packets via the traditional forwarding path supported by the switch FLOOD : flood along the minimum spanning tree 137

138 Matching (1) Packet In Start at table 0 Yes Match in table n? Yes Update counters Execute instructions: update action set update packet/match set fields update metadata Goto table n? No No Table-miss flow entry exists? Yes Execute action set No Drop packet 138

139 Matching (2) Every flow entry has a 16-bit priority value associated with it It is possible that a packet may match more than one flow entry. Only the highest-priority flow entry matching the packet is used as the matching flow entry for the packet. 139

140 Pipeline processing (1) Definition: the set of linked tables that provide matching, forwarding and packet modifications in an OpenFlow switch Matching starts at the first table and may continue to other tables If a matching entry is found, the instructions associated with the flow entry are executed. The instructions may explicitly direct the packet to another flow table. 140

141 Pipeline processing (2) Pipeline processing stops when the instruction set associated with a matching flow entry does not specify a next table. The packet s 'Action Set' is processed and it is forwarded at this point. If no match is found (called a table miss), the behaviour depends on the table-miss flow entry in the table. The actions may include: forwarding to the controller continuing to the next table being dropped 141

142 Pipeline processing (3) Packet In Ingress port Action set = {} Flow Table 0 Packet+ Ingress port + metadata Action set Flow Table 1 Flow Table n Packet Action set Execute Action Set Packet Out OpenFlow Switch 142

143 Pipeline processing (4) Per-table packet processing: A Find highest-priority matching flow entry Match fields: Ingress port + metadata + packet headers Action set Flow Table A B Match fields: Ingress port + metadata + packet headers Action set C B Apply instructions: i. Modify packet and update match fields (APPLY- ACTIONS) ii. Update action set (CLEAR- ACTIONS, WRITE-ACTIONS) iii. Update metadata C Send match data and action set to next table 143

144 Flow table example Ethernet learning switch Header Fields Actions Input port Eth Src Eth Dest Ether Type VID PCP IP Src * * 12:34:56:AB:CD:EF * * * * * * * * * Output to port 1/2 * * 11:22:33:44:55:66 * * * * * * * * * Output to port 3/8 IP Dest IP Proto IP ToS L4 Src Port L4 Dest Port Firewall Header Fields Actions Input port Eth Src Eth Dest Ether Type VID PCP IP Src IP Dest IP Prot o * * * 0x0800 * * / /32 6 * * 80 Forward * * * 0x0800 * * / 16 IP ToS L4 Src Port L4 Dest Port /32 6 * * 80 Forward * * * * * * * * * * * * Drop 144

145 OpenFlow Channel This is the logical interface that connects each OpenFlow switch to an OpenFlow controller. The OpenFlow controller uses this interface to: Configure and manage the switch Add, delete and modify flow entries Receive events from the switch Send packets out the switch There is one OpenFlow channel per OpenFlow controller 145

146 OpenFlow Connection A TLS or TCP network connection that is used by the OpenFlow channel to carry OpenFlow messages between a switch and a controller. An OpenFlow channel has a main connection (tcp or tls) and optionally, a number of auxiliary connections (tcp, tls, dtls or udp), in order to exploit parallelism Auxiliary connections on non-reliable transport, such as dtls or udp, can only support a small subset of the OpenFlow protocol e.g. they can be used to read stats 146

147 Multiple controllers Multiple controllers are supported to improve reliability Communication between controllers is not specified by the OpenFlow specification Controller roles: EQUAL: controller has complete access to the switch and is equal to all other controllers in the same role SLAVE: controller only has read-only access to the switch MASTER: controller has complete access to the switch; there can only be one controller with this role A switch may be simultaneously connected to multiple controllers in Equal state, multiple controllers in Slave state and at most a single controller in Master state. 147

148 Connection setup A TLS connection is established by the switch to a configured IP address and TCP port 6653 (as of version 1.3.3). Prior to v1.3.3, TCP port 6633 was used. Traffic to/from the secure channel is not processed by the flow table 148

149 Connection interruption If connectivity with the controller is lost, the switch enters either fail-secure or fail-standalone mode The concept of emergency mode was deprecated in v1.1.0 Fail-secure mode: In all packets and messages destined to the controller are dropped. Flow entries continue to be used and expire based on their timeouts. Fail-standalone mode: All packets are processed via the NORMAL port i.e. the switch acts as a traditional Ethernet switch or router Applies only to OpenFlow-hybrid switches 149

150 OpenFlow protocol messages Protocol defines three types of messages. Controller-to-switch: Are initiated by the controller and used to configure the switch or query its state Asynchronous: Are initiated by the switch and used to notify the controller about network events or changes to the switch state Symmetric: Can be initiated by either the controller or the switch and sent without solicitation 150

151 Controller-to-Switch messages Initiated by the controller and may or may not require a response from the switch. Messages include: Features: used by the controller to discover the capabilities supported by the switch Configuration: used to set and query configuration parameters Modify-state: sent by the controller to manage state on the switch. Main purpose is to add/delete/modify flows Read-state: used by the controller to query stats from the switch Packet-out: used by the controller to send a packet out of a specified port of the switch Barrier: used to ensure message dependencies Role-Request: used by the controller to set or query the role of its OpenFlow channel Asynchronous-Configuration: used by the controller to filter asynchronous messages it receives Messages new to v1.3.x are in bold 151

152 Asynchronous messages Initiated by the switch without solicitation from the controller. Messages include: Packet-in: sent to the controller for all packets that: do not have a matching flow entry OR are explicitly sent to the controller Flow-removed: sent when flows are removed from the flow-table. May be due to expiration or explicit deletion. Port-status: sent by the switch on port configuration or state changes. Errors: sent when errors are detected 152

153 Symmetric messages Can be initiated by either the controller or the switch and sent without solicitation Messages include: Hello: sent between the controller and switch upon connection establishment Echo: echo request/reply messages can be sent from either the switch or the controller; request messages must be responded to with a reply. Experimenter: vendor-specific messages 153

154 OpenFlow protocol Common OpenFlow packet header All OpenFlow messages start with this header 32 bits 8 bits 8 bits 16 bits version type length xid Version: Type: Length: xid: - version of OpenFlow protocol - type of OpenFlow protocol message - total length of message in octets - transaction ID used to match responses with requests 154

155 OpenFlow version numbers Version number has incremented with every major release of the OpenFlow specification Version of specification 1.0.x 1.1.x 1.2.x 1.3.x 1.4.x 1.5.x OpenFlow protocol version 0x01 0x02 0x03 0x04 0x05 0x06 OpenFlow versions are NOT backwards-compatible. For example, a device running version 0x03 will not fall back to 0x01 to interwork with a device that only supports 0x

156 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Multipart Request 19 Multipart Reply ID Type 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply 24 Role Request 25 Role Reply 26 Get Async Request 27 Get Async Reply 28 Set Async 29 Meter Mod Messages new to v1.3.x are in bold. 156

157 Version negotiation On connection establishment: Each side sends a Hello message with the version set to the highest OpenFlow version supported by the sender. The Hello message can optionally include a version bitmap that specifies all the versions supported by the sender. If (the version bitmap is supported by both sides) AND (the two bitmaps have some common bits set) negotiated version = highest version set in both bitmaps Else negotiated version = minimum (version number that was sent, version number that was received) 157

158 Understanding switch capabilities Due to the large number of required and optional OpenFlow capabilities, it is important for the controller to understand the features supported by the switch it is managing. A features/capabilities discovery is done via a handshake to acquire this information. 158

159 Handshake Once TLS session is established, the controller sends a Features Request message. The switch responds with a Features Reply message: #tables 32 bits datapath id #buffers auxiliary ID Padding Capabilities Reserved Datapath ID: - Uniquely identifies a datapath. Lower 48-bits are the switch MAC address. Capabilities: - Types of stats supported etc. Ports: - Array of OpenFlowenabled physical ports 159

160 Flow table modification messages 5 possible operations: Add: instantiates a new flow entry in the flow table Modify: modifies elements of all (existing) matching flow entries Modify-Strict: modifies elements of flow entries that exactly match all fields including wildcards and priority Delete: deletes all (existing) matching flow entries Delete-Strict: deletes flow entries that exactly match all fields including wildcards and priority 160

161 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits cookie cookie mask table id command idle timeout hard timeout priority buffer id output port output group flags padding Cookie: - Opaque value set by the controller Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority flow match descriptor instructions descriptor 161

162 Flow match descriptor Flow match descriptor structure Payload is a set of OXM (OpenFlow Extensible Match) flow match fields 32 bits type padding OXM TLVs length oxm_class: - Specifies a set of related match types - OFPXMC_OPENFLOW_BASIC: contains the basic set of OpenFlow match fields oxm_class oxm_field length OXM TLV header oxm_field: - Match field within the oxm_class payload oxm_type: - combination of oxm_class and oxm_type 162

163 OXM example OXM TLV for TCP source port oxm_class=0x8000 (OFPXMC_OPENFLOW_BASIC) oxm_field=13 (TCP source port)_ H M length payload 163

164 Flow match field prerequisites The matching of header fields of a protocol can only be done if the OpenFlow match also explicitly matches the corresponding protocol For example, a match for the TCP source port is only allowed if it is preceded by: A match for an IP Ethertype (either 0x0800 or ox86dd) AND A match for IP protocol = 6 (TCP) In other words, matching on the TCP port is only allowed if the EtherType is IP and the IP protocol is TCP 164

165 Flow action descriptor Flow action descriptor structure Structures used to describe flow actions 32 bits 32 bits Type= OUTPUT Length Output port Max Length Padding Type= SET MPLS TTL TTL Length Padding 32 bits 32 bits Type= SET QUEUE Length Type= SET FIELD Length Queue ID OXM TLV 165

166 Proactive vs reactive flow entries Entries in the flow table can be installed either a priori (proactive) or on demand (reactive): Proactive Applicable when flow patterns are known ahead of time More suitable for aggregate traffic flows May require larger tables to allow a complete set of flow entries No delays with flow installation Reactive May be more applicable to dynamic flow patterns Optimises flow table usage as inactive flows may be timed out Delays may be experienced with flow installation as first packet needs to be sent to controller Uninterrupted connection to controller is essential It is also possible to have a combination of proactive and reactive flow entries 166

167 Flow removal All flow entries have two timers associated with them: idle_timeout: maximum time that can elapse without a flow matching the flow entry hard_timeout: maximum time that a flow entry can remain in the flow table A Flow-Removed message is sent by the switch to the controller when a flow entry is removed from the flow table 167

168 Packet-In Message Packet-In message structure For packets sent from the switch to the controller 32 bits buffer id total length reason cookie match fields (OXM TLVs) data (ethernet frame) table ID Buffer ID: - Identifies where packet is buffered Reason: - One of: - no match - explicit action - TTL expired Match fields: - Pipeline fields associated with the packet Data: - Initial portion of packet 168

169 Packet-Out Message Packet-Out message structure For packets sent from the controller to the switch 32 bits buffer id input port length of actions array padding action list Buffer ID: - Same as the buffer ID in the original Packet-In message Packet data: - Initial portion of packet packet data 169

170 Multipart Request/Reply Messages Replace Stats-Request and Stats-Reply messages in earlier versions Used to encode requests or replies that may carry a large amount of data which may not be able to fit within a single OpenFlow message (max length of 64KB) A request or reply can span multiple messages and must use the same xid (transaction ID) for all messages in the message sequence 170

171 Multipart message types Type DESC FLOW AGGREGATE TABLE PORT_STATS QUEUE GROUP GROUP_DESC GROUP_FEATURES METER METER_CONFIG METER_FEATURES TABLE_FEATURES PORT_DESC EXPERIMENTER Description Information about the switch manufacturer, hardware revision, software revision, serial number etc Individual flow statistics Statistics about multiple flow entries Table statistics Port statistics Queue statistics Group statistics Lists the set of groups in the switch together with their bucket actions Capabilities of groups on a switch Meter statistics Configuration for one more more meters Set of features of the metering system Capabilities of the currently configured tables e.g. supported actions, instructions, match fields etc. Description of all the standard ports of the OpenFlow switch Experimenter-defined behaviour 171

172 Echo Request/Reply Messages Echo Request may be initiated by either the controller or the switch May be used for a number of reasons: To determine latency of connection between controller and switch As a liveness detection mechanism to verify liveness of the connection between controller and switch 172

173 QoS structures: queues Limited QoS support is provided through a simple queuing mechanism Flows can be mapped to queues which attach to a port The only queue configuration available is: min-rate: minimum guaranteed data-rate 173

174 QoS structures: meters Definition: switch elements that can measure and control the rate of packets The meter triggers a meter band if the rate passing through the meter exceeds a predefined threshold 174

175 Switch bootstrapping OpenFlow switches need to be configured with: URI or <IP address>:<port> of OpenFlow controllers Can be accomplished via OF-CONFIG For OpenFlow-hybrid switches: OpenFlow-capable ports need to be identified and configured A mechanism must exist to channel flows to either OpenFlow processing or normal processing Ports and queues need to be configured For topology discovery via LLDP (de-facto mechanism): A flow entry to direct all received LLDP packets to the controller should be installed 175

176 Message flow example Controller Switch Initial exchange of Hellos with version negotiation Hello Hello Features Request Features Reply Discovery of switch features Reaction to unknown packet flow Packet- Out Packet- In Flow- Mod Installation of new flow entry 176

177 Topology discovery (1) The challenge: How can an OpenFlow controller discover the topology of a network comprising of OpenFlow switches in the absence of a distributed control plane? OpenFlow Discovery Protocol (OFDP): Not a formally specified protocol (topology discovery is not specified in any OpenFlow specification documents) The concept was inherited from the first implementation of an OpenFlow controller (the NOX implementation) 177

178 Topology discovery (2) Switches need to be bootstrapped as follows: URI or <IP address>:<port> of OpenFlow controllers A proactive rule is instantiated on all switches to allow dealing with LLDP packets: If ethertype=lldp, output to CONTROLLER In other words, if a packet is received with an ethertype of 0x88cc, it must be encapsulated within a Packet-In frame and sent to the controller 178

179 Aside: LLDP Standardised by IEEE 802.1ab Single-hop neighbour discovery protocol Operates at Layer 2 (Ethernet layer) Allows nodes to advertise their identities and capabilities and learn the identities and capabilities of directlyconnected neighbours Uses an Ethertype of 0x88cc and a destination multicast address of C E 179

180 Aside: LLDP (2) LLDP Ethertype LLDPDU Chassis ID TLV Port ID TLV TTL TLV Optional TLV End of LLDPDU TLV Chassis ID TLV Port ID TLV TTL TLV End of LLDPDU Description Identifier of the switch that sends the LLDP packet Identifier of the port through which the packet is sent Time validity of the information in the LLDP frame Indicates end of the payload in the LLDP frame 180

181 Topology discovery process (1) 1 Switches establish OpenFlow channel with the controller 1 Controller 1 1 OpenFlow Switch 1 (OFS1) p1 p1 OpenFlow Switch 3 (OFS3) p2 p2 p1 OpenFlow Switch 2 (OFS2) p2 181

182 Topology discovery process (2) 2 Controller learns of all active ports on all switches via Features Reply message Controller 2 2 OpenFlow Switch 1 (OFS1) p1 p1 OpenFlow Switch 3 (OFS3) p2 2 p2 p1 OpenFlow Switch 2 (OFS2) p2 182

183 Topology discovery process (3) 3 Input port Flow entry installed to forward all LLDP packets to controller Eth Src Eth Dest Ether Type VID PCP IP Src OpenFlow Switch 1 (OFS1) p2 IP Dest 3 IP Proto p1 IP ToS L4 Src Port 3 Controller L4 Dest Port * * * 0x88cc * * * * * * * * Send to controller Action p1 3 OpenFlow Switch 3 (OFS3) p2 p1 OpenFlow Switch 2 (OFS2) p2 183

184 Topology discovery process (4) 4 Controller generates a Packet-Out message (with an encapsulated LLDP packet) for each active port on each switch (only switch OFS1 shown) Packet-Out Output port: p1 Encapsulated packet: LLDP Chassis ID: OFS 1 Port ID: p1 OpenFlow Switch 1 (OFS1) 4 p1 Controller Packet-Out Output port: p2 Encapsulated packet: LLDP Chassis ID: OFS 1 Port ID: p2 p2 184

185 Topology discovery process (5) 5 Switch sends encapsulated LLDP packet out of each active port (only switch OFS1 shown) Controller OpenFlow Switch 1 (OFS1) p2 5 p1 5 LLDP Chassis ID: OFS 1 Port ID: p1 p1 OpenFlow Switch 3 (OFS3) p2 LLDP Chassis ID: OFS 1 Port ID: p2 p1 OpenFlow Switch 2 (OFS2) p2 185

186 Topology discovery process (6) 6 Switches OFS1 and OFS2 forward received LLDP packets to controller via Packet-In message Controller Packet-In Input port: p1 Encapsulated packet: LLDP Chassis ID: OFS 1 Port ID: p1 6 OpenFlow Switch 1 (OFS1) p2 p1 6 Packet-In Input port: p1 Encapsulated packet: LLDP Chassis ID: OFS 1 Port ID: p2 p1 OpenFlow Switch 3 (OFS3) p2 p1 OpenFlow Switch 2 (OFS2) p2 186

187 Topology discovery process (7) Controller learns that: Port p1 of OFS1 is directly connected to port p1 of OFS3 Port p2 of OFS1 is directly connected to port p1 of OFS2 Port p2 of OFS2 is directly connected to port p2 of OFS3 Port p1 of OFS3 is directly connected to port p1 of OFS1 Port p1 of OFS2 is directly connected to port p2 of OFS1 Port p2 of OFS3 is directly connected to port p2 of OFS2 187

188 Exclusions What OpenFlow does not do (or specify): Communication between controllers when using multiple controllers (v1.2.0+) How OpenFlow is used by northbound applications Topology discovery How to bootstrap the network Construction of paths that traverse multiple OpenFlow switches Configuration of OpenFlow switches (some of this is enabled by OF- CONFIG) 188

189 OpenFlow v1.4.x 189

190 OpenFlow v1.4.x Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

191 New features More extensible wire protocol (TLV-based structures) Optical port properties Flow monitoring to allow better co-ordination between multiple controllers Eviction and vacancy events to deal with finite table capacity Message bundling Table synchronisation 191

192 Flow eviction Mechanism used to reclaim switch resources Has to be explicitly enabled Flow entries are selected for eviction based on: A new flow entry field importance. Flow entries with lower importance will always be evicted before entries with higher importance. The remaining lifetime of the flow entry. Flow entries with shorter remaining lifetimes will be evicted before entries with longer remaining lifetimes. 192

193 Table vacancy events Generates events (TABLE_STATUS) messages based on occupancy of flow tables vacancy_down: generated when the remaining space in the flow table falls to less than the configured threshold vacancy_up: generated when the remaining space in the flow table increases to more than the configured threshold The specification does not define the behaviour of controllers on receiving these messages. 193

194 Flow monitoring (1) Flow monitoring allows a controller to keep track of changes to flow tables Useful for multi-controller environments where controllers can be made aware of changes made to the flow table by other controllers Flow monitors can be created to match a subset of flow entries in selected flow tables. Events are generated for any changes to matching flow entries. Flow monitoring requests are done via multipart messages. 194

195 Flow monitoring (2) Types of flow monitors: Initial: all flow entries matching the flow monitor at the time of the request Add: new additions of flow entries matching the flow monitor Removed: removal of flow entries matching the flow monitor Modification: modification of flow entries matching the flow monitor 195

196 Flow table synchronisation (1) Allows flow entries in a table to be synchronised with another table: Flow entries in the synchronised table are automatically updated to reflect changes in the table it is synchronised with Enables multiple matches on different views of the same data at different points of the OpenFlow pipeline. Synchronisation can be uni-directional or bi-directional. When a flow entry is added, modified or removed in the source table, a corresponding flow entry is automatically added, modified or removed in the synchronised table 196

197 Flow table synchronisation (2) Entries in the synchronised table may not be identical to the corresponding entry in the source flow table e.g. transposed source/destination matches, different instruction sets etc. Recommended to be created as permanent flow entries (expiry timers set to zero) so that the lifetime of the corresponding flow entries is also synchronised Flow entry sychronisation can be unidirectional or bidirectional. 197

198 Bundle messages Bundle: sequence of OpenFlow modification messages that are applied as a single OpenFlow operation Provides a degree of atomicity (either all changes are applied or none at all) Example: 1. OFPBCT_OPEN_REQUEST bundle_id 2. OFPT_BUNDLE_ADD_MESSAGE bundle_id modication 1 3. OFPT_BUNDLE_ADD_MESSAGE bundle_id OFPT_BUNDLE_ADD_MESSAGE bundle_id modication n 5. OFPBCT_CLOSE_REQUEST bundle_id 6. OFPBCT_COMMIT_REQUEST bundle_id 198

199 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status 30 Role status 31 Table status 32 Request Forward Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Multipart Request 19 Multipart Reply ID Type 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply 24 Role Request 25 Role Reply 26 Get Async Request 27 Get Async Reply 28 Set Async 29 Meter Mod 33 Bundle Control 34 Bundle Add Messages new to v1.4.0 are in bold. 199

200 Basic OpenFlow match fields OXM_field types for the OXM_class: OFPXMC_OPENFLOW_BASIC Ingress port Ingress physical port Metadata Ethernet destination address Ethernet source address Ethertype VLAN ID VLAN priority IP DSCP IP ECN IP protocol IPv4 source address IPv4 destination address Fields new to v1.4.0 are in bold. TCP source port TCP destination port UDP source port UDP destination port SCTP source port SCTP destination port ICMPv4 type ICMPv4 code ARP OP ARP SPA ARP TPA ARP SHA ARP THA IPv6 source address IPv6 destination address IPv6 Flow Label ICMPv6 type ICMPv6 code IPv6 ND Target IPv6 ND SLL IPv6 ND TLL MPLS label MPLS TC MPLS BoS PBB ISID Tunnel ID IPv4 Ext Header PBB UCA 200

201 Modify Flow Entry Message Flow Mod message structure Structure used to add/delete/modify flow entries 32 bits cookie cookie mask table id command idle timeout hard timeout priority buffer id output port output group flags importance flow match descriptor instructions descriptor Cookie: - Opaque value set by the controller Command - Add/Modify/Modifystrict/Delete/Deletestrict Priority: - Priority of flow entry. Higher numerical value implies higher priority importance: - Used for flow eviction purposes Fields new to v1.4.0 are in bold. 201

202 OpenFlow v1.5.x 202

203 OpenFlow v1.5.x Version 1.5.x Version 1.4.x Version 1.3.x Version 1.2.x Version 1.1.x Version 1.0.x

204 OpenFlow v1.5.x Sixth major release version Wire protocol 0x06 December 19, 2014 Latest release of the specification This section highlights deltas of version (March 26, 2015) from the previous release

205 New features Egress tables Packet Type-aware pipeline Extensible flow entry statistics: OpenFlow extensible Statistics (OXS) Flow entry statistics trigger Copy-Field action to copy between two OXM fields Packet Register pipeline fields Scheduled Bundles Meter action 205

206 OpenFlow components Controller Controller OpenFlow Protocol OpenFlow Protocol OpenFlow Channel Control Channel OpenFlow Channel Group Table Datapath Meter Table Port Port Flow Table Flow Table Flow Table Port Port Pipeline OpenFlow Switch 206

207 Basic OpenFlow match fields OXM_field types for the OXM_class: OFPXMC_OPENFLOW_BASIC Ingress port Ingress physical port Metadata Ethernet destination address Ethernet source address Ethertype VLAN ID VLAN priority IP DSCP IP ECN IP protocol IPv4 source address IPv4 destination address TCP source port Fields new to v1.5.0 are in bold. TCP destination port UDP source port UDP destination port SCTP source port SCTP destination port ICMPv4 type ICMPv4 code ARP OP ARP SPA ARP TPA ARP SHA ARP THA IPv6 source address IPv6 destination address IPv6 Flow Label ICMPv6 type ICMPv6 code IPv6 ND Target IPv6 ND SLL IPv6 ND TLL MPLS label MPLS TC MPLS BoS PBB ISID Tunnel ID IPv4 Ext Header PBB UCA TCP Flags Action Set Output port Packet Type 207

208 Instructions (1) Definition: attached to a flow entry as part of an Instruction Set and describe the OpenFlow processing that takes place when a packet matches the flow entry. Each instruction either: Modifies pipeline processing e.g. directing the packet to another flow table OR Contains a set of actions to add to the 'Action Set' OR Contains a list of actions to apply immediately to the packet 208

209 Instructions (2) Supported instructions include: Apply-Actions: immediately applies the specified actions. The Action Set is not modified. Clear-Actions: immediately clears all actions in the Action Set Write-Actions: merges the specified actions into the current Action Set. Write-Metadata: writes to the metadata field Stat-Trigger: generate events based on stats thresholds Goto-Table: indicates that the packet should next be processed through the specified table Each instruction type may only appear once in the Instruction Set. Instructions new to v1.5.x are in bold 209

210 Instructions (3) Find highestpriority matching flow entry Match flow entry flow entry flow entry flow entry Flow Table Apply instructions Action Set Pipeline fields Packet Extract header fields flow entry table-miss flow entry Apply-actions {list of actions} modify packet update match fields update pipeline fields if output or group -> clone packet Clear-actions empty action set Write-actions {set of actions} merge in action set Goto-table {table-id} Flow Table Execute Action Set Packet clones Egress 210

211 Actions Definition: an operation that acts on a packet Forward (output) Set-Queue Drop Group Mandatory Optional Mandatory Mandatory Forwarding of packet to physical or virtual ports Forward a packet through a specified queue attached to a port Implicit action associated with a flow-entry that has no specified action Processes the packet through the specified group Push/Pop-Tag Set-field Meter Copy-field Optional Optional Optional Optional Push/pop of VLAN and MPLS headers Set packet header fields, manipulate TTL etc. Directs the packet to the specified meter Copies data between pipeline or header fields 211

212 Supported Actions Action Description Action Description Output Copy TTL out Copy TTL in Set MPLS TTL Decrement MPLS TTL Push VLAN Output to switch port Copy TTL from next-tooutermost to outermost header Copy TTL from outermost header to next-to-outermost Set value of the MPLS TTL Decrement MPLS TTL Push a new VLAN tag Set Network TTL Decrement Network TTL Set Field Push PBB Pop PBB Set value of the IP TTL Decrement IP TTL Set a header field using OXM TLV format Push a new PBB service tag (Itag) Pop the outer PBB service tag (I-tag) Pop VLAN Push MPLS Pop MPLS Pop the outer VLAN tag Push a new MPLS label Pop the outer MPLS label Copy Field Meter Copy value between header and register Apply meter (rate limiter) Set Queue ID Set queue ID when outputting to a port Set Group Apply group Actions new to v1.5.0 are in bold. 212

213 Egress tables In older versions, all processing was done in the context of the input port Egress tables allow processing to be done in the context of the output port 213

214 Matching (1) Packet In Clear action set Initialise pipeline fields Start at table 0 Match in table n? No Yes Update counters Execute instruction set: update action set update packet headers update match set fields update pipeline fields as needed, clone packet to egress Yes Goto table n? No Execute action set: update packet headers update match set fields update pipeline fields Table-miss flow entry exists? Yes Group Action? No Yes No Drop packet Drop packet No Output Action? Ingress Processing Yes Egress Processing (next slide) 214

215 Matching (2) Ingress Processing (previous slide) Egress Processing Yes Egress tables exist? No Start egress processing action set = {output port} start at first egress table Match in table n? No Yes Update counters Execute instruction set: update action set update packet headers update match set fields update pipeline fields as needed, clone packet to egress Yes Goto table n? No Execute action set: update packet headers update match set fields update pipeline fields Table-miss flow entry exists? Yes Drop packet No Output Action? Yes No Packet Out Drop packet 215

216 Matching (3) Every flow entry has a 16-bit priority value associated with it It is possible that a packet may match more than one flow entry. Only the highest-priority flow entry matching the packet is used as the matching flow entry for the packet. 216

217 Packet Type-aware pipeline First release to support non-ethernet packets: IPv4 and IPv6 New OXM pipeline field identifies the packet type. Namespace ns_type Match description Packet-in and packet-out format 0 0 Ethernet packet (default) Ethernet header and Ethernet payload 1 0x0800 IPv4 packet (no preceding header) 1 0x86dd IPv6 packet (no preceding header) IPv4 header and IPv4 payload IPv6 header and IPv6 payload 0 1 No packet Empty 0 0xFFFF Experimenter-defined Experimenter-defined 217

218 Pipeline processing (1) Definition: the set of linked tables that provide matching, forwarding and packet modifications in an OpenFlow switch Pipeline processing happens in two stages: ingress processing and egress processing Separation between the two stages is indicated by the first egress table Ingress tables: table IDs < first egress table Egress tables: table IDs first egress table 218

219 Pipeline processing (2) Pipeline processing starts with ingress processing at at the first table and may continue to other tables If a matching entry is found, the instructions associated with the flow entry are executed. The instructions may explicitly direct the packet to another flow table. Pipeline processing stops when the instruction set associated with a matching flow entry does not specify a next table. The packet s action set is processed at this point. 219

220 Pipeline processing (3) If the outcome of ingress processing is to forward the packet to an output port, (optional) egress processing may be performed in the context of that output port. If no match is found (called a table miss), the behaviour depends on the table-miss flow entry in the table. The actions may include: forwarding to the controller continuing to the next table being dropped 220

221 Pipeline processing (4) Packet In Ingress Port Ingress processing Set Ingress port Action set = {} Flow Table 0 Flow Table 1 Packet + pipeline fields (Ingress port, metadata etc.) Action set Flow Table n Action set Execute Action Set Group Table Egress processing Set output port Action set = {output} Flow Table e Flow Table e+1 e = first egress table ID Packet + pipeline fields (output port, metadata etc.) Action set Flow Table e+m Action set Execute Action Set Packet Out Output Port 221

222 Pipeline processing (5) Per-table packet processing: A Find highest-priority matching flow entry Match fields: Ingress port + metadata + packet headers Action set Flow Table A B Match fields: Ingress port + metadata + packet headers Action set C B Apply instructions: i. Modify packet and update match fields (APPLY- ACTIONS) ii. Update action set (CLEAR- ACTIONS, WRITE-ACTIONS) iii. Update metadata C Send match data and action set to next table 222

223 Ingress and egress processing Similarities in behaviour: Flow table matching Execution of instructions Table-miss processing Differences: At the beginning of ingress processing, the Action Set is empty At the beginning of egress processing, the 'Action Set' is initialised to contain only the Output action for the current output port 223

224 Asynchronous messages Initiated by the switch without solicitation from the controller. Messages include: Packet-in: sent to the controller for all packets that: do not have a matching flow entry OR are explicitly sent to the controller Flow-removed: sent when flows are removed from the flow-table. May be due to expiration or explicit deletion. Port-status: sent by the switch on port configuration or state changes. Role-status: informs the controller of a change in its role Controller-status: informs the controller when the status of an OpenFlow channel changes Flow-monitor: informs the controller of a change in a flow Errors: sent when errors are detected Messages new to v1.5.x are in bold 224

225 OpenFlow message types Symmetric ID Type 0 Hello 1 Error 2 Echo Request 3 Echo Reply 4 Experimenter Asynchronous ID Type 10 Packet-In 11 Flow-removed 12 Port status 30 Role status 31 Table status 32 Request Forward 35 Controller Status Controller-to-Switch ID Type 5 Features Request 6 Features Reply 7 Get Config Request 8 Get Config Reply 9 Set Config 13 Packet-out 14 Flow Mod 15 Group Mod 16 Port Mod 17 Table Mod 18 Multipart Request 19 Multipart Reply ID Type 20 Barrier Request 21 Barrier Reply 22 Queue Get Config Request 23 Queue Get Config Reply 24 Role Request 25 Role Reply 26 Get Async Request 27 Get Async Reply 28 Set Async 29 Meter Mod 33 Bundle Control 34 Bundle Add Messages new to v1.5.0 are in bold. 225

226 OpenFlow extensible Statistics (OXS) Extensible flow entry statistics: OpenFlow extensible Statistics (OXS) Similar concept to OXM Allows encoding of arbitrary flow statistics 226

227 Flow stats descriptor Flow stats descriptor structure Payload is a set of OXS (OpenFlow Extensible Stats) flow stats fields 32 bits reserved padding OXS TLVs length oxs_class: - Specifies a set of related stats types - OFPXSC_OPENFLOW_BASIC: contains the basic set of OpenFlow stats oxs_class oxs_field length OXS TLV header oxs_field: - Stats field within the oxs_class payload oxs_type: - combination of oxs_class and oxs_type 227

228 Limitations of OpenFlow Matching and action process in OpenFlow is not advanced enough to describe the rich set of capabilities of contemporary routers and switches. As OpenFlow cannot express all necessary packet operations, it must be augmented by device-specific mechanisms e.g. application of advanced QoS features Large number of optional features so it s not easy to work out a maximal set of intersecting features 228

229 OF-CONFIG 229

230 OF-CONFIG OF-CONFIG: OpenFlow Management and Configuration Protocol Companion protocol to OpenFlow Motivation of the protocol is to enable the remote configuration of OpenFlow switches Latest version of 1.2 supports up to OpenFlow v

231 OF-CONFIG components OpenFlow Configuration Point OF-CONFIG OpenFlow Controller OpenFlow Protocol OpenFlow Switch An OpenFlow Configuration Point communicates with an operational context which is capable of supporting an OpenFlow switch using the OpenFlow Configuration and Management Protocol )OF- CONFIG) Operational Context 231

232 Components (1) OpenFlow Logical Switch (OFLS): Abstraction defined by OF-CONFIG OF-CONFIG enables the configuration of the essential elements of an OpenFlow Logical Switch so that an OpenFlow controller can communicate with and control the switch via the OpenFlow protocol OpenFlow Capable Switch (OFCS): Physical or virtual network element that provides an operational context to host one or more OpenFlow Logical Switches by partitioning a set of OpenFlow-related resources such as ports and queues between the hosted OpenFlow Logical Switches. 232

233 Components (2) OpenFlow Configuration Point (OFCP): Service which sends OF-CONFIG messages to an OpenFlow Capable Switch OpenFlow resource: An element of an OpenFlow Capable Switch (e.g. ports, queues) that can be associated with an OpenFlow Logical Switch 233

234 Relationships OpenFlow Configuration Point OpenFlow Controller (s) OpenFlow Controller (s) OF-CONFIG OpenFlow Protocol OpenFlow Protocol OpenFlow Logical Switch OpenFlow Logical Switch OF resource (e.g port) OF resource (e.g port) OF resource (e.g port) OF resource (e.g port) OpenFlow Capable Switch 234

235 Scope The basic scope of OF-CONFIG is a set of functions required to configure an OpenFlow v1.3 logical switch: Discovery of capabilities of an OpenFlow Logical Switch Assignment of one or more controllers to an OpenFlow Logical Switch Assignment of resources of an OpenFlow Capable Switch to one or more OpenFlow Logical Switches Configuration of queues and ports Ability to remotely configure properties of ports Configuration of certificates for secure operation of OpenFlow Logical Switches and OpenFlow controllers 235

236 OFLS instantiation Initially, the OFCS owns all the resources of the switch and does not have any data planes instantiated. Using OF-CONFIG, the OFCP can instantiate one or more OFLS and assign resources such as queues and ports to it. 236

237 Transport protocol NETCONF is used as the transport protocol for OF- CONFIG OpenFlow Capable Switches need to implement SSH as the transport protocol as required by NETCONF NETCONF defines a set of operations on top of a messaging layer (RPC remote procedure calls) 237

238 NETCONF Layer Example Content <capable-switch> </capable-switch> Operations <get-config>,<set-config>,<notification> RPC <rpc>,<rpc-reply> Transport Protocol SSH, TLS, BEEP, SOAP 238

239 Data model Data model for OF-CONFIG 1.2 is encoded in an XML schema (YANG model is also available) Structured into classes and attributes of classes All NETCONF base protocol operations are supported: editconfig, get-config, copy-config, delete-config 239

240 Core Data model One or more OpenFlow controllers are associated with each OpenFlow Logical Switch One or more instances of OpenFlow Logical Switches are contained within the OpenFlow Capable Switch UML (Unified Modeling Language) Association Inheritance Aggregation Composition Switch contains different types of resources: ports, queues, certificates, flow tables Reproduced from OF-CONFIG 1.2 specification ( 240

241 Example: OpenFlow Capable Switch Reproduced from OF-CONFIG 1.2 specification ( <capable-switch> <id>capableswitch0</id> <configuration-points>... </configuration-points> <resources>... </resources> <logical-switches>... </logical-switches> </capable-switch> XML Example 241

242 Example: OpenFlow Logical Switch <logical-switch> <id>logicalswitch5</id> <capabilities>... <capabilities> Reproduced from OF-CONFIG 1.2 specification ( <datapath-id>datapath-id0</datapath-id> <enabled>true</enabled> <check-controller-certificate>false</checkcontroller-certificate> <lost-connection-behavior>failsecuremode</lostconnection-behavior> <controllers>... </controllers> <resources> <port>port2</port> <port>port3</port> <queue>queue0</queue> <queue>queue1</queue> <certificate>ownedcertificate4</certificate> <flow-table>1</flow-table> <flow-table>2</flow-table>... <flow-table>255</flow-table> </resources> </logical-switch> XML Example 242

243 XML Example Example Replacing the ipaddress element of the controller <?xml version="1.0" encoding="utf-8"?> <rpc message-id="1" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"> <edit-config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"> <target> <candidate/> </target> <default-operation>merge</default-operation> <config> <capable-switch xmlns="urn:onf:of12:config:yang"> <logical-switches> <switch> <id>logic-switch-1</id> <controllers> <controller> <id>controller-0</id> <ip-address operation="replace"> </ip-address> </controller> </controllers> </switch> </logical-switches> </capable-switch> </config> </edit-config> </rpc> <rpc-reply message-id="1" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"> <ok/> </rpc-reply Reproduced from OF-CONFIG 1.2 specification ( 243

244 References 244