Secure Science DMZ using Event-Driven SDN Tae Hwang Technical Solutions Architect @ Cisco
Typical Science DMZ Architecture 1.0 What is the biggest challenge with this architecture? Internet Firewall I2 AL2S/ AL3S Campus Traffic is managed via simple ACL or Flow Rule UC S 5 1 0 8 1 2 3 4 6 5 7 8 OK FAIL OK FAIL OK FAIL OK FAIL DMZ Switch UC S 5 1 0 8 1 2 3 4 6 5 7 8 OK FAIL OK FAIL OK FAIL OK FAIL DTN/Servers/Storage/perfSONAR 2
Science DMZ 2.0 SciPass Operation - Phase 3 n - Phase 2 Bypassing Campus Firewall for Large Flows SDN SOLUTIONS SHOWCASE SDN SOLUTIONS SHOWCASE SciPass Architecture: Combined with Brocade OF Switch (typically), SciPass inserts bypass Bro, PerfSONAR, and SciPass controller (Indiana University) d OpenFlow forwarding rules Traffic not sent to Traffic not sent to Firewall SciPass: Controller 100G Throughput improves 100G SciPass: Controller Feedback 10G OpenFlow Switch 10G 10G 100G Firewall Firewall Feedback 10G OpenFlow Switch 10G 10G 100G PerfSONAR PerfSONAR 3
Flow Detection Method IPS//FW/Router Insert whitelist/acl to match a packet with specific header information Data Transfer Node (DTN) Get a notification from DTN that is about to start a data transfer Globus Get a notification from Globus or similar tools. 4
FW/IPS Bypass Methods Option 1: Enable OpenFlow feature on Cisco OpenFlow Hybrid Switch Option 2: Use a dedicated OpenFlow Switch if the current device doesn t support OF. Option 3: Use PBR with NXAPI. Option 4: Use VACL and Redirect with NXAPI 5
How to Secure Science DMZ and Campus Q. Science DMZ is directly connected to the Internet. How can we secure Science DMZ and the campus? A. Leverage security devices to detect the threats and log threats to Event server, such as Splunk. Necessary actions against the threat are triggered by apps in the event server, actions could be Blackholeling BGP routes on routers, or applying OpenFlow rules on the OF switches, or both. 6
Science DMZ Reference Implementa3on Commodity Internet Internet2/AL2S Next Genera=on Firewall Commodity: In- Line Internet 2: In- Line or OOB w/steering BGP Nexus 3K OpenFlow BGP Null Routes High- Throughput Science s DMZ ASR 1K ASR 9K Nexus 9K Flow No3fica3on Ac3ve Blocking Secure Corporate s Compute DTN ASA 5585 Event Correlation Log Storage Auditing Analysis Corporate DC Campus External Services 7
Splunk as an SDN Application Logically sits on top of COSC to provide application intelligence Likely already sending events to central logging Has the most informed view of the status of the network, servers, and apps. Provides event correlation Consolidates the number of devices sending REST commands Correlates by severity, rate, and between events Provides for auditing and reporting capabilities Leverage existing skill by writing logic in Splunk search language 8
Example Event Actions Real-Time, Immediate Action: e.g. High Priority Event: Block Host Immediately From Real Time With Sliding Window and Threshold: e.g. SYN Attacks: Block host after 100 improper SYNs in 60 seconds From FW Scheduled with Fixed Window: e.g. Block Timeout: Unblock host if it has not been seen in last 24 hours 9 9
Globus for Data Transfer A key service in the research networking ecosystem with more than 10,000 active endpoints Software-as-a-Service (SaaS) solution to manage transfers where users can direct requests to transfer or synchronize files and directories between two locations Uses GridFTP to provide secure, reliable, and efficient transfer of data across wide-area distributed networks GridFTP extensions provides parallelism (i.e., the use of multiple socket connections between pairs of data movers), restart markers, and data channel security. GridFTP control plane provides the source and destination information for the flows it sets up Effectively authenticates flows before they bypass security 10
OpenFlow Data Flow Steering Base setup depending on mode: Out-Of-Band : <priority>100</priority> <in-port>54</in-port> <output-node-connector>52</output-node-connector> Outside 54 25 <output-node-connector>25</output-node-connector> In-Band Firewall/IPS: <priority>100</priority> <in-port>54</in-port> <output-node-connector>25</output-node-connector> 52 Inside Out-Of-Band <in-port>25</in-port> <output-node-connector>52</output-node-connector> Bypass operation the same for both modes <priority>200</priority> <in-port>54</in-port> <output-node-connector>52</output-node-connector> Outside 54 52 Inside In-Band FW/IPS 25 FW/IPS 11
Bypass Flows in Tap Switch Flow start notification: Jun 10 10:53:43 localhost splunk_odl_action: log_level=info, action=start, flow=199.66.189.10:50368-128.55.29.41:42600, status_code=200 Flows added to Nexus 3000: Flow: 4 Match: Actions: Priority: 200 tcp,in_port=54,nw_src=199.66.189.10,nw_dst=128.55.29.41,tp_src=50368,tp_dst=42600 output:52 Flow: 5 Match: tcp,in_port=52,nw_src=128.55.29.41,nw_dst=199.66.189.10,tp_src=42600,tp_dst=50368 Actions: output:54 Priority: 200 Flow stop notification: Jun 10 10:54:51 localhost splunk_odl_action: log_level=info, action=stop, flow=199.66.189.10:50368-128.55.29.41:42600, status_code=200 12
Remotely Triggered Black Hole Routing Static routes added by COSC through Netconf on ASR 9000: router static address-family ipv4 unicast 1.0.184.115/32 Null0 tag 666 1.161.169.139/32 Null0 tag 666 2.25.74.127/32 Null0 tag 666 2.50.153.67/32 Null0 tag 666 12.197.32.116/32 Null0 tag 666 Export the Null routes setting next-hop to black hole IP: route-policy as-11017-out if tag is 666 then else endif set next-hop 192.0.2.1 set community (no-export) additive pass pass end-policy Enable urpf on WAN interface on ASR 9000: ipv4 verify unicast source reachable-via any allow-default Route Black Hole IP to NULL 0 on other border routers: ip route 192.0.2.1 255.255.255.255 Null0 Enable urpf on WAN interface on ASR 1000: ip verify unicast source reachable-via any 13
Cisco Open SDN Controller Application 1 Application 2 Application 3 Application 4 Application n Applications REST APIs DLux User Interface BASE NETWORK SERVICE FUNCTIONS Topology Statistics Manager Manager FRM L2 Switch AAA Service GBP Service Host Tracker 3 rd PARTY NETWORK SERVICE FUNCTIONS Service 1 Service 2 Service 3 Service 4 Service n Cisco Open SDN Controller Platform Model Driven Service Abstraction Layer (Plugin Manager, Capacity Abstraction, Flow Programming, Inventory, etc) OpenFlow Interface OVSDB Interface NETCONF Interface BGPLS Interface PCEP Interface OpenFlow Enabled Devices Open vswitches Cisco and 3 rd Virtual and Physical Devices Data Plane Elements 14
Splunk Screenshot 1 15
Splunk Screenshot 2 16
Splunk Screenshot 3 17