The balancing act: Research vs. Produc7on. Kobus Van der Merwe

Transcription

1 The balancing act: Research vs. Produc7on Kobus Van der Merwe 1

2 The balancing act: and Research vs. Produc7on Kobus Van der Merwe 2

3 Context All started with 3

4 Context All started with NSF program manager You guys have a number of C*C* projects surely there are many lessons learned 4

5 Context All started with NSF program manager You guys have a number of C*C* projects surely there are many lessons learned First lesson: When your NSF PM asks for something, you know the right answer 5

6 Context More seriously: Struggle with the tension between doing research and being involved with produc7on opera7ons ATempt to do both... 6

7 Context More seriously: Struggle with the tension between doing research and being involved with produc7on opera7ons ATempt to do both... Plan: Talk about aspects from three C*C* related project we have Get back to lessons learned (or observa+ons) part 7

8 Science Slices: Conver7ng Network Research Innova7on into Enhanced Capability for Computa7onal Science and Engineering Project in CC-NIE program (# ) Tom Cheatham, Steve Corbato, Robert Ricci, Adam Bolton and Kobus Van der Merwe 8

9 Science Slices Hardware refresh (100 Gbps) Science Slices: Using network virtualiza7on Science DMZ Suppor7ng Sloan Digital Sky Survey, Genomics & Gene7cs research Network and systems research Suppor7ng CloudLab & APT testbeds Conven+onal campus tenants 9

10 Science Slices: Conven7onal tenant slice Planned: SDN campus deployment Undergraduate residen7al environment Opt-in services Actual: Wired and wireless SDN Two campus buildings Explored SDN in campus environment Conven7onal networking combined with SDN White-box networking equipment Knowledge-centric network management Fine grained, dynamic, policy driven access control KnowU 10

11 KnowU deployment Internet Campus Core Building A Building B Building Z 11

12 KnowU deployment Internet Campus Core Building A Building B Building Z 12

13 KnowU deployment Core Router Distribution Router Distribution Router Access Router/Switch Access Router/Switch Access Router/Switch Access Router/Switch WiFi AP WiFi AP 13

14 KnowU deployment Distribution Router Core Router Access Router/Switch Distribution Router Access Router/Switch Access Router/Switch Access Router/Switch WiFi AP Access Router/Switch WiFi AP WiFi AP Access Router/Switch WiFi AP 14

15 KnowU deployment Internet KnowU Controller Campus Core Access Router/Switch Access Router/Switch WiFi AP WiFi AP 15

16 KnowU deployment Internet KnowU Controller Campus Core OF Switch OF Switch OF Switch WiFi AP OF Switch WiFi AP AP AP 16

17 KnowU: Policy driven access control Different users/services associated with different policies Network segrega7on: separate groups of Informa7on Assets, IT resources, Servers, Informa7on Systems, Users Control access and informa7on flow between domains Current solu7ons Sta7c network par77ons Typically 7ed to network loca7on, not users Human in the loop between policy and network realiza7on Low level realiza7on unconnected with high level policy Policy changes not reflected in network 17

18 KnowU: Policy driven access control Goals: Systema7c policy specifica7on Specifica7on drives network realiza7on Policies 7ed to groups of en77es (including users) Dynamically apply policies independent of network loca7on Policy changes applied to network automa7cally 18

19 KnowU: Policy driven access control Policies Security group E.g., users, servers etc. Allowed/disallowed communica7on between security groups Default rules Allows for excep7ons 19

20 KnowU: Policy driven access control Example: Regular Campus Network Flux Infrastructure Flux Resource CHPC Infrastructure AP AP AP FluxStaff Laptop 20

21 KnowU: Policy driven access control Example: Regular Campus Network Flux Infrastructure Flux Resource CHPC Infrastructure AP AP AP FluxStaff Laptop FluxStaff Laptop 21

22 KnowU: Policy driven access control Example: Regular Campus Network Flux Infrastructure Flux Resource CHPC Infrastructure AP AP AP FluxStaff Laptop FluxStaff Laptop 22

23 KnowU: Policy driven access control Example: { groupid": FluxGroupResources", "defaultcommunica7on": "false", groups": [ { "dstgroupid": FluxStaff", "isallowed": "true }, { dstgroupid : FluxStudents isallowed : false "excep7ons": [ { "isallowed": "true", endpointid": "server1", userid": "student1" } ]}]} { groupid": FluxStaff", defaultcommunica7on : true } { groupid": FluxStudents", defaultcommunica7on : true } 23

24 KnowU: System Architecture ODL CONTROLLER Radius Server ADMIN Policies DB Group DB User Details Ac7ve Users Ac7ve Directory Switch Config Switch Details DHCP Lease DB DHCP Server Campus Core Talac AP 24

25 KnowU: System Architecture ODL CONTROLLER Radius Server ADMIN Policies DB Group DB User Details Ac7ve Users Ac7ve Directory Switch Config Switch Details DHCP Lease DB DHCP Server Campus Core Talac AP User 1. User Connects 25

26 KnowU: System Architecture ODL CONTROLLER Radius Server ADMIN Policies DB Group DB User Details Ac7ve Users Ac7ve Directory Switch Config Switch Details DHCP Lease DB 2. Authen+cate DHCP Server Campus Core Talac AP User 1. User Connects 26

27 KnowU: System Architecture ODL CONTROLLER Radius Server ADMIN Policies DB Group DB User Details Ac7ve Users Ac7ve Directory Switch Config Switch Details DHCP Lease DB 2. Authen+cate DHCP Server Campus Core Talac AP 3. DHCP Request User 1. User Connects 27

28 KnowU: System Architecture ODL CONTROLLER Radius Server ADMIN Policies DB Group DB User Details Ac7ve Users Ac7ve Directory Switch Config 4. Install Flows Switch Details DHCP Lease DB 2. Authen+cate DHCP Server Campus Core Talac AP 3. DHCP Request User 1. User Connects 28

29 KnowU: Status Basic func7onality implemented and lab tested Currently doing staged deployment Target: serving friendly users by end of month 29

30 KnowU Policy driven access control Specific instance of more general campus/ enterprise management and opera7ons problem 30

31 NetSecOps Policy-driven, Knowledge-Centric, Holis7c Network Security Opera7ons Projects in CICI program (# and # ) Collabora7on between University of Utah and University of Kentucky Kobus Van der Merwe, Joe Breen and Corey Roach Jim Griffioen, Jane Hayes and Cody Bumgardner 31

32 The problem 32

33 The problem 33

34 The problem 34

35 Closing the loop 35

36 Closing the loop 36

37 Closing the loop 37

38 Closing the loop 38

39 Closing the loop 39

40 NetSecOps Policy Documents Knowledge Discovery Apps Policy Generation & Verification Apps Network Control Apps Knowledge Store Network Control Existing Data Sources Network 40

41 NetSecOps Policy Documents Knowledge Discovery Apps Policy Generation & Verification Apps Network Control Apps Knowledge Store Network Control Existing Data Sources Network 41

42 NetSecOps Policy Documents Knowledge Discovery Apps Policy Generation & Verification Apps Network Control Apps Knowledge Store Network Control Existing Data Sources Network 42

43 NetSecOps Policy Documents Knowledge Discovery Apps Policy Generation & Verification Apps Network Control Apps Knowledge Store Network Control Existing Data Sources Network 43

44 NetSecOps Policy Documents Knowledge Discovery Apps Policy Generation & Verification Apps Network Control Apps Knowledge Store Network Control Existing Data Sources Network 44

45 NetSecOps Policy Documents Knowledge Discovery Apps Policy Generation & Verification Apps Network Control Apps Knowledge Store Network Control Existing Data Sources Network 45

46 CapNet Secure Scien7fic Workloads with Capability Enabled Networks Project in CICI program (# ) Anton Burtsev and Kobus Van der Merwe 46

47 The problem Scien7fic workflows: across mul7ple infrastructures lab -> cloud -> HPC cluster across mul7ple ins7tu7ons sharing data complex policies associated with access Concerns: fragile security ad hoc manual solu7ons data might be sensi7ve medical records, genome data 47

48 Policy driven secure scien7fic workflows CapNet Controller 48

49 SDN-enabled capability access control Cloud plavorm Node A 64-bit Capability Node B Connec7vity controlled Name send(,...) recv (,...) through a capability system SDN Network Realized as a capabilityenabled SDN controller SDN Controller and 7e in with cloud for other cloud resources Capability System Capability Workflow applica7ons Object Nodes Rendezvous Flows Proxies manipulate resources and connec7vity through CapNet API CSpace Enables sophis7cated mul7-party workflows/ interac7ons 49

50 SDN-enabled capability access control Capability object types: Node Correspond to a VM (or physical machine) in a cloud environment Flow Unidirec7onal communica7on channel Rendezvous Point Allows principals to exchange capabili7es (associated with objects) Basic example: For two nodes to enable bidirec7onal communica7on: Create flow objects and exchange the capabili7es to those objects via a Rendezvous Point 50

51 SDN-enabled capability access control Capability object types (cont.): Membrane Allows controlled exchange of capabili7es: Destroy membrane: all capabili7es that passed through the membrane is destroyed. However: object capabili7es that were created on the inside of the membrane remain Example: Appliance as a service A creates a set of nodes Creates a membrane and send node capabili7es to B B access nodes and set up Hadoop cluster A revokes membrane B no longer has access to nodes, but Hadoop setup remains intact 51

52 SDN-enabled capability access control Capability object types (cont.): Sealer/unsealer Used to protect a capability so that it can be exchanged through untrusted environment Sealed capability need to be unsealed before it can be used 52

53 More involved example: sharing EHR Medical researchers different organiza7ons Different research agendas Want to share electronic health records (EHRs) from each organiza7on Concerns: Only EHR that meet certain criteria may be used E.g., consent to use in research, sa7sfy criteria for study etc. Only data needed for study should be extracted Plus might have to be anonymized Data from other EHRs should not be extracted/leaked Algorithms used to perform extrac7on might be proprietary Inspired by: MEdical Records and Genomics (emerge) Network: htps://emerge.mc.vanderbilt.edu 53

54 Simplified example: sharing EHR EHRs A EHRs B Extraction Algorithms A Validation Proxy B Combined EHRs A & B Goal is to construct workflow with following proper7es: EHRs A & EHRs B: isolated from everything, connected to secured Extrac+on Algorithms A Extrac+on Algorithms A isolated everything, connected to secured Valida+on Proxy B 54

55 Simplified example: sharing EHR EHRs A EHRs B Extraction Algorithms A Validation Proxy B Combined EHRs A & B Approach: Series of capability enabled cloud opera7ons to construct A deploys data in isolated node(s) B deploys data in isolated node(s) A asks B to deploy Proxy Within B s process: asks A to deploy Algorithms Allows A s Algorithm node to connect to isolated data nodes 55

56 Simplified example: sharing EHR EHRs A A s ac7ons: Obtains a node Performs reset opera7on on node node is now isolated Install data Create a flow capability to node Seal flow capability capability can safely be passed around requires unseal capability to use 56

57 Simplified example: sharing EHR EHRs A EHRs B B performs the same ac7ons: Obtains a node Performs reset opera7on on node node is now isolated Install data Create a flow capability to node Seal flow capability capability can safely be passed around requires unseal capability to use 57

58 Simplified example: sharing EHR EHRs A EHRs B Validation Proxy B A obtains node Creates a membrane Allows B to install Proxy through membrane Once B is done, A can remove membrane B s access to node is removed all configura7on B did on the node through the membrane remains 58

59 Simplified example: sharing EHR EHRs A EHRs B Extraction Algorithms A Validation Proxy B As part of B s ac7ons: recursively asks A to install Algorithms Obtains a node Creates a membrane Allows A to install Algorithms through membrane 59

60 Simplified example: sharing EHR EHRs A EHRs B Extraction Algorithms A Validation Proxy B As part of A s ac7ons: Pass in the sealed flow capabili7es for A and B s data nodes Create flow capability to allow connec7vity with Proxy node As before, once A is done, B can remove membrane B s access to node is removed all configura7on B did on the node through the membrane remains Once B removes membrane: A s Algorithm node unseal flow capabili7es and connects to data nodes 60

61 CapNet Status Implemented single cloud infrastructure func7onality SDN network: Open MUL controller Connec7vity controlled through CapNet Capability controller Built on capability library Integrated with OpenStack cloud plavorm Example workflow applica7ons: Appliance as a service Computa7on over joint data User Core Interface Driver XNet ML2 Agent Workflow Agents Horizon Glance Keystone Nova Neutron Metadata Manager Network Node SDN Control Manager Controller Node DHCP Agent WF Agent Metadata XNet ML2 Plugin MUL SDN Controller XNet Capability Control Application Novacompute Agent XNet ML2 Agent Core Interface Driver Physical Network Compute Node VM VM XNet Capability Protocol OpenStack ML2 RPC OpenFlow Control Other control New/modified Component started as part of XNet instance 61

62 Observa7ons 62

63 Three projects Science Slices CapNet NetSecOps 63

64 Three projects Science Slices CapNet NetSecOps 64

65 Broader context APT CloudLab TCloud XCap Science Slices CapNet Proto Geni SeaCat NetSecOps KnowOps 65

66 Broader context APT CloudLab TCloud XCap Science Slices CapNet Proto Geni SeaCat NetSecOps KnowOps Produc7on/ Infrastructure Research 66

67 Observa7ons Argue that atemp7ng to do both research and produc7on/infrastructure projects: posi7ve impact on both Requirements to make that work: Collabora7on! Folks who build/operate produc7on systems is a great source of interes7ng problems Find the human connectors People who might not do research, but understand where you are coming from (or the other way around) Know what side of the aisle you are opera7ng in Not all research efforts should end up in produc7on Produc7on efforts might not get you publica7ons 67

68 Thanks! 68