Interfacing CoUniverse & Internet2 DCN

Transcription

1 Interfacing CoUniverse & Internet2 DCN Petr Holub, Miloš Liška, Andrew Lake, Chris Tracey, Tom Lehman, John Vollbrecht, Brian Cashman Sitola Laboratory of Advanced Networking Technologies CESNET, Internet2, Max Gigapop, Masaryk University Demo supported by: TAMU, NLM, TMC Internet2 Spring Member Meeting Washington DC, /20

2 Talk Overview Motivation Demo CoUniverse CoUniverse & Internet2 DCN Implementation Future Work 2/20

3 Motivation: CoUniverse High-definition collaborative environments Using high-quality, high-definition media streams to build collaborative environment bandwidth demands comparable to network link speeds (10 GbE) requires careful planing and configuration of infrastructure lacks adaptivity to changing networking conditions Large numbers of components needed to build the environment each one of them needs to be configured hard to orchestrate them manually to build the desired environment virtually impossible to cope with network events manualy 3/20

4 Motivation: CoUniverse Couniverse is a framework to orchestrate interactive network-centric applications GLIF and SuperComputing demonstrations Introduction to High-Performance Computing class Fields where CoUniverse can help audio/video transmissions remote instrument control generic network-centric application encapsulation... for scientific collaboration, telemedicine, etc. 4/20

5 Motivation: CoUniverse & Internet2 DCN Application-driven network allocation if we can control applications, why not the network? user should not be forced to allocate it manually CoUniverse is ideal for implementing this it s just another component to orchestrate framework is general enough to implement it 5/20

6 CoUniverse & Internet2 DCN Joint Tech Demo Network Topology Low-latency 1080i HD video using a network like this... for switch addressing: /24 to be used on vlan3440 for debug and test I2 IDC OC192 I2 Ciena Chicago 10GE LEARN: TAMU: Internet2 Dynamic Circuit Network LEARN Domain LEARN POP Houston, Texas 10GE learn-sw Te1/0/1 c3750e 3 I2 Ciena Houston learn-idc Wehner Building College Station, Texas DCN Workshop /29 VLANs: 3440/3441 tamu-sw 1Gig tamu-idc Cu c /29 eth0 gig1/0/ Memorial eth2 Internet Student Center couniverse College Station, /29 Texas tamu-sw c gig1/0/50 vlan3441 gig1/0/49 gig1/0/28 gig1/0/26 vlan3440 gig1/0/27 gig1/0/23 gig1/0/24 TAMU Domain 20 Force10 E Prague, Czech Republic Starlight E1200 C15454 Transponder DWDM Backbone 21 C Transponder Force10 HP5412 S Brno, Czech Republic hds.fi.muni.cz 6/20

7 Internet2 SMM Demo Washington, DC Bethesda, MD collaboration with National Library of Medicine Washington, DC Houston, TX collaboration with Memorial Hermann Texas Medical Center What do we need to make this happen? 1. network initialization 2. monitor network and wait until it comes up 3. startup of all applications and components 4. monitor everything and react to events 5. termination of all applications and components 6. network tear-down... we don t want this by hand, do we? 7/20

8 Internet2 SMM Demo DCN - CoUniverse Presentation With Just-In-Time Medical demos VLAN Ultragrid Video ( /24) VLAN Monitoring ( /24) VLAN 3305 Polycom/Ultragrid ( /24) National LIbrary of Medicine Bethesda, MD Memorial Hermann Texas Medical Center location Houston, TX static VLAN config Crystal Gateway Marriott Arlington, VA Ultragrid Sender Polycom 30 vlan Ultragrid Receiver static VLAN config vlan 3305 NLM darma-sw1 Switch NLM darma-sw2 Switch DRAGON clpk-raptor Switch static VLAN config DRAGON mcln-raptor Switch vlan 3205 LEARN DCN Switch I2 DCN Washington DCN I2 DCN Houston Internet2 1 Ann Arbor monitoring Polycom vlan 3305 Hotel HP2848 Switch MAX clpk-6509 Switch production IP VLANs: 246,247,248 IP IDC circuit endpoints: tagged nlm.nih.gov or urn:ogf:network:domain=dragon.maxgigapop.net:node=clpk:port=1-3-3:link=* tagged smm09.internet2.edu or urn:ogf:network:domain=dragon.maxgigapop.net:node=clpk6509:port=0-4-16:link=* untagged utest.anna.internet2.edu or urn:ogf:network:domain=anna.internet2.edu:node=raptor1:port=utest:link=* untagged memorialhermann.org or urn:ogf:network:domain=gigapop.gen.tx:node=switch1:port=0-1-22:link=* MAX clpk-t640 Router MAX lvl3-t640 Router I2 Router Washington MAX/DRAGON IDC URL: Wed Apr /20

9 Internet2 SMM Demo CoUniverse manages network: DCN circuits UltraGrid , 60 Hz, interlaced DXT compressed video (decompression runs entirely in GPU) 250 Mbps bandwidth 130 ms latency end-to-end Mac sender and Linux receiver, over GE Polycom HDX , compressed 4 Mbps bandwidth noticeably higher end-to-end latency 9/20

10 CoUniverse Building Blocks Organization of the CoUniverse Collaborative Universes where actual collaboration takes place partitions virtual space (like, e.g., Virtual Venues) provides privacy for users (may enforce authentication and authorization of users) limits size of the system Multiverse registration and lookup of Collaborative Universes automatically joined by each node Network organization control plane based on P2P substrate maximize robustness of the network distributes control messages, updates from monitoring, etc. one or more data planes running over native network maximize performance for the applications (typically throughput, latency, jitter) 10/20

11 CoUniverse Building Blocks Components nodes: physical nodes, having one or more network interfaces, running individual orchestrated applications proxy nodes: proxies and controllers for devices that can t run CoUniverse directly (Polycoms, microscopes, etc.) sites: aggregate of network nodes (e.g., you can tell you want stream from a specific site) application groups: aggregate of applications 11/20

12 CoUniverse Building Blocks Application Group Controller (AGC) controls operations in the Universe contains media streams scheduler if needed one AGC per application group 12/20

13 Self-organization in CoUniverse Dynamic media streams scheduling: Schedule media streams produced by media applications on particular network links (plan step) scheduling media streams using bandwidth close to physical link capacity is hard scheduling based on set of constraints producer constraints, consumer constraints, data distribution constraints, network link constraints Resilience: ability to react to changes/failures in the network infrastructure, media applications etc. achieved by monitoring, infrastructure changes and/or failures lead to new media streams schedule 13/20

14 Visualisation in CoUniverse overview of actual CoUniverse state for the user network topology visualization actually scheduled media streams GLIF 2007 visualisation Current development visualisation 14/20

15 CoUniverse & Internet2 DCN DCN interfaces web interface for humans web service interface (w/ security!) for machines Initializing and tearing circuits network link can have associated one or more lambda links DCN-specific lambda link: two endpoints (including identification, IDC and (tagged untagged) interface) and requested bandwidth Integration of on-demand circuits with CoUniverse brings another level of uncertainty into scheduling should I count on a network link I m uncertain to get? should I preallocate the network? (but there are n2 n 2 bidirecitonal links!)... see future work 15/20

16 Notes on Implementation State of the CoUniverse goal is to have usable proof-of-concept implementation research on self-organization, application orchestration, scheduling real-life applications (science, education) Open-source Implemented in Java, works on Linux, MacOS X, Windows P2P control plane based on JXTA joining takes 2 30 s depending on the network Internet2 DCN OSCARS API using Axis and Rampart 16/20

17 Future Work CoUniverse ongoing research into scheduling constraint based, graph based, genetic algorithms, user-friendly workflow support improving code quality direct support for more applications: embedding various types of streaming applications (microscopes, media streaming applications) UltraGrid support for audio support for other compressed formats (JPEG2000) support for higher resolution (2K, 4K) 17/20

18 Future Work DCN security needs to be scalable and developer-friendly improvements in allocation speed of the DCN circuits important for interactive applications ( I want the circuit NOW! ) multi-point issues renewing/modification of reservations notification API instead of status polling 18/20

19 Acknowledgments This project has been supported by a research intent Optical Network of National Research and Its New Applications (MŠM ) and Parallel and Distributed Systems (MŠM ). 19/20

20 Thank you for your attention! Q?/A! 20/20