Challenges in Moving (slowly) Towards Production Mode Jon MacLaren GHPN-RG at OGF20, Manchester, UK 9 May 2007

Transcription

1 asdb Challenges in Moving (slowly) Towards Production Mode Jon MacLaren GHPN-RG at OGF20, Manchester, UK 9 May 2007 NSF seed-funded project

2 EnLIGHTened Introduction Network research, driven by concrete application projects, all of which critically require progress in network technologies and tools that utilize them EnLIGHTened testbed: 10 Gbps optical networks running over NLR and Louisiana Optical Network Initiative (LONI), connected via four all-photonic Calient switches, all using GMPLS control plane technologies Global alliance of partners Will develop, test, and disseminate advanced software and underlying technologies to: Provide generic applications with the ability to be aware of their network, Grid environment and capabilities, and to make dynamic, adaptive and optimized use (monitor & abstract, request & control) of networks connecting various high end resources Provide vertical integration from the application to the optical control plane, including extending GMPLS Will examine how to distribute the network intelligence among the network control plane, management plane, and the Grid middleware

3 EnLIGHTened testbed connectivity diagram, with partners To Asia To Canada To Europe SEA POR BOI SVL CAVE wave OGD DEN KAN Cisco/UltraLight wave CHI CLE PIT EnLIGHTened wave (Cisco/NLR) WDC San Diego TUL LONI wave DAL Members: - MCNC GCNS - LSU CCT - NCSU - RENCI International Partners Phosphorus - EC G-lambda - Japan -GLIF Official Partners: HOU - AT&T Research - SURA NSF Project Partners - NRL - OptIPuter - Cisco Systems - UltraLight - Calient Networks - DRAGON - IBM - Cheetah

4 Applications Edge Routers Workflow Engines Application Abstraction Layer (API) Translate app request to policy Resource Allocation Resource Manager Co-Scheduler Feedback Loop For SLA Monitoring Resource Monitoring Abstraction Discovery Performance Policy

5 EnLIGHTened to extend to PHOSPHORUS Testbed

6 HARC: Highly Available Resource Co-allocator Extensible, open-sourced co-allocation system Can already reserve: Time on supercomputers (advance reservation), and Dedicated paths on GMPLS-based networks with simple topologies Uses Paxos Commit to atomically reserve multiple resources, while providing a highly-available service Used to coordinate bookings across EnLIGHTened and G-lambda testbeds in largest demonstration of its kind to date (more later) Used for setting up the network for Thomas Sterling s HPC Class which goes out live in HD (more later)

7 Network Components

8 NRM View of Network

9 Scheduling a link Scheduling is only one part of the puzzle Input is a request to make a connection from one endpoint to another, e.g. RA1 to BT2 The NRM decides the path that this will use Currently it does this by looking up the path in a table Sends the Explicit Route Object (ERO) as part of the TL1 command So the middleware is deciding the path, not GMPLS XML snippet: <Work> <Path> <From>RA1</From><To>BT2</To> <BandwidthMbs>10240</BandwidthMbs> </Path> </Work>

10 Resource Map FUK KAN X2N OSA X2 2G X2S NR3 KMF Japan North Japan South TKB X1N 4G X1 5G 5G X1S AKB X1U CH1 (SL) (UO2) (UO1) RA1 (MCNC) 0.11a.6 US 0.11a.7 (UR1) (UO3) (UO4) Back up 10.16a a.2 LA1 (Caltech) (UR3) LA Foundry VC1 (NCSU) (UR2) BT1 (LSU) Pelican BT2 (LSU) Santaka BT3 (LSU) Viz Machine Client Credit: Tomohiro Kudoh KHN

11 EL-GL Middleware Interoperability Japan Application EL GL wrapper EL App. Launcher US Application GL Grid Resource Scheduler KDDI NRM GNS-WSI NTT NRM GL EL CRM wrapper JAPAN GL EL GNS-WSI wrapper US HARC Acceptor EL Grid Resource Coordinator EL NRM CRM CRM CRM CRM CRM CRM CRM CRM Credit: Tomohiro Kudoh GL: G-lambda EL: Enlightened Computing CRM: Compute Resource Manager HARC: Highly-Available Resource Co-allocator GNS-WSI: Grid Network Service-Web Services Interface NRM: Network Resource Manager

12 RNDS Display of G-lambda Reservations Credit: Tomohiro Kudoh

13 RNDS Display of EnLIGHTened Reservations Credit: Tomohiro Kudoh

14 G-Lambda and EnLIGHTened GMPLS E-NNI Demonstrations Collaborative effort between NTT, KDDI Research and EnLIGHTened Computing Goal: to investigate potential for interdomain provisioning GLIF2006 and SC06 - demonstrated single-vendor interoperation between Japan North (KDDI Research) and Enlightened Automated simultaneous in-advance reservation of network bandwidth between the US and Japan, and computing resources in the US and Japan World s first inter-domain coordination of resource mangers for in-advance reservation Resource managers have different interfaces and are independently developed December tested three domain multi-vendor provisioning between Japan South (NTT) and Enlightened, with Japan North as transit domain. GL s GNS-WSI between the Grid Resource Scheduler (GRS) and the Network Resource Manager (NRM) EL s Highly-Available Resource Co-allocator (HARC) uses Paxos Commit to reserve resources

15 Application (MPI) JAPAN US Application (Visualization) Request Network bandwidth and Computers Request Network bandwidth and Computers Reservation From xx:xx to yy:yy Reservation From xx:xx to yy:yy KDDI NRM NTT NRM EL NRM CRM CRM CRM CRM CRM CRM CRM CRM

16 HD Class Spring HPC Class Thomas Sterling, LSU, instructor Students at LSU (2 sites), LA Tech, University of Arkansas, Masaryk University Simulcast as HDVC, Access Grid, QT Streaming (NCast), Webex (graphics only), ichat (text only for inbound messages) Broadcast streams 1:N HD video broadcast from LSU (each 1.5 Gbps) N:1 HD video broadcast to LSU N:N audio distribution Uses EnLIGHTened middleware to build and tear-down network twice weekly System developed as collaboration between CCT/LSU and Masaryk Univ. (Brno, Czech Republic)

17 HD Video over IP Transmission System Packetization specified in RFC 4175 ( RTP Payload Format for Uncompressed Video ) encapsulation: payload/rtp/udp/ip augmenting common RTP headers with payload headers (e.g., additional packet numbers because of fast RTP counter wrap-arounds 0.5 s for HD-SDI) Reasonable to use jumbo frames (best > 8500B) decreases packetization size overhead decreases host load due to decreasing number of pps Linux-based implementation Comes from UltraGrid by Perkins & Gharai extended to support full-hd 1080i support for SW display including color space down-sampling and eld de-interlacing (assembly optimized for AMD) other enhancements Used with DVS Centaurus HD capture cards problems with latency, since the card doesn t support DMA and requires buffering at least 4 fields for reliable operation quite expensive there are other cards but not supported in Linux :( End-to-end (camera-to-display) latency: 175±5 ms

18 HDTV Video Distribution Based on UDP packet reflectors designed as user-empowered solution relatively scalable with respect to number of users Simple design for HDTV video distribution read from network socket, write to other network sockets (loop) optimizations for 1.5 Gbps streams - reduced per-packet overhead (system calls/packet) can multiply the 1.5 Gbps stream for 4 users (limited by the network card performance) latency increase as low as 13 ms

19 HDTV Video Class Advantages Low network latencies Measurements based on ICMP ping averages with 8500 B packets: LSU StarLight: ms StarLight Masaryk University: ms LSU Masaryk University: ms Multi-way interaction Quality far exceeds Access Grid you can see the students facial expressions

20 HDTV Video Distribution

21 Configuration Changes...

22 The EnLIGHTened Team Yufeng Xin Steve Thorpe Bonnie Hurst Joel Dunn Gigi Karmous-Edwards Mark Johnson John Moore Carla Hunt Lina Battestilli Andrew Mabe Ed Seidel Gabrielle Allen Seung Jong Park Jon MacLaren Andrei Hutanu Lonnie Leger Dan Katz Steven Hunter Savera Tanwir Harry Perros Mladen Vouk Olivier Jerphagnon John Bowers Dan Reed Alan Blatecky Chris Heermann Javad Boroumand Russ Gyurek Wayne Clark Kevin McGrattan Peter Tompsu Rick Schlichting John Strand Matti Hiltunen Yang Xia Xun Su