Optical Switching and Routing for the Data Center. Madeleine Glick Intel Labs Pittsburgh
|
|
|
- Spencer Tyler Daniel
- 6 years ago
- Views:
Transcription
1 Optical Switching and Routing for the Data Center Madeleine Glick Intel Labs Pittsburgh
2 Optical Switching and Routing for the Data Center? Data center challenge Motivation and background Applications Data Center Prior art and alternative approaches Our approach Hybrid Network Conclusion 2
3 The Challenge Current data center networks cannot affordably satisfy the bandwidth requirements of upcoming applications Bandwidth intensive applications Low cost commodity hardware Goal Can optical switching improve data center performance in a cost and energy efficient manner? Design cost and power efficient, high bandwidth optical network Meet data center criteria Understand application requirements Adapts to application communication patterns 3
4 Optical Switching and Routing for the Data Center? Data center challenge Motivation and background Applications Address new, upcoming applications What are the requirements? What are the problems? Data Center Prior art and alternative approaches Our approach Hybrid network Conclusion 4
5 Looking Forward: New data-rich applications will cause bottlenecks Internet traffic video included Traffic volume Time Ref Effect of video on service traffic (Donald Lee, Google, OFC 08) This result for the internet but increased traffic likely to affect data center traffic and design 5
6 Example: Real time event recognition in video Goal: enable interactive applications driven by real-time processing of high-rate streaming data Robot actuation and control based on real-time object recognition Virtual reality studio process hundreds of video feeds Natural gesture interfaces pointing without props (Pinpoint) Face recognition fast, accurate Parallelizes compute tasks across a cluster of compute nodes Large amounts of data on the data center network 6
7 Video Event Detection Goal: reliably identify actions in the midst of visual clutter Algorithm: event detection [Ke, Sukthankar, Hebert, ICCV 07] Robustly matches target event in large spatio-temporal volumes Key problem: need to match a 3D volume and scan across space and time Y T X Can we run this fast enough to use in interactive settings? 7
8 Object Recognition Requires significant hardware to recognize even 5 objects Uses 112 cores, each 2.8GHz to enable 25frames/s on 640x480 video stream Usable object recognition will require hundreds if not thousands of cores 8
9 Real Time Object Recognition Requires significant hardware to recognize even 5 objects Split, sometimes replicate image Uses 112 cores, each 2.8GHz to enable 25fps on 640x480 video stream Usable object recognition will require hundreds if not thousands of cores 9
10 Real Time Object Recognition Requires significant hardware to recognize even 5 objects Split, sometimes replicate image Uses 112 cores, each 2.8GHz to enable 25fps on 640x480 video stream Usable object recognition will require hundreds if not thousands of cores Extract lighting, pose-invariant representation 10
11 Real Time Object Recognition Requires significant hardware to recognize even 5 objects Split, sometimes replicate image Uses 112 cores, each 2.8GHz to enable 25fps on 640x480 video stream Usable object recognition will require hundreds if not thousands of cores Extract lighting, pose-invariant representation Match against database of reference images 11
12 Real Time Object Recognition Requires significant hardware to recognize even 5 objects Split, sometimes replicate image Uses 112 cores, each 2.8GHz to enable 25fps on 640x480 video stream Usable object recognition will require hundreds if not thousands of cores Extract lighting, pose-invariant representation Match against database of reference images 12 Different stages have different bandwidth and processing requirements
13 Face recognilon applicalon graph Connector type Local Remote Synchronous Asynchronous Connector rate O(bytes)/item 1 MB 100 KB 10 KB 1 KB 100 B 10 B ~2MB / frame (960x720x3) Video Source Frame Copy Frame Tiler Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Detect Face Merger Graph Spli5er Face Display Face Copy Token Generator Add User Similarity Similarity Branislav Kveton (ILSC), Lily Mummert (ILP), Casey Helfrich (ILP), Michal Valko (Uni Pittsburgh), Matthai Philipose (ILS), Ken Lafond (ILSA) Recognize 13
14 PinPoint applicalon graph Connector type Local Remote Cam1 Bayer Scale ~1MB / frame (640x480x3) Scale Cam2 Bayer Synchronous Asynchronous Frame copy Frame copy Connector rate O(bytes)/item Gray scale Image Ller UDP Merge vectors Gray scale Image Ller Token Copy 1 MB 100 KB 10 KB 1 KB 100 B 10 B KLT KLT KLT KLT KLT KLT KLT merge KLT copy Hand mergea CollectA Hand mergeb CollectB KLT KLT KLT KLT KLT KLT KLT merge KLT copy Token And PointA PointB PointB PointA TokenGen Displays TokenGen KeySplit PinPoint (CMU, ILP): Pyry Matikainen, Martial Hebert, Rahul Sukthankar, Babu Pillai, Lily Mummert, Casey Helfrich 14
15 Applications requirements Bandwidth requirements Vary in time Vary between processors depending on the task Many applications have latency requirements on human time scale 15
16 Applications requirements Bandwidth requirements Vary in time Vary between processors depending on the task Many applications have latency requirements on human time scale Other applications Scientific computing VM migration 16
17 Optical Switching and Routing for the Data Center? Data center challenge Motivation and background Applications Data Center Bandwidth bottlenecks Scaling Prior art and alternative approaches Our approach Conclusion 17
18 A Typical Data Center Tree architecture Core Switch Access Router Internet End of Row Switch Top of Rack Switch 18
19 A typical data center Ethernet switch Internet switch switch switch Top of rack switch blade blade blade blade blade blade blade blade blade blade blade blade Rack of blades or servers with compute or storage nodes 19
20 Pittsburgh Big Data Cluster 45 Mb/s T3 to Internet Switch 48 Gb/s Switch 48 Gb/s 1 Gb/s (x4) Switch 48 Gb/s 1 Gb/s (x4) 1 Gb/s (x4) Switch 48 Gb/s 1 Gb/s (x4) Switch 48 Gb/s 1 Gb/s (x5 p2p) 1 Gb/s (x4 p2p) 1 Gb/s (x4 p2p) 1 Gb/s (x15 p2p) 1 Gb/s (x15 p2p) Rack of 40 blade compute/ storage nodes Rack of 40 blade compute/ storage nodes Rack of 15 1u compute/ storage nodes Rack of 15 2u compute/ storage nodes Rack of 5 3u storage nodes 20
21 A Typical Data Center Core Switch End of Row Switch Top of Rack Switch Access Router Internet Enormous effort is spent trying to optimize data centers Ideal goal: equal high bandwidth between all pairs of nodes Current data center designs do not meet goal Constrained by: Cost, Physical layout, Power 21
22 Warehouse scale systems Typical elements in warehouse-scale systems From The Datacenter as a Computer An Introduction to the Design of Warehouse-Scale Machines, Luiz André Barroso and Urs Hölzle 22
23 Container based systems Google s container based system has ~45k servers
24 A Typical Data Center Core Switch End of Row Switch Top of Rack Switch Access Router Internet The data center architecture must change to accommodate new high bandwidth applications and changes in scale. 24
25 Optical Switching and Routing for the Data Center? Data center challenge Motivation and background Applications Data Center Prior art and alternative approaches Our approach Hybrid Network Conclusion 25
26 The data center architecture must change to accommodate new applications and changes in scale Alternative choices All electrical upgrade Higher data rate links Add more links (scale out rather than scale up) Fat Tree BCube Fat Tree BCube Al Fares et al., Sigcomm 2008 C. Guo et al., Sigcomm
27 The data center architecture must change to accommodate new applications and changes in scale Alternative choices All electrical upgrade Higher data rate links More powerful master switch Add more links Fat Tree These solutions are complex in terms of wiring, switching and BCube software implementation N. Farrington et al., Hot Interconnects
28 The data center architecture must change to accommodate new applications and changes in scale Alternative choices All electrical upgrade Higher data rate links More powerful master switch Add more links Software Placement of data relative to compute nodes 28
29 The data center architecture must change to accommodate new applications and changes in scale Alternative choices All electrical upgrade Higher data rate links More powerful master switch Add more links Software Placement of data relative to compute nodes All optical All optical switch on packet time scales 29
30 The data center architecture must change to accommodate new applications and changes in scale Alternative choices All electrical upgrade Higher data rate links More powerful master switch Add more links Software Placement of data relative to compute nodes All optical All optical switch on packet time scales Hybrid Electrical/Optical Fast, low bandwidth electrical switch (standard low cost electrical switch) in conjunction with Slow (reconfiguration time), high bandwidth optical switch 30
31 Optical Switching and Routing for the Data Center? Data center challenge Motivation and background Applications Data Center Prior art and alternative approaches Our approach Hybrid Network Focus on Interfaces Conclusion 31
32 Hybrid electrical/optical network Electrical packet-switched network Optical circuit-switched network Hybrid network: Optical circuit-switched paths for data intensive transfer, latency tolerant, long lived flows Electrical packet-switched paths for latency intolerant, small packets 32
33 Hybrid electrical/optical network High-Bandwidth links connect switches A B C D E F Electrical packet-switched network Optical circuit-switched network A few well placed high bandwidth links would improve performance few to reduce cost, scaling requirements reconfigurable to accommodate changes in communication patterns over time 33
34 Hybrid electrical/optical network Electrical packet-switched network 34 A B C D E F Hybrid switched network Does it improve performance? Reduce execution time, improve throughput Optical circuit-switched network How best to use the optics? Identify communication patterns Design initial application interface Build Ethernet based prototype for feasibility in cluster G Wang et al., Your Data Center Is a Router: The Case for Reconfigurable Optical Circuit Switched Paths, Hot Topics in Networks (HotNets-VIII)
35 Hybrid electrical/optical network Electrical packet-switched network A B C D E F Optical circuit-switched network Optical links High bandwidth low-cost low-power Low cost photonic elements CMOS based drivers DSP for increased bandwidth using advanced modulation formats Y. Benlachtar et al. ECOC 09 Postdeadline session 35
36 Hybrid electrical/optical network Electrical packet-switched network A B C D E F Optical circuit-switched network Optical switch Eliminates optical/electrical/optical conversions All optical data path Start with MEMs? Semiconductor based switches, Challenges; scaling, integration, high volume manufacture 36 All optical hybrid packet and circuit switched network platform H. Wang et al. OFC 2010
37 Conclusion Data Centers will need higher bandwidth capabilities Opportunity for optics Optically switched networks are a logical next step Data rate requirements increasing Photonic components becoming less costly but need a better understanding of applications to design viable solutions We are investigating Hybrid electrical/ optical (packet/circuit) solution a small number of high bandwidth links match data center traffic patterns and upcoming application requirements Network implications and solutions Physical layer requirements and implementations 37
38 Collaborators Intel Labs Pittsburgh Madeleine Glick, Michael Kaminsky, Michael Kozuch, Lily Mummert, Dina Papagiannaki, Michael Ryan Carnegie Mellon University- ECE and CS Markus Püschel, James Hoe, Robert Koutsoyannis, Peter Milder, Christian Berger, (Deepak Rangaraj, Anthony Cartolano) Dave Andersen Columbia University- Lightwave Research Laboratory Keren Bergman, Howard Wang, Ajay Garg, Caroline Lai University College London Optical Networks Group Robert, Killey, Yannis Benlachtar, Rashid Bouziane, Phil Watts, Ramanan Thiruneelakandan Rice University - CS Guohui Wang, T. S. Euguen Ng 38
39 Thank you 39
c-through: Part-time Optics in Data Centers
Data Center Network Architecture c-through: Part-time Optics in Data Centers Guohui Wang 1, T. S. Eugene Ng 1, David G. Andersen 2, Michael Kaminsky 3, Konstantina Papagiannaki 3, Michael Kozuch 3, Michael
Running Interactive Perception Applications on Open Cirrus
Running Interactive Perception Applications on Open Cirrus Qian Zhu Nezih Yigitbasi Padmanabhan Pillai Accenture Technology Labs qian.zhu@accenture.com Delft University of Technology M.N.Yigitbasi@tudelft.nl
Data Center Fundamentals: The Datacenter as a Computer
Data Center Fundamentals: The Datacenter as a Computer George Porter CSE 124 Feb 10, 2017 Includes material from (1) Barroso, Clidaras, and Hölzle, as well as (2) Evrard (Michigan), used with permission
Network Design Considerations for Grid Computing
Network Design Considerations for Grid Computing Engineering Systems How Bandwidth, Latency, and Packet Size Impact Grid Job Performance by Erik Burrows, Engineering Systems Analyst, Principal, Broadcom
Optical Interconnection Networks in Data Centers: Recent Trends and Future Challenges
Optical Interconnection Networks in Data Centers: Recent Trends and Future Challenges Speaker: Lin Wang Research Advisor: Biswanath Mukherjee Kachris C, Kanonakis K, Tomkos I. Optical interconnection networks
CSE 124: THE DATACENTER AS A COMPUTER. George Porter November 20 and 22, 2017
CSE 124: THE DATACENTER AS A COMPUTER George Porter November 20 and 22, 2017 ATTRIBUTION These slides are released under an Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) Creative
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff and Shun Tak Leung Google* Shivesh Kumar Sharma fl4164@wayne.edu Fall 2015 004395771 Overview Google file system is a scalable distributed file system
High Performance Datacenter Networks
M & C Morgan & Claypool Publishers High Performance Datacenter Networks Architectures, Algorithms, and Opportunity Dennis Abts John Kim SYNTHESIS LECTURES ON COMPUTER ARCHITECTURE Mark D. Hill, Series
Prop-Free Pointing Detection in Dynamic Cluttered Environments
Prop-Free Pointing Detection in Dynamic Cluttered Environments Pyry Matikainen, Padmanabhan Pillai, Lily Mummert, Rahul Sukthankar, Martial Hebert Intel Labs Pittsburgh, Robotics Institute, Carnegie Mellon
Hybrid On-chip Data Networks. Gilbert Hendry Keren Bergman. Lightwave Research Lab. Columbia University
Hybrid On-chip Data Networks Gilbert Hendry Keren Bergman Lightwave Research Lab Columbia University Chip-Scale Interconnection Networks Chip multi-processors create need for high performance interconnects
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung SOSP 2003 presented by Kun Suo Outline GFS Background, Concepts and Key words Example of GFS Operations Some optimizations in
The Impact of Optics on HPC System Interconnects
The Impact of Optics on HPC System Interconnects Mike Parker and Steve Scott Hot Interconnects 2009 Manhattan, NYC Will cost-effective optics fundamentally change the landscape of networking? Yes. Changes
Yuval Carmel Tel-Aviv University "Advanced Topics in Storage Systems" - Spring 2013
Yuval Carmel Tel-Aviv University "Advanced Topics in About & Keywords Motivation & Purpose Assumptions Architecture overview & Comparison Measurements How does it fit in? The Future 2 About & Keywords
15-744: Computer Networking. Data Center Networking II
15-744: Computer Networking Data Center Networking II Overview Data Center Topology Scheduling Data Center Packet Scheduling 2 Current solutions for increasing data center network bandwidth FatTree BCube
A Network-aware Scheduler in Data-parallel Clusters for High Performance
A Network-aware Scheduler in Data-parallel Clusters for High Performance Zhuozhao Li, Haiying Shen and Ankur Sarker Department of Computer Science University of Virginia May, 2018 1/61 Data-parallel clusters
Cutting the Cord: A Robust Wireless Facilities Network for Data Centers
Cutting the Cord: A Robust Wireless Facilities Network for Data Centers Yibo Zhu, Xia Zhou, Zengbin Zhang, Lin Zhou, Amin Vahdat, Ben Y. Zhao and Haitao Zheng U.C. Santa Barbara, Dartmouth College, U.C.
Enabling High Performance Data Centre Solutions and Cloud Services Through Novel Optical DC Architectures. Dimitra Simeonidou
Enabling High Performance Data Centre Solutions and Cloud Services Through Novel Optical DC Architectures Dimitra Simeonidou Challenges and Drivers for DC Evolution Data centres are growing in size and
A Scalable, Commodity Data Center Network Architecture
A Scalable, Commodity Data Center Network Architecture B Y M O H A M M A D A L - F A R E S A L E X A N D E R L O U K I S S A S A M I N V A H D A T P R E S E N T E D B Y N A N X I C H E N M A Y. 5, 2 0
Achieving Lightweight Multicast in Asynchronous Networks-on-Chip Using Local Speculation
Achieving Lightweight Multicast in Asynchronous Networks-on-Chip Using Local Speculation Kshitij Bhardwaj Dept. of Computer Science Columbia University Steven M. Nowick 2016 ACM/IEEE Design Automation
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung December 2003 ACM symposium on Operating systems principles Publisher: ACM Nov. 26, 2008 OUTLINE INTRODUCTION DESIGN OVERVIEW
Cutting the Cord: A Robust Wireless Facilities Network for Data Centers
Cutting the Cord: A Robust Wireless Facilities Network for Data Centers Yibo Zhu, Xia Zhou, Zengbin Zhang, Lin Zhou, Amin Vahdat, Ben Y. Zhao and Haitao Zheng U.C. Santa Barbara, Dartmouth College, U.C.
The Google File System (GFS)
1 The Google File System (GFS) CS60002: Distributed Systems Antonio Bruto da Costa Ph.D. Student, Formal Methods Lab, Dept. of Computer Sc. & Engg., Indian Institute of Technology Kharagpur 2 Design constraints
MAPREDUCE FOR BIG DATA PROCESSING BASED ON NETWORK TRAFFIC PERFORMANCE Rajeshwari Adrakatti
International Journal of Computer Engineering and Applications, ICCSTAR-2016, Special Issue, May.16 MAPREDUCE FOR BIG DATA PROCESSING BASED ON NETWORK TRAFFIC PERFORMANCE Rajeshwari Adrakatti 1 Department
Distributed Systems. 05r. Case study: Google Cluster Architecture. Paul Krzyzanowski. Rutgers University. Fall 2016
Distributed Systems 05r. Case study: Google Cluster Architecture Paul Krzyzanowski Rutgers University Fall 2016 1 A note about relevancy This describes the Google search cluster architecture in the mid
Ultra-Low Latency, Bit-Parallel Message Exchange in Optical Packet Switched Interconnection Networks
Ultra-Low Latency, Bit-Parallel Message Exchange in Optical Packet Switched Interconnection Networks O. Liboiron-Ladouceur 1, C. Gray 2, D. Keezer 2 and K. Bergman 1 1 Department of Electrical Engineering,
Crossing the Chasm: Sneaking a parallel file system into Hadoop
Crossing the Chasm: Sneaking a parallel file system into Hadoop Wittawat Tantisiriroj Swapnil Patil, Garth Gibson PARALLEL DATA LABORATORY Carnegie Mellon University In this work Compare and contrast large
XCo: Explicit Coordination for Preventing Congestion in Data Center Ethernet
XCo: Explicit Coordination for Preventing Congestion in Data Center Ethernet Vijay Shankar Rajanna, Smit Shah, Anand Jahagirdar and Kartik Gopalan Computer Science, State University of New York at Binghamton
FAWN as a Service. 1 Introduction. Jintian Liang CS244B December 13, 2017
Liang 1 Jintian Liang CS244B December 13, 2017 1 Introduction FAWN as a Service FAWN, an acronym for Fast Array of Wimpy Nodes, is a distributed cluster of inexpensive nodes designed to give users a view
Georgia Institute of Technology ECE6102 4/20/2009 David Colvin, Jimmy Vuong
Georgia Institute of Technology ECE6102 4/20/2009 David Colvin, Jimmy Vuong Relatively recent; still applicable today GFS: Google s storage platform for the generation and processing of data used by services
Solace JMS Broker Delivers Highest Throughput for Persistent and Non-Persistent Delivery
Solace JMS Broker Delivers Highest Throughput for Persistent and Non-Persistent Delivery Java Message Service (JMS) is a standardized messaging interface that has become a pervasive part of the IT landscape
Google File System. Arun Sundaram Operating Systems
Arun Sundaram Operating Systems 1 Assumptions GFS built with commodity hardware GFS stores a modest number of large files A few million files, each typically 100MB or larger (Multi-GB files are common)
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google SOSP 03, October 19 22, 2003, New York, USA Hyeon-Gyu Lee, and Yeong-Jae Woo Memory & Storage Architecture Lab. School
The Software Defined Hybrid Packet Optical Datacenter Network BIG DATA AT LIGHT SPEED TM CALIENT Technologies
The Software Defined Hybrid Packet Optical Datacenter Network BIG DATA AT LIGHT SPEED TM 2012-13 CALIENT Technologies www.calient.net 1 INTRODUCTION In datacenter networks, video, mobile data, and big
Optical switching for scalable and programmable data center networks
Optical switching for scalable and programmable data center networks Paraskevas Bakopoulos National Technical University of Athens Photonics Communications Research Laboratory @ pbakop@mail.ntua.gr Please
Data Center Virtualization: VirtualWire
Data Center Virtualization: VirtualWire Hakim Weatherspoon Assistant Professor, Dept of Computer Science CS 5413: High Performance Systems and Networking November 21, 2014 Slides from USENIX Workshop on
Lecture 10.1 A real SDN implementation: the Google B4 case. Antonio Cianfrani DIET Department Networking Group netlab.uniroma1.it
Lecture 10.1 A real SDN implementation: the Google B4 case Antonio Cianfrani DIET Department Networking Group netlab.uniroma1.it WAN WAN = Wide Area Network WAN features: Very expensive (specialized high-end
Impact of TCP Window Size on a File Transfer
Impact of TCP Window Size on a File Transfer Introduction This example shows how ACE diagnoses and visualizes application and network problems; it is not a step-by-step tutorial. If you have experience
Cache Management for TelcoCDNs. Daphné Tuncer Department of Electronic & Electrical Engineering University College London (UK)
Cache Management for TelcoCDNs Daphné Tuncer Department of Electronic & Electrical Engineering University College London (UK) d.tuncer@ee.ucl.ac.uk 06/01/2017 Agenda 1. Internet traffic: trends and evolution
Building petabit/s data center network with submicroseconds latency by using fast optical switches Miao, W.; Yan, F.; Dorren, H.J.S.; Calabretta, N.
Building petabit/s data center network with submicroseconds latency by using fast optical switches Miao, W.; Yan, F.; Dorren, H.J.S.; Calabretta, N. Published in: Proceedings of 20th Annual Symposium of
CS550. TA: TBA Office: xxx Office hours: TBA. Blackboard:
CS550 Advanced Operating Systems (Distributed Operating Systems) Instructor: Xian-He Sun Email: sun@iit.edu, Phone: (312) 567-5260 Office hours: 1:30pm-2:30pm Tuesday, Thursday at SB229C, or by appointment
Authors : Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung Presentation by: Vijay Kumar Chalasani
The Authors : Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung Presentation by: Vijay Kumar Chalasani CS5204 Operating Systems 1 Introduction GFS is a scalable distributed file system for large data intensive
IP Video Network Gateway Solutions
IP Video Network Gateway Solutions INTRODUCTION The broadcast systems of today exist in two separate and largely disconnected worlds: a network-based world where audio/video information is stored and passed
FAWN. A Fast Array of Wimpy Nodes. David Andersen, Jason Franklin, Michael Kaminsky*, Amar Phanishayee, Lawrence Tan, Vijay Vasudevan
FAWN A Fast Array of Wimpy Nodes David Andersen, Jason Franklin, Michael Kaminsky*, Amar Phanishayee, Lawrence Tan, Vijay Vasudevan Carnegie Mellon University *Intel Labs Pittsburgh Energy in computing
Lecture 26: Interconnects. James C. Hoe Department of ECE Carnegie Mellon University
18 447 Lecture 26: Interconnects James C. Hoe Department of ECE Carnegie Mellon University 18 447 S18 L26 S1, James C. Hoe, CMU/ECE/CALCM, 2018 Housekeeping Your goal today get an overview of parallel
Crossing the Chasm: Sneaking a parallel file system into Hadoop
Crossing the Chasm: Sneaking a parallel file system into Hadoop Wittawat Tantisiriroj Swapnil Patil, Garth Gibson PARALLEL DATA LABORATORY Carnegie Mellon University In this work Compare and contrast large
CSE 124: Networked Services Fall 2009 Lecture-19
CSE 124: Networked Services Fall 2009 Lecture-19 Instructor: B. S. Manoj, Ph.D http://cseweb.ucsd.edu/classes/fa09/cse124 Some of these slides are adapted from various sources/individuals including but
SAP High-Performance Analytic Appliance on the Cisco Unified Computing System
Solution Overview SAP High-Performance Analytic Appliance on the Cisco Unified Computing System What You Will Learn The SAP High-Performance Analytic Appliance (HANA) is a new non-intrusive hardware and
ClearCube White Paper Best Practices Pairing Virtualization and Collaboration
ClearCube White Paper Best Practices Pairing Virtualization and Collaboration Increasing VDI Audio/Video Performance for the Defense Connect Online User Community Introduction In Quarter 4, 2011, significant
Distributed Filesystem
Distributed Filesystem 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributing Code! Don t move data to workers move workers to the data! - Store data on the local disks of nodes in the
Distributed Data Infrastructures, Fall 2017, Chapter 2. Jussi Kangasharju
Distributed Data Infrastructures, Fall 2017, Chapter 2 Jussi Kangasharju Chapter Outline Warehouse-scale computing overview Workloads and software infrastructure Failures and repairs Note: Term Warehouse-scale
ECE/CS 757: Advanced Computer Architecture II Interconnects
ECE/CS 757: Advanced Computer Architecture II Interconnects Instructor:Mikko H Lipasti Spring 2017 University of Wisconsin-Madison Lecture notes created by Natalie Enright Jerger Lecture Outline Introduction
A Single Chip Shared Memory Switch with Twelve 10Gb Ethernet Ports
A Single Chip Shared Memory Switch with Twelve 10Gb Ethernet Ports Takeshi Shimizu, Yukihiro Nakagawa, Sridhar Pathi, Yasushi Umezawa, Takashi Miyoshi, Yoichi Koyanagi, Takeshi Horie, Akira Hattori Hot
Challenges for Future Interconnection Networks Hot Interconnects Panel August 24, Dennis Abts Sr. Principal Engineer
Challenges for Future Interconnection Networks Hot Interconnects Panel August 24, 2006 Sr. Principal Engineer Panel Questions How do we build scalable networks that balance power, reliability and performance
Data Center Network Topologies II
Data Center Network Topologies II Hakim Weatherspoon Associate Professor, Dept of Computer cience C 5413: High Performance ystems and Networking April 10, 2017 March 31, 2017 Agenda for semester Project
CS 61C: Great Ideas in Computer Architecture (Machine Structures) Warehouse-Scale Computing
CS 61C: Great Ideas in Computer Architecture (Machine Structures) Warehouse-Scale Computing Instructors: Nicholas Weaver & Vladimir Stojanovic http://inst.eecs.berkeley.edu/~cs61c/ Coherency Tracked by
CSE 124: Networked Services Lecture-16
Fall 2010 CSE 124: Networked Services Lecture-16 Instructor: B. S. Manoj, Ph.D http://cseweb.ucsd.edu/classes/fa10/cse124 11/23/2010 CSE 124 Networked Services Fall 2010 1 Updates PlanetLab experiments
Alternative Switching Technologies: Wireless Datacenters Hakim Weatherspoon
Alternative Switching Technologies: Wireless Datacenters Hakim Weatherspoon Assistant Professor, Dept of Computer Science CS 5413: High Performance Systems and Networking October 22, 2014 Slides from the
Meet in the Middle: Leveraging Optical Interconnection Opportunities in Chip Multi Processors
Meet in the Middle: Leveraging Optical Interconnection Opportunities in Chip Multi Processors Sandro Bartolini* Department of Information Engineering, University of Siena, Italy bartolini@dii.unisi.it
Cloud Computing and Hadoop Distributed File System. UCSB CS170, Spring 2018
Cloud Computing and Hadoop Distributed File System UCSB CS70, Spring 08 Cluster Computing Motivations Large-scale data processing on clusters Scan 000 TB on node @ 00 MB/s = days Scan on 000-node cluster
Knowledge-Defined Network Orchestration in a Hybrid Optical/Electrical Datacenter Network
Knowledge-Defined Network Orchestration in a Hybrid Optical/Electrical Datacenter Network Wei Lu (Postdoctoral Researcher) On behalf of Prof. Zuqing Zhu University of Science and Technology of China, Hefei,
LazyBase: Trading freshness and performance in a scalable database
LazyBase: Trading freshness and performance in a scalable database (EuroSys 2012) Jim Cipar, Greg Ganger, *Kimberly Keeton, *Craig A. N. Soules, *Brad Morrey, *Alistair Veitch PARALLEL DATA LABORATORY
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google* 정학수, 최주영 1 Outline Introduction Design Overview System Interactions Master Operation Fault Tolerance and Diagnosis Conclusions
SPAIN: High BW Data-Center Ethernet with Unmodified Switches. Praveen Yalagandula, HP Labs. Jayaram Mudigonda, HP Labs
SPAIN: High BW Data-Center Ethernet with Unmodified Switches Jayaram Mudigonda, HP Labs Mohammad Al-Fares, UCSD Praveen Yalagandula, HP Labs Jeff Mogul, HP Labs 1 Copyright Copyright 2010 Hewlett-Packard
Two-Aggregator Topology Optimization without Splitting in Data Center Networks
Two-Aggregator Topology Optimization without Splitting in Data Center Networks Soham Das and Sartaj Sahni Department of Computer and Information Science and Engineering University of Florida Gainesville,
Memory-Based Cloud Architectures
Memory-Based Cloud Architectures ( Or: Technical Challenges for OnDemand Business Software) Jan Schaffner Enterprise Platform and Integration Concepts Group Example: Enterprise Benchmarking -) *%'+,#$)
Staggeringly Large File Systems. Presented by Haoyan Geng
Staggeringly Large File Systems Presented by Haoyan Geng Large-scale File Systems How Large? Google s file system in 2009 (Jeff Dean, LADIS 09) - 200+ clusters - Thousands of machines per cluster - Pools
There Is More Consensus in Egalitarian Parliaments
There Is More Consensus in Egalitarian Parliaments Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Fault tolerance Redundancy State Machine Replication 3 State Machine
Microprocessors LCD Parallel Port USB Port
Microprocessors LCD Parallel Port USB Port H. Abdoli Bu-Ali Sina University 1 New LCDs Interfacing Lower prices Display numbers, characters, graphics Integrated refreshing controller Ease of programming
Distributed File Systems II
Distributed File Systems II To do q Very-large scale: Google FS, Hadoop FS, BigTable q Next time: Naming things GFS A radically new environment NFS, etc. Independence Small Scale Variety of workloads Cooperation
Google is Really Different.
COMP 790-088 -- Distributed File Systems Google File System 7 Google is Really Different. Huge Datacenters in 5+ Worldwide Locations Datacenters house multiple server clusters Coming soon to Lenior, NC
Future Routing Schemes in Petascale clusters
Future Routing Schemes in Petascale clusters Gilad Shainer, Mellanox, USA Ola Torudbakken, Sun Microsystems, Norway Richard Graham, Oak Ridge National Laboratory, USA Birds of a Feather Presentation Abstract
Utilizing Datacenter Networks: Centralized or Distributed Solutions?
Utilizing Datacenter Networks: Centralized or Distributed Solutions? Costin Raiciu Department of Computer Science University Politehnica of Bucharest We ve gotten used to great applications Enabling Such
Different network topologies
Network Topology Network topology is the arrangement of the various elements of a communication network. It is the topological structure of a network and may be depicted physically or logically. Physical
Warehouse-Scale Computing
ecture 31 Computer Science 61C Spring 2017 April 7th, 2017 Warehouse-Scale Computing 1 New-School Machine Structures (It s a bit more complicated!) Software Hardware Parallel Requests Assigned to computer
Computer and Information Sciences College / Computer Science Department CS 207 D. Computer Architecture. Lecture 9: Multiprocessors
Computer and Information Sciences College / Computer Science Department CS 207 D Computer Architecture Lecture 9: Multiprocessors Challenges of Parallel Processing First challenge is % of program inherently
Technical Document. What You Need to Know About Ethernet Audio
Technical Document What You Need to Know About Ethernet Audio Overview Designing and implementing an IP-Audio Network can be a daunting task. The purpose of this paper is to help make some of these decisions
Beyond fat-trees without antennae, mirrors, and disco-balls. Simon Kassing, Asaf Valadarsky, Gal Shahaf, Michael Schapira, Ankit Singla
Beyond fat-trees without antennae, mirrors, and disco-balls Simon Kassing, Asaf Valadarsky, Gal Shahaf, Michael Schapira, Ankit Singla Skewed traffic within data centers 2 [Google] Skewed traffic within
Module 6: INPUT - OUTPUT (I/O)
Module 6: INPUT - OUTPUT (I/O) Introduction Computers communicate with the outside world via I/O devices Input devices supply computers with data to operate on E.g: Keyboard, Mouse, Voice recognition hardware,
Odessa: Enabling Interactive Perception Applications on Mobile Devices
Odessa: Enabling Interactive Perception Applications on Mobile Devices Moo-Ryong Ra, Anmol Sheth, Lily Mummert, Padmanabhan Pillai, David Wetherall and Ramesh Govindan University of Southern California
ASPERA HIGH-SPEED TRANSFER. Moving the world s data at maximum speed
ASPERA HIGH-SPEED TRANSFER Moving the world s data at maximum speed ASPERA HIGH-SPEED FILE TRANSFER 80 GBIT/S OVER IP USING DPDK Performance, Code, and Architecture Charles Shiflett Developer of next-generation
PacketShader: A GPU-Accelerated Software Router
PacketShader: A GPU-Accelerated Software Router Sangjin Han In collaboration with: Keon Jang, KyoungSoo Park, Sue Moon Advanced Networking Lab, CS, KAIST Networked and Distributed Computing Systems Lab,
Sinbad. Leveraging Endpoint Flexibility in Data-Intensive Clusters. Mosharaf Chowdhury, Srikanth Kandula, Ion Stoica. UC Berkeley
Sinbad Leveraging Endpoint Flexibility in Data-Intensive Clusters Mosharaf Chowdhury, Srikanth Kandula, Ion Stoica UC Berkeley Communication is Crucial for Analytics at Scale Performance Facebook analytics
A Reconfigurable Crossbar Switch with Adaptive Bandwidth Control for Networks-on
A Reconfigurable Crossbar Switch with Adaptive Bandwidth Control for Networks-on on-chip Donghyun Kim, Kangmin Lee, Se-joong Lee and Hoi-Jun Yoo Semiconductor System Laboratory, Dept. of EECS, Korea Advanced
RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store
RAMCube: Exploiting Network Proximity for RAM-Based Key-Value Store Yiming Zhang, Rui Chu @ NUDT Chuanxiong Guo, Guohan Lu, Yongqiang Xiong, Haitao Wu @ MSRA June, 2012 1 Background Disk-based storage
This is a repository copy of PON Data Centre Design with AWGR and Server Based Routing.
This is a repository copy of PON Data Centre Design with AWGR and Server Based Routing. White Rose Research Online URL for this paper: http://eprints.whiterose.ac.uk/116818/ Version: Accepted Version Proceedings
Users Application Virtual Machine Users Application Virtual Machine Users Application Virtual Machine Private Cloud Users Application Virtual Machine On-Premise Service Providers Private Cloud Users Application
Low-Power Reconfigurable Network Architecture for On-Chip Photonic Interconnects
Low-Power Reconfigurable Network Architecture for On-Chip Photonic Interconnects I. Artundo, W. Heirman, C. Debaes, M. Loperena, J. Van Campenhout, H. Thienpont New York, August 27th 2009 Iñigo Artundo,
Intel: Driving the Future of IT Technologies. Kevin C. Kahn Senior Fellow, Intel Labs Intel Corporation
Research @ Intel: Driving the Future of IT Technologies Kevin C. Kahn Senior Fellow, Intel Labs Intel Corporation kp Intel Labs Mission To fuel Intel s growth, we deliver breakthrough technologies that
Google File System. Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google fall DIP Heerak lim, Donghun Koo
Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google 2017 fall DIP Heerak lim, Donghun Koo 1 Agenda Introduction Design overview Systems interactions Master operation Fault tolerance
Net-Centric 2017 Data-center network (DCN) architectures with Reduced Power Consumption
Data-center network (DCN) architectures with Reduced Power Consumption Flow/Application triggered SDN controlled electrical/optical hybrid switching data-center network: HOLST Satoru Okamoto, Keio University
Database Architecture 2 & Storage. Instructor: Matei Zaharia cs245.stanford.edu
Database Architecture 2 & Storage Instructor: Matei Zaharia cs245.stanford.edu Summary from Last Time System R mostly matched the architecture of a modern RDBMS» SQL» Many storage & access methods» Cost-based
End-to-End Adaptive Packet Aggregation for High-Throughput I/O Bus Network Using Ethernet
Hot Interconnects 2014 End-to-End Adaptive Packet Aggregation for High-Throughput I/O Bus Network Using Ethernet Green Platform Research Laboratories, NEC, Japan J. Suzuki, Y. Hayashi, M. Kan, S. Miyakawa,
Themes. The Network 1. Energy in the DC: ~15% network? Energy by Technology
Themes The Network 1 Low Power Computing David Andersen Carnegie Mellon University Last two classes: Saving power by running more slowly and sleeping more. This time: Network intro; saving power by architecting
Lecture 9: MIMD Architectures
Lecture 9: MIMD Architectures Introduction and classification Symmetric multiprocessors NUMA architecture Clusters Zebo Peng, IDA, LiTH 1 Introduction MIMD: a set of general purpose processors is connected
Bigtable. A Distributed Storage System for Structured Data. Presenter: Yunming Zhang Conglong Li. Saturday, September 21, 13
Bigtable A Distributed Storage System for Structured Data Presenter: Yunming Zhang Conglong Li References SOCC 2010 Key Note Slides Jeff Dean Google Introduction to Distributed Computing, Winter 2008 University
THE ZADARA CLOUD. An overview of the Zadara Storage Cloud and VPSA Storage Array technology WHITE PAPER
WHITE PAPER THE ZADARA CLOUD An overview of the Zadara Storage Cloud and VPSA Storage Array technology Zadara 6 Venture, Suite 140, Irvine, CA 92618, USA www.zadarastorage.com EXECUTIVE SUMMARY The IT
EEE for 40G/100G NGOPTX
EEE for 40G/100G NGOPTX Open Issues and Objective Proposal IEEE 40G and 100G Next Generation Optics Study Group Michael J. Bennett Lawrence Berkeley National Laboratory Wael William Diab Broadcom Corporation
Why Data Center Requires Both, OPS and OCS!
Why Data Center Requires Both, OPS and OCS! K. Kitayama 1, Y.-C. Huang 1, Y. Yoshida 1, R. Takahashi 2, and A. Hiramatsu 3 1. Osaka University, Japan kitayama@comm.eng.osaka-u.ac.jp 2. NTT Device Technology
Row Buffer Locality Aware Caching Policies for Hybrid Memories. HanBin Yoon Justin Meza Rachata Ausavarungnirun Rachael Harding Onur Mutlu
Row Buffer Locality Aware Caching Policies for Hybrid Memories HanBin Yoon Justin Meza Rachata Ausavarungnirun Rachael Harding Onur Mutlu Executive Summary Different memory technologies have different
Arista 7060X, 7060X2, 7260X and 7260X3 series: Q&A
Arista 7060X, 7060X2, 7260X and 7260X3 series: Q&A Product Overview What are the 7060X, 7060X2, 7260X & 7260X3 series? The Arista 7060X Series, comprising of the 7060X, 7060X2, 7260X and 7260X3, are purpose-built