RAMP: RDMA Migration Platform
|
|
|
- Kimberly Hopkins
- 5 years ago
- Views:
Transcription
1 RAMP: RDMA Migration Platform Babar Naveed Memon, Xiayue Charles Lin, Arshia Mufti, Arthur Scott Wesley, Tim Brecht, Kenneth Salem, Bernard Wong, and Benjamin Cassell firstname.lastname@uwaterloo.ca
2 RDMA and RDMA-based Systems What and why? 2
3 RDMA and RDMA-based Systems What and why? What is the right programming model? 3
4 RDMA and RDMA-based Systems What and why? What is the right programming model? Shared Memory FaRM NAM-DB 4
5 Motivation 5
6 Motivation Support Configuration Operations in Loosely Coupled Distributed Systems 6
7 Motivation Support Configuration Operations in Loosely Coupled Distributed Systems Loosely Coupled Applications 7
8 Cluster of Servers Motivation Server 1 Server 2 Server 3 Partition1 Partition4 Partition7 Support Configuration Operations in Loosely Coupled Distributed Systems Partition2 Partition3 Partition5 Partition6 Partition8 Partition9 Loosely Coupled Applications Client 1 Client 2 Client 3 Client n 8
9 Motivation Support Configuration Operations in Loosely Coupled Distributed Systems Cluster of Servers Server 1 Server 2 Server 3 Partition1 Partition2 Partition3 Partition4 Partition5 Partition6 Partition7 Partition8 Partition9 Loosely Coupled Applications Configuration Operations Scale out, scale in or load balance Client 1 Client 2 Client 3 Client n 9
10 Motivation Support Configuration Operations in Loosely Coupled Distributed Systems Cluster of Servers Server 1 Server 2 Server 3 Partition1 Partition2 Partition3 Partition4 Partition5 Partition6 Partition7 Partition8 Partition9 Loosely Coupled Applications Configuration Operations Scale out, scale in or load balance Is shared memory overkill? Client 1 Client 2 Client 3 Client n 10
11 Desired Properties for RAMP 11
12 Desired Properties for RAMP On-The-Fly Bulk Data Movement Minimize interference with on-going application workload Particularly at the source 12
13 Desired Properties for RAMP On-The-Fly Bulk Data Movement Minimize interference with on-going application workload Particularly at the source Non-Intrusive Stay out of the way, except during configuration options Avoid shared storage approach Local memory access faster than RDMA 13
14 Desired Properties for RAMP On-The-Fly Bulk Data Movement Minimize interference with on-going application workload Particularly at the source Non-Intrusive Stay out of the way, except during configuration options Avoid shared storage approach Local memory access faster than RDMA Application-Managed Application controls when data moves, and where it moves 14
15 RAMP: The Big Picture 15
16 RAMP: The Big Picture Coordinated memory segments Single reader/writer Contains application data 16
17 RAMP: The Big Picture Coordinated memory segments Single reader/writer Contains application data Segments are migratable 17
18 RAMP: The Big Picture Coordinated memory segments Single reader/writer Contains application data Segments are migratable No serialization/deserialization of application data during migration 18
19 RAMP Functionality 19
20 RAMP Functionality Cluster of Servers Server 1 Server 2 Server 3 Partition1 Partition2 Partition3 Partition4 Partition5 Partition6 Partition7 Partition8 Partition9 Client 1 Client 2 Client 3 Client n 20
21 RAMP Memory Segment Allocation Server 1 Server 2 Server 3 21
22 RAMP Memory Segment Allocation Server 1 Server 2 Server 3 RAMP Arena 22
23 RAMP Memory Segment Allocation RAMP Segment Server 1 Server 2 Server 3 RAMP Arena 23
24 RAMP Memory Segment Allocation RAMP Segment Server 1 Server 2 Server 3 RAMP Arena RAMP Segment 24
25 Using RAMP Segments 25
26 Using RAMP Segments Server 1 Application Data Structures 26
27 Using RAMP Segments Server 1 Application Data Structures Optional RAMP-provided in-segment C++ containers (vectors, maps) using custom memory allocators 27
28 Normal Operation Cluster of Servers Server 1 Server 2 Server 3 Partition1 Partition2 Partition3 Partition4 Partition5 Partition6 Partition7 Partition8 Partition9 Client 1 Client 2 Client 3 Client n 28
29 RAMP Segment Migration (Phase 1) Server 1 Server 2 29
30 RAMP Segment Migration (Phase 1) CONNECT Server 1 Server 2 30
31 RAMP Segment Migration (Phase 1) CONNECT Server 1 Server 2 ACCEPT 31
32 RAMP Segment Migration (Phase 1) CONNECT Server 1 Server 2 ACCEPT Registration has a high latency cost (100 s ms) but segment remains available 32
33 RAMP Segment Migration (Phase 2) Server 1 Server 2 33
34 RAMP Segment Migration (Phase 2) Server TRANSFER 1 Server 2 34
35 RAMP Segment Migration (Phase 2) Server TRANSFER 1 Server Transfers segment 2 ownership (not data) Low latency operation (20 microseconds) 35
36 RAMP Segment Migration (Phase 3) Server 1 Server 2 36
37 RAMP Segment Migration (Phase 3) Server 1 Server 2 PULL 37
38 RAMP Segment Migration (Phase 3) Server 1 Server 2 PULL PULL 38
39 RAMP Segment Migration (Phase 3) Server 1 Server 2 Implemented using one-sided RDMA reads Application managed vs RAMP managed PULL PULL 39
40 RAMP Segment Migration (Phase 4) CLOSE Server 1 Server 2 40
41 RAMP Segment Migration (Phase 4) CLOSE Server 1 Server 2 Clean up 41
42 Segment Migration Performance 42
43 Segment Migration Performance STL map with 8B keys and 128B values in 256MB segment 43
44 Segment Migration Performance STL map with 8B keys and 128B values in 256MB segment Single thread at receiver starts using the map immediately after TRANSFER 44
45 Segment Migration Performance STL map with 8B keys and 128B values in 256MB segment Single thread at receiver starts using the map immediately after TRANSFER Latency of map get operations, as a function of time since TRANSFER 45
46 Segment Migration Performance STL map with 8B keys and 128B values in 256MB segment Single thread at receiver starts using the map immediately after TRANSFER Latency of map get operations, as a function of time since TRANSFER 46
47 Segment Migration Performance Paging first access 40 µs Stop-and-copy first access 310 ms 47
48 Conclusion Overview of RAMP, lightweight support for configuration operations in loosely coupled systems 48
49 Conclusion Overview of RAMP, lightweight support for configuration operations in loosely coupled systems Coordinated allocation of segments, fast ownership transfer, application-managed data movement 49
50 Conclusion Overview of RAMP, lightweight support for configuration operations in loosely coupled systems Coordinated allocation of segments, fast ownership transfer, application-managed data movement In the paper: 50
51 Conclusion Overview of RAMP, lightweight support for configuration operations in loosely coupled systems Coordinated allocation of segments, fast ownership transfer, application-managed data movement In the paper: Many more details 51
52 Conclusion Overview of RAMP, lightweight support for configuration operations in loosely coupled systems Coordinated allocation of segments, fast ownership transfer, application-managed data movement In the paper: Many more details Rcached: memcached-like in-memory k/v store, using RAMP for load balancing 52
53 Feedback The right abstraction for the application Is shared memory abstraction overkill for loosely coupled data intensive applications? 53
54 Thank You 54
55 Rcached Memcached with RAMP based Hash-Maps Ability to migrate partitions 128 partitions hashed across 4 servers 40 million keys (key = 8 Bytes, Value = 128 Byte) 100 closed loop clients Per server latency noted over request windows 55
56 Rcached (2) 56
RAMP: A Lightweight RDMA Abstraction for Loosely Coupled Applications
RAMP: A Lightweight RDMA Abstraction for Loosely Coupled Applications Babar Naveed Memon, Xiayue Charles Lin, Arshia Mufti, Arthur Scott Wesley Tim Brecht, Kenneth Salem, Bernard Wong, Benjamin Cassell
MicroFuge: A Middleware Approach to Providing Performance Isolation in Cloud Storage Systems
1 MicroFuge: A Middleware Approach to Providing Performance Isolation in Cloud Storage Systems Akshay Singh, Xu Cui, Benjamin Cassell, Bernard Wong and Khuzaima Daudjee July 3, 2014 2 Storage Resources
Track Join. Distributed Joins with Minimal Network Traffic. Orestis Polychroniou! Rajkumar Sen! Kenneth A. Ross
Track Join Distributed Joins with Minimal Network Traffic Orestis Polychroniou Rajkumar Sen Kenneth A. Ross Local Joins Algorithms Hash Join Sort Merge Join Index Join Nested Loop Join Spilling to disk
No Tradeoff Low Latency + High Efficiency
No Tradeoff Low Latency + High Efficiency Christos Kozyrakis http://mast.stanford.edu Latency-critical Applications A growing class of online workloads Search, social networking, software-as-service (SaaS),
Memory-Based Cloud Architectures
Memory-Based Cloud Architectures ( Or: Technical Challenges for OnDemand Business Software) Jan Schaffner Enterprise Platform and Integration Concepts Group Example: Enterprise Benchmarking -) *%'+,#$)
Jinho Hwang (IBM Research) Wei Zhang, Timothy Wood, H. Howie Huang (George Washington Univ.) K.K. Ramakrishnan (Rutgers University)
Jinho Hwang (IBM Research) Wei Zhang, Timothy Wood, H. Howie Huang (George Washington Univ.) K.K. Ramakrishnan (Rutgers University) Background: Memory Caching Two orders of magnitude more reads than writes
WORKLOAD CHARACTERIZATION OF INTERACTIVE CLOUD SERVICES BIG AND SMALL SERVER PLATFORMS
WORKLOAD CHARACTERIZATION OF INTERACTIVE CLOUD SERVICES ON BIG AND SMALL SERVER PLATFORMS Shuang Chen*, Shay Galon**, Christina Delimitrou*, Srilatha Manne**, and José Martínez* *Cornell University **Cavium
Rocksteady: Fast Migration for Low-Latency In-memory Storage. Chinmay Kulkarni, Aniraj Kesavan, Tian Zhang, Robert Ricci, Ryan Stutsman
Rocksteady: Fast Migration for Low-Latency In-memory Storage Chinmay Kulkarni, niraj Kesavan, Tian Zhang, Robert Ricci, Ryan Stutsman 1 Introduction Distributed low-latency in-memory key-value stores are
High Speed Asynchronous Data Transfers on the Cray XT3
High Speed Asynchronous Data Transfers on the Cray XT3 Ciprian Docan, Manish Parashar and Scott Klasky The Applied Software System Laboratory Rutgers, The State University of New Jersey CUG 2007, Seattle,
A Distributed Hash Table for Shared Memory
A Distributed Hash Table for Shared Memory Wytse Oortwijn Formal Methods and Tools, University of Twente August 31, 2015 Wytse Oortwijn (Formal Methods and Tools, AUniversity Distributed of Twente) Hash
Analysis and Optimization. Carl Waldspurger Irfan Ahmad CloudPhysics, Inc.
PRESENTATION Practical Online TITLE GOES Cache HERE Analysis and Optimization Carl Waldspurger Irfan Ahmad CloudPhysics, Inc. SNIA Legal Notice The material contained in this tutorial is copyrighted by
Data Transformation and Migration in Polystores
Data Transformation and Migration in Polystores Adam Dziedzic, Aaron Elmore & Michael Stonebraker September 15th, 2016 Agenda Data Migration for Polystores: What & Why? How? Acceleration of physical data
The End of a Myth: Distributed Transactions Can Scale
The End of a Myth: Distributed Transactions Can Scale Erfan Zamanian, Carsten Binnig, Tim Harris, and Tim Kraska Rafael Ketsetsides Why distributed transactions? 2 Cheaper for the same processing power
Jinho Hwang and Timothy Wood George Washington University
Jinho Hwang and Timothy Wood George Washington University Background: Memory Caching Two orders of magnitude more reads than writes Solution: Deploy memcached hosts to handle the read capacity 6. HTTP
Stateless Network Functions:
Stateless Network Functions: Breaking the Tight Coupling of State and Processing Murad Kablan, Azzam Alsudais, Eric Keller, Franck Le University of Colorado IBM Networks Need Network Functions Firewall
Managing VMware Distributed Resource Scheduler
Managing VMware Distributed Resource Scheduler This chapter contains the following sections: About VMware Distributed Resource Scheduler, page 1 Using DRS Affinity Rules, page 1 Enabling or Disabling DRS,
YCSB++ Benchmarking Tool Performance Debugging Advanced Features of Scalable Table Stores
YCSB++ Benchmarking Tool Performance Debugging Advanced Features of Scalable Table Stores Swapnil Patil Milo Polte, Wittawat Tantisiriroj, Kai Ren, Lin Xiao, Julio Lopez, Garth Gibson, Adam Fuchs *, Billie
RAMCloud: A Low-Latency Datacenter Storage System Ankita Kejriwal Stanford University
RAMCloud: A Low-Latency Datacenter Storage System Ankita Kejriwal Stanford University (Joint work with Diego Ongaro, Ryan Stutsman, Steve Rumble, Mendel Rosenblum and John Ousterhout) a Storage System
Near Memory Key/Value Lookup Acceleration MemSys 2017
Near Key/Value Lookup Acceleration MemSys 2017 October 3, 2017 Scott Lloyd, Maya Gokhale Center for Applied Scientific Computing This work was performed under the auspices of the U.S. Department of Energy
MOHA: Many-Task Computing Framework on Hadoop
Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction
Shared Address Space I/O: A Novel I/O Approach for System-on-a-Chip Networking
Shared Address Space I/O: A Novel I/O Approach for System-on-a-Chip Networking Di-Shi Sun and Douglas M. Blough School of Electrical and Computer Engineering Georgia Institute of Technology Atlanta, GA
YCSB++ benchmarking tool Performance debugging advanced features of scalable table stores
YCSB++ benchmarking tool Performance debugging advanced features of scalable table stores Swapnil Patil M. Polte, W. Tantisiriroj, K. Ren, L.Xiao, J. Lopez, G.Gibson, A. Fuchs *, B. Rinaldi * Carnegie
Tailwind: Fast and Atomic RDMA-based Replication. Yacine Taleb, Ryan Stutsman, Gabriel Antoniu, Toni Cortes
Tailwind: Fast and Atomic RDMA-based Replication Yacine Taleb, Ryan Stutsman, Gabriel Antoniu, Toni Cortes In-Memory Key-Value Stores General purpose in-memory key-value stores are widely used nowadays
Nessie: A Decoupled, Client-Driven Key-Value Store Using RDMA
TO APPEAR IN: IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 1 Nessie: A Decoupled, Client-Driven Key-Value Store Using RDMA Benjamin Cassell*, Tyler Szepesi*, Bernard Wong*, Tim Brecht*, Jonathan
Architecture of a Real-Time Operational DBMS
Architecture of a Real-Time Operational DBMS Srini V. Srinivasan Founder, Chief Development Officer Aerospike CMG India Keynote Thane December 3, 2016 [ CMGI Keynote, Thane, India. 2016 Aerospike Inc.
Facebook Tao Distributed Data Store for the Social Graph
L. Lancia, G. Salillari Cloud Computing Master Degree in Data Science Sapienza Università di Roma Facebook Tao Distributed Data Store for the Social Graph L. Lancia & G. Salillari 1 / 40 Table of Contents
E-Store: Fine-Grained Elastic Partitioning for Distributed Transaction Processing Systems
E-Store: Fine-Grained Elastic Partitioning for Distributed Transaction Processing Systems Rebecca Taft, Essam Mansour, Marco Serafini, Jennie Duggan, Aaron J. Elmore, Ashraf Aboulnaga, Andrew Pavlo, Michael
Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work
Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Today (2014):
SHARDS & Talus: Online MRC estimation and optimization for very large caches
SHARDS & Talus: Online MRC estimation and optimization for very large caches Nohhyun Park CloudPhysics, Inc. Introduction Efficient MRC Construction with SHARDS FAST 15 Waldspurger at al. Talus: A simple
<Insert Picture Here> Enterprise Data Management using Grid Technology
Enterprise Data using Grid Technology Kriangsak Tiawsirisup Sales Consulting Manager Oracle Corporation (Thailand) 3 Related Data Centre Trends. Service Oriented Architecture Flexibility
ARM Vision for Thermal Management and Energy Aware Scheduling on Linux
ARM Vision for Management and Energy Aware Scheduling on Linux Charles Garcia-Tobin, Software Power Architect, ARM Thomas Molgaard, Director of Product Management, ARM ARM Tech Symposia China 2015 November
SWAP: EFFECTIVE FINE-GRAIN MANAGEMENT
: EFFECTIVE FINE-GRAIN MANAGEMENT OF SHARED LAST-LEVEL CACHES WITH MINIMUM HARDWARE SUPPORT Xiaodong Wang, Shuang Chen, Jeff Setter, and José F. Martínez Computer Systems Lab Cornell University Page 1
MDHIM: A Parallel Key/Value Store Framework for HPC
MDHIM: A Parallel Key/Value Store Framework for HPC Hugh Greenberg 7/6/2015 LA-UR-15-25039 HPC Clusters Managed by a job scheduler (e.g., Slurm, Moab) Designed for running user jobs Difficult to run system
Be Fast, Cheap and in Control with SwitchKV Xiaozhou Li
Be Fast, Cheap and in Control with SwitchKV Xiaozhou Li Raghav Sethi Michael Kaminsky David G. Andersen Michael J. Freedman Goal: fast and cost-effective key-value store Target: cluster-level storage for
How we build TiDB. Max Liu PingCAP Amsterdam, Netherlands October 5, 2016
How we build TiDB Max Liu PingCAP Amsterdam, Netherlands October 5, 2016 About me Infrastructure engineer / CEO of PingCAP Working on open source projects: TiDB: https://github.com/pingcap/tidb TiKV: https://github.com/pingcap/tikv
Infiniswap. Efficient Memory Disaggregation. Mosharaf Chowdhury. with Juncheng Gu, Youngmoon Lee, Yiwen Zhang, and Kang G. Shin
Infiniswap Efficient Memory Disaggregation Mosharaf Chowdhury with Juncheng Gu, Youngmoon Lee, Yiwen Zhang, and Kang G. Shin Rack-Scale Computing Datacenter-Scale Computing Geo-Distributed Computing Coflow
Database Workload. from additional misses in this already memory-intensive databases? interference could be a problem) Key question:
Database Workload + Low throughput (0.8 IPC on an 8-wide superscalar. 1/4 of SPEC) + Naturally threaded (and widely used) application - Already high cache miss rates on a single-threaded machine (destructive
Application Programming
Multicore Application Programming For Windows, Linux, and Oracle Solaris Darryl Gove AAddison-Wesley Upper Saddle River, NJ Boston Indianapolis San Francisco New York Toronto Montreal London Munich Paris
Leveraging Flash in HPC Systems
Leveraging Flash in HPC Systems IEEE MSST June 3, 2015 This work was performed under the auspices of the U.S. Department of Energy by under Contract DE-AC52-07NA27344. Lawrence Livermore National Security,
PAC485 Managing Datacenter Resources Using the VirtualCenter Distributed Resource Scheduler
PAC485 Managing Datacenter Resources Using the VirtualCenter Distributed Resource Scheduler Carl Waldspurger Principal Engineer, R&D This presentation may contain VMware confidential information. Copyright
VoltDB vs. Redis Benchmark
Volt vs. Redis Benchmark Motivation and Goals of this Evaluation Compare the performance of several distributed databases that can be used for state storage in some of our applications Low latency is expected
Data Processing on Emerging Hardware
Data Processing on Emerging Hardware Gustavo Alonso Systems Group Department of Computer Science ETH Zurich, Switzerland 3 rd International Summer School on Big Data, Munich, Germany, 2017 www.systems.ethz.ch
Shared Memory and Distributed Multiprocessing. Bhanu Kapoor, Ph.D. The Saylor Foundation
Shared Memory and Distributed Multiprocessing Bhanu Kapoor, Ph.D. The Saylor Foundation 1 Issue with Parallelism Parallel software is the problem Need to get significant performance improvement Otherwise,
Ceph: A Scalable, High-Performance Distributed File System
Ceph: A Scalable, High-Performance Distributed File System S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long Presented by Philip Snowberger Department of Computer Science and Engineering University
Oncilla - a Managed GAS Runtime for Accelerating Data Warehousing Queries
Oncilla - a Managed GAS Runtime for Accelerating Data Warehousing Queries Jeffrey Young, Alex Merritt, Se Hoon Shon Advisor: Sudhakar Yalamanchili 4/16/13 Sponsors: Intel, NVIDIA, NSF 2 The Problem Big
Flash Storage Complementing a Data Lake for Real-Time Insight
Flash Storage Complementing a Data Lake for Real-Time Insight Dr. Sanhita Sarkar Global Director, Analytics Software Development August 7, 2018 Agenda 1 2 3 4 5 Delivering insight along the entire spectrum
Reducing CPU and network overhead for small I/O requests in network storage protocols over raw Ethernet
Reducing CPU and network overhead for small I/O requests in network storage protocols over raw Ethernet Pilar González-Férez and Angelos Bilas 31 th International Conference on Massive Storage Systems
Tuning WebHound 4.0 and SAS 8.2 for Enterprise Windows Systems James R. Lebak, Unisys Corporation, Malvern, PA
Paper 272-27 Tuning WebHound 4.0 and SAS 8.2 for Enterprise Windows Systems James R. Lebak, Unisys Corporation, Malvern, PA ABSTRACT Windows is SAS largest and fastest growing platform. Windows 2000 Advanced
Unevenly Distributed. Adrian
Unevenly Distributed Adrian Colyer @adriancolyer blog.acolyer.org 350 Foundations Frontiers 5 Reasons to
Asymmetry-aware execution placement on manycore chips
Asymmetry-aware execution placement on manycore chips Alexey Tumanov Joshua Wise, Onur Mutlu, Greg Ganger CARNEGIE MELLON UNIVERSITY Introduction: Core Scaling? Moore s Law continues: can still fit more
Virtual SQL Servers. Actual Performance. 2016
@kleegeek davidklee.net heraflux.com linkedin.com/in/davidaklee Specialties / Focus Areas / Passions: Performance Tuning & Troubleshooting Virtualization Cloud Enablement Infrastructure Architecture Health
Voldemort. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Overview Design Evaluation
Voldemort Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi Leader Election 1/29 Outline 1 2 3 Smruti R. Sarangi Leader Election 2/29 Data
A Comparison of Capacity Management Schemes for Shared CMP Caches
A Comparison of Capacity Management Schemes for Shared CMP Caches Carole-Jean Wu and Margaret Martonosi Princeton University 7 th Annual WDDD 6/22/28 Motivation P P1 P1 Pn L1 L1 L1 L1 Last Level On-Chip
Distributed Systems COMP 212. Lecture 18 Othon Michail
Distributed Systems COMP 212 Lecture 18 Othon Michail Virtualisation & Cloud Computing 2/27 Protection rings It s all about protection rings in modern processors Hardware mechanism to protect data and
Simplified Storage Migration for Microsoft Cluster Server
Simplified Storage Migration for Microsoft Cluster Server Using VERITAS Volume Manager for Windows 2000 with Microsoft Cluster Server V E R I T A S W H I T E P A P E R June 2001 Table of Contents Overview...................................................................................1
Inexpensive Coordination in Hardware
Consensus in a Box Inexpensive Coordination in Hardware Zsolt István, David Sidler, Gustavo Alonso, Marko Vukolic * Systems Group, Department of Computer Science, ETH Zurich * Consensus IBM in Research,
Properties of Processes
CPU Scheduling Properties of Processes CPU I/O Burst Cycle Process execution consists of a cycle of CPU execution and I/O wait. CPU burst distribution: CPU Scheduler Selects from among the processes that
XBS Application Development Platform
Introduction to XBS Application Development Platform By: Liu, Xiao Kang (Ken) Xiaokang Liu Page 1/10 Oct 2011 Overview The XBS is an application development platform. It provides both application development
DIDO: Dynamic Pipelines for In-Memory Key-Value Stores on Coupled CPU-GPU Architectures
DIDO: Dynamic Pipelines for In-Memory Key-Value Stores on Coupled CPU-GPU rchitectures Kai Zhang, Jiayu Hu, Bingsheng He, Bei Hua School of Computing, National University of Singapore School of Computer
L3.4. Data Management Techniques. Frederic Desprez Benjamin Isnard Johan Montagnat
Grid Workflow Efficient Enactment for Data Intensive Applications L3.4 Data Management Techniques Authors : Eddy Caron Frederic Desprez Benjamin Isnard Johan Montagnat Summary : This document presents
Big and Fast. Anti-Caching in OLTP Systems. Justin DeBrabant
Big and Fast Anti-Caching in OLTP Systems Justin DeBrabant Online Transaction Processing transaction-oriented small footprint write-intensive 2 A bit of history 3 OLTP Through the Years relational model
Sun N1: Storage Virtualization and Oracle
OracleWorld 2003 Session 36707 - Sun N1: Storage Virtualization and Oracle Glenn Colaco Performance Engineer Sun Microsystems Performance and Availability Engineering September 9, 2003 Background PAE works
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff and Shun Tak Leung Google* Shivesh Kumar Sharma fl4164@wayne.edu Fall 2015 004395771 Overview Google file system is a scalable distributed file system
Fluxo. Improving the Responsiveness of Internet Services with Automa7c Cache Placement
Fluxo Improving the Responsiveness of Internet Services with Automac Cache Placement Alexander Rasmussen UCSD (Presenng) Emre Kiciman MSR Redmond Benjamin Livshits MSR Redmond Madanlal Musuvathi MSR Redmond
Introducing the Cray XMT. Petr Konecny May 4 th 2007
Introducing the Cray XMT Petr Konecny May 4 th 2007 Agenda Origins of the Cray XMT Cray XMT system architecture Cray XT infrastructure Cray Threadstorm processor Shared memory programming model Benefits/drawbacks/solutions
Main Memory (Part II)
Main Memory (Part II) Amir H. Payberah amir@sics.se Amirkabir University of Technology (Tehran Polytechnic) Amir H. Payberah (Tehran Polytechnic) Main Memory 1393/8/17 1 / 50 Reminder Amir H. Payberah
SCALE OUT AND CONQUER: ARCHITECTURAL DECISIONS BEHIND DISTRIBUTED IN-MEMORY SYSTEMS VLADIMIR OZEROV YAKOV ZHDANOV
SCALE OUT AND CONQUER: ARCHITECTURAL DECISIONS BEHIND DISTRIBUTED IN-MEMORY SYSTEMS VLADIMIR OZEROV YAKOV ZHDANOV WHO? Yakov Zhdanov: - GridGain s Product Development VP - With GridGain since 2010 - Apache
Data Centers and Cloud Computing
Data Centers and Cloud Computing CS677 Guest Lecture Tim Wood 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises for server applications Internet
Data Centers and Cloud Computing. Slides courtesy of Tim Wood
Data Centers and Cloud Computing Slides courtesy of Tim Wood 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises for server applications Internet
Atlas: Baidu s Key-value Storage System for Cloud Data
Atlas: Baidu s Key-value Storage System for Cloud Data Song Jiang Chunbo Lai Shiding Lin Liqiong Yang Guangyu Sun Jason Cong Wayne State University Zhenyu Hou Can Cui Peking University University of California
Memory hierarchy review. ECE 154B Dmitri Strukov
Memory hierarchy review ECE 154B Dmitri Strukov Outline Cache motivation Cache basics Six basic optimizations Virtual memory Cache performance Opteron example Processor-DRAM gap in latency Q1. How to deal
Most real programs operate somewhere between task and data parallelism. Our solution also lies in this set.
for Windows Azure and HPC Cluster 1. Introduction In parallel computing systems computations are executed simultaneously, wholly or in part. This approach is based on the partitioning of a big task into
HyperDex. A Distributed, Searchable Key-Value Store. Robert Escriva. Department of Computer Science Cornell University
HyperDex A Distributed, Searchable Key-Value Store Robert Escriva Bernard Wong Emin Gün Sirer Department of Computer Science Cornell University School of Computer Science University of Waterloo ACM SIGCOMM
Using FPGAs as Microservices
Using FPGAs as Microservices David Ojika, Ann Gordon-Ross, Herman Lam, Bhavesh Patel, Gaurav Kaul, Jayson Strayer (University of Florida, DELL EMC, Intel Corporation) The 9 th Workshop on Big Data Benchmarks,
Data Centers and Cloud Computing. Data Centers
Data Centers and Cloud Computing Slides courtesy of Tim Wood 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises for server applications Internet
Loosely coupled: asynchronous processing, decoupling of tiers/components Fan-out the application tiers to support the workload Use cache for data and content Reduce number of requests if possible Batch
Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency
Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency Jialin Li, Naveen Kr. Sharma, Dan R. K. Ports and Steven D. Gribble February 2, 2015 1 Introduction What is Tail Latency? What
Goals. Facebook s Scaling Problem. Scaling Strategy. Facebook Three Layer Architecture. Workload. Memcache as a Service.
Goals Memcache as a Service Tom Anderson Rapid application development - Speed of adding new features is paramount Scale Billions of users Every user on FB all the time Performance Low latency for every
SCYLLA: NoSQL at Ludicrous Speed. 主讲人 :ScyllaDB 软件工程师贺俊
SCYLLA: NoSQL at Ludicrous Speed 主讲人 :ScyllaDB 软件工程师贺俊 Today we will cover: + Intro: Who we are, what we do, who uses it + Why we started ScyllaDB + Why should you care + How we made design decisions to
On Smart Query Routing: For Distributed Graph Querying with Decoupled Storage
On Smart Query Routing: For Distributed Graph Querying with Decoupled Storage Arijit Khan Nanyang Technological University (NTU), Singapore Gustavo Segovia ETH Zurich, Switzerland Donald Kossmann Microsoft
Lecture S3: File system data layout, naming
Lecture S3: File system data layout, naming Review -- 1 min Intro to I/O Performance model: Log Disk physical characteristics/desired abstractions Physical reality Desired abstraction disks are slow fast
Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work
Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Today (2014):
Presented by: Alvaro Llanos E
Presented by: Alvaro Llanos E Motivation and Overview Frangipani Architecture overview Similar DFS PETAL: Distributed virtual disks Overview Design Virtual Physical mapping Failure tolerance Frangipani
This presentation covers Gen Z Buffer operations. Buffer operations are used to move large quantities of data between components.
This presentation covers Gen Z Buffer operations. Buffer operations are used to move large quantities of data between components. 1 2 Buffer operations are used to get data put or push up to 4 GiB of data
IX: A Protected Dataplane Operating System for High Throughput and Low Latency
IX: A Protected Dataplane Operating System for High Throughput and Low Latency Adam Belay et al. Proc. of the 11th USENIX Symp. on OSDI, pp. 49-65, 2014. Presented by Han Zhang & Zaina Hamid Challenges
Software-Defined Data Infrastructure Essentials
Software-Defined Data Infrastructure Essentials Cloud, Converged, and Virtual Fundamental Server Storage I/O Tradecraft Greg Schulz Server StorageIO @StorageIO 1 of 13 Contents Preface Who Should Read
Exercises: April 11. Hermann Härtig, TU Dresden, Distributed OS, Load Balancing
Exercises: April 11 1 PARTITIONING IN MPI COMMUNICATION AND NOISE AS HPC BOTTLENECK LOAD BALANCING DISTRIBUTED OPERATING SYSTEMS, SCALABILITY, SS 2017 Hermann Härtig THIS LECTURE Partitioning: bulk synchronous
LECTURE 3:CPU SCHEDULING
LECTURE 3:CPU SCHEDULING 1 Outline Basic Concepts Scheduling Criteria Scheduling Algorithms Multiple-Processor Scheduling Real-Time CPU Scheduling Operating Systems Examples Algorithm Evaluation 2 Objectives
Aerie: Flexible File-System Interfaces to Storage-Class Memory [Eurosys 2014] Operating System Design Yongju Song
![Aerie: Flexible File-System Interfaces to Storage-Class Memory [Eurosys 2014] Operating System Design Yongju Song](/thumbs/93/111822298.jpg "Aerie: Flexible File-System Interfaces to Storage-Class Memory [Eurosys 2014] Operating System Design Yongju Song")
Aerie: Flexible File-System Interfaces to Storage-Class Memory [Eurosys 2014] Operating System Design Yongju Song Outline 1. Storage-Class Memory (SCM) 2. Motivation 3. Design of Aerie 4. File System Features
High Performance Packet Processing with FlexNIC
High Performance Packet Processing with FlexNIC Antoine Kaufmann, Naveen Kr. Sharma Thomas Anderson, Arvind Krishnamurthy University of Washington Simon Peter The University of Texas at Austin Ethernet
Adapted from: TRENDS AND ATTRIBUTES OF HORIZONTAL AND VERTICAL COMPUTING ARCHITECTURES
Adapted from: TRENDS AND ATTRIBUTES OF HORIZONTAL AND VERTICAL COMPUTING ARCHITECTURES Tom Atwood Business Development Manager Sun Microsystems, Inc. Takeaways Understand the technical differences between
Exadata X3 in action: Measuring Smart Scan efficiency with AWR. Franck Pachot Senior Consultant
Exadata X3 in action: Measuring Smart Scan efficiency with AWR Franck Pachot Senior Consultant 16 March 2013 1 Exadata X3 in action: Measuring Smart Scan efficiency with AWR Exadata comes with new statistics
MARACAS: A Real-Time Multicore VCPU Scheduling Framework
: A Real-Time Framework Computer Science Department Boston University Overview 1 2 3 4 5 6 7 Motivation platforms are gaining popularity in embedded and real-time systems concurrent workload support less
Performance Evaluation of NoSQL Databases
Performance Evaluation of NoSQL Databases A Case Study - John Klein, Ian Gorton, Neil Ernst, Patrick Donohoe, Kim Pham, Chrisjan Matser February 2015 PABS '15: Proceedings of the 1st Workshop on Performance
A Practical Scalable Distributed B-Tree
A Practical Scalable Distributed B-Tree CS 848 Paper Presentation Marcos K. Aguilera, Wojciech Golab, Mehul A. Shah PVLDB 08 March 8, 2010 Presenter: Evguenia (Elmi) Eflov Presentation Outline 1 Background
Exam Guide COMPSCI 386
FOUNDATIONS We discussed in broad terms the three primary responsibilities of an operating system. Describe each. What is a process? What is a thread? What parts of a process are shared by threads? What
Data Partitioning on Heterogeneous Multicore and Multi-GPU systems Using Functional Performance Models of Data-Parallel Applictions
Data Partitioning on Heterogeneous Multicore and Multi-GPU systems Using Functional Performance Models of Data-Parallel Applictions Ziming Zhong Vladimir Rychkov Alexey Lastovetsky Heterogeneous Computing
Penalized Graph Partitioning for Static and Dynamic Load Balancing
Penalized Graph Partitioning for Static and Dynamic Load Balancing Tim Kiefer, Dirk Habich, Wolfgang Lehner Euro-Par 06, Grenoble, France, 06-08-5 Task Allocation Challenge Application (Workload) = Set
FaRM: Fast Remote Memory
FaRM: Fast Remote Memory Problem Context DRAM prices have decreased significantly Cost effective to build commodity servers w/hundreds of GBs E.g. - cluster with 100 machines can hold tens of TBs of main
@ Massachusetts Institute of Technology All rights reserved.
..... CPHASH: A Cache-Partitioned Hash Table with LRU Eviction by Zviad Metreveli MASSACHUSETTS INSTITUTE OF TECHAC'LOGY JUN 2 1 2011 LIBRARIES Submitted to the Department of Electrical Engineering and