Achieving the Potential of a Fully Distributed Storage System

Transcription

1 Achieving the Potential of a Fully Distributed Storage System HPCN Workshop 2013, DLR Braunschweig, 7-8 May 2013 Slide 1

2 Scality Quick Facts Founded 2009 Experienced management team HQ in the San Francisco, Global reach 50+ Employees, 20 engineers in Paris 24 x 7 support team US patents $13M invested in Scality to-date 120% annual growth Industry Associations Aggressive use of a scale-out architecture like that enabled by Scality's RING architecture will become more prevalent, as IT organizations develop best practices that boost storage asset use, reduce operational overhead, and meet high data availability expectations. Slide 2

3 Customers in US, Europe and Japan Service Providers Cloud Providers (s3 Compatible, FileSync, Cloud Backup ) Consumer Internet Big Data Hardware Alliances Slide 3

4 Distributed Architecture P2P Scality RING 1 node manages 1/36th of the key space servers (6) storage nodes (ex: 6/server, total=36) 36 storage nodes projected on a ring From Servers to Storage Nodes RING Topology, P2P Architecture Limitless Scale-Out Storage based on Shared Nothing model Fully Distributed Storage (Data and Meta-data) Slide 4

5 Scality Basics A Software Technology using generic X86 hardware and disks Simple Object based model: Key-Object Store with CRUD interface: Create, Read, Update, Delete Keys are 20 bytes and can be chosen by the application Fully Distributed: No assumption about immutability manage version consistency Slide 5

6 State of the Art distributed protocol RING Protocol VERBS Clustering Transac:ons Data Healing LOCATE HEARTBEAT CHECK RLOCK RWLOCK RESERVE UPDATELOCK VERSIONS GET, PUT, DELETE GET, PUT, DELETE MD UPDATE PURGE REBUILD BALANCE 6

7 End-to-End Parallelism Parallel Connectors access to Storage Nodes Performance aggregation Redundant Data Path Multiple Storage Nodes per server Minimum 6 to increase parallelism and data independence Fast and easy rebuild Multiple IO Daemons per Server Control Physical and Boost IO APPLICATIONS / CONNECTORS STORAGE NODES I/O DAEMONS SSD APPLICATIONS / CONNECTORS I/O DAEMONS STORAGE NODES I/O DAEMONS APPLICATIONS / CONNECTORS SATA STORAGE NODES I/O DAEMONS APPLICATIONS / CONNECTORS I/O DAEMONS APPLICATIONS / CONNECTORS STORAGE NODES TIERED STORAGE I/O DAEMONS Scality Parallelism Factor #Storage Nodes x #IO Daemons vs Simple server node with only 1 IO Engine Independent Performance and Capacity Scalability Slide 7

8 The Transac:onal Database Data Model 8

9 The POSIX Filesystem Data Model Fileysystem Metadata uses database model Files can use Several object models: Chunking (gives unlimited file size) Sparse Files (gives random access R/W) ARC (Gives Erasure Code based Protec:on) Slide 9

10 Advanced Redundancy Configura:on ARC Slide 10

11 Data Metadata Stack Op:ons Slide 11

12 Distribution Challenge: Consistency ACID: Atomicity, Consistency, Isolation, Durability CRUD: Create, Read, Update, Delete Brewer s CAP Theorem Consistency, Availability, Partition Tolerance Par::on! No- contact Not consistent Slide 12

13 Distributed Consistency Central Lock Service Distributed Locking Two-Phase Commits and Vector Clocks Paxos and Multi-Paxos Roadmap next step Directory Level,file level and block level locking via election of a lock manager, supports partitionning Slide 13

14 Strong Eventual Consistency Why Bother? Serialization/Sync is Expensive Systems fail New Models (Note: eventual = sometime not maybe) ACID 2.0 : Associative, Commutative, Idempotent, and Distributed CR: Create & Read model Can Create Consistent Results via: Problem Sharding CALM : consistency as logical monotonicity, Bloom L language LWW: Last Writer Wins CRDT Conflict-free Replicated Data Types Slide 14

15 Scality RING 4 HPC StaaS Digital Media Big Data Enterprise & Cloud System Scale Out File System S3 & CDMI API Origin Server Data Processing with Hadoop Scality RING Organic Storage 4 P2P MD DATA ARC S3 CDMI Ring Topology End-to-End Parallelism Object Storage MESA NewSQL DB Replication Erasure Coding Geo Redundancy Tiering Standard Management x86 Slide 15

16 Scality Hadoop Technical View Get_Blocks_Locations() 2 List (Range ) 1 Job Tracker CDMI/SOFS server 3 Block Data Ingest without Hadoop components Hadoop Job Tracker + Scality CDMI server (Dewpoint) and Scale Out File System 3 Block Scality RING Task Tracker Storage Node CDMIfs CDMI/SOFS server Hadoop Task Trackers on RING Storage Nodes Hadoop Scality RING Scality extension for Hadoop Hadoop Cluster on Scality RING (example: 12 Hadoop nodes for 12 storage nodes) Slide 16

17 Scality Solution for Hadoop NoSQL DB (Hbase) Scripting (Pig) Query/SQL (Hive) Distributed Processing (MapReduce) Metadata Services (HCatalog) NewSQL DB* (Scality MESA) Distributed Storage (Scality Scale Out File System) * Future Slide 17

18 Thank You Brad King, Chief Architect Slide 18