KerData: Scalable Data Management on Clouds and Beyond

Transcription

1 KerData: Scalable Data Management on Clouds and Beyond Gabriel Antoniu INRIA Rennes Bretagne Atlantique Research Centre Franco-British Workshop on Big Data in Science, 6-7 November 2012

2 The French Institute for Research in ICST Information and Communication Science and Technologies RESEARCH TECHNOLOGY DEVELOPMENT AND EXPERIMENTATION EDUCATION AND TRAINING TRANSFER AND INNOVATION A scientific and technological public institution under the dual authority of the Ministry of Research and the Ministry of Industry 2

3 Inria Key figures 4,290 People (60 % paid by INRIA) 8 Research Centers in France 41 International conferences 69 Associated teams throughout the world 4,850 Scientific publications 3,430 Scientists 1,000 Doctoral students 100 Post-Doctoral 300 R&D engineers 171 Project teams A BUDGET OF 252M Of which more than 25% from external resources 271 Active patents (in total) 1,000 Software 3

4 5 Main research topics 1 Applied Mathematics, Computation and Simulation 2 Algorithmics, Programming, Software and Architecture 3 Networks, Systems and Services, Distributed Computing 4 Perception, Cognition, Interaction 5 Computational Sciences for Biology, Medicine and Environment 4

5 Inria s Research Centres Inria LILLE Nord Europe Inria PARIS - Rocquencourt Inria NANCY Grand Est Inria SACLAY Île-de-France Inria RENNES Bretagne Atlantique Inria GRENOBLE Rhône-Alpes Inria BORDEAUX Sud-Ouest Inria SOPHIA ANTIPOLIS Méditerranée 5

6 Inria Project-Team Inria Project Team 10 to 30 people working under a scientific leader supervision Focused scientific theme and international evaluation A limited lifespan : average 8 years, maximum 12 years Well-defined objectives and work program Linked to and cooperating with industrial and scientific partners in France and around the world A priori and a posteriori evaluation 171 Inria Project- Teams in in partnership An organization that complements the universities 6

7 Context for KerData: the Data Deluge Deliver the capability to mine, search and analyze this data in near real time Science itself is evolving Credits: Microsoft 7

8 The Data Science: The 4 th Paradigm for Scientific Discovery. a a 2 = 4πGρ Κ 3 c a 2 2 Thousand years ago Description of natural phenomena Last few hundred years Newton s laws, Maxwell s equations Last few decades Simulation of complex phenomena Today and the Future Unify theory, experiment and simulation with large multidisciplinary Data Using data exploration and data mining (from instruments, sensors, humans ) Crédits: Dennis Gannon Distributed Communities 8

9 The Data Science: The 4 th Paradigm for Scientific Discovery. a a 2 = 4πGρ Κ 3 c a 2 2 Thousand years ago Description of natural phenomena Last few hundred years Newton s laws, Maxwell s equations Last few decades Simulation of complex phenomena Today and the Future Unify theory, experiment and simulation with large multidisciplinary Data Using data exploration and data mining (from instruments, sensors, humans ) Distributed Communities 9

10 KerData s Focus: How to efficiently store and share data for new-generation, data-intensive applications? Scientific challenges Massive data (1 object = 1 TB) Geographically distributed Fine-grain access (MB) for reading and writing High concurrency (10³ concurrent clients) Without locking Major goal: high-throughput under heavy concurrency Our contribution Design and implementation of distributed algorithms Validation with real apps on real platforms with real users 10

11 KerData s Core Research Challenges Applications Massive data analysis: clouds (e.g. MapReduce) Post-Petascale HPC simulations: supercomputers Focus 1: Scalable data storage on IaaS and PaaS clouds Challenge : understand how to reconcile performance, scalability, security and quality of service according to the requirements of data-intensive applications Our Answer: BlobSeer Focus 2: Scalable data storage on Post-Petascale HPC systems Challenge: go beyond the limitations of current file-based approaches Our Answer: Damaris Key aspect : multi-disciplinary collaborative research with application communities 11

12 Focus 1: Scalable Data Storage on IaaS and Paas Clouds

13 BlobSeer: A Software Platform for Scalable, Distributed BLOB Management Started in 2008, 6 PhD theses (Gilles Kahn/SPECIF PhD Thesis Award in 2011) Main goal: optimized for concurrent accesses under heavy concurrency Three key ideas Decentralized metadata management Lock-free concurrent writes (enabled by versioning) - Write = create new version of the data Data and metadata patching rather than updating A back-end for higher-level data management systems Short term: highly scalable distributed file systems Middle term: storage for cloud services Our approach Design and implementation of distributed algorithms Experiments on the Grid 5000 grid/cloud testbed Validation with real apps on real platforms: Nimbus, Azure, OpenNebula clouds 13

14 Impact of BlobSeer: MapReduce BlobSeer improves Hadoop Gain (execution time) : 35% ANR MapReduce Project ( ) Lead: G. Antoniu (KerData) Partners: INRIA (AVALON), Argonne National Lab, U. Illinois Urbana- Champaign, IBM, JLPC, IBCP, MEDIT Strong collaboration with the Nimbus team from Argonne National Lab BlobSeer integrated with the Nimbus cloud toolkit BlobSeer used for efficient VM deployment and snapshotting Validation : Grid 5000 with Nimbus, FutureGrid (USA), Open Cirrus (USA) 14

15 Impact of BlobSeer: The A-Brain Microsoft Research Inria Project KerData, PARIETAL teams at INRIA Brain image Genetic data European Microsoft Innovation Center (Aachen) p(, ) Y Anatomical MRI Functional MRI Diffusion MRI q~ p~10 6 X DNA array (SNP/CNV) gene expression data others... N~2000 TomusBlobs software (based on BlobSeer) Gain / Blobs Azure : 45% Scalability : 350 cores Demo available! 15

16 Scalable Storage on Clouds: Our 4-Year Strategy BlobSeer Nimbus ANR MapReduce ADT BlobSeer Clouds Microsoft Azure A-Brain OpenNebula FP7 SCALUS PaaS : advanced services for data storage, sharing and processing IaaS : VM deployment and checkpointing services Challenge : understand how to reconcile performance, scalability, security and quality of service according to the application requirements Autonomy, adaptive consistency, dynamic elasticity Trade-offs exposed to the user Our tools: algorithmes/distributed protocols, software, distributed platforms Collaboration with application communities 16

17 Focus 2: Scalable Data Storage for Post-Petascale HPC

18 Context : Data Management in Post-Petascale HPC Systems cores PetaBytes of data ~ cores ² Problem: simulations generate TB/min ² How to store et transfer data? ² How to analyze, visualize and extract knowledge? 18 18

19 Context: Data Management in Post-Petascale HPC Systems cores PetaBytes of data Does not scale! ~ cores ² Problem: simulations generate TB/min ² How to store et transfer data? ² How to analyze, visualize and extract knowledge? What is difficult? Too many files (e.g. Blue Waters files/min) Too much data Unpredictable I/O performance 19 19

20 Damaris: A Middleware-Level Approach to I/O on Multicore HPC Systems Idea : one dedicated I/O core per multicore node Originality : shared memory, asynchronous processing Implementation: software library Application: Tornado simulation (Blue Waters) Tests on Kraken (11th in Top PF) Scales on 10,000+ cores x12 less files Overhead-free compression Predictable performance 20

21 Damaris: Early Impact First results of the Joint Lab for Petascale Computing transfered to Blue Waters (2011) 2 nd Award for Matthieu Dorier at the ACM Student Research Competition (ICS 2011) «This work is practically very useful (and novel) to the field of computational meteorology and probably fluid dynamics. I think Damaris is going to be the best I/O option for these unprecedented supercomputing simulations.» Leigh Orf, atmospheric scientist, Central Michigan University 21

22 Scalable Storage and I/O on HPC Systems: Our 4-Year Strategy INRIA-ANL (FACCTS, PUF, EA) Damaris, BlobSeer Blue Waters JLPC Post-Petascale HPC ANR-JST FP3C Optimized I/O library for postpetascale HPC Middleware for storing multidimensional data Challenge: go beyond the limitations of current file-based approaches Explore dedicated I/O cores for in-situ visualization - collaboration : UIUC, USA Automatic analysis and image generation Adaptive image generation (recognize «interesting» data subsets) Interactive visualization Coupling with alternative approaches (I/O forwarding, etc.) collaboration : ANL, USA Collaboration with application communities Associate Team Data@Exascale with UIUC/ANL submitted ( ) 22 22

23 Longer-Term Directions Scalable storage and processing for structured data Structural/semantic metadata Collaboration with the database community (NoSQL, ) Performance modeling for scalable storage systems Capture interactions between system components Predict storage/transfer costs on clouds, self-optimization Collaboration with the scheduling community 23