RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP

Size: px
Start display at page:

Download "RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP"

Transcription

1 RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP

2 THE EUROPEAN ORGANISATION FOR PARTICLE PHYSICS RESEARCH (CERN) 2 THE LARGE HADRON COLLIDER

3 THE LARGE HADRON COLLIDER 3 TUNNEL VISION 27 km circumference 100 m underground 180MW power consumption 7 TeV per beam

4 THE LARGE HADRON COLLIDER ALICE, ATLAS, CMS AND LHCB DETECTORS 4

5 THE LARGE HADRON COLLIDER 5 CMS DETECTOR INNER BARREL

6 THE LARGE HADRON COLLIDER 6 SUPER COLLIDER Super Collider, Mustaine et al. (2013, Universal Records)

7 THE LARGE HADRON COLLIDER 7 EXPERIMENT DATA CHALLENGE 100 Million channels, bunch crossing every 25 ns 1 PB/s internal data rate 5 PB data / year recorded (plus derived data sets) 100 PB / year by 2025 (x20) 5 million lines of code / experiment

8 THE WORLDWIDE LHC COMPUTING GRID GLOBALLY DISTRIBUTED Worldwide LHC Compute Grid live map 42 countries, 170 computing centres, 2 million jobs run each day 8

9 LHC EXPERIMENT SOFTWARE STACKS 9 KEY FIGURES Hundreds of developers ~10^8 binaries ~1TB / day of nightly builds ~ machines world-wide Daily production releases, remain available

10 THE CERNVM FILE SYSTEM 10

11 THE CERNVM FILE SYSTEM 11 A FILE SYSTEM APPROACH TO DISTRIBUTING SOFTWARE BASIC SYSTEM UTILITIES OS KERNEL CERNVM FS FUSE GLOBAL HTTP CACHE HIERARCHY FILE SYSTEM MEMORY BUFFER (~100MB) CERNVM-FS PERSISTENT CACHE (~20GB) REPOSITORY (HTTP OR S3) ~1-10TB ~ clients FUSE based, independent mount points, e. g. /cvmfs/atlas.cern.ch Clients have a read-only view; single writer into repository HTTP transport, access and caching on demand

12 THE CERNVM FILE SYSTEM 12 MAIN COMPONENTS Client: FUSE module (with cache plugins) Server tools (command line tools) Standard HTTP server HTTP caches

13 THE CERNVM FILE SYSTEM 13 DESIGN Data store: Immutable content-addressed blobs (*) Compression, deduplication Metadata: Catalogs: state of the entire repository at a given moment in time is encoded in a Merkle tree Digitally signed manifest Versioning, snapshots etc. PULL based!

14 CVMFS PUBLICATION WORKFLOW 14

15 THE CERNVM FILE SYSTEM 15 PUBLISHING Single writer (stateless command line utilities) A read/write view is constructed with a union mount (OverlayFS, Aufs) Files are compressed and hashed, and written to repository storage New metadata catalogs are created and published Repository manifest is updated (atomic operation)

16 PUBLISHING TO CVMFS REPOSITORIES 16 EXISTING WORKFLOW Centralised release manager machine Direct interaction with the release manager: $ ssh my-cvmfs-server.cern.ch $ cvmfs_server transaction $ vim /cvmfs/my-cvmfs-server.cern.ch/some_file.org (Make changes to files in the R/W mount) $ cvmfs_server publish

17 PUBLISHING TO CVMFS REPOSITORIES 17 EXISTING WORKFLOW PROS: Straightforward to use Good for scripting Hides somewhat the distributed nature of the system CONS: No support for concurrent writing Can be unsafe (shell access to machine with repository storage) Performance issues for large change-sets

18 PUBLISHING TO CVMFS REPOSITORIES 18 PROPERTIES AND CONSTRAINTS 1. The system (repository + cache + clients) is eventually consistent 2. Concurrency can be further exploited due to: Immutability of CAS Pushing objects is idempotent Directory tree structure 3. The critical section involves updating the metadata catalog and swapping the manifest

19 PUBLISHING TO CVMFS REPOSITORIES 19 EXISTING ARCHITECTURE HTTP SSH NFS, S3 USER MACHINE RELEASE MANAGER AND GATEWAY CVMFS FUSE CVMFS SERVER AUTHORITATIVE STORAGE STRATA 1

20 PUBLISHING TO CVMFS REPOSITORIES 20 AN IMPROVED ARCHITECTURE USER MACHINE RELEASE MANAGER AND GATEWAY CVMFS FUSE CVMFS SERVER CVMFS SERVICE API Gateway CVMFS Gateway Services CVMFS Gateway Services CVMFS Services STORAGE GATEWAY USER MACHINE RELEASE MANAGER AND GATEWAY CVMFS FUSE CVMFS SERVER REPLICAS AUTH. STORAGE STRATA 1

21 PUBLISHING TO CVMFS REPOSITORIES 21 AN IMPROVED WORKFLOW $ ssh my-cvmfs-1.cern.ch $ cvmfs_server transaction /lcg/58 (Make changes to files in the R/W mount) $ vim /cvmfs/my-cvmfs.cern.ch/lcg/58/ some_file.org $ cvmfs_server publish $ ssh my-cvmfs-2.cern.ch $ cvmfs_server transaction /lcg/60 (Make changes to files in the R/W mount) $ vim /cvmfs/my-cvmfs.cern.ch/lcg/60/ some_file.org $ cvmfs_server publish

22 CVMFS SERVICE ARCHITECTURE 22 CVMFS STORAGE GATEWAY Serves as a distributed lock manager Checks rights of clients to modify repositories Assigns exclusive leases to clients on repository subpaths Receives files (object packs) from clients, writes them to authoritative storage

23 CVMFS SERVICES IMPLEMENTATION 23 ERLANG/OTP: DISTRIBUTED GLUE Language (Erlang) and framework (OTP) designed for concurrent and distributed applications: Actor model: lightweight processes with memory isolation Immutability of values Supervision trees Erlang/OTP/BEAM are battle-tested, 30+ years of use at Ericsson Excellent C/C++ interoperability

24 CVMFS SERVICES IMPLEMENTATION 24 GATEWAY APPLICATION ARCHITECTURE HTTP FRONT-END (COWBOY) BACK-END (MULTIPLEXER) AUTH LEASE RECEIVER (WORKER POOL) PERSIST (MNESIA) WORKER (C++) WORKER (C++) WORKER (C++)

25 CVMFS SERVICES IMPLEMENTATION 25 DEVELOPER EXPERIENCE WITH ERLANG/OTP Great: OTP Tracing, inspection etc. Immutability, Functional language Very simple to write concurrent programs Use Dialyzer, CommonTest, QuickCheck etc. Easy integration with C++

26 CVMFS SERVICES IMPLEMENTATION 26 DEVELOPER EXPERIENCE WITH ERLANG/OTP Less great: Dynamic typing is strange, coming from C++ Deciphering Erlang errors is an acquired taste (use Lager for logging) Large APIs in OTP, some parts feel less clearly documented

27 TEXT 27 DEVELOPER EXPERIENCE WITH ERLANG/OTP Overall impression is very positive! Would definitely use it for other new components Looking forward to more operational experience

28 OTHER CERNVM-FS PROJECTS AND ACTIVITIES 28 DOCKER GRAPHDRIVER PLUGIN Docker Graphdriver plugin for CernVM-FS (Nikola Hardi): Store the contents of Docker image layers inside CernVM FS repositories Instead of having to download the entire layers, mount a CernVM FS repository and download individual files on-demand

29 OTHER CERNVM-FS PROJECTS AND ACTIVITIES 29 CERN VM 10TH ANNIVERSARY! Next year, CernVM is turning 10 Jan 30th -> Feb 1st 2018: CernVM Open to anyone Talks by users and developers of CernVM and related projects

30 30 THE CERNVM TEAM (LEFT TO RIGHT) Radu Popescu Jakob Blomer Gerardo Ganis Petr Jirout (former) Nikola Hardi (former)

31 TEXT 31 THANK YOU CernVM-FS:

32 ERLANG/OTP CONCURRENCY PATTERNS 32 CRITICAL SECTIONS Erlang (only) provides processes and message passing for concurrency No locks, semaphores, condition variables etc. What if a exclusive access to a resource is needed? OTP gen_server works as a critical section

33 ERLANG/OTP CONCURRENCY PATTERNS 33 MULTIPLEXING REQUESTS/REPLIES ON GEN_SERVER OTP gen_server with concurrency? In gen_server:handle_call, spawn a process per request, and return {noreply, } The spawned process later returns a value with gen_server:reply. Does not maintain order of requests Concurrency adaptor between Cowboy and C++ worker pool

Global Software Distribution with CernVM-FS

Global Software Distribution with CernVM-FS Global Software Distribution with CernVM-FS Jakob Blomer CERN 2016 CCL Workshop on Scalable Computing October 19th, 2016 jblomer@cern.ch CernVM-FS 1 / 15 The Anatomy of a Scientific Software Stack (In

More information

STATUS OF PLANS TO USE CONTAINERS IN THE WORLDWIDE LHC COMPUTING GRID

STATUS OF PLANS TO USE CONTAINERS IN THE WORLDWIDE LHC COMPUTING GRID The WLCG Motivation and benefits Container engines Experiments status and plans Security considerations Summary and outlook STATUS OF PLANS TO USE CONTAINERS IN THE WORLDWIDE LHC COMPUTING GRID SWISS EXPERIENCE

More information

Recent Developments in the CernVM-FS Server Backend

Recent Developments in the CernVM-FS Server Backend Recent Developments in the CernVM-FS Server Backend René Meusel Jakob Blomer, Gerardo Ganis, Predrag Buncic, Seppo Heikkila ACAT 2014 - Prague, 4th of September 1 Usage Statistics and Adoption 2 New Challenges

More information

CernVM-FS beyond LHC computing

CernVM-FS beyond LHC computing CernVM-FS beyond LHC computing C Condurache, I Collier STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot, OX11 0QX, UK E-mail: catalin.condurache@stfc.ac.uk Abstract. In the last three years

More information

Recent Developments in the CernVM-File System Server Backend

Recent Developments in the CernVM-File System Server Backend Journal of Physics: Conference Series PAPER OPEN ACCESS Recent Developments in the CernVM-File System Server Backend To cite this article: R Meusel et al 2015 J. Phys.: Conf. Ser. 608 012031 Recent citations

More information

Using CernVM-FS to deploy Euclid processing S/W on Science Data Centres

Using CernVM-FS to deploy Euclid processing S/W on Science Data Centres Using CernVM-FS to deploy Euclid processing S/W on Science Data Centres M. Poncet (CNES) Q. Le Boulc h (IN2P3) M. Holliman (ROE) On behalf of Euclid EC SGS System Team ADASS 2016 1 Outline Euclid Project

More information

Application of Virtualization Technologies & CernVM. Benedikt Hegner CERN

Application of Virtualization Technologies & CernVM. Benedikt Hegner CERN Application of Virtualization Technologies & CernVM Benedikt Hegner CERN Virtualization Use Cases Worker Node Virtualization Software Testing Training Platform Software Deployment }Covered today Server

More information

Worldwide Production Distributed Data Management at the LHC. Brian Bockelman MSST 2010, 4 May 2010

Worldwide Production Distributed Data Management at the LHC. Brian Bockelman MSST 2010, 4 May 2010 Worldwide Production Distributed Data Management at the LHC Brian Bockelman MSST 2010, 4 May 2010 At the LHC http://op-webtools.web.cern.ch/opwebtools/vistar/vistars.php?usr=lhc1 Gratuitous detector pictures:

More information

CouchDB-based system for data management in a Grid environment Implementation and Experience

CouchDB-based system for data management in a Grid environment Implementation and Experience CouchDB-based system for data management in a Grid environment Implementation and Experience Hassen Riahi IT/SDC, CERN Outline Context Problematic and strategy System architecture Integration and deployment

More information

CERN openlab II. CERN openlab and. Sverre Jarp CERN openlab CTO 16 September 2008

CERN openlab II. CERN openlab and. Sverre Jarp CERN openlab CTO 16 September 2008 CERN openlab II CERN openlab and Intel: Today and Tomorrow Sverre Jarp CERN openlab CTO 16 September 2008 Overview of CERN 2 CERN is the world's largest particle physics centre What is CERN? Particle physics

More information

Storage Resource Sharing with CASTOR.

Storage Resource Sharing with CASTOR. Storage Resource Sharing with CASTOR Olof Barring, Benjamin Couturier, Jean-Damien Durand, Emil Knezo, Sebastien Ponce (CERN) Vitali Motyakov (IHEP) ben.couturier@cern.ch 16/4/2004 Storage Resource Sharing

More information

Singularity in CMS. Over a million containers served

Singularity in CMS. Over a million containers served Singularity in CMS Over a million containers served Introduction The topic of containers is broad - and this is a 15 minute talk! I m filtering out a lot of relevant details, particularly why we are using

More information

Volunteer Computing at CERN

Volunteer Computing at CERN Volunteer Computing at CERN BOINC workshop Sep 2014, Budapest Tomi Asp & Pete Jones, on behalf the LHC@Home team Agenda Overview Status of the LHC@Home projects Additional BOINC projects Service consolidation

More information

Virtualizing a Batch. University Grid Center

Virtualizing a Batch. University Grid Center Virtualizing a Batch Queuing System at a University Grid Center Volker Büge (1,2), Yves Kemp (1), Günter Quast (1), Oliver Oberst (1), Marcel Kunze (2) (1) University of Karlsruhe (2) Forschungszentrum

More information

Geant4 on Azure using Docker containers

Geant4 on Azure using Docker containers http://www.geant4.org Geant4 on Azure using Docker containers Andrea Dotti (adotti@slac.stanford.edu) ; SD/EPP/Computing 1 Outlook Motivation/overview Docker + G4 Azure + G4 Conclusions 2 Motivation/overview

More information

THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES

THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES 1 THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES Vincent Garonne, Mario Lassnig, Martin Barisits, Thomas Beermann, Ralph Vigne, Cedric Serfon Vincent.Garonne@cern.ch ph-adp-ddm-lab@cern.ch XLDB

More information

From raw data to new fundamental particles: The data management lifecycle at the Large Hadron Collider

From raw data to new fundamental particles: The data management lifecycle at the Large Hadron Collider From raw data to new fundamental particles: The data management lifecycle at the Large Hadron Collider Andrew Washbrook School of Physics and Astronomy University of Edinburgh Dealing with Data Conference

More information

PROOF-Condor integration for ATLAS

PROOF-Condor integration for ATLAS PROOF-Condor integration for ATLAS G. Ganis,, J. Iwaszkiewicz, F. Rademakers CERN / PH-SFT M. Livny, B. Mellado, Neng Xu,, Sau Lan Wu University Of Wisconsin Condor Week, Madison, 29 Apr 2 May 2008 Outline

More information

Scientific data processing at global scale The LHC Computing Grid. fabio hernandez

Scientific data processing at global scale The LHC Computing Grid. fabio hernandez Scientific data processing at global scale The LHC Computing Grid Chengdu (China), July 5th 2011 Who I am 2 Computing science background Working in the field of computing for high-energy physics since

More information

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of

More information

Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF

Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment

More information

Distributed File Systems II

Distributed File Systems II Distributed File Systems II To do q Very-large scale: Google FS, Hadoop FS, BigTable q Next time: Naming things GFS A radically new environment NFS, etc. Independence Small Scale Variety of workloads Cooperation

More information

CSCS CERN videoconference CFD applications

CSCS CERN videoconference CFD applications CSCS CERN videoconference CFD applications TS/CV/Detector Cooling - CFD Team CERN June 13 th 2006 Michele Battistin June 2006 CERN & CFD Presentation 1 TOPICS - Some feedback about already existing collaboration

More information

Evaluation of the Huawei UDS cloud storage system for CERN specific data

Evaluation of the Huawei UDS cloud storage system for CERN specific data th International Conference on Computing in High Energy and Nuclear Physics (CHEP3) IOP Publishing Journal of Physics: Conference Series 53 (4) 44 doi:.88/74-6596/53/4/44 Evaluation of the Huawei UDS cloud

More information

Distributed Systems 16. Distributed File Systems II

Distributed Systems 16. Distributed File Systems II Distributed Systems 16. Distributed File Systems II Paul Krzyzanowski pxk@cs.rutgers.edu 1 Review NFS RPC-based access AFS Long-term caching CODA Read/write replication & disconnected operation DFS AFS

More information

Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine

Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine Journal of Physics: Conference Series OPEN ACCESS Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine To cite this article: Henrik Öhman et al 2014 J. Phys.: Conf.

More information

and the GridKa mass storage system Jos van Wezel / GridKa

and the GridKa mass storage system Jos van Wezel / GridKa and the GridKa mass storage system / GridKa [Tape TSM] staging server 2 Introduction Grid storage and storage middleware dcache h and TSS TSS internals Conclusion and further work 3 FZK/GridKa The GridKa

More information

Using S3 cloud storage with ROOT and CvmFS

Using S3 cloud storage with ROOT and CvmFS Journal of Physics: Conference Series PAPER OPEN ACCESS Using S cloud storage with ROOT and CvmFS To cite this article: María Arsuaga-Ríos et al 05 J. Phys.: Conf. Ser. 66 000 View the article online for

More information

Overview. About CERN 2 / 11

Overview. About CERN 2 / 11 Overview CERN wanted to upgrade the data monitoring system of one of its Large Hadron Collider experiments called ALICE (A La rge Ion Collider Experiment) to ensure the experiment s high efficiency. They

More information

ISTITUTO NAZIONALE DI FISICA NUCLEARE

ISTITUTO NAZIONALE DI FISICA NUCLEARE ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Perugia INFN/TC-05/10 July 4, 2005 DESIGN, IMPLEMENTATION AND CONFIGURATION OF A GRID SITE WITH A PRIVATE NETWORK ARCHITECTURE Leonello Servoli 1,2!, Mirko

More information

Batch Services at CERN: Status and Future Evolution

Batch Services at CERN: Status and Future Evolution Batch Services at CERN: Status and Future Evolution Helge Meinhard, CERN-IT Platform and Engineering Services Group Leader HTCondor Week 20 May 2015 20-May-2015 CERN batch status and evolution - Helge

More information

The CMS Computing Model

The CMS Computing Model The CMS Computing Model Dorian Kcira California Institute of Technology SuperComputing 2009 November 14-20 2009, Portland, OR CERN s Large Hadron Collider 5000+ Physicists/Engineers 300+ Institutes 70+

More information

Security in the CernVM File System and the Frontier Distributed Database Caching System

Security in the CernVM File System and the Frontier Distributed Database Caching System Security in the CernVM File System and the Frontier Distributed Database Caching System D Dykstra 1 and J Blomer 2 1 Scientific Computing Division, Fermilab, Batavia, IL 60510, USA 2 PH-SFT Department,

More information

The LHC Computing Grid

The LHC Computing Grid The LHC Computing Grid Visit of Finnish IT Centre for Science CSC Board Members Finland Tuesday 19 th May 2009 Frédéric Hemmer IT Department Head The LHC and Detectors Outline Computing Challenges Current

More information

Storage and I/O requirements of the LHC experiments

Storage and I/O requirements of the LHC experiments Storage and I/O requirements of the LHC experiments Sverre Jarp CERN openlab, IT Dept where the Web was born 22 June 2006 OpenFabrics Workshop, Paris 1 Briefly about CERN 22 June 2006 OpenFabrics Workshop,

More information

N. Marusov, I. Semenov

N. Marusov, I. Semenov GRID TECHNOLOGY FOR CONTROLLED FUSION: CONCEPTION OF THE UNIFIED CYBERSPACE AND ITER DATA MANAGEMENT N. Marusov, I. Semenov Project Center ITER (ITER Russian Domestic Agency N.Marusov@ITERRF.RU) Challenges

More information

The CORAL Project. Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008

The CORAL Project. Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008 The CORAL Project Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008 Outline CORAL - a foundation for Physics Database Applications in the LHC Computing Grid (LCG)

More information

13th International Workshop on Advanced Computing and Analysis Techniques in Physics Research ACAT 2010 Jaipur, India February

13th International Workshop on Advanced Computing and Analysis Techniques in Physics Research ACAT 2010 Jaipur, India February LHC Cloud Computing with CernVM Ben Segal 1 CERN 1211 Geneva 23, Switzerland E mail: b.segal@cern.ch Predrag Buncic CERN E mail: predrag.buncic@cern.ch 13th International Workshop on Advanced Computing

More information

Evolution of Cloud Computing in ATLAS

Evolution of Cloud Computing in ATLAS The Evolution of Cloud Computing in ATLAS Ryan Taylor on behalf of the ATLAS collaboration 1 Outline Cloud Usage and IaaS Resource Management Software Services to facilitate cloud use Sim@P1 Performance

More information

Docker 101 Workshop. Eric Smalling - Solution Architect, Docker

Docker 101 Workshop. Eric Smalling - Solution Architect, Docker Docker 101 Workshop Eric Smalling - Solution Architect, Docker Inc. @ericsmalling Who Am I? Eric Smalling Solution Architect Docker Customer Success Team ~25 years in software development, architecture,

More information

Travelling securely on the Grid to the origin of the Universe

Travelling securely on the Grid to the origin of the Universe 1 Travelling securely on the Grid to the origin of the Universe F-Secure SPECIES 2007 conference Wolfgang von Rüden 1 Head, IT Department, CERN, Geneva 24 January 2007 2 CERN stands for over 50 years of

More information

Towards Reproducible Research Data Analyses in LHC Particle Physics

Towards Reproducible Research Data Analyses in LHC Particle Physics Towards Reproducible Research Data Analyses in LHC Particle Physics Tibor Šimko CERN ILIDE 2017 Jasná, Slovakia 3 5 April 2017 @tiborsimko 1 / 30 Preserving research data @tiborsimko 2 / 30 Why? Scientific

More information

Monitoring system for geographically distributed datacenters based on Openstack. Gioacchino Vino

Monitoring system for geographically distributed datacenters based on Openstack. Gioacchino Vino Monitoring system for geographically distributed datacenters based on Openstack Gioacchino Vino Tutor: Dott. Domenico Elia Tutor: Dott. Giacinto Donvito Borsa di studio GARR Orio Carlini 2016-2017 INFN

More information

News From the OTP TEAM. Kenneth Lundin, Erlang/OTP, Ericsson Erlang User Conference, Stockholm 2017

News From the OTP TEAM. Kenneth Lundin, Erlang/OTP, Ericsson Erlang User Conference, Stockholm 2017 News From the OTP TEAM Kenneth Lundin, Erlang/OTP, Ericsson Erlang User Conference, Stockholm 17 1 Outline Planned Releases Next major release () OTP 21 and beyond Public 17-06-08 Page 2 2 Planned Releases

More information

Opportunities for container environments on Cray XC30 with GPU devices

Opportunities for container environments on Cray XC30 with GPU devices Opportunities for container environments on Cray XC30 with GPU devices Cray User Group 2016, London Sadaf Alam, Lucas Benedicic, T. Schulthess, Miguel Gila May 12, 2016 Agenda Motivation Container technologies,

More information

CernVM a virtual software appliance for LHC applications

CernVM a virtual software appliance for LHC applications CernVM a virtual software appliance for LHC applications P Buncic 1, C Aguado Sanchez 1, J Blomer 1, L Franco 1, A Harutyunian 2,3, P Mato 1, Y Yao 3 1 CERN, 1211 Geneve 23, Geneva, Switzerland 2 Armenian

More information

Grid Data Management

Grid Data Management Grid Data Management Week #4 Hardi Teder hardi@eenet.ee University of Tartu March 6th 2013 Overview Grid Data Management Where the Data comes from? Grid Data Management tools 2/33 Grid foundations 3/33

More information

CernVM-FS. Catalin Condurache STFC RAL UK

CernVM-FS. Catalin Condurache STFC RAL UK CernVM-FS Catalin Condurache STFC RAL UK Outline Introduction Brief history EGI CernVM-FS infrastructure The users Recent developments Plans 2 Outline Introduction Brief history EGI CernVM-FS infrastructure

More information

WORK PROJECT REPORT: TAPE STORAGE AND CRC PROTECTION

WORK PROJECT REPORT: TAPE STORAGE AND CRC PROTECTION WORK PROJECT REPORT: TAPE STORAGE AND CRC PROTECTION CERN Summer Student Programme 2014 Student: Main supervisor: Second supervisor: Division: Karel Ha Julien Marcel Leduc

More information

BUILDING A SCALABLE MOBILE GAME BACKEND IN ELIXIR. Petri Kero CTO / Ministry of Games

BUILDING A SCALABLE MOBILE GAME BACKEND IN ELIXIR. Petri Kero CTO / Ministry of Games BUILDING A SCALABLE MOBILE GAME BACKEND IN ELIXIR Petri Kero CTO / Ministry of Games MOBILE GAME BACKEND CHALLENGES Lots of concurrent users Complex interactions between players Persistent world with frequent

More information

The evolving role of Tier2s in ATLAS with the new Computing and Data Distribution model

The evolving role of Tier2s in ATLAS with the new Computing and Data Distribution model Journal of Physics: Conference Series The evolving role of Tier2s in ATLAS with the new Computing and Data Distribution model To cite this article: S González de la Hoz 2012 J. Phys.: Conf. Ser. 396 032050

More information

The Lion of storage systems

The Lion of storage systems The Lion of storage systems Rakuten. Inc, Yosuke Hara Mar 21, 2013 1 The Lion of storage systems http://www.leofs.org LeoFS v0.14.0 was released! 2 Table of Contents 1. Motivation 2. Overview & Inside

More information

Data services for LHC computing

Data services for LHC computing Data services for LHC computing SLAC 1 Xavier Espinal on behalf of IT/ST DAQ to CC 8GB/s+4xReco Hot files Reliable Fast Processing DAQ Feedback loop WAN aware Tier-1/2 replica, multi-site High throughout

More information

Modules and Front-End Electronics Developments for the ATLAS ITk Strips Upgrade

Modules and Front-End Electronics Developments for the ATLAS ITk Strips Upgrade Modules and Front-End Electronics Developments for the ATLAS ITk Strips Upgrade Carlos García Argos, on behalf of the ATLAS ITk Collaboration University of Freiburg International Conference on Technology

More information

ATLAS Experiment and GCE

ATLAS Experiment and GCE ATLAS Experiment and GCE Google IO Conference San Francisco, CA Sergey Panitkin (BNL) and Andrew Hanushevsky (SLAC), for the ATLAS Collaboration ATLAS Experiment The ATLAS is one of the six particle detectors

More information

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9b: Distributed File Systems INTRODUCTION. Transparency: Flexibility: Slide 1. Slide 3.

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9b: Distributed File Systems INTRODUCTION. Transparency: Flexibility: Slide 1. Slide 3. CHALLENGES Transparency: Slide 1 DISTRIBUTED SYSTEMS [COMP9243] Lecture 9b: Distributed File Systems ➀ Introduction ➁ NFS (Network File System) ➂ AFS (Andrew File System) & Coda ➃ GFS (Google File System)

More information

Big Data Analytics and the LHC

Big Data Analytics and the LHC Big Data Analytics and the LHC Maria Girone CERN openlab CTO Computing Frontiers 2016, Como, May 2016 DOI: 10.5281/zenodo.45449, CC-BY-SA, images courtesy of CERN 2 3 xx 4 Big bang in the laboratory We

More information

CSE 124: Networked Services Lecture-16

CSE 124: Networked Services Lecture-16 Fall 2010 CSE 124: Networked Services Lecture-16 Instructor: B. S. Manoj, Ph.D http://cseweb.ucsd.edu/classes/fa10/cse124 11/23/2010 CSE 124 Networked Services Fall 2010 1 Updates PlanetLab experiments

More information

Software installation and condition data distribution via CernVM File System in ATLAS

Software installation and condition data distribution via CernVM File System in ATLAS Journal of Physics: Conference Series Software installation and condition data distribution via CernVM File System in ATLAS To cite this article: A De Salvo et al 2012 J. Phys.: Conf. Ser. 396 032030 View

More information

DISTRIBUTED FILE SYSTEMS CARSTEN WEINHOLD

DISTRIBUTED FILE SYSTEMS CARSTEN WEINHOLD Department of Computer Science Institute of System Architecture, Operating Systems Group DISTRIBUTED FILE SYSTEMS CARSTEN WEINHOLD OUTLINE Classical distributed file systems NFS: Sun Network File System

More information

Interoperating AliEn and ARC for a distributed Tier1 in the Nordic countries.

Interoperating AliEn and ARC for a distributed Tier1 in the Nordic countries. for a distributed Tier1 in the Nordic countries. Philippe Gros Lund University, Div. of Experimental High Energy Physics, Box 118, 22100 Lund, Sweden philippe.gros@hep.lu.se Anders Rhod Gregersen NDGF

More information

The EU DataGrid Testbed

The EU DataGrid Testbed The EU DataGrid Testbed The European DataGrid Project Team http://www.eudatagrid.org DataGrid is a project funded by the European Union Grid Tutorial 4/3/2004 n 1 Contents User s Perspective of the Grid

More information

CERN s Business Computing

CERN s Business Computing CERN s Business Computing Where Accelerated the infinitely by Large Pentaho Meets the Infinitely small Jan Janke Deputy Group Leader CERN Administrative Information Systems Group CERN World s Leading Particle

More information

The Grid: Processing the Data from the World s Largest Scientific Machine

The Grid: Processing the Data from the World s Largest Scientific Machine The Grid: Processing the Data from the World s Largest Scientific Machine 10th Topical Seminar On Innovative Particle and Radiation Detectors Siena, 1-5 October 2006 Patricia Méndez Lorenzo (IT-PSS/ED),

More information

LHCb experience running jobs in virtual machines

LHCb experience running jobs in virtual machines LHCb experience running jobs in virtual machines Andrew McNab, University of Manchester Federico Stagni & Cinzia Luzzi, CERN on behalf of the LHCb collaboration Overview Starting from DIRAC + Grid CernVM

More information

WLCG Transfers Dashboard: a Unified Monitoring Tool for Heterogeneous Data Transfers.

WLCG Transfers Dashboard: a Unified Monitoring Tool for Heterogeneous Data Transfers. WLCG Transfers Dashboard: a Unified Monitoring Tool for Heterogeneous Data Transfers. J Andreeva 1, A Beche 1, S Belov 2, I Kadochnikov 2, P Saiz 1 and D Tuckett 1 1 CERN (European Organization for Nuclear

More information

DISTRIBUTED FILE SYSTEMS CARSTEN WEINHOLD

DISTRIBUTED FILE SYSTEMS CARSTEN WEINHOLD Department of Computer Science Institute of System Architecture, Operating Systems Group DISTRIBUTED FILE SYSTEMS CARSTEN WEINHOLD OUTLINE Classical distributed file systems NFS: Sun Network File System

More information

The LHC Computing Grid

The LHC Computing Grid The LHC Computing Grid Gergely Debreczeni (CERN IT/Grid Deployment Group) The data factory of LHC 40 million collisions in each second After on-line triggers and selections, only 100 3-4 MB/event requires

More information

Andrea Sciabà CERN, Switzerland

Andrea Sciabà CERN, Switzerland Frascati Physics Series Vol. VVVVVV (xxxx), pp. 000-000 XX Conference Location, Date-start - Date-end, Year THE LHC COMPUTING GRID Andrea Sciabà CERN, Switzerland Abstract The LHC experiments will start

More information

Preparing for High-Luminosity LHC. Bob Jones CERN Bob.Jones <at> cern.ch

Preparing for High-Luminosity LHC. Bob Jones CERN Bob.Jones <at> cern.ch Preparing for High-Luminosity LHC Bob Jones CERN Bob.Jones cern.ch The Mission of CERN Push back the frontiers of knowledge E.g. the secrets of the Big Bang what was the matter like within the first

More information

Data Transfers Between LHC Grid Sites Dorian Kcira

Data Transfers Between LHC Grid Sites Dorian Kcira Data Transfers Between LHC Grid Sites Dorian Kcira dkcira@caltech.edu Caltech High Energy Physics Group hep.caltech.edu/cms CERN Site: LHC and the Experiments Large Hadron Collider 27 km circumference

More information

CC-IN2P3: A High Performance Data Center for Research

CC-IN2P3: A High Performance Data Center for Research April 15 th, 2011 CC-IN2P3: A High Performance Data Center for Research Toward a partnership with DELL Dominique Boutigny Agenda Welcome Introduction to CC-IN2P3 Visit of the computer room Lunch Discussion

More information

State of Containers. Convergence of Big Data, AI and HPC

State of Containers. Convergence of Big Data, AI and HPC State of Containers Convergence of Big Data, AI and HPC Technology ReCap Comparison of Hypervisor and Container Virtualization VM1 VM2 appa appb Userland Userland Kernel Kernel Operational Abstraction

More information

Large Scale Software Building with CMake in ATLAS

Large Scale Software Building with CMake in ATLAS 1 Large Scale Software Building with CMake in ATLAS 2 3 4 5 6 7 J Elmsheuser 1, A Krasznahorkay 2, E Obreshkov 3, A Undrus 1 on behalf of the ATLAS Collaboration 1 Brookhaven National Laboratory, USA 2

More information

Construct a High Efficiency VM Disaster Recovery Solution. Best choice for protecting virtual environments

Construct a High Efficiency VM Disaster Recovery Solution. Best choice for protecting virtual environments Construct a High Efficiency VM Disaster Recovery Solution Best choice for protecting virtual environments About NAKIVO Established in the USA since 2012 Provides data protection solutions for VMware, Hyper-V

More information

Changing Requirements for Distributed File Systems in Cloud Storage

Changing Requirements for Distributed File Systems in Cloud Storage Changing Requirements for Distributed File Systems in Cloud Storage Wesley Leggette Cleversafe Presentation Agenda r About Cleversafe r Scalability, our core driver r Object storage as basis for filesystem

More information

Computing at the Large Hadron Collider. Frank Würthwein. Professor of Physics University of California San Diego November 15th, 2013

Computing at the Large Hadron Collider. Frank Würthwein. Professor of Physics University of California San Diego November 15th, 2013 Computing at the Large Hadron Collider Frank Würthwein Professor of Physics of California San Diego November 15th, 2013 Outline The Science Software & Computing Challenges Present Solutions Future Solutions

More information

CLOUD-SCALE FILE SYSTEMS

CLOUD-SCALE FILE SYSTEMS Data Management in the Cloud CLOUD-SCALE FILE SYSTEMS 92 Google File System (GFS) Designing a file system for the Cloud design assumptions design choices Architecture GFS Master GFS Chunkservers GFS Clients

More information

Storage Virtualization. Eric Yen Academia Sinica Grid Computing Centre (ASGC) Taiwan

Storage Virtualization. Eric Yen Academia Sinica Grid Computing Centre (ASGC) Taiwan Storage Virtualization Eric Yen Academia Sinica Grid Computing Centre (ASGC) Taiwan Storage Virtualization In computer science, storage virtualization uses virtualization to enable better functionality

More information

18-hdfs-gfs.txt Thu Oct 27 10:05: Notes on Parallel File Systems: HDFS & GFS , Fall 2011 Carnegie Mellon University Randal E.

18-hdfs-gfs.txt Thu Oct 27 10:05: Notes on Parallel File Systems: HDFS & GFS , Fall 2011 Carnegie Mellon University Randal E. 18-hdfs-gfs.txt Thu Oct 27 10:05:07 2011 1 Notes on Parallel File Systems: HDFS & GFS 15-440, Fall 2011 Carnegie Mellon University Randal E. Bryant References: Ghemawat, Gobioff, Leung, "The Google File

More information

New strategies of the LHC experiments to meet the computing requirements of the HL-LHC era

New strategies of the LHC experiments to meet the computing requirements of the HL-LHC era to meet the computing requirements of the HL-LHC era NPI AS CR Prague/Rez E-mail: adamova@ujf.cas.cz Maarten Litmaath CERN E-mail: Maarten.Litmaath@cern.ch The performance of the Large Hadron Collider

More information

Bigtable: A Distributed Storage System for Structured Data By Fay Chang, et al. OSDI Presented by Xiang Gao

Bigtable: A Distributed Storage System for Structured Data By Fay Chang, et al. OSDI Presented by Xiang Gao Bigtable: A Distributed Storage System for Structured Data By Fay Chang, et al. OSDI 2006 Presented by Xiang Gao 2014-11-05 Outline Motivation Data Model APIs Building Blocks Implementation Refinement

More information

Grid Computing: dealing with GB/s dataflows

Grid Computing: dealing with GB/s dataflows Grid Computing: dealing with GB/s dataflows Jan Just Keijser, Nikhef janjust@nikhef.nl David Groep, NIKHEF 21 March 2011 Graphics: Real Time Monitor, Gidon Moont, Imperial College London, see http://gridportal.hep.ph.ic.ac.uk/rtm/

More information

Efficient HTTP based I/O on very large datasets for high performance computing with the Libdavix library

Efficient HTTP based I/O on very large datasets for high performance computing with the Libdavix library Efficient HTTP based I/O on very large datasets for high performance computing with the Libdavix library Authors Devresse Adrien (CERN) Fabrizio Furano (CERN) Typical HPC architecture Computing Cluster

More information

ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective

ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective Part II: Data Center Software Architecture: Topic 1: Distributed File Systems GFS (The Google File System) 1 Filesystems

More information

Data Management for the World s Largest Machine

Data Management for the World s Largest Machine Data Management for the World s Largest Machine Sigve Haug 1, Farid Ould-Saada 2, Katarina Pajchel 2, and Alexander L. Read 2 1 Laboratory for High Energy Physics, University of Bern, Sidlerstrasse 5,

More information

Visita delegazione ditte italiane

Visita delegazione ditte italiane Visita delegazione ditte italiane CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Massimo Lamanna/CERN IT department - Data Storage Services group Innovation in Computing in High-Energy

More information

Getting Started with Hadoop

Getting Started with Hadoop Getting Started with Hadoop May 28, 2018 Michael Völske, Shahbaz Syed Web Technology & Information Systems Bauhaus-Universität Weimar 1 webis 2018 What is Hadoop Started in 2004 by Yahoo Open-Source implementation

More information

The CMS data quality monitoring software: experience and future prospects

The CMS data quality monitoring software: experience and future prospects The CMS data quality monitoring software: experience and future prospects Federico De Guio on behalf of the CMS Collaboration CERN, Geneva, Switzerland E-mail: federico.de.guio@cern.ch Abstract. The Data

More information

Summary of the LHC Computing Review

Summary of the LHC Computing Review Summary of the LHC Computing Review http://lhc-computing-review-public.web.cern.ch John Harvey CERN/EP May 10 th, 2001 LHCb Collaboration Meeting The Scale Data taking rate : 50,100, 200 Hz (ALICE, ATLAS-CMS,

More information

ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective

ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective Part II: Software Infrastructure in Data Centers: Distributed File Systems 1 Permanently stores data Filesystems

More information

Stephen J. Gowdy (CERN) 12 th September 2012 XLDB Conference FINDING THE HIGGS IN THE HAYSTACK(S)

Stephen J. Gowdy (CERN) 12 th September 2012 XLDB Conference FINDING THE HIGGS IN THE HAYSTACK(S) Stephen J. Gowdy (CERN) 12 th September 2012 XLDB Conference FINDING THE HIGGS IN THE HAYSTACK(S) Overview Large Hadron Collider (LHC) Compact Muon Solenoid (CMS) experiment The Challenge Worldwide LHC

More information

CS November 2017

CS November 2017 Bigtable Highly available distributed storage Distributed Systems 18. Bigtable Built with semi-structured data in mind URLs: content, metadata, links, anchors, page rank User data: preferences, account

More information

Weak Consistency and Disconnected Operation in git. Raymond Cheng

Weak Consistency and Disconnected Operation in git. Raymond Cheng Weak Consistency and Disconnected Operation in git Raymond Cheng ryscheng@cs.washington.edu Motivation How can we support disconnected or weakly connected operation? Applications File synchronization across

More information

Distributing Software in a Massively Parallel Environment

Distributing Software in a Massively Parallel Environment Distributing Software in a Massively Parallel Environment LISA 2014 Dinah McNutt Release Engineer, Google, Inc. November 12, 2014 Problem: Reliably and consistently distributing software in a Laaaaaaaaaaaarge

More information

CSE 124: Networked Services Fall 2009 Lecture-19

CSE 124: Networked Services Fall 2009 Lecture-19 CSE 124: Networked Services Fall 2009 Lecture-19 Instructor: B. S. Manoj, Ph.D http://cseweb.ucsd.edu/classes/fa09/cse124 Some of these slides are adapted from various sources/individuals including but

More information

CA485 Ray Walshe Google File System

CA485 Ray Walshe Google File System Google File System Overview Google File System is scalable, distributed file system on inexpensive commodity hardware that provides: Fault Tolerance File system runs on hundreds or thousands of storage

More information

The creation of a Tier-1 Data Center for the ALICE experiment in the UNAM. Lukas Nellen ICN-UNAM

The creation of a Tier-1 Data Center for the ALICE experiment in the UNAM. Lukas Nellen ICN-UNAM The creation of a Tier-1 Data Center for the ALICE experiment in the UNAM Lukas Nellen ICN-UNAM lukas@nucleares.unam.mx 3rd BigData BigNetworks Conference Puerto Vallarta April 23, 2015 Who Am I? ALICE

More information

RUSSIAN DATA INTENSIVE GRID (RDIG): CURRENT STATUS AND PERSPECTIVES TOWARD NATIONAL GRID INITIATIVE

RUSSIAN DATA INTENSIVE GRID (RDIG): CURRENT STATUS AND PERSPECTIVES TOWARD NATIONAL GRID INITIATIVE RUSSIAN DATA INTENSIVE GRID (RDIG): CURRENT STATUS AND PERSPECTIVES TOWARD NATIONAL GRID INITIATIVE Viacheslav Ilyin Alexander Kryukov Vladimir Korenkov Yuri Ryabov Aleksey Soldatov (SINP, MSU), (SINP,

More information

arxiv: v1 [cs.dc] 7 Apr 2014

arxiv: v1 [cs.dc] 7 Apr 2014 arxiv:1404.1814v1 [cs.dc] 7 Apr 2014 CernVM Online and Cloud Gateway: a uniform interface for CernVM contextualization and deployment G Lestaris 1, I Charalampidis 2, D Berzano, J Blomer, P Buncic, G Ganis

More information