Data Handling for LHC: Plans and Reality

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Data Handling for LHC: Plans and Reality"

Transcription

1 Data Handling for LHC: Plans and Reality Tony Cass Leader, Database Services Group Information Technology Department 11 th July

2 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique In outline In more detail Towards the Future Summary Outline 2

3 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique In outline In more detail Towards the Future Summary Outline 3

4 Familiar, but not Fundamental Periodic Table courtesy of wikipedia 4

5 5 5

6 The Standard Model Fundamental and well tested, but Why do particles have mass? Why is there no antimatter? Are these the only particles? 4 th Generation? LEP Discovery Do Fermions have bosonic partners and vice-versa? How does Gravity fit in? 6

7 Other interesting questions How do quarks LEP and gluons behave at ultra-high temperatures and densities? LHC What is dark matter? Supersymmetric particles? 7

8 How to find the answers? Smash things together! Images courtesy of hyperphysics 8

9 CERN Methodology The fastest racetrack on the planet Trillions of protons will race around the 27km ring in opposite directions over 11,000 times a second, travelling at per cent the speed of light. 9

10 Energy of a 1TeV Proton 10 10

11 Energy of 7TeV Beams Two nominal beams together can melt ~1,000kg of copper. Current beams: ~100kg of copper

12 CERN Methodology The emptiest space in the solar system To accelerate protons to almost the speed of light requires a vacuum as empty as interplanetary space. There is 10 times more atmosphere on the moon than there will be in the LHC. 12

13 CERN Methodology One of the coldest places in the universe With an operating temperature of about -271 degrees Celsius, just 1.9 degrees above absolute zero, the LHC is colder than outer space. 13

14 CERN Methodology The hottest spots in the galaxy When two beams of protons collide, they will generate temperatures 1000 million times hotter than the heart of the sun, but in a minuscule space. 14

15 CERN Methodology The biggest most sophisticated detectors ever built To sample and record the debris from up to 600 million proton collisions per second, scientists are building gargantuan devices that measure particles with micron precision. 15

16 Compact Detectors! 16

17 17

18

19 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique In outline In more detail Towards the Future Summary Outline 19

20 We are looking for rare events! number of events = Luminosity Cross section 2010 Luminosity: 45pb billion pb à 3 trillion events! * * N.B. only a very small fraction saved! ~250x more events to date Higgs (m H =120 GeV) : 17 pb à 750 events 7 e.g. potentially ~1 Higgs in every 300 billion interactions! Emily Nurse ATLAS 20

21 So the four LHC Experiments ATLAS - General purpose - Origin of mass - Supersymmetry - 2,000 scientists from 34 countries CMS - General purpose - Origin of mass - Supersymmetry - 1,800 scientists from over 150 institutes ALICE LHCb - heavy ion collisions, to create quark-gluon plasmas - to study the differences between matter and antimatter - 50,000 particles in each collision - will detect over 100 million b and b-bar mesons each year 21

22 So the four LHC Experiments 22

23 generate lots of data The accelerator generates 40 million particle collisions (events) every second at the centre of each of the four experiments detectors 23

24 generate lots of data CASTOR data wri.en, reduced by online computers to01/01/2010 to 29/6/2012 (in PB) 60 a few hundred good events per 50 second. 40 ATLAS Zà μμ event from 2012 data with 25 reconstructed vertices USER NTOF NA61 NA48 LHCB 30 COMPASS CMS 20 ATLAS Which are recorded on disk and magnetic tape Zà μμ at 100-1,000 MegaBytes/sec ~15 PetaBytes per year 0 for all four experiments 10 AMS ALICE Current forecast ~ PB / year, M files / year ~ 20-25K 1 TB tapes / year Archive will need to store 0.1 EB in 2014, ~1Billion files in

25 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique In outline In more detail Towards the Future Summary Outline 25

26 What is the technique? Break up a Massive Data Set 26

27 What is the technique? into lots of small pieces and distribute them around the world 27

28 What is the technique? analyse in parallel 28

29 What is the technique? gather the results 29

30 What is the technique? a and discover the Higgs boson: Nice result, but is it novel? 30

31 Is it Novel? Maybe not novel as such, but the implementation is Terrascale computing that is widely appreciated! 31

32 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique In outline In more detail Towards the Future Summary Outline 32

33 Requirements! Computing Challenges Summary of Computing Resource Requirements All experiments From 100,000 LCG TDR - June PCs PB/year to tape CERN All Tier-1s All Tier-2s Total CPU (MSPECint2000s) Disk O(100PB) (PetaBytes) disk cache Tape (PetaBytes) Worldwide Collaboration A Problem and a Solution Tier1s 4,000HS06 = 1MSPECint

34 Timely Technology! The WLCG project deployed to meet LHC computing needs. The EDG and EGEE projects organised development in Europe. (OSG and others in the US.) The Grid 34

35 Compute Element Grid Middleware Basics Standard interface to local workload management systems (batch scheduler) Storage Element Standard interface to local mass storage systems Resource Broker Tool to analyse user job requests (input data sets, cpu time, data output requirements) and route these to sites according to data and cpu time availability. Many implementations of the basic principles: Globus, VDT, EDG/EGEE, NorduGrid, OSG 35

36 Job Scheduling in Practice Issue Grid sites generally want to maintain a high average CPU utilisation; easiest to do this if there is a local queue of work to select from when another job ends. Users are generally interested in turnround times as well as job throughput. Turnround is reduced if jobs are held centrally until a processing slot is known to be free at a target site. Solution: Pilot job frameworks. Per-experiment code submits a job which chooses a work unit to run from a per-experiment queue when it is allocated an execution slot at a site. Pilot job frameworks separate out site responsibility for allocating CPU resources from Experiment responsibility for allocating priority between different research sub-groups

37 Data Issues Reception and long-term storage Delivery for processing and export Distribution Metadata distribution 700MB/s 700MB/s 420MB/s 2600MB/s (3600MB/s) (>4000MB/s) 1430MB/s Scheduled work only and we need ability to support 2x for recovery! 37

38 (Mass) Storage Systems After evaluation of commercial alternatives in the late 1990s, two tape-capable Mass storage systems have been developed for HEP: CASTOR: an integrated mass storage system dcache: a disk pool manager that interfaces to multiple tape archives FNAL, IBM s TSM) dcache is also used a basic disk storage manager Tier2s along with the simpler DPM 38

39 A Word About Tape Our data set may be massive, but CERN Archive file size distribution, in % ~195MB average only increasing slowly after LHC startup! It is made up of many small files Drive write performance, CASTOR tape format (ANSI AUL) which is bad for tape speeds: Write speed (KB/s) IBM AUL SUN AUL Average write drive speed: < 40MB/s (cf native drive speeds: MB/s) Small increases with new drive generations file size (MB) 39

40 Tape Drive Efficiency So we have to change tape writing policy Drive write performance, buffered vs nonbuffered tape marks 140 Average drive performance (MB/s) for CERN Archive files speed, MB/s CASTOR present (3sync/file) CASTOR new (1sync/ file) CASTOR future (1 sync / 4GB) file size, MB 0 3 sync/file 1 sync/file 1 sync / 4GB 40

41 Users aren t the only writers! Bulk data storage requires space! Fortunately Tape capacity Repack in 1 will year: continue to double every 2-3yrs 35 & 35 MB/s tape demonstrations in 2010 CERN has ~50K slots: ~0.25EB with new T10KC cartridges Unfortunately You have to copy data Repack from in 1 year: old cartridges to 500M-1G new or ~28 drives you run out of 63 MB/s 100M-500M 10M-100M Data rates for repack will soon Repack exceed in 1 year: LHC rates drive / days Repack in 1 year small files (<500M) 2012: 55PB = 1.7GB/s sustained 2015: 120PB = 3.8GB/s sustained 5000 time to migrate 55 PB (2012), drive/days, by file size ~55 drives ~ MB/s C.f. PP LHC rates: ~0.7GB/s; PbPb peak rate of 2.5GB/s And! 0 All LEP data fits on ~150 cartridges, or 30 new T10KCs 3 TM / file 1 TM / file TM / 4GB Automatic data duplication becomes a necessity >2G 1G-2G 1M-10M 100K-1M 10K-100K <10K 41 41

42 Media Verification Data in the archive cannot just be written and forgotten about. Q: can you retrieve my file? A: let me check err, sorry, we lost it. Proactive and regular verification of archive data required Ensure cartridges can be mounted Ensure data can be read and verified against metadata (checksum, size, ) Do not wait until media migration to detect problems Opportunistic scanning when resources available

43 Storage vs Recall Efficiency Efficient data acceptance: Have lots of input streams, spread across a number of storage servers, wait until the storage servers are ~full, and write the data from each storage server to tape. Result: data recorded at the same time is scattered over many tapes. How is the data read back? Generally, files grouped by time of creation. How to optimise for this? Group files on to a small number of tapes. Ooops 43 43

44 Keep users away from tape 44 44

45 CASTOR & EOS 45

46 Data Access Realism Mass Storage systems work well for recording, export and retrieval of production data. Good: This is what they were designed for! But some features of the CASTOR system developed at CERN are unused or ill-adapted experiments want to manage data availability file sizes, file-placement policies and access patterns interact badly alleviated by experiment management of data transfer between tape and disk analysis use favours low latency over guaranteed data rates aggravated by experiment management of data; automated replication of busy datasets is disabled. But we should not be too surprised: storage systems were designed many years before analysis patterns were understood. (If they are even today ) 46 46

47 Data Distribution The LHC experiments need to distribute millions of files between the different sites. The File Transfer System automates this handling failures of the underlying distribution technology (gridftp) ensuring effective use of the bandwidth with multiple streams, and managing the bandwidth use ensuring ATLAS, say, is guaranteed 50% of the available bandwidth between two sites if there is data to transfer 47

48 Data Distribution FTS uses the Storage Resource Manager as an abstract interface to the different storage systems A Good Idea but this is not (IMHO) a complete storage abstraction layer and anyway cannot hide fundamental differences in approaches to MSS design Lots of interest in the Amazon S3 interface these days; this doesn t try to do as much as SRM, but HEP should try to adopt de facto standards. Once you have distributed the data, a file catalogue is needed to record which files are available where. LFC, the LCG File Catalogue was designed for this role as a distributed catalogue to avoid a single point of failure, but other solutions are also used And as many other services rely on CERN, the need for a distributed catalogue is no longer (seen as ) so important. 48

49 Looking more widely I Only a small subset of data distributed is actually used Experiments don t know a priori which dataset will be popular CMS has 8 orders magnitude in access between most and least popular Dynamic data replication: create copies of popular datasets at multiple sites

50 University n.10 6 MIPS m Tbyte Robot" Looking more widely II Network capacity is readily available 622 Mbits/s" and it is reliable: FNAL MIPS Desk" 110 Tbyte So let s simply tops" Robot" copy data from another site if Desk" tops" it is not available locally rather than recalling from tape or failing the job. N x 622 Mbits/s" Inter-connectedness is increasing with the design of LHCOne to deliver (multi-) 10Gb links Desk" CERN between tops" Tier2s. n.10 7 MIPS m Pbyte Robot" MONARC Fibre 2000 cut during tests in 2009 Capacity reduced, but alternative links took over 50 50

51 Metadata Distribution Conditions data is needed to make sense of the raw data from Average the experiments Streams Throughput Data on items such as temperatures, detector voltages and gas compositions is needed to turn the ~100M Pixel image of the event into a meaningful description in terms of particles, tracks and momenta. LCR/s This data is in an RDBMS, Oracle at CERN, and presents interesting distribution challenges One 5000 cannot tightly 4600 couple databases across the loosely coupled 0 WLCG sites, for example row size = 100B row size = 500B row size = 1000B Oracle streams technology improved to deliver the Oracle 10g Oracle 11gR2 Oracle 11g R2 (opnmized) necessary performance, and http caching systems developed to address need for cross-dbms distribution. 51

52 Job Execution Environment Jobs submitted to sites depend on large, rapidly changing libraries of experiment specific code Major problems ensue if updated code is not distributed to every server across the grid (remember, there are x0,000 servers ) Shared filesystems can become a bottleneck if used as a distribution mechanism within a site. Approaches 2011 ATLAS Today: 22/1.8M files ATLAS Today: 921/115GB Pilot job framework can check to see if the execution host has the correct environment A global caching file system: CernVM-FS

53 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique In outline In more detail Towards the Future Summary Outline 53

54 Learning from our mistakes We have just completed a review of WLCG operations and services based on 2+ years of operations with the aim to simplify and harmonise during the forthcoming long shutdown. Key areas to improve are data management & access and exploiting many/multi-core architectures, especially with use of virtualisation. Clouds Towards the Future Identity Management 54

55 Learning from our mistakes We have just completed a review of WLCG operations and services based on 2+ years of operations with the aim to simplify and harmonise during the forthcoming long shutdown. Key areas to improve are data management & access and exploiting many/multi-core architectures, especially with use of virtualisation. Clouds Towards the Future Identity Management 55

56 Learning from our mistakes We have just completed a review of WLCG operations and services based on 2+ years of operations with the aim to simplify and harmonise during the forthcoming long shutdown. Key areas to improve are data management & access and exploiting many/multi-core architectures, especially with use of virtualisation. Clouds Towards the Future Identity Management 56

57 Integrating With The Cloud? User Site A Slide courtesy of Ulrich Schwickerath Central Task Queue Payload pull Instance requests VO service Site B Site C Shared Image Repository (VMIC) Image maintainer Cloud bursting Commercial cloud 57

58 Learning from our mistakes We have just completed a review of WLCG operations and services based on 2+ years of operations with the aim to simplify and harmonise during the forthcoming long shutdown. Key areas to improve are data management & access and exploiting many/multi-core architectures, especially with use of virtualisation. Clouds Towards the Future Identity Management 58

59 Learning from our mistakes We have just completed a review of WLCG operations and services based on 2+ years of operations with the aim to simplify and harmonise during the forthcoming long shutdown. Key areas to improve are data management & access and exploiting many/multi-core architectures, especially with use of virtualisation. Clouds Towards the Future Identity Management 59

60 Compute Element Grid Middleware Basics Standard interface to local workload management systems (batch scheduler) Storage Element Standard interface to local mass storage systems Resource Broker Tool to analyse user job requests (input data sets, cpu time, data output requirements) and route these to sites according to data and cpu time availability. Many implementations of the basic principles: Globus, VDT, EDG/EGEE, NorduGrid, OSG 60

61 Trust! 61

62 One step beyond? 62

63 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique In outline In more detail Towards the Future Summary Outline 63

64 Summary WLCG has delivered the capability to manage and distribute the large volumes of data generated by the LHC experiments and the excellent WLCG performance has enabled physicists to deliver results rapidly. HEP datasets may not be the most complex or (any longer) massive, but in addressing the LHC computing challenges, the community has delivered the world s largest computing Grid, practical solutions to requirements for large-scale data storage, distribution and access, and a global trust federation enabling world-wide collaboration

65 Thank You! And thanks to Vlado Bahyl, German Cancio, Ian Bird, Jakob Blomer, Eva Dafonte Perez, Fabiola Gianotti, Frédéric Hemmer, Jan Iven, Alberto Pace and Romain Wartel of CERN, Elisa Lanciotti of PIC and K. De, T. Maeno, and S. Panitkin of ATLAS for various unattributed graphics and slides. 65

Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF

Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment

More information

The LHC Computing Grid. Slides mostly by: Dr Ian Bird LCG Project Leader 18 March 2008

The LHC Computing Grid. Slides mostly by: Dr Ian Bird LCG Project Leader 18 March 2008 The LHC Computing Grid Slides mostly by: Dr Ian Bird LCG Project Leader 18 March 2008 The LHC Computing Grid February 2008 Some precursors Computing for HEP means data handling Fixed-target experiments

More information

The LHC Computing Grid

The LHC Computing Grid The LHC Computing Grid Visit of Finnish IT Centre for Science CSC Board Members Finland Tuesday 19 th May 2009 Frédéric Hemmer IT Department Head The LHC and Detectors Outline Computing Challenges Current

More information

Batch Services at CERN: Status and Future Evolution

Batch Services at CERN: Status and Future Evolution Batch Services at CERN: Status and Future Evolution Helge Meinhard, CERN-IT Platform and Engineering Services Group Leader HTCondor Week 20 May 2015 20-May-2015 CERN batch status and evolution - Helge

More information

Data Transfers Between LHC Grid Sites Dorian Kcira

Data Transfers Between LHC Grid Sites Dorian Kcira Data Transfers Between LHC Grid Sites Dorian Kcira dkcira@caltech.edu Caltech High Energy Physics Group hep.caltech.edu/cms CERN Site: LHC and the Experiments Large Hadron Collider 27 km circumference

More information

CHIPP Phoenix Cluster Inauguration

CHIPP Phoenix Cluster Inauguration TheComputing Environment for LHC Data Analysis The LHC Computing Grid CHIPP Phoenix Cluster Inauguration Manno, Switzerland 30 May 2008 Les Robertson IT Department - CERN CH-1211 Genève 23 les.robertson@cern.ch

More information

The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland

The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland The LCG 3D Project Maria Girone, CERN The rd Open Grid Forum - OGF 4th June 2008, Barcelona Outline Introduction The Distributed Database (3D) Project Streams Replication Technology and Performance Availability

More information

Big Data Analytics and the LHC

Big Data Analytics and the LHC Big Data Analytics and the LHC Maria Girone CERN openlab CTO Computing Frontiers 2016, Como, May 2016 DOI: 10.5281/zenodo.45449, CC-BY-SA, images courtesy of CERN 2 3 xx 4 Big bang in the laboratory We

More information

CouchDB-based system for data management in a Grid environment Implementation and Experience

CouchDB-based system for data management in a Grid environment Implementation and Experience CouchDB-based system for data management in a Grid environment Implementation and Experience Hassen Riahi IT/SDC, CERN Outline Context Problematic and strategy System architecture Integration and deployment

More information

ATLAS Distributed Computing Experience and Performance During the LHC Run-2

ATLAS Distributed Computing Experience and Performance During the LHC Run-2 ATLAS Distributed Computing Experience and Performance During the LHC Run-2 A Filipčič 1 for the ATLAS Collaboration 1 Jozef Stefan Institute, Jamova 39, 1000 Ljubljana, Slovenia E-mail: andrej.filipcic@ijs.si

More information

IEPSAS-Kosice: experiences in running LCG site

IEPSAS-Kosice: experiences in running LCG site IEPSAS-Kosice: experiences in running LCG site Marian Babik 1, Dusan Bruncko 2, Tomas Daranyi 1, Ladislav Hluchy 1 and Pavol Strizenec 2 1 Department of Parallel and Distributed Computing, Institute of

More information

The Grid: Processing the Data from the World s Largest Scientific Machine

The Grid: Processing the Data from the World s Largest Scientific Machine The Grid: Processing the Data from the World s Largest Scientific Machine 10th Topical Seminar On Innovative Particle and Radiation Detectors Siena, 1-5 October 2006 Patricia Méndez Lorenzo (IT-PSS/ED),

More information

CERN openlab II. CERN openlab and. Sverre Jarp CERN openlab CTO 16 September 2008

CERN openlab II. CERN openlab and. Sverre Jarp CERN openlab CTO 16 September 2008 CERN openlab II CERN openlab and Intel: Today and Tomorrow Sverre Jarp CERN openlab CTO 16 September 2008 Overview of CERN 2 CERN is the world's largest particle physics centre What is CERN? Particle physics

More information

Grid Computing Activities at KIT

Grid Computing Activities at KIT Grid Computing Activities at KIT Meeting between NCP and KIT, 21.09.2015 Manuel Giffels Karlsruhe Institute of Technology Institute of Experimental Nuclear Physics & Steinbuch Center for Computing Courtesy

More information

Grid Data Management

Grid Data Management Grid Data Management Week #4 Hardi Teder hardi@eenet.ee University of Tartu March 6th 2013 Overview Grid Data Management Where the Data comes from? Grid Data Management tools 2/33 Grid foundations 3/33

More information

First Experience with LCG. Board of Sponsors 3 rd April 2009

First Experience with LCG. Board of Sponsors 3 rd April 2009 First Experience with LCG Operation and the future... CERN openlab Board of Sponsors 3 rd April 2009 Ian Bird LCG Project Leader The LHC Computing Challenge Signal/Noise: 10-9 Data volume High rate * large

More information

Grid Computing a new tool for science

Grid Computing a new tool for science Grid Computing a new tool for science CERN, the European Organization for Nuclear Research Dr. Wolfgang von Rüden Wolfgang von Rüden, CERN, IT Department Grid Computing July 2006 CERN stands for over 50

More information

Big Computing and the Mitchell Institute for Fundamental Physics and Astronomy. David Toback

Big Computing and the Mitchell Institute for Fundamental Physics and Astronomy. David Toback Big Computing and the Mitchell Institute for Fundamental Physics and Astronomy Texas A&M Big Data Workshop October 2011 January 2015, Texas A&M University Research Topics Seminar 1 Outline Overview of

More information

The grid for LHC Data Analysis

The grid for LHC Data Analysis The grid for LHC Data Analysis ICAP 2006 Conference Chamonix 5 October 2006 Les Robertson - CERN LHC Computing Grid Project Leader The LHC Computing Challenges 1. Data After reduction by triggers and data

More information

Physics Computing at CERN. Helge Meinhard CERN, IT Department OpenLab Student Lecture 27 July 2010

Physics Computing at CERN. Helge Meinhard CERN, IT Department OpenLab Student Lecture 27 July 2010 Physics Computing at CERN Helge Meinhard CERN, IT Department OpenLab Student Lecture 27 July 2010 Location Building 513 (opposite of restaurant no. 2) Building Large building with 2700 m 2 surface for

More information

Data Management for the World s Largest Machine

Data Management for the World s Largest Machine Data Management for the World s Largest Machine Sigve Haug 1, Farid Ould-Saada 2, Katarina Pajchel 2, and Alexander L. Read 2 1 Laboratory for High Energy Physics, University of Bern, Sidlerstrasse 5,

More information

RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP

RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP THE EUROPEAN ORGANISATION FOR PARTICLE PHYSICS RESEARCH (CERN) 2 THE LARGE HADRON COLLIDER THE LARGE HADRON COLLIDER

More information

How to discover the Higgs Boson in an Oracle database. Maaike Limper

How to discover the Higgs Boson in an Oracle database. Maaike Limper How to discover the Higgs Boson in an Oracle database Maaike Limper 2 Introduction CERN openlab is a unique public-private partnership between CERN and leading ICT companies. Its mission is to accelerate

More information

Lessons Learned in the NorduGrid Federation

Lessons Learned in the NorduGrid Federation Lessons Learned in the NorduGrid Federation David Cameron University of Oslo With input from Gerd Behrmann, Oxana Smirnova and Mattias Wadenstein Creating Federated Data Stores For The LHC 14.9.12, Lyon,

More information

The LHC Computing Grid

The LHC Computing Grid The LHC Computing Grid Gergely Debreczeni (CERN IT/Grid Deployment Group) The data factory of LHC 40 million collisions in each second After on-line triggers and selections, only 100 3-4 MB/event requires

More information

AGIS: The ATLAS Grid Information System

AGIS: The ATLAS Grid Information System AGIS: The ATLAS Grid Information System Alexey Anisenkov 1, Sergey Belov 2, Alessandro Di Girolamo 3, Stavro Gayazov 1, Alexei Klimentov 4, Danila Oleynik 2, Alexander Senchenko 1 on behalf of the ATLAS

More information

Streamlining CASTOR to manage the LHC data torrent

Streamlining CASTOR to manage the LHC data torrent Streamlining CASTOR to manage the LHC data torrent G. Lo Presti, X. Espinal Curull, E. Cano, B. Fiorini, A. Ieri, S. Murray, S. Ponce and E. Sindrilaru CERN, 1211 Geneva 23, Switzerland E-mail: giuseppe.lopresti@cern.ch

More information

Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science

Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science T. Maeno, K. De, A. Klimentov, P. Nilsson, D. Oleynik, S. Panitkin, A. Petrosyan, J. Schovancova, A. Vaniachine,

More information

Towards Network Awareness in LHC Computing

Towards Network Awareness in LHC Computing Towards Network Awareness in LHC Computing CMS ALICE CERN Atlas LHCb LHC Run1: Discovery of a New Boson LHC Run2: Beyond the Standard Model Gateway to a New Era Artur Barczyk / Caltech Internet2 Technology

More information

Evaluation of the computing resources required for a Nordic research exploitation of the LHC

Evaluation of the computing resources required for a Nordic research exploitation of the LHC PROCEEDINGS Evaluation of the computing resources required for a Nordic research exploitation of the LHC and Sverker Almehed, Chafik Driouichi, Paula Eerola, Ulf Mjörnmark, Oxana Smirnova,TorstenÅkesson

More information

Summary of the LHC Computing Review

Summary of the LHC Computing Review Summary of the LHC Computing Review http://lhc-computing-review-public.web.cern.ch John Harvey CERN/EP May 10 th, 2001 LHCb Collaboration Meeting The Scale Data taking rate : 50,100, 200 Hz (ALICE, ATLAS-CMS,

More information

LHC Computing Grid today Did it work?

LHC Computing Grid today Did it work? Did it work? Sept. 9th 2011, 1 KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association Institut www.kit.edu Abteilung Large Hadron Collider and Experiments

More information

Data services for LHC computing

Data services for LHC computing Data services for LHC computing SLAC 1 Xavier Espinal on behalf of IT/ST DAQ to CC 8GB/s+4xReco Hot files Reliable Fast Processing DAQ Feedback loop WAN aware Tier-1/2 replica, multi-site High throughout

More information

Data Reconstruction in Modern Particle Physics

Data Reconstruction in Modern Particle Physics Data Reconstruction in Modern Particle Physics Daniel Saunders, University of Bristol 1 About me Particle Physics student, final year. CSC 2014, tcsc 2015, icsc 2016 Main research interests. Detector upgrades

More information

ISTITUTO NAZIONALE DI FISICA NUCLEARE

ISTITUTO NAZIONALE DI FISICA NUCLEARE ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Perugia INFN/TC-05/10 July 4, 2005 DESIGN, IMPLEMENTATION AND CONFIGURATION OF A GRID SITE WITH A PRIVATE NETWORK ARCHITECTURE Leonello Servoli 1,2!, Mirko

More information

where the Web was born Experience of Adding New Architectures to the LCG Production Environment

where the Web was born Experience of Adding New Architectures to the LCG Production Environment where the Web was born Experience of Adding New Architectures to the LCG Production Environment Andreas Unterkircher, openlab fellow Sverre Jarp, CTO CERN openlab Industrializing the Grid openlab Workshop

More information

Grid Computing at the IIHE

Grid Computing at the IIHE BNC 2016 Grid Computing at the IIHE The Interuniversity Institute for High Energies S. Amary, F. Blekman, A. Boukil, O. Devroede, S. Gérard, A. Ouchene, R. Rougny, S. Rugovac, P. Vanlaer, R. Vandenbroucke

More information

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy

30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of

More information

Outline. ASP 2012 Grid School

Outline. ASP 2012 Grid School Distributed Storage Rob Quick Indiana University Slides courtesy of Derek Weitzel University of Nebraska Lincoln Outline Storage Patterns in Grid Applications Storage

More information

CERN Lustre Evaluation

CERN Lustre Evaluation CERN Lustre Evaluation Arne Wiebalck Sun HPC Workshop, Open Storage Track Regensburg, Germany 8 th Sep 2009 www.cern.ch/it Agenda A Quick Guide to CERN Storage Use Cases Methodology & Initial Findings

More information

High Throughput WAN Data Transfer with Hadoop-based Storage

High Throughput WAN Data Transfer with Hadoop-based Storage High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San

More information

Physics Computing at CERN. Helge Meinhard CERN, IT Department OpenLab Student Lecture 21 July 2011

Physics Computing at CERN. Helge Meinhard CERN, IT Department OpenLab Student Lecture 21 July 2011 Physics Computing at CERN Helge Meinhard CERN, IT Department OpenLab Student Lecture 21 July 2011 Location (1) Building 513 (opposite of restaurant no. 2) Building 513 (1) Large building with 2700 m 2

More information

CernVM-FS beyond LHC computing

CernVM-FS beyond LHC computing CernVM-FS beyond LHC computing C Condurache, I Collier STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot, OX11 0QX, UK E-mail: catalin.condurache@stfc.ac.uk Abstract. In the last three years

More information

Grids and Security. Ian Neilson Grid Deployment Group CERN. TF-CSIRT London 27 Jan

Grids and Security. Ian Neilson Grid Deployment Group CERN. TF-CSIRT London 27 Jan Grids and Security Ian Neilson Grid Deployment Group CERN TF-CSIRT London 27 Jan 2004-1 TOC Background Grids Grid Projects Some Technical Aspects The three or four A s Some Operational Aspects Security

More information

ATLAS distributed computing: experience and evolution

ATLAS distributed computing: experience and evolution Journal of Physics: Conference Series OPEN ACCESS ATLAS distributed computing: experience and evolution To cite this article: A Nairz and the Atlas Collaboration 2014 J. Phys.: Conf. Ser. 523 012020 View

More information

Online data storage service strategy for the CERN computer Centre G. Cancio, D. Duellmann, M. Lamanna, A. Pace CERN, Geneva, Switzerland

Online data storage service strategy for the CERN computer Centre G. Cancio, D. Duellmann, M. Lamanna, A. Pace CERN, Geneva, Switzerland Online data storage service strategy for the CERN computer Centre G. Cancio, D. Duellmann, M. Lamanna, A. Pace CERN, Geneva, Switzerland Abstract. The Data and Storage Services group at CERN is conducting

More information

EGEE and Interoperation

EGEE and Interoperation EGEE and Interoperation Laurence Field CERN-IT-GD ISGC 2008 www.eu-egee.org EGEE and glite are registered trademarks Overview The grid problem definition GLite and EGEE The interoperability problem The

More information

Operating the Distributed NDGF Tier-1

Operating the Distributed NDGF Tier-1 Operating the Distributed NDGF Tier-1 Michael Grønager Technical Coordinator, NDGF International Symposium on Grid Computing 08 Taipei, April 10th 2008 Talk Outline What is NDGF? Why a distributed Tier-1?

More information

The LHC computing model and its evolution. Dr Bob Jones CERN

The LHC computing model and its evolution. Dr Bob Jones CERN The LHC computing model and its evolution Dr Bob Jones CERN Bob.Jones CERN.ch CERN was founded 1954: 12 European States Today: 20 Member States ~ 2300 staff ~ 790 other paid personnel > 10000 users

More information

Computing: new records broken! Data in Tier-0 vs time

Computing: new records broken! Data in Tier-0 vs time Data in Tier-0 vs time Computing: new records broken! Jamie.Shiers@cern.ch Data transfer 20 GB/s x10 design! WLCG today: ~ 170 sites (40 countries) ~ 500k CPU cores, 500 PB storage > 2 million jobs/days,

More information

Data preservation for the HERA experiments at DESY using dcache technology

Data preservation for the HERA experiments at DESY using dcache technology Journal of Physics: Conference Series PAPER OPEN ACCESS Data preservation for the HERA experiments at DESY using dcache technology To cite this article: Dirk Krücker et al 2015 J. Phys.: Conf. Ser. 66

More information

Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine

Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine Journal of Physics: Conference Series OPEN ACCESS Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine To cite this article: Henrik Öhman et al 2014 J. Phys.: Conf.

More information

Evolution of Database Replication Technologies for WLCG

Evolution of Database Replication Technologies for WLCG Evolution of Database Replication Technologies for WLCG Zbigniew Baranowski, Lorena Lobato Pardavila, Marcin Blaszczyk, Gancho Dimitrov, Luca Canali European Organisation for Nuclear Research (CERN), CH-1211

More information

Federated data storage system prototype for LHC experiments and data intensive science

Federated data storage system prototype for LHC experiments and data intensive science Federated data storage system prototype for LHC experiments and data intensive science A. Kiryanov 1,2,a, A. Klimentov 1,3,b, D. Krasnopevtsev 1,4,c, E. Ryabinkin 1,d, A. Zarochentsev 1,5,e 1 National

More information

Evolution of Database Replication Technologies for WLCG

Evolution of Database Replication Technologies for WLCG Journal of Physics: Conference Series PAPER OPEN ACCESS Evolution of Database Replication Technologies for WLCG To cite this article: Zbigniew Baranowski et al 2015 J. Phys.: Conf. Ser. 664 042032 View

More information

Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008

Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008 Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, 13-14 November 2008 Outline Introduction SRM Storage Elements in glite LCG File Catalog (LFC) Information System Grid Tutorial, 13-14

More information

CC-IN2P3: A High Performance Data Center for Research

CC-IN2P3: A High Performance Data Center for Research April 15 th, 2011 CC-IN2P3: A High Performance Data Center for Research Toward a partnership with DELL Dominique Boutigny Agenda Welcome Introduction to CC-IN2P3 Visit of the computer room Lunch Discussion

More information

Distributed Monte Carlo Production for

Distributed Monte Carlo Production for Distributed Monte Carlo Production for Joel Snow Langston University DOE Review March 2011 Outline Introduction FNAL SAM SAMGrid Interoperability with OSG and LCG Production System Production Results LUHEP

More information

Overview of ATLAS PanDA Workload Management

Overview of ATLAS PanDA Workload Management Overview of ATLAS PanDA Workload Management T. Maeno 1, K. De 2, T. Wenaus 1, P. Nilsson 2, G. A. Stewart 3, R. Walker 4, A. Stradling 2, J. Caballero 1, M. Potekhin 1, D. Smith 5, for The ATLAS Collaboration

More information

UK Tier-2 site evolution for ATLAS. Alastair Dewhurst

UK Tier-2 site evolution for ATLAS. Alastair Dewhurst UK Tier-2 site evolution for ATLAS Alastair Dewhurst Introduction My understanding is that GridPP funding is only part of the story when it comes to paying for a Tier 2 site. Each site is unique. Aim to

More information

The ATLAS Production System

The ATLAS Production System The ATLAS MC and Data Rodney Walker Ludwig Maximilians Universität Munich 2nd Feb, 2009 / DESY Computing Seminar Outline 1 Monte Carlo Production Data 2 3 MC Production Data MC Production Data Group and

More information

Application of Virtualization Technologies & CernVM. Benedikt Hegner CERN

Application of Virtualization Technologies & CernVM. Benedikt Hegner CERN Application of Virtualization Technologies & CernVM Benedikt Hegner CERN Virtualization Use Cases Worker Node Virtualization Software Testing Training Platform Software Deployment }Covered today Server

More information

The JINR Tier1 Site Simulation for Research and Development Purposes

The JINR Tier1 Site Simulation for Research and Development Purposes EPJ Web of Conferences 108, 02033 (2016) DOI: 10.1051/ epjconf/ 201610802033 C Owned by the authors, published by EDP Sciences, 2016 The JINR Tier1 Site Simulation for Research and Development Purposes

More information

Global Software Distribution with CernVM-FS

Global Software Distribution with CernVM-FS Global Software Distribution with CernVM-FS Jakob Blomer CERN 2016 CCL Workshop on Scalable Computing October 19th, 2016 jblomer@cern.ch CernVM-FS 1 / 15 The Anatomy of a Scientific Software Stack (In

More information

Constant monitoring of multi-site network connectivity at the Tokyo Tier2 center

Constant monitoring of multi-site network connectivity at the Tokyo Tier2 center Constant monitoring of multi-site network connectivity at the Tokyo Tier2 center, T. Mashimo, N. Matsui, H. Matsunaga, H. Sakamoto, I. Ueda International Center for Elementary Particle Physics, The University

More information

Introduction to High Performance Parallel I/O

Introduction to High Performance Parallel I/O Introduction to High Performance Parallel I/O Richard Gerber Deputy Group Lead NERSC User Services August 30, 2013-1- Some slides from Katie Antypas I/O Needs Getting Bigger All the Time I/O needs growing

More information

Deep Learning Photon Identification in a SuperGranular Calorimeter

Deep Learning Photon Identification in a SuperGranular Calorimeter Deep Learning Photon Identification in a SuperGranular Calorimeter Nikolaus Howe Maurizio Pierini Jean-Roch Vlimant @ Williams College @ CERN @ Caltech 1 Outline Introduction to the problem What is Machine

More information

CA485 Ray Walshe Google File System

CA485 Ray Walshe Google File System Google File System Overview Google File System is scalable, distributed file system on inexpensive commodity hardware that provides: Fault Tolerance File system runs on hundreds or thousands of storage

More information

Installation of CMSSW in the Grid DESY Computing Seminar May 17th, 2010 Wolf Behrenhoff, Christoph Wissing

Installation of CMSSW in the Grid DESY Computing Seminar May 17th, 2010 Wolf Behrenhoff, Christoph Wissing Installation of CMSSW in the Grid DESY Computing Seminar May 17th, 2010 Wolf Behrenhoff, Christoph Wissing Wolf Behrenhoff, Christoph Wissing DESY Computing Seminar May 17th, 2010 Page 1 Installation of

More information

Distributed File Systems II

Distributed File Systems II Distributed File Systems II To do q Very-large scale: Google FS, Hadoop FS, BigTable q Next time: Naming things GFS A radically new environment NFS, etc. Independence Small Scale Variety of workloads Cooperation

More information

Understanding the T2 traffic in CMS during Run-1

Understanding the T2 traffic in CMS during Run-1 Journal of Physics: Conference Series PAPER OPEN ACCESS Understanding the T2 traffic in CMS during Run-1 To cite this article: Wildish T and 2015 J. Phys.: Conf. Ser. 664 032034 View the article online

More information

The Google File System

The Google File System The Google File System Sanjay Ghemawat, Howard Gobioff and Shun Tak Leung Google* Shivesh Kumar Sharma fl4164@wayne.edu Fall 2015 004395771 Overview Google file system is a scalable distributed file system

More information

Improved Solutions for I/O Provisioning and Application Acceleration

Improved Solutions for I/O Provisioning and Application Acceleration 1 Improved Solutions for I/O Provisioning and Application Acceleration August 11, 2015 Jeff Sisilli Sr. Director Product Marketing jsisilli@ddn.com 2 Why Burst Buffer? The Supercomputing Tug-of-War A supercomputer

More information

Chapter 4:- Introduction to Grid and its Evolution. Prepared By:- NITIN PANDYA Assistant Professor SVBIT.

Chapter 4:- Introduction to Grid and its Evolution. Prepared By:- NITIN PANDYA Assistant Professor SVBIT. Chapter 4:- Introduction to Grid and its Evolution Prepared By:- Assistant Professor SVBIT. Overview Background: What is the Grid? Related technologies Grid applications Communities Grid Tools Case Studies

More information

WHEN the Large Hadron Collider (LHC) begins operation

WHEN the Large Hadron Collider (LHC) begins operation 2228 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 53, NO. 4, AUGUST 2006 Measurement of the LCG2 and Glite File Catalogue s Performance Craig Munro, Birger Koblitz, Nuno Santos, and Akram Khan Abstract When

More information

Oracle at CERN CERN openlab summer students programme 2011

Oracle at CERN CERN openlab summer students programme 2011 Oracle at CERN CERN openlab summer students programme 2011 Eric Grancher eric.grancher@cern.ch CERN IT department Image courtesy of Forschungszentrum Jülich / Seitenplan, with material from NASA, ESA and

More information

How physicists analyze massive data: LHC + brain + ROOT = Higgs. Axel Naumann, CERN - 33C3, 2016 (but almost 2017)

How physicists analyze massive data: LHC + brain + ROOT = Higgs. Axel Naumann, CERN - 33C3, 2016 (but almost 2017) How physicists analyze massive data: LHC + brain + ROOT = Higgs Axel Naumann, CERN - axel@cern.ch 33C3, 2016 (but almost 2017) CERN, People, Code Axel Naumann, CERN - axel@cern.ch 33C3, 2016 (but almost

More information

Grid Interoperation and Regional Collaboration

Grid Interoperation and Regional Collaboration Grid Interoperation and Regional Collaboration Eric Yen ASGC Academia Sinica Taiwan 23 Jan. 2006 Dreams of Grid Computing Global collaboration across administrative domains by sharing of people, resources,

More information

Spark and HPC for High Energy Physics Data Analyses

Spark and HPC for High Energy Physics Data Analyses Spark and HPC for High Energy Physics Data Analyses Marc Paterno, Jim Kowalkowski, and Saba Sehrish 2017 IEEE International Workshop on High-Performance Big Data Computing Introduction High energy physics

More information

LCG Conditions Database Project

LCG Conditions Database Project Computing in High Energy and Nuclear Physics (CHEP 2006) TIFR, Mumbai, 13 Feb 2006 LCG Conditions Database Project COOL Development and Deployment: Status and Plans On behalf of the COOL team (A.V., D.Front,

More information

Data Analysis in ATLAS. Graeme Stewart with thanks to Attila Krasznahorkay and Johannes Elmsheuser

Data Analysis in ATLAS. Graeme Stewart with thanks to Attila Krasznahorkay and Johannes Elmsheuser Data Analysis in ATLAS Graeme Stewart with thanks to Attila Krasznahorkay and Johannes Elmsheuser 1 ATLAS Data Flow into Analysis RAW detector data and simulated RDO data are reconstructed into our xaod

More information

The CORAL Project. Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008

The CORAL Project. Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008 The CORAL Project Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008 Outline CORAL - a foundation for Physics Database Applications in the LHC Computing Grid (LCG)

More information

Volunteer Computing at CERN

Volunteer Computing at CERN Volunteer Computing at CERN BOINC workshop Sep 2014, Budapest Tomi Asp & Pete Jones, on behalf the LHC@Home team Agenda Overview Status of the LHC@Home projects Additional BOINC projects Service consolidation

More information

Interoperating AliEn and ARC for a distributed Tier1 in the Nordic countries.

Interoperating AliEn and ARC for a distributed Tier1 in the Nordic countries. for a distributed Tier1 in the Nordic countries. Philippe Gros Lund University, Div. of Experimental High Energy Physics, Box 118, 22100 Lund, Sweden philippe.gros@hep.lu.se Anders Rhod Gregersen NDGF

More information

File Access Optimization with the Lustre Filesystem at Florida CMS T2

File Access Optimization with the Lustre Filesystem at Florida CMS T2 Journal of Physics: Conference Series PAPER OPEN ACCESS File Access Optimization with the Lustre Filesystem at Florida CMS T2 To cite this article: P. Avery et al 215 J. Phys.: Conf. Ser. 664 4228 View

More information

An SQL-based approach to physics analysis

An SQL-based approach to physics analysis Journal of Physics: Conference Series OPEN ACCESS An SQL-based approach to physics analysis To cite this article: Dr Maaike Limper 2014 J. Phys.: Conf. Ser. 513 022022 View the article online for updates

More information

The Google File System

The Google File System The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung SOSP 2003 presented by Kun Suo Outline GFS Background, Concepts and Key words Example of GFS Operations Some optimizations in

More information

Configuring Short RPO with Actifio StreamSnap and Dedup-Async Replication

Configuring Short RPO with Actifio StreamSnap and Dedup-Async Replication CDS and Sky Tech Brief Configuring Short RPO with Actifio StreamSnap and Dedup-Async Replication Actifio recommends using Dedup-Async Replication (DAR) for RPO of 4 hours or more and using StreamSnap for

More information

Experience of Data Grid simulation packages using.

Experience of Data Grid simulation packages using. Experience of Data Grid simulation packages using. Nechaevskiy A.V. (SINP MSU), Korenkov V.V. (LIT JINR) Dubna, 2008 Contant Operation of LCG DataGrid Errors of FTS services of the Grid. Primary goals

More information

Computing. DOE Program Review SLAC. Rainer Bartoldus. Breakout Session 3 June BaBar Deputy Computing Coordinator

Computing. DOE Program Review SLAC. Rainer Bartoldus. Breakout Session 3 June BaBar Deputy Computing Coordinator Computing DOE Program Review SLAC Breakout Session 3 June 2004 Rainer Bartoldus BaBar Deputy Computing Coordinator 1 Outline The New Computing Model (CM2) New Kanga/ROOT event store, new Analysis Model,

More information

MOHA: Many-Task Computing Framework on Hadoop

MOHA: Many-Task Computing Framework on Hadoop Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction

More information

Introduction to OS. File Management. MOS Ch. 4. Mahmoud El-Gayyar. Mahmoud El-Gayyar / Introduction to OS 1

Introduction to OS. File Management. MOS Ch. 4. Mahmoud El-Gayyar. Mahmoud El-Gayyar / Introduction to OS 1 Introduction to OS File Management MOS Ch. 4 Mahmoud El-Gayyar elgayyar@ci.suez.edu.eg Mahmoud El-Gayyar / Introduction to OS 1 File Management Objectives Provide I/O support for a variety of storage device

More information

RESEARCH DATA DEPOT AT PURDUE UNIVERSITY

RESEARCH DATA DEPOT AT PURDUE UNIVERSITY Preston Smith Director of Research Services RESEARCH DATA DEPOT AT PURDUE UNIVERSITY May 18, 2016 HTCONDOR WEEK 2016 Ran into Miron at a workshop recently.. Talked about data and the challenges of providing

More information

Track reconstruction with the CMS tracking detector

Track reconstruction with the CMS tracking detector Track reconstruction with the CMS tracking detector B. Mangano (University of California, San Diego) & O.Gutsche (Fermi National Accelerator Laboratory) Overview The challenges The detector Track reconstruction

More information

13th International Workshop on Advanced Computing and Analysis Techniques in Physics Research ACAT 2010 Jaipur, India February

13th International Workshop on Advanced Computing and Analysis Techniques in Physics Research ACAT 2010 Jaipur, India February LHC Cloud Computing with CernVM Ben Segal 1 CERN 1211 Geneva 23, Switzerland E mail: b.segal@cern.ch Predrag Buncic CERN E mail: predrag.buncic@cern.ch 13th International Workshop on Advanced Computing

More information

Tape in the Microsoft Datacenter: The Good and Bad of Tape as a Target for Cloud-based Archival Storage

Tape in the Microsoft Datacenter: The Good and Bad of Tape as a Target for Cloud-based Archival Storage Tape in the Microsoft Datacenter: The Good and Bad of Tape as a Target for Cloud-based Archival Storage Marvin McNett Principal Development Manager Microsoft Azure Storage Content I. Azure Archival Storage

More information

Distributed File Systems Part IV. Hierarchical Mass Storage Systems

Distributed File Systems Part IV. Hierarchical Mass Storage Systems Distributed File Systems Part IV Daniel A. Menascé Hierarchical Mass Storage Systems On-line data requirements Mass Storage Systems Concepts Mass storage system architectures Example systems Performance

More information

Implementing a Digital Video Archive Based on the Sony PetaSite and XenData Software

Implementing a Digital Video Archive Based on the Sony PetaSite and XenData Software Based on the Sony PetaSite and XenData Software The Video Edition of XenData Archive Series software manages a Sony PetaSite tape library on a Windows Server 2003 platform to create a digital video archive

More information

5 Fundamental Strategies for Building a Data-centered Data Center

5 Fundamental Strategies for Building a Data-centered Data Center 5 Fundamental Strategies for Building a Data-centered Data Center June 3, 2014 Ken Krupa, Chief Field Architect Gary Vidal, Solutions Specialist Last generation Reference Data Unstructured OLTP Warehouse

More information

FLASHARRAY//M Business and IT Transformation in 3U

FLASHARRAY//M Business and IT Transformation in 3U FLASHARRAY//M Business and IT Transformation in 3U TRANSFORM IT Who knew that moving to all-flash storage could help reduce the cost of IT? FlashArray//m makes server and workload investments more productive,

More information