CERN Tape Archive (CTA) :

Size: px
Start display at page:

Download "CERN Tape Archive (CTA) :"

Transcription

1 CERN Tape Archive (CTA) : From Development to Production Deployment Michael Davis, Vladimír Bahyl, Germán Cancio, Eric Cano, Julien Leduc and Steven Murray CHEP 2018, Sofia, Bulgaria 9 July 2018

2 Changing Use Cases for Archival Storage 1. Scaling up for Run 3 and HL LHC 1/12 Predicted T0 Tape Archival Storage Needs 4.3 EB 0.25 EB 0.5 EB 1.3 EB 3.1 EB

3 Changing Use Cases for Archival Storage 2. Data for online analysis stored on tape ( Data Carousel ) 2/12 Source: Tape Usage, Xin Zhao (Brookhaven National Laboratory), ADC Technical Coordination Board Meeting, 28 May 2018

4 Changing Use Cases for Archival Storage 2. Data for online analysis stored on tape ( Data Carousel ) 2/12 Source: Tape Usage, Xin Zhao (Brookhaven National Laboratory), ADC Technical Coordination Board Meeting, 28 May 2018

5 Use Cases CASTOR to CTA Testing and Deployment Summary Changing Use Cases for Archival Storage 2. Data for online analysis stored on tape ( Data Carousel ) Source: Fascinating Vintage 20 Cassette Carousel from 1972 : Panasonic RS 296US 2/12

6 CASTOR Architecture 3/12 Client Disk Servers Disk Cache Management/ Disk Cache : Queues + Catalogue Central Services NameServer VMGR VDQM

7 CERN Tape Archive Architecture 4/12 Client CTA Frontend : Catalogue Object Store

8 CTA Architecture 5/12 CTA offers the Best of Both Worlds User interface, file access and disk pool management from EOS Tape system management from CASTOR New scalable, robust queuing system to link the two CTA design principles Simplicity Scalability Performance Full details: An efficient, modular and simple tape archiving solution for LHC Run 3, Steven Murray et al. (CERN), CHEP 2016

9 CTA Architecture 6/12 CASTOR Scheduling decisions made at time of user request. Tape drive may not be available when job reaches the front of the queue. High-priority jobs cannot interrupt running jobs. CTA Scheduling decisions made at time of tape mount. Tape drive allocated when job reaches the front of the queue. Reduced latency for users. High-priority jobs can preempt lower-priority jobs. Can switch from repack to data taking and back without operator intervention. System operates at full capacity at all times.

10 Use Cases CASTOR to CTA Testing and Deployment Summary 7/12 CASTOR Deployment Model JobManager Scheduler RR Mover Disk Servers Job Disk Cache Subsystem Error Qry Central Services NameServer RR Mover Disk Servers VMGR VDQM Job Disk Cache Subsystem Error Qry Central Services NameServer RR Mover Disk Servers VMGR VDQM Job Disk Cache Subsystem Error Qry Central Services NameServer RR Mover Disk Servers VMGR VDQM Job Disk Cache Subsystem Error Qry Central Services NameServer RR Mover Disk Servers VMGR VDQM RH Job GC JobManager Scheduler RH Job GC JobManager Scheduler RH Job GC JobManager Scheduler RH Job GC JobManager Scheduler RH Job Job Disk Cache Subsystem Central Services NameServer VMGR CASTOR ALICE CASTOR ATLAS CASTOR CMS CASTOR LHCb CASTOR PUBLIC Experiment Error Qry GC VDQM

11 Use Cases CASTOR to CTA Testing and Deployment Summary 8/12 CTA Deployment Model MGM MGM FST CTA Frontend Tape Catalogue Object Store MGM FST CTA Frontend Tape Catalogue Object Store MGM FST CTA Frontend Tape Catalogue Object Store MGM FST CTA Frontend Tape Catalogue CTA Frontend Tape Catalogue Object Store Object Store EOS+CTA ALICE EOS+CTA ATLAS EOS+CTA CMS EOS+CTA LHCb EOS+CTA PUBLIC Experiment + CTA FST

12 System Testing 9/12 Scale tests and stress tests : 10 million files archived in 27 hours Files archived to tape per second 100 Hz Time (hours)

13 Field Testing 10/12 Goal of user testing is to ensure that all use cases are covered Rucio/File Transfer Service (FTS) tests with ATLAS have started Next : Agree schedule for field testing with all CERN experiments

14 Migration Schedule 11/ Run LS LS Run 3 CTA FIELD TESTING Migration during period of reduced data taking. Migration is a metadata-only operation. No physical movement of data on tape. DATA TAKING AND RETRIEVAL : CASTOR DATA TAKING AND RETRIEVAL : CTA

15 CERN Tape Archive : Summary 12/12 Use cases for tape archival are changing Increased rate of data taking for Run 3 and HL-LHC Data for online analysis accessed via Data Carousel CTA is the Best of Both Worlds EOS disk and CASTOR tape Simplicity Scalability Performance Deployment Now: Field test instances with redundant copies of data LS2: Migration from CASTOR to CTA

Storage Resource Sharing with CASTOR.

Storage Resource Sharing with CASTOR. Storage Resource Sharing with CASTOR Olof Barring, Benjamin Couturier, Jean-Damien Durand, Emil Knezo, Sebastien Ponce (CERN) Vitali Motyakov (IHEP) ben.couturier@cern.ch 16/4/2004 Storage Resource Sharing

More information

Data services for LHC computing

Data services for LHC computing Data services for LHC computing SLAC 1 Xavier Espinal on behalf of IT/ST DAQ to CC 8GB/s+4xReco Hot files Reliable Fast Processing DAQ Feedback loop WAN aware Tier-1/2 replica, multi-site High throughout

More information

CASTOR: A Distributed Storage Resource Facility for High Performance Data Processing at CERN

CASTOR: A Distributed Storage Resource Facility for High Performance Data Processing at CERN CASTOR: A Distributed Storage Resource Facility for High Performance Data Processing at CERN Giuseppe Lo Presti, Olof Bärring, Alasdair Earl, Rosa María García Rioja, Sébastien Ponce, Giulia Taurelli,

More information

The Software Defined Online Storage System at the GridKa WLCG Tier-1 Center

The Software Defined Online Storage System at the GridKa WLCG Tier-1 Center The Software Defined Online Storage System at the GridKa WLCG Tier-1 Center CHEP 2018, Sofia Jan Erik Sundermann, Jolanta Bubeliene, Ludmilla Obholz, Andreas Petzold STEINBUCH CENTRE FOR COMPUTING (SCC)

More information

Andrea Sciabà CERN, Switzerland

Andrea Sciabà CERN, Switzerland Frascati Physics Series Vol. VVVVVV (xxxx), pp. 000-000 XX Conference Location, Date-start - Date-end, Year THE LHC COMPUTING GRID Andrea Sciabà CERN, Switzerland Abstract The LHC experiments will start

More information

Summary of the LHC Computing Review

Summary of the LHC Computing Review Summary of the LHC Computing Review http://lhc-computing-review-public.web.cern.ch John Harvey CERN/EP May 10 th, 2001 LHCb Collaboration Meeting The Scale Data taking rate : 50,100, 200 Hz (ALICE, ATLAS-CMS,

More information

CouchDB-based system for data management in a Grid environment Implementation and Experience

CouchDB-based system for data management in a Grid environment Implementation and Experience CouchDB-based system for data management in a Grid environment Implementation and Experience Hassen Riahi IT/SDC, CERN Outline Context Problematic and strategy System architecture Integration and deployment

More information

CERN European Organization for Nuclear Research, 1211 Geneva, CH

CERN European Organization for Nuclear Research, 1211 Geneva, CH Disk storage at CERN L Mascetti, E Cano, B Chan, X Espinal, A Fiorot, H González Labrador, J Iven, M Lamanna, G Lo Presti, JT Mościcki, AJ Peters, S Ponce, H Rousseau and D van der Ster CERN European Organization

More information

Online data storage service strategy for the CERN computer Centre G. Cancio, D. Duellmann, M. Lamanna, A. Pace CERN, Geneva, Switzerland

Online data storage service strategy for the CERN computer Centre G. Cancio, D. Duellmann, M. Lamanna, A. Pace CERN, Geneva, Switzerland Online data storage service strategy for the CERN computer Centre G. Cancio, D. Duellmann, M. Lamanna, A. Pace CERN, Geneva, Switzerland Abstract. The Data and Storage Services group at CERN is conducting

More information

Streamlining CASTOR to manage the LHC data torrent

Streamlining CASTOR to manage the LHC data torrent Streamlining CASTOR to manage the LHC data torrent G. Lo Presti, X. Espinal Curull, E. Cano, B. Fiorini, A. Ieri, S. Murray, S. Ponce and E. Sindrilaru CERN, 1211 Geneva 23, Switzerland E-mail: giuseppe.lopresti@cern.ch

More information

PROOF-Condor integration for ATLAS

PROOF-Condor integration for ATLAS PROOF-Condor integration for ATLAS G. Ganis,, J. Iwaszkiewicz, F. Rademakers CERN / PH-SFT M. Livny, B. Mellado, Neng Xu,, Sau Lan Wu University Of Wisconsin Condor Week, Madison, 29 Apr 2 May 2008 Outline

More information

The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland

The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland The LCG 3D Project Maria Girone, CERN The rd Open Grid Forum - OGF 4th June 2008, Barcelona Outline Introduction The Distributed Database (3D) Project Streams Replication Technology and Performance Availability

More information

WLCG Transfers Dashboard: a Unified Monitoring Tool for Heterogeneous Data Transfers.

WLCG Transfers Dashboard: a Unified Monitoring Tool for Heterogeneous Data Transfers. WLCG Transfers Dashboard: a Unified Monitoring Tool for Heterogeneous Data Transfers. J Andreeva 1, A Beche 1, S Belov 2, I Kadochnikov 2, P Saiz 1 and D Tuckett 1 1 CERN (European Organization for Nuclear

More information

Federated Data Storage System Prototype based on dcache

Federated Data Storage System Prototype based on dcache Federated Data Storage System Prototype based on dcache Andrey Kiryanov, Alexei Klimentov, Artem Petrosyan, Andrey Zarochentsev on behalf of BigData lab @ NRC KI and Russian Federated Data Storage Project

More information

PoS(EGICF12-EMITC2)106

PoS(EGICF12-EMITC2)106 DDM Site Services: A solution for global replication of HEP data Fernando Harald Barreiro Megino 1 E-mail: fernando.harald.barreiro.megino@cern.ch Simone Campana E-mail: simone.campana@cern.ch Vincent

More information

Worldwide Production Distributed Data Management at the LHC. Brian Bockelman MSST 2010, 4 May 2010

Worldwide Production Distributed Data Management at the LHC. Brian Bockelman MSST 2010, 4 May 2010 Worldwide Production Distributed Data Management at the LHC Brian Bockelman MSST 2010, 4 May 2010 At the LHC http://op-webtools.web.cern.ch/opwebtools/vistar/vistars.php?usr=lhc1 Gratuitous detector pictures:

More information

Overview of ATLAS PanDA Workload Management

Overview of ATLAS PanDA Workload Management Overview of ATLAS PanDA Workload Management T. Maeno 1, K. De 2, T. Wenaus 1, P. Nilsson 2, G. A. Stewart 3, R. Walker 4, A. Stradling 2, J. Caballero 1, M. Potekhin 1, D. Smith 5, for The ATLAS Collaboration

More information

Challenges and Evolution of the LHC Production Grid. April 13, 2011 Ian Fisk

Challenges and Evolution of the LHC Production Grid. April 13, 2011 Ian Fisk Challenges and Evolution of the LHC Production Grid April 13, 2011 Ian Fisk 1 Evolution Uni x ALICE Remote Access PD2P/ Popularity Tier-2 Tier-2 Uni u Open Lab m Tier-2 Science Uni x Grid Uni z USA Tier-2

More information

Scientific data processing at global scale The LHC Computing Grid. fabio hernandez

Scientific data processing at global scale The LHC Computing Grid. fabio hernandez Scientific data processing at global scale The LHC Computing Grid Chengdu (China), July 5th 2011 Who I am 2 Computing science background Working in the field of computing for high-energy physics since

More information

The CMS Computing Model

The CMS Computing Model The CMS Computing Model Dorian Kcira California Institute of Technology SuperComputing 2009 November 14-20 2009, Portland, OR CERN s Large Hadron Collider 5000+ Physicists/Engineers 300+ Institutes 70+

More information

Federated data storage system prototype for LHC experiments and data intensive science

Federated data storage system prototype for LHC experiments and data intensive science Federated data storage system prototype for LHC experiments and data intensive science A. Kiryanov 1,2,a, A. Klimentov 1,3,b, D. Krasnopevtsev 1,4,c, E. Ryabinkin 1,d, A. Zarochentsev 1,5,e 1 National

More information

WORK PROJECT REPORT: TAPE STORAGE AND CRC PROTECTION

WORK PROJECT REPORT: TAPE STORAGE AND CRC PROTECTION WORK PROJECT REPORT: TAPE STORAGE AND CRC PROTECTION CERN Summer Student Programme 2014 Student: Main supervisor: Second supervisor: Division: Karel Ha Julien Marcel Leduc

More information

Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF

Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment

More information

The ATLAS EventIndex: Full chain deployment and first operation

The ATLAS EventIndex: Full chain deployment and first operation The ATLAS EventIndex: Full chain deployment and first operation Álvaro Fernández Casaní Instituto de Física Corpuscular () Universitat de València CSIC On behalf of the ATLAS Collaboration 1 Outline ATLAS

More information

Towards Network Awareness in LHC Computing

Towards Network Awareness in LHC Computing Towards Network Awareness in LHC Computing CMS ALICE CERN Atlas LHCb LHC Run1: Discovery of a New Boson LHC Run2: Beyond the Standard Model Gateway to a New Era Artur Barczyk / Caltech Internet2 Technology

More information

An Analysis of Storage Interface Usages at a Large, MultiExperiment Tier 1

An Analysis of Storage Interface Usages at a Large, MultiExperiment Tier 1 Journal of Physics: Conference Series PAPER OPEN ACCESS An Analysis of Storage Interface Usages at a Large, MultiExperiment Tier 1 Related content - Topical Review W W Symes - MAP Mission C. L. Bennett,

More information

Data Transfers Between LHC Grid Sites Dorian Kcira

Data Transfers Between LHC Grid Sites Dorian Kcira Data Transfers Between LHC Grid Sites Dorian Kcira dkcira@caltech.edu Caltech High Energy Physics Group hep.caltech.edu/cms CERN Site: LHC and the Experiments Large Hadron Collider 27 km circumference

More information

The High Performance Archiver for the LHC Experiments

The High Performance Archiver for the LHC Experiments The High Performance Archiver for the LHC Experiments Manuel Gonzalez Berges CERN, Geneva (Switzerland) CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Outline Context Archiving in PVSS

More information

New strategies of the LHC experiments to meet the computing requirements of the HL-LHC era

New strategies of the LHC experiments to meet the computing requirements of the HL-LHC era to meet the computing requirements of the HL-LHC era NPI AS CR Prague/Rez E-mail: adamova@ujf.cas.cz Maarten Litmaath CERN E-mail: Maarten.Litmaath@cern.ch The performance of the Large Hadron Collider

More information

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. Presented by Manfred Alef Contributions of Jos van Wezel, Andreas Heiss

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. Presented by Manfred Alef Contributions of Jos van Wezel, Andreas Heiss Site Report Presented by Manfred Alef Contributions of Jos van Wezel, Andreas Heiss Grid Computing Centre Karlsruhe (GridKa) Forschungszentrum Karlsruhe Institute for Scientific Computing Hermann-von-Helmholtz-Platz

More information

THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES

THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES 1 THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES Vincent Garonne, Mario Lassnig, Martin Barisits, Thomas Beermann, Ralph Vigne, Cedric Serfon Vincent.Garonne@cern.ch ph-adp-ddm-lab@cern.ch XLDB

More information

Benoit DELAUNAY Benoit DELAUNAY 1

Benoit DELAUNAY Benoit DELAUNAY 1 Benoit DELAUNAY 20091023 Benoit DELAUNAY 1 CC-IN2P3 provides computing and storage for the 4 LHC experiments and many others (astro particles...) A long history of service sharing between experiments Some

More information

Overcoming Obstacles to Petabyte Archives

Overcoming Obstacles to Petabyte Archives Overcoming Obstacles to Petabyte Archives Mike Holland Grau Data Storage, Inc. 609 S. Taylor Ave., Unit E, Louisville CO 80027-3091 Phone: +1-303-664-0060 FAX: +1-303-664-1680 E-mail: Mike@GrauData.com

More information

CERN s Business Computing

CERN s Business Computing CERN s Business Computing Where Accelerated the infinitely by Large Pentaho Meets the Infinitely small Jan Janke Deputy Group Leader CERN Administrative Information Systems Group CERN World s Leading Particle

More information

The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data

The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data D. Barberis 1*, J. Cranshaw 2, G. Dimitrov 3, A. Favareto 1, Á. Fernández Casaní 4, S. González de la Hoz 4, J.

More information

dcache Ceph Integration

dcache Ceph Integration dcache Ceph Integration Paul Millar for dcache Team ADC TIM at CERN 2016 06 16 https://indico.cern.ch/event/438205/ Many slides stolen fromdonated by Tigran Mkrtchyan dcache as Storage System Provides

More information

Storage Management Solutions. The Case For Efficient Data Management In Broadcast

Storage Management Solutions. The Case For Efficient Data Management In Broadcast Storage Management Solutions The Case For Efficient Data Management In Broadcast Overview Definitions Data Management Subsystem Integrated Environment Data Content or Data as used in this presentation,

More information

Grid Computing Activities at KIT

Grid Computing Activities at KIT Grid Computing Activities at KIT Meeting between NCP and KIT, 21.09.2015 Manuel Giffels Karlsruhe Institute of Technology Institute of Experimental Nuclear Physics & Steinbuch Center for Computing Courtesy

More information

Data Handling for LHC: Plans and Reality

Data Handling for LHC: Plans and Reality Data Handling for LHC: Plans and Reality Tony Cass Leader, Database Services Group Information Technology Department 11 th July 2012 1 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique

More information

where the Web was born Experience of Adding New Architectures to the LCG Production Environment

where the Web was born Experience of Adding New Architectures to the LCG Production Environment where the Web was born Experience of Adding New Architectures to the LCG Production Environment Andreas Unterkircher, openlab fellow Sverre Jarp, CTO CERN openlab Industrializing the Grid openlab Workshop

More information

Evolution of Database Replication Technologies for WLCG

Evolution of Database Replication Technologies for WLCG Evolution of Database Replication Technologies for WLCG Zbigniew Baranowski, Lorena Lobato Pardavila, Marcin Blaszczyk, Gancho Dimitrov, Luca Canali European Organisation for Nuclear Research (CERN), CH-1211

More information

and the GridKa mass storage system Jos van Wezel / GridKa

and the GridKa mass storage system Jos van Wezel / GridKa and the GridKa mass storage system / GridKa [Tape TSM] staging server 2 Introduction Grid storage and storage middleware dcache h and TSS TSS internals Conclusion and further work 3 FZK/GridKa The GridKa

More information

SXL-4205Q LTO-8 Digital Archive

SXL-4205Q LTO-8 Digital Archive SXL-4205Q LTO-8 Digital Archive 540 TB LTO Archive System with 1 or 2 LTO-8 Drives Functionality 540 TB library capacity 1 or 2 IBM LTO-8 Drives Manages Unlimited Offline LTO Overview The SXL-4205Q system

More information

CSC Operating Systems Spring Lecture - XIX Storage and I/O - II. Tevfik Koşar. Louisiana State University.

CSC Operating Systems Spring Lecture - XIX Storage and I/O - II. Tevfik Koşar. Louisiana State University. CSC 4103 - Operating Systems Spring 2007 Lecture - XIX Storage and I/O - II Tevfik Koşar Louisiana State University April 10 th, 2007 1 RAID Structure As disks get cheaper, adding multiple disks to the

More information

RAID Structure. RAID Levels. RAID (cont) RAID (0 + 1) and (1 + 0) Tevfik Koşar. Hierarchical Storage Management (HSM)

RAID Structure. RAID Levels. RAID (cont) RAID (0 + 1) and (1 + 0) Tevfik Koşar. Hierarchical Storage Management (HSM) CSC 4103 - Operating Systems Spring 2007 Lecture - XIX Storage and I/O - II Tevfik Koşar RAID Structure As disks get cheaper, adding multiple disks to the same system provides increased storage space,

More information

A Simple Mass Storage System for the SRB Data Grid

A Simple Mass Storage System for the SRB Data Grid A Simple Mass Storage System for the SRB Data Grid Michael Wan, Arcot Rajasekar, Reagan Moore, Phil Andrews San Diego Supercomputer Center SDSC/UCSD/NPACI Outline Motivations for implementing a Mass Storage

More information

Understanding StoRM: from introduction to internals

Understanding StoRM: from introduction to internals Understanding StoRM: from introduction to internals 13 November 2007 Outline Storage Resource Manager The StoRM service StoRM components and internals Deployment configuration Authorization and ACLs Conclusions.

More information

Solution Brief: Archiving with Harmonic Media Application Server and ProXplore

Solution Brief: Archiving with Harmonic Media Application Server and ProXplore Solution Brief: Archiving with Harmonic Media Application Server and ProXplore Summary Harmonic Media Application Server (MAS) provides management of content across the Harmonic server and storage infrastructure.

More information

A scalable storage element and its usage in HEP

A scalable storage element and its usage in HEP AstroGrid D Meeting at MPE 14 15. November 2006 Garching dcache A scalable storage element and its usage in HEP Martin Radicke Patrick Fuhrmann Introduction to dcache 2 Project overview joint venture between

More information

CASTORFS - A filesystem to access CASTOR

CASTORFS - A filesystem to access CASTOR Journal of Physics: Conference Series CASTORFS - A filesystem to access CASTOR To cite this article: Alexander Mazurov and Niko Neufeld 2010 J. Phys.: Conf. Ser. 219 052023 View the article online for

More information

Evolution of Database Replication Technologies for WLCG

Evolution of Database Replication Technologies for WLCG Journal of Physics: Conference Series PAPER OPEN ACCESS Evolution of Database Replication Technologies for WLCG To cite this article: Zbigniew Baranowski et al 2015 J. Phys.: Conf. Ser. 664 042032 View

More information

UW-ATLAS Experiences with Condor

UW-ATLAS Experiences with Condor UW-ATLAS Experiences with Condor M.Chen, A. Leung, B.Mellado Sau Lan Wu and N.Xu Paradyn / Condor Week, Madison, 05/01/08 Outline Our first success story with Condor - ATLAS production in 2004~2005. CRONUS

More information

Storage on the Lunatic Fringe. Thomas M. Ruwart University of Minnesota Digital Technology Center Intelligent Storage Consortium

Storage on the Lunatic Fringe. Thomas M. Ruwart University of Minnesota Digital Technology Center Intelligent Storage Consortium Storage on the Lunatic Fringe Thomas M. Ruwart University of Minnesota Digital Technology Center Intelligent Storage Consortium tmruwart@dtc.umn.edu Orientation Who are the lunatics? What are their requirements?

More information

System level traffic shaping in disk servers with heterogeneous protocols

System level traffic shaping in disk servers with heterogeneous protocols Journal of Physics: Conference Series OPEN ACCESS System level traffic shaping in disk servers with heterogeneous protocols To cite this article: Eric Cano and Daniele Francesco Kruse 14 J. Phys.: Conf.

More information

Evolution of Cloud Computing in ATLAS

Evolution of Cloud Computing in ATLAS The Evolution of Cloud Computing in ATLAS Ryan Taylor on behalf of the ATLAS collaboration 1 Outline Cloud Usage and IaaS Resource Management Software Services to facilitate cloud use Sim@P1 Performance

More information

ISTITUTO NAZIONALE DI FISICA NUCLEARE

ISTITUTO NAZIONALE DI FISICA NUCLEARE ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Perugia INFN/TC-05/10 July 4, 2005 DESIGN, IMPLEMENTATION AND CONFIGURATION OF A GRID SITE WITH A PRIVATE NETWORK ARCHITECTURE Leonello Servoli 1,2!, Mirko

More information

Cooperation among ALICE Storage Elements: current status and directions (The ALICE Global Redirector: a step towards real storage robustness).

Cooperation among ALICE Storage Elements: current status and directions (The ALICE Global Redirector: a step towards real storage robustness). Cooperation among ALICE Storage Elements: current status and directions (The ALICE Global Redirector: a step towards real storage robustness). 1 CERN Geneve 23, CH-1211, Switzerland E-mail: fabrizio.furano@cern.ch

More information

High-Energy Physics Data-Storage Challenges

High-Energy Physics Data-Storage Challenges High-Energy Physics Data-Storage Challenges Richard P. Mount SLAC SC2003 Experimental HENP Understanding the quantum world requires: Repeated measurement billions of collisions Large (500 2000 physicist)

More information

The INFN Tier1. 1. INFN-CNAF, Italy

The INFN Tier1. 1. INFN-CNAF, Italy IV WORKSHOP ITALIANO SULLA FISICA DI ATLAS E CMS BOLOGNA, 23-25/11/2006 The INFN Tier1 L. dell Agnello 1), D. Bonacorsi 1), A. Chierici 1), M. Donatelli 1), A. Italiano 1), G. Lo Re 1), B. Martelli 1),

More information

LCG data management at IN2P3 CC FTS SRM dcache HPSS

LCG data management at IN2P3 CC FTS SRM dcache HPSS jeudi 26 avril 2007 LCG data management at IN2P3 CC FTS SRM dcache HPSS Jonathan Schaeffer / Lionel Schwarz dcachemaster@cc.in2p3.fr dcache Joint development by FNAL and DESY Cache disk manager with unique

More information

LHCb Computing Strategy

LHCb Computing Strategy LHCb Computing Strategy Nick Brook Computing Model 2008 needs Physics software Harnessing the Grid DIRC GNG Experience & Readiness HCP, Elba May 07 1 Dataflow RW data is reconstructed: e.g. Calo. Energy

More information

Operating system Dr. Shroouq J.

Operating system Dr. Shroouq J. 2.2.2 DMA Structure In a simple terminal-input driver, when a line is to be read from the terminal, the first character typed is sent to the computer. When that character is received, the asynchronous-communication

More information

Multi-threaded, discrete event simulation of distributed computing systems

Multi-threaded, discrete event simulation of distributed computing systems Multi-threaded, discrete event simulation of distributed computing systems Iosif C. Legrand California Institute of Technology, Pasadena, CA, U.S.A Abstract The LHC experiments have envisaged computing

More information

Batch Services at CERN: Status and Future Evolution

Batch Services at CERN: Status and Future Evolution Batch Services at CERN: Status and Future Evolution Helge Meinhard, CERN-IT Platform and Engineering Services Group Leader HTCondor Week 20 May 2015 20-May-2015 CERN batch status and evolution - Helge

More information

Lecture Topics. Announcements. Today: Uniprocessor Scheduling (Stallings, chapter ) Next: Advanced Scheduling (Stallings, chapter

Lecture Topics. Announcements. Today: Uniprocessor Scheduling (Stallings, chapter ) Next: Advanced Scheduling (Stallings, chapter Lecture Topics Today: Uniprocessor Scheduling (Stallings, chapter 9.1-9.3) Next: Advanced Scheduling (Stallings, chapter 10.1-10.4) 1 Announcements Self-Study Exercise #10 Project #8 (due 11/16) Project

More information

Oracle StorageTek. Dr. James Cates Senior Vice President Archive Development. Public

Oracle StorageTek. Dr. James Cates Senior Vice President Archive Development. Public Oracle StorageTek Dr. James Cates Senior Vice President Archive Development james.cates@oracle.com Public Don't try to frighten us with your sorcerer's ways, Lord Vader. Your sad devotion to that ancient

More information

Virtualizing a Batch. University Grid Center

Virtualizing a Batch. University Grid Center Virtualizing a Batch Queuing System at a University Grid Center Volker Büge (1,2), Yves Kemp (1), Günter Quast (1), Oliver Oberst (1), Marcel Kunze (2) (1) University of Karlsruhe (2) Forschungszentrum

More information

ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective. Part I: Operating system overview: Processes and threads

ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective. Part I: Operating system overview: Processes and threads ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective Part I: Operating system overview: Processes and threads 1 Overview Process concept Process scheduling Thread

More information

ANSE: Advanced Network Services for [LHC] Experiments

ANSE: Advanced Network Services for [LHC] Experiments ANSE: Advanced Network Services for [LHC] Experiments Artur Barczyk California Institute of Technology Joint Techs 2013 Honolulu, January 16, 2013 Introduction ANSE is a project funded by NSF s CC-NIE

More information

Distributing storage of LHC data - in the nordic countries

Distributing storage of LHC data - in the nordic countries Distributing storage of LHC data - in the nordic countries Gerd Behrmann INTEGRATE ASG Lund, May 11th, 2016 Agenda WLCG: A world wide computing grid for the LHC NDGF: The Nordic Tier 1 dcache: Distributed

More information

Operating the Distributed NDGF Tier-1

Operating the Distributed NDGF Tier-1 Operating the Distributed NDGF Tier-1 Michael Grønager Technical Coordinator, NDGF International Symposium on Grid Computing 08 Taipei, April 10th 2008 Talk Outline What is NDGF? Why a distributed Tier-1?

More information

CLOUDS OF JINR, UNIVERSITY OF SOFIA AND INRNE JOIN TOGETHER

CLOUDS OF JINR, UNIVERSITY OF SOFIA AND INRNE JOIN TOGETHER CLOUDS OF JINR, UNIVERSITY OF SOFIA AND INRNE JOIN TOGETHER V.V. Korenkov 1, N.A. Kutovskiy 1, N.A. Balashov 1, V.T. Dimitrov 2,a, R.D. Hristova 2, K.T. Kouzmov 2, S.T. Hristov 3 1 Laboratory of Information

More information

Storage and I/O requirements of the LHC experiments

Storage and I/O requirements of the LHC experiments Storage and I/O requirements of the LHC experiments Sverre Jarp CERN openlab, IT Dept where the Web was born 22 June 2006 OpenFabrics Workshop, Paris 1 Briefly about CERN 22 June 2006 OpenFabrics Workshop,

More information

The JINR Tier1 Site Simulation for Research and Development Purposes

The JINR Tier1 Site Simulation for Research and Development Purposes EPJ Web of Conferences 108, 02033 (2016) DOI: 10.1051/ epjconf/ 201610802033 C Owned by the authors, published by EDP Sciences, 2016 The JINR Tier1 Site Simulation for Research and Development Purposes

More information

Comp 310 Computer Systems and Organization

Comp 310 Computer Systems and Organization Comp 310 Computer Systems and Organization Lecture #9 Process Management (CPU Scheduling) 1 Prof. Joseph Vybihal Announcements Oct 16 Midterm exam (in class) In class review Oct 14 (½ class review) Ass#2

More information

The PanDA System in the ATLAS Experiment

The PanDA System in the ATLAS Experiment 1a, Jose Caballero b, Kaushik De a, Tadashi Maeno b, Maxim Potekhin b, Torre Wenaus b on behalf of the ATLAS collaboration a University of Texas at Arlington, Science Hall, PO Box 19059, Arlington, TX

More information

Operating System: Chap13 I/O Systems. National Tsing-Hua University 2016, Fall Semester

Operating System: Chap13 I/O Systems. National Tsing-Hua University 2016, Fall Semester Operating System: Chap13 I/O Systems National Tsing-Hua University 2016, Fall Semester Outline Overview I/O Hardware I/O Methods Kernel I/O Subsystem Performance Application Interface Operating System

More information

Tier 3 batch system data locality via managed caches

Tier 3 batch system data locality via managed caches Journal of Physics: Conference Series PAPER OPEN ACCESS Tier 3 batch system data locality via managed caches To cite this article: Max Fischer et al 2015 J. Phys.: Conf. Ser. 608 012018 Recent citations

More information

Preparing for High-Luminosity LHC. Bob Jones CERN Bob.Jones <at> cern.ch

Preparing for High-Luminosity LHC. Bob Jones CERN Bob.Jones <at> cern.ch Preparing for High-Luminosity LHC Bob Jones CERN Bob.Jones cern.ch The Mission of CERN Push back the frontiers of knowledge E.g. the secrets of the Big Bang what was the matter like within the first

More information

DIRAC data management: consistency, integrity and coherence of data

DIRAC data management: consistency, integrity and coherence of data Journal of Physics: Conference Series DIRAC data management: consistency, integrity and coherence of data To cite this article: M Bargiotti and A C Smith 2008 J. Phys.: Conf. Ser. 119 062013 Related content

More information

IBM Spectrum Protect Version Introduction to Data Protection Solutions IBM

IBM Spectrum Protect Version Introduction to Data Protection Solutions IBM IBM Spectrum Protect Version 8.1.2 Introduction to Data Protection Solutions IBM IBM Spectrum Protect Version 8.1.2 Introduction to Data Protection Solutions IBM Note: Before you use this information

More information

LCG Conditions Database Project

LCG Conditions Database Project Computing in High Energy and Nuclear Physics (CHEP 2006) TIFR, Mumbai, 13 Feb 2006 LCG Conditions Database Project COOL Development and Deployment: Status and Plans On behalf of the COOL team (A.V., D.Front,

More information

Chapter 2 CommVault Data Management Concepts

Chapter 2 CommVault Data Management Concepts Chapter 2 CommVault Data Management Concepts 10 - CommVault Data Management Concepts The Simpana product suite offers a wide range of features and options to provide great flexibility in configuring and

More information

SXL-8: LTO-8 Archive System. managed by XenData6 Server software. Eight cartridge library simplifies management of your offline files.

SXL-8: LTO-8 Archive System. managed by XenData6 Server software. Eight cartridge library simplifies management of your offline files. SXL-8: LTO-8 Archive System LTO-8 Archive System managed by XenData6 Server software Eight cartridge library simplifies management of your offline files Functionality Unlimited Offline LTO capacity 84

More information

IBM Tivoli Storage Manager Version Introduction to Data Protection Solutions IBM

IBM Tivoli Storage Manager Version Introduction to Data Protection Solutions IBM IBM Tivoli Storage Manager Version 7.1.6 Introduction to Data Protection Solutions IBM IBM Tivoli Storage Manager Version 7.1.6 Introduction to Data Protection Solutions IBM Note: Before you use this

More information

White Paper. Major Performance Tuning Considerations for Weblogic Server

White Paper. Major Performance Tuning Considerations for Weblogic Server White Paper Major Performance Tuning Considerations for Weblogic Server Table of Contents Introduction and Background Information... 2 Understanding the Performance Objectives... 3 Measuring your Performance

More information

Monitoring system for geographically distributed datacenters based on Openstack. Gioacchino Vino

Monitoring system for geographically distributed datacenters based on Openstack. Gioacchino Vino Monitoring system for geographically distributed datacenters based on Openstack Gioacchino Vino Tutor: Dott. Domenico Elia Tutor: Dott. Giacinto Donvito Borsa di studio GARR Orio Carlini 2016-2017 INFN

More information

CERN Open Data and Data Analysis Knowledge Preservation

CERN Open Data and Data Analysis Knowledge Preservation CERN Open Data and Data Analysis Knowledge Preservation Tibor Šimko Digital Library 2015 21 23 April 2015 Jasná, Slovakia @tiborsimko @inveniosoftware 1 / 26 1 Invenio @tiborsimko @inveniosoftware 2 /

More information

Chapter 1: Introduction. Operating System Concepts 8 th Edition,

Chapter 1: Introduction. Operating System Concepts 8 th Edition, Chapter 1: Introduction Operating System Concepts 8 th Edition, Silberschatz, Galvin and Gagne 2009 Operating-System Operations Interrupt driven by hardware Software error or system request creates exception

More information

SXL-8500 Series: LTO Digital Video Archives

SXL-8500 Series: LTO Digital Video Archives SXL-8500 Series: LTO Digital Video Archives Scalable LTO-7 Archive System 420 TB to 3.3 PB managed by XenData6 Server software Functionality Standard File-Folder Interface Object Storage Interface Writes

More information

Ideas for evolution of replication CERN

Ideas for evolution of replication CERN Ideas for evolution of replication technology @ CERN Openlab Minor Review December 14 th, 2010 Zbigniew Baranowski, IT-DB Outline Replication use cases at CERN Motivation for evolution of replication Oracle

More information

SXL-6500 Series: LTO Digital Video Archives

SXL-6500 Series: LTO Digital Video Archives SXL-6500 Series: LTO Digital Video Archives Scalable LTO-7 Archive System 180 TB to 1.8 PB managed by XenData6 Server software Functionality Standard File-Folder Interface Object Storage Interface Writes

More information

HammerCloud: A Stress Testing System for Distributed Analysis

HammerCloud: A Stress Testing System for Distributed Analysis HammerCloud: A Stress Testing System for Distributed Analysis Daniel C. van der Ster 1, Johannes Elmsheuser 2, Mario Úbeda García 1, Massimo Paladin 1 1: CERN, Geneva, Switzerland 2: Ludwig-Maximilians-Universität

More information

Tests of PROOF-on-Demand with ATLAS Prodsys2 and first experience with HTTP federation

Tests of PROOF-on-Demand with ATLAS Prodsys2 and first experience with HTTP federation Journal of Physics: Conference Series PAPER OPEN ACCESS Tests of PROOF-on-Demand with ATLAS Prodsys2 and first experience with HTTP federation To cite this article: R. Di Nardo et al 2015 J. Phys.: Conf.

More information

XenData Product Brief: SX-550 Series Servers for Sony Optical Disc Archives

XenData Product Brief: SX-550 Series Servers for Sony Optical Disc Archives XenData Product Brief: SX-550 Series Servers for Sony Optical Disc Archives The SX-550 Series of Archive Servers creates highly scalable Optical Disc Digital Video Archives that are optimized for broadcasters,

More information

Detector Control LHC

Detector Control LHC Detector Control Systems @ LHC Matthias Richter Department of Physics, University of Oslo IRTG Lecture week Autumn 2012 Oct 18 2012 M. Richter (UiO) DCS @ LHC Oct 09 2012 1 / 39 Detectors in High Energy

More information

The LHC Computing Grid

The LHC Computing Grid The LHC Computing Grid Gergely Debreczeni (CERN IT/Grid Deployment Group) The data factory of LHC 40 million collisions in each second After on-line triggers and selections, only 100 3-4 MB/event requires

More information

RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP

RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP THE EUROPEAN ORGANISATION FOR PARTICLE PHYSICS RESEARCH (CERN) 2 THE LARGE HADRON COLLIDER THE LARGE HADRON COLLIDER

More information

Milestone Solution Partner IT Infrastructure Components Certification Report

Milestone Solution Partner IT Infrastructure Components Certification Report Milestone Solution Partner IT Infrastructure Components Certification Report HUAWEI OceanStor 2800 V3 27-08-2015 Table of Contents Executive Summary:... 4 Abstract... 4 Certified Products... 4 Key Findings...

More information

XenData SX-10 V2 LTO Video Archive Appliance

XenData SX-10 V2 LTO Video Archive Appliance XenData SX-10 V2 LTO Video Archive Appliance Managed by XenData6 Server Software Overview The XenData SX-10 V2 appliance manages a robotic LTO tape library or stand-alone LTO tape drives and creates a

More information