Grid Data Management
|
|
- Liliana Bradley
- 6 years ago
- Views:
Transcription
1 Grid Data Management Week #4 Hardi Teder University of Tartu March 6th 2013
2 Overview Grid Data Management Where the Data comes from? Grid Data Management tools 2/33
3 Grid foundations 3/33
4 Where the data comes from? CERN's LHC CMS experiment example CERN European Organization for Nuclear Research LHC Large Hadron Collider CMS Compact Muon Solenoid 4/33
5 Grid acronyms EGI Glossary Goole search helps EGI Security Policy Glossary of Terms 5/33
6 Large Hadron Collider (LHC) 6/33
7 Smash things together, see what happens! 7/33
8 Discover particles Quarks Leptons Leptons Quarks up charm electron muon electron neutrino muon neutrino tau top down strange bottom tau neutrino 8/33
9 Large Hadron Collider (LHC) 9/33
10 CMS detector Took ~2000 scientists and engineers more than 20 years to design and build Is about 15 metres wide and 21.5 metres long Weighs twice as much as the Eiffel Tower about 14000t Uses the largest, most powerful magnet of its kind ever made 10/33
11 11/33
12 12/33
13 Collisions in CMS 13/33
14 CMS in production volume: ~250 TB/day among dozens of Tiers # files: ~19M logical files (but total of replicas so far is ~27M) throughput: GB/s aggregate (weekly averages) in peak weeks in /33
15 Worldwide LHC Computing Grid (WLCG) Tier0 at CERN 11 Tier1 sites 138 Tier2 sites 15/33
16 WLCG 15 Petabytes of data annually generated 16/33
17 There are more projects DNA experiments Radio telescopes Sensor networks Digitalizing data: books, documents, images 17/33
18 Grid foundations 18/33
19 Data management Data access and transfer Simple, automatic multi-protocol file transfer tools: Integrated with Resource Management service Move data from local machine to remote machine, where the job is executed (input file staging) Move the output files from the remote computer to the local machines (output file staging) Pull executable from a remote location To have a secure, high-performance, reliable file transfer over modern WANs: GridFTP Data replication and management 19/33
20 ARC Computing Element (CE) Universal frontend for different batch systems Standard and custom interfaces Status information publishing File handling 20/33
21 ARC CE and data handling Data are moved by the users and/or by the ARC Frequently used files are cached at the execution sites Cached files are indexed 21/33
22 ARC CE internals All services are only in the frontend Grid users are mapped to local identities Use /tmp/user for files witch are actively used 22/33
23 ARC UI data manipulation arcls to list contents and view some attributes of objects of a specified (by a URL) remote directory arccp a tool to copy files over the Grid arcrm allows users to erase files and directories at any location specified by a valid URL arcmkdir allows users to create directories, if the protocol of the specified URL supports it 23/33
24 ARC URLs ftp ordinary File Transfer Protocol (FTP) gsiftp GridFTP, the Globus - enhanced FTP protocol with security, encryption, etc. developed by The Globus Alliance http ordinary Hyper-Text Transfer Protocol (HTTP) with PUT and GET methods using multiple streams https HTTP with SSL v3 httpg HTTP with Globus GSI ldap ordinary Lightweight Data Access Protocol (LDAP) [9] lfc LFC catalog and indexing service of glite [1] srm Storage Resource Manager (SRM) service [7] root Xrootd protocol (read-only, available in ARC and later) file local to the host le name with a full path 24/33
25 An URL can be used: In standard form: protocol://[host[:port]]/file Or, to enhance the performance protocol://[host[:port]][;option[;option[...]]]/file protocol://[url[ /lfn[:metadataoption[:metadataoption[...]]] protocol://[;commonoption[;commonoption] ][url[ [;option[;option[...]]/lfn[:metadataoption[:metadataoption[...]]] 25/33
26 URL examples ARC UI arcls lfc://lfc.balticgrid.org/grid/balticgrid/bgcc2013/lab4/ arcls -l gsiftp://se.grid.eenet.ee/storage/balticgrid/bgcc2013 XRSL to store the job output to storage (optputfiles=("jobhugeoutputfile.tgz" "gsiftp://se.grid.eenet.ee/storage/balticgrid/bgcc2013/user/")) 26/33
27 GridFTP The GSIFTP protocol offers the functionalities of FTP, but with support for GSI. Supported by all VOs in Gird arccp gsiftp://lscf.nbi.dk:2811/jobs/ /jo b.out job.out
28 File Catalogue (LFC) Users and applications need to locate files (or replicas) on the Grid. The File Catalogue is the service which maintains mappings between LFN(s), GUID and SURL(s). lfc://lfc.balticgrid.org/grid/balticgrid/bgcc2013/lab4/p 4_data.test Lfc:P4_data.test
29 Relationships between tables 29/33
30 LFC environment!/bin/bash export LCG_GFAL_INFOSYS=bdii.balticgrid.org:2170 export LCG_CATALOG_TYPE=lfc export LFC_HOST=lfc.balticgrid.org echo -e 'Prindin muutujaid: LCG_GFAL_INFOSYS; LCG_CATALOG_TYPE; LFC_HOST \n' echo $LCG_GFAL_INFOSYS; echo $LCG_CATALOG_TYPE; echo $LFC_HOST export LFC_HOME=/grid/balticgrid/BGCC2012/Hardi_Teder
31 Clean up after yourself Delete the files you don't use any more 31/33
32 References I used several pictures from: CMS experiment public presentations: NorduGrid repository FREEIMAGES.co.uk More information about ARC Data Management: 32/33
33 Thank you More information from: Hardi Teder 33/33
Scientific data management
Scientific data management Storage and data management components Application database Certificate Certificate Authorised users directory Certificate Certificate Researcher Certificate Policies Information
More informationIntroduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008
Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, 13-14 November 2008 Outline Introduction SRM Storage Elements in glite LCG File Catalog (LFC) Information System Grid Tutorial, 13-14
More informationData Transfers Between LHC Grid Sites Dorian Kcira
Data Transfers Between LHC Grid Sites Dorian Kcira dkcira@caltech.edu Caltech High Energy Physics Group hep.caltech.edu/cms CERN Site: LHC and the Experiments Large Hadron Collider 27 km circumference
More information30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy
Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of
More informationOperating the Distributed NDGF Tier-1
Operating the Distributed NDGF Tier-1 Michael Grønager Technical Coordinator, NDGF International Symposium on Grid Computing 08 Taipei, April 10th 2008 Talk Outline What is NDGF? Why a distributed Tier-1?
More informationLessons Learned in the NorduGrid Federation
Lessons Learned in the NorduGrid Federation David Cameron University of Oslo With input from Gerd Behrmann, Oxana Smirnova and Mattias Wadenstein Creating Federated Data Stores For The LHC 14.9.12, Lyon,
More informationPoS(EGICF12-EMITC2)106
DDM Site Services: A solution for global replication of HEP data Fernando Harald Barreiro Megino 1 E-mail: fernando.harald.barreiro.megino@cern.ch Simone Campana E-mail: simone.campana@cern.ch Vincent
More informationData transfer over the wide area network with a large round trip time
Journal of Physics: Conference Series Data transfer over the wide area network with a large round trip time To cite this article: H Matsunaga et al 1 J. Phys.: Conf. Ser. 219 656 Recent citations - A two
More informationIntroduction to SRM. Riccardo Zappi 1
Introduction to SRM Grid Storage Resource Manager Riccardo Zappi 1 1 INFN-CNAF, National Center of INFN (National Institute for Nuclear Physic) for Research and Development into the field of Information
More informationChallenges and Evolution of the LHC Production Grid. April 13, 2011 Ian Fisk
Challenges and Evolution of the LHC Production Grid April 13, 2011 Ian Fisk 1 Evolution Uni x ALICE Remote Access PD2P/ Popularity Tier-2 Tier-2 Uni u Open Lab m Tier-2 Science Uni x Grid Uni z USA Tier-2
More informationCernVM-FS beyond LHC computing
CernVM-FS beyond LHC computing C Condurache, I Collier STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot, OX11 0QX, UK E-mail: catalin.condurache@stfc.ac.uk Abstract. In the last three years
More informationGrid services. Enabling Grids for E-sciencE. Dusan Vudragovic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia
Grid services Dusan Vudragovic dusan@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Sep. 19, 2008 www.eu-egee.org Set of basic Grid services Job submission/management
More informationWorldwide Production Distributed Data Management at the LHC. Brian Bockelman MSST 2010, 4 May 2010
Worldwide Production Distributed Data Management at the LHC Brian Bockelman MSST 2010, 4 May 2010 At the LHC http://op-webtools.web.cern.ch/opwebtools/vistar/vistars.php?usr=lhc1 Gratuitous detector pictures:
More informationGrid Computing at the IIHE
BNC 2016 Grid Computing at the IIHE The Interuniversity Institute for High Energies S. Amary, F. Blekman, A. Boukil, O. Devroede, S. Gérard, A. Ouchene, R. Rougny, S. Rugovac, P. Vanlaer, R. Vandenbroucke
More informationData Storage. Paul Millar dcache
Data Storage Paul Millar dcache Overview Introducing storage How storage is used Challenges and future directions 2 (Magnetic) Hard Disks 3 Tape systems 4 Disk enclosures 5 RAID systems 6 Types of RAID
More informationglite Grid Services Overview
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite Grid Services Overview Antonio Calanducci INFN Catania Joint GISELA/EPIKH School for Grid Site Administrators Valparaiso,
More informationCMS Computing Model with Focus on German Tier1 Activities
CMS Computing Model with Focus on German Tier1 Activities Seminar über Datenverarbeitung in der Hochenergiephysik DESY Hamburg, 24.11.2008 Overview The Large Hadron Collider The Compact Muon Solenoid CMS
More informationStephen J. Gowdy (CERN) 12 th September 2012 XLDB Conference FINDING THE HIGGS IN THE HAYSTACK(S)
Stephen J. Gowdy (CERN) 12 th September 2012 XLDB Conference FINDING THE HIGGS IN THE HAYSTACK(S) Overview Large Hadron Collider (LHC) Compact Muon Solenoid (CMS) experiment The Challenge Worldwide LHC
More informationWHEN the Large Hadron Collider (LHC) begins operation
2228 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 53, NO. 4, AUGUST 2006 Measurement of the LCG2 and Glite File Catalogue s Performance Craig Munro, Birger Koblitz, Nuno Santos, and Akram Khan Abstract When
More informationCouchDB-based system for data management in a Grid environment Implementation and Experience
CouchDB-based system for data management in a Grid environment Implementation and Experience Hassen Riahi IT/SDC, CERN Outline Context Problematic and strategy System architecture Integration and deployment
More informationScientific data processing at global scale The LHC Computing Grid. fabio hernandez
Scientific data processing at global scale The LHC Computing Grid Chengdu (China), July 5th 2011 Who I am 2 Computing science background Working in the field of computing for high-energy physics since
More informationAGIS: The ATLAS Grid Information System
AGIS: The ATLAS Grid Information System Alexey Anisenkov 1, Sergey Belov 2, Alessandro Di Girolamo 3, Stavro Gayazov 1, Alexei Klimentov 4, Danila Oleynik 2, Alexander Senchenko 1 on behalf of the ATLAS
More informationOutline. ASP 2012 Grid School
Distributed Storage Rob Quick Indiana University Slides courtesy of Derek Weitzel University of Nebraska Lincoln Outline Storage Patterns in Grid Applications Storage
More informationGrid Computing. MCSN - N. Tonellotto - Distributed Enabling Platforms
Grid Computing 1 Resource sharing Elements of Grid Computing - Computers, data, storage, sensors, networks, - Sharing always conditional: issues of trust, policy, negotiation, payment, Coordinated problem
More informationUnderstanding StoRM: from introduction to internals
Understanding StoRM: from introduction to internals 13 November 2007 Outline Storage Resource Manager The StoRM service StoRM components and internals Deployment configuration Authorization and ACLs Conclusions.
More informationSPINOSO Vincenzo. Optimization of the job submission and data access in a LHC Tier2
EGI User Forum Vilnius, 11-14 April 2011 SPINOSO Vincenzo Optimization of the job submission and data access in a LHC Tier2 Overview User needs Administration issues INFN Bari farm design and deployment
More informationArchitecture Proposal
Nordic Testbed for Wide Area Computing and Data Handling NORDUGRID-TECH-1 19/02/2002 Architecture Proposal M.Ellert, A.Konstantinov, B.Kónya, O.Smirnova, A.Wäänänen Introduction The document describes
More informationExtended Resource Specification Language Reference Manual for ARC versions 0.8 and above
NORDUGRID NORDUGRID-MANUAL-4 7/7/2014 Extended Resource Specification Language Reference Manual for ARC versions 0.8 and above 2 Contents 1 Introduction 5 2 xrsl syntax and rules 7 2.1 Syntax...............................................
More informationData Management. Enabling Grids for E-sciencE. Vladimir Slavnic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia
Data Management Vladimir Slavnic slavnic@scl.rs Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Academic and Educational Grid Initiative of Serbia Feb. 19, 2009 A E G I S www.eu-egee.org
More informationHigh Throughput WAN Data Transfer with Hadoop-based Storage
High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San
More informationExtended Resource Specification Language Reference Manual for ARC versions 0.8 and above
NORDUGRID NORDUGRID-MANUAL-4 7/2/2018 Extended Resource Specification Language Reference Manual for ARC versions 0.8 and above 2 Contents 1 Introduction 5 2 xrsl syntax and rules 7 2.1 Syntax...............................................
More informationBenchmarking third-party-transfer protocols with the FTS
Benchmarking third-party-transfer protocols with the FTS Rizart Dona CERN Summer Student Programme 2018 Supervised by Dr. Simone Campana & Dr. Oliver Keeble 1.Introduction 1 Worldwide LHC Computing Grid
More informationData Management for the World s Largest Machine
Data Management for the World s Largest Machine Sigve Haug 1, Farid Ould-Saada 2, Katarina Pajchel 2, and Alexander L. Read 2 1 Laboratory for High Energy Physics, University of Bern, Sidlerstrasse 5,
More information( PROPOSAL ) THE AGATA GRID COMPUTING MODEL FOR DATA MANAGEMENT AND DATA PROCESSING. version 0.6. July 2010 Revised January 2011
( PROPOSAL ) THE AGATA GRID COMPUTING MODEL FOR DATA MANAGEMENT AND DATA PROCESSING version 0.6 July 2010 Revised January 2011 Mohammed Kaci 1 and Victor Méndez 1 For the AGATA collaboration 1 IFIC Grid
More informationEGI-InSPIRE. ARC-CE IPv6 TESTBED. Barbara Krašovec, Jure Kranjc ARNES. EGI-InSPIRE RI
EGI-InSPIRE ARC-CE IPv6 TESTBED Barbara Krašovec, Jure Kranjc ARNES Why ARC-CE over IPv6? - IPv4 exhaustion - On Friday 14 th, RIPE NCC has announced that the last /8 is being distributed from available
More informationThe LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland
The LCG 3D Project Maria Girone, CERN The rd Open Grid Forum - OGF 4th June 2008, Barcelona Outline Introduction The Distributed Database (3D) Project Streams Replication Technology and Performance Availability
More informationConference The Data Challenges of the LHC. Reda Tafirout, TRIUMF
Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment
More informationThe CMS Computing Model
The CMS Computing Model Dorian Kcira California Institute of Technology SuperComputing 2009 November 14-20 2009, Portland, OR CERN s Large Hadron Collider 5000+ Physicists/Engineers 300+ Institutes 70+
More informationATLAS Experiment and GCE
ATLAS Experiment and GCE Google IO Conference San Francisco, CA Sergey Panitkin (BNL) and Andrew Hanushevsky (SLAC), for the ATLAS Collaboration ATLAS Experiment The ATLAS is one of the six particle detectors
More informationBookkeeping and submission tools prototype. L. Tomassetti on behalf of distributed computing group
Bookkeeping and submission tools prototype L. Tomassetti on behalf of distributed computing group Outline General Overview Bookkeeping database Submission tools (for simulation productions) Framework Design
More informationThe glite middleware. Ariel Garcia KIT
The glite middleware Ariel Garcia KIT Overview Background The glite subsystems overview Security Information system Job management Data management Some (my) answers to your questions and random rumblings
More informationVirtualizing a Batch. University Grid Center
Virtualizing a Batch Queuing System at a University Grid Center Volker Büge (1,2), Yves Kemp (1), Günter Quast (1), Oliver Oberst (1), Marcel Kunze (2) (1) University of Karlsruhe (2) Forschungszentrum
More informationThe LHC Computing Grid
The LHC Computing Grid Visit of Finnish IT Centre for Science CSC Board Members Finland Tuesday 19 th May 2009 Frédéric Hemmer IT Department Head The LHC and Detectors Outline Computing Challenges Current
More informationA data Grid testbed environment in Gigabit WAN with HPSS
Computing in High Energy and Nuclear Physics, 24-28 March 23, La Jolla, California A data Grid testbed environment in Gigabit with HPSS Atsushi Manabe, Setsuya Kawabata, Youhei Morita, Takashi Sasaki,
More informationThe evolving role of Tier2s in ATLAS with the new Computing and Data Distribution model
Journal of Physics: Conference Series The evolving role of Tier2s in ATLAS with the new Computing and Data Distribution model To cite this article: S González de la Hoz 2012 J. Phys.: Conf. Ser. 396 032050
More informationHigh Performance Computing Course Notes Grid Computing I
High Performance Computing Course Notes 2008-2009 2009 Grid Computing I Resource Demands Even as computer power, data storage, and communication continue to improve exponentially, resource capacities are
More informationFile Access Optimization with the Lustre Filesystem at Florida CMS T2
Journal of Physics: Conference Series PAPER OPEN ACCESS File Access Optimization with the Lustre Filesystem at Florida CMS T2 To cite this article: P. Avery et al 215 J. Phys.: Conf. Ser. 664 4228 View
More informationRADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP
RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP THE EUROPEAN ORGANISATION FOR PARTICLE PHYSICS RESEARCH (CERN) 2 THE LARGE HADRON COLLIDER THE LARGE HADRON COLLIDER
More informationDistributing storage of LHC data - in the nordic countries
Distributing storage of LHC data - in the nordic countries Gerd Behrmann INTEGRATE ASG Lund, May 11th, 2016 Agenda WLCG: A world wide computing grid for the LHC NDGF: The Nordic Tier 1 dcache: Distributed
More informationFrom raw data to new fundamental particles: The data management lifecycle at the Large Hadron Collider
From raw data to new fundamental particles: The data management lifecycle at the Large Hadron Collider Andrew Washbrook School of Physics and Astronomy University of Edinburgh Dealing with Data Conference
More informationIEPSAS-Kosice: experiences in running LCG site
IEPSAS-Kosice: experiences in running LCG site Marian Babik 1, Dusan Bruncko 2, Tomas Daranyi 1, Ladislav Hluchy 1 and Pavol Strizenec 2 1 Department of Parallel and Distributed Computing, Institute of
More informationGrid Architectural Models
Grid Architectural Models Computational Grids - A computational Grid aggregates the processing power from a distributed collection of systems - This type of Grid is primarily composed of low powered computers
More informationPoS(ACAT2010)039. First sights on a non-grid end-user analysis model on Grid Infrastructure. Roberto Santinelli. Fabrizio Furano.
First sights on a non-grid end-user analysis model on Grid Infrastructure Roberto Santinelli CERN E-mail: roberto.santinelli@cern.ch Fabrizio Furano CERN E-mail: fabrzio.furano@cern.ch Andrew Maier CERN
More informationARC integration for CMS
ARC integration for CMS ARC integration for CMS Erik Edelmann 2, Laurence Field 3, Jaime Frey 4, Michael Grønager 2, Kalle Happonen 1, Daniel Johansson 2, Josva Kleist 2, Jukka Klem 1, Jesper Koivumäki
More informationARC middleware. The NorduGrid Collaboration
ARC middleware The NorduGrid Collaboration Abstract The paper describes the existing components of ARC, discusses some of the new components, functionalities and enhancements currently under development,
More informationHigh Energy Physics data analysis
escience Intrastructure T2-T3 T3 for High Energy Physics data analysis Presented by: Álvaro Fernandez Casani (Alvaro.Fernandez@ific.uv.es) IFIC Valencia (Spain) Santiago González de la Hoz, Gabriel Amorós,
More informationDepartment of Physics & Astronomy
Department of Physics & Astronomy Experimental Particle Physics Group Kelvin Building, University of Glasgow, Glasgow, G1 8QQ, Scotland Telephone: +44 ()141 339 8855 Fax: +44 ()141 33 5881 GLAS-PPE/7-3
More informationData Quality Monitoring at CMS with Machine Learning
Data Quality Monitoring at CMS with Machine Learning July-August 2016 Author: Aytaj Aghabayli Supervisors: Jean-Roch Vlimant Maurizio Pierini CERN openlab Summer Student Report 2016 Abstract The Data Quality
More informationExperience of Data Grid simulation packages using.
Experience of Data Grid simulation packages using. Nechaevskiy A.V. (SINP MSU), Korenkov V.V. (LIT JINR) Dubna, 2008 Contant Operation of LCG DataGrid Errors of FTS services of the Grid. Primary goals
More informationThe ATLAS PanDA Pilot in Operation
The ATLAS PanDA Pilot in Operation P. Nilsson 1, J. Caballero 2, K. De 1, T. Maeno 2, A. Stradling 1, T. Wenaus 2 for the ATLAS Collaboration 1 University of Texas at Arlington, Science Hall, P O Box 19059,
More informationReliability Engineering Analysis of ATLAS Data Reprocessing Campaigns
Journal of Physics: Conference Series OPEN ACCESS Reliability Engineering Analysis of ATLAS Data Reprocessing Campaigns To cite this article: A Vaniachine et al 2014 J. Phys.: Conf. Ser. 513 032101 View
More informationAccess the power of Grid with Eclipse
Access the power of Grid with Eclipse Harald Kornmayer (Forschungszentrum Karlsruhe GmbH) Markus Knauer (Innoopract GmbH) October 11th, 2006, Eclipse Summit, Esslingen 2006 by H. Kornmayer, M. Knauer;
More informationThe ATLAS Tier-3 in Geneva and the Trigger Development Facility
Journal of Physics: Conference Series The ATLAS Tier-3 in Geneva and the Trigger Development Facility To cite this article: S Gadomski et al 2011 J. Phys.: Conf. Ser. 331 052026 View the article online
More informationA distributed tier-1. International Conference on Computing in High Energy and Nuclear Physics (CHEP 07) IOP Publishing. c 2008 IOP Publishing Ltd 1
A distributed tier-1 L Fischer 1, M Grønager 1, J Kleist 2 and O Smirnova 3 1 NDGF - Nordic DataGrid Facilty, Kastruplundgade 22(1), DK-2770 Kastrup 2 NDGF and Aalborg University, Department of Computer
More informationTravelling securely on the Grid to the origin of the Universe
1 Travelling securely on the Grid to the origin of the Universe F-Secure SPECIES 2007 conference Wolfgang von Rüden 1 Head, IT Department, CERN, Geneva 24 January 2007 2 CERN stands for over 50 years of
More informationEvaluation of the computing resources required for a Nordic research exploitation of the LHC
PROCEEDINGS Evaluation of the computing resources required for a Nordic research exploitation of the LHC and Sverker Almehed, Chafik Driouichi, Paula Eerola, Ulf Mjörnmark, Oxana Smirnova,TorstenÅkesson
More informationMonte Carlo Production on the Grid by the H1 Collaboration
Journal of Physics: Conference Series Monte Carlo Production on the Grid by the H1 Collaboration To cite this article: E Bystritskaya et al 2012 J. Phys.: Conf. Ser. 396 032067 Recent citations - Monitoring
More informationGRID COMPUTING APPLIED TO OFF-LINE AGATA DATA PROCESSING. 2nd EGAN School, December 2012, GSI Darmstadt, Germany
GRID COMPUTING APPLIED TO OFF-LINE AGATA DATA PROCESSING M. KACI mohammed.kaci@ific.uv.es 2nd EGAN School, 03-07 December 2012, GSI Darmstadt, Germany GRID COMPUTING TECHNOLOGY THE EUROPEAN GRID: HISTORY
More informationDistributed Data Management on the Grid. Mario Lassnig
Distributed Data Management on the Grid Mario Lassnig Who am I? Mario Lassnig Computer scientist main field of study was theoretical (algorithm design) working on/with distributed and embedded systems
More informationISTITUTO NAZIONALE DI FISICA NUCLEARE
ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Perugia INFN/TC-05/10 July 4, 2005 DESIGN, IMPLEMENTATION AND CONFIGURATION OF A GRID SITE WITH A PRIVATE NETWORK ARCHITECTURE Leonello Servoli 1,2!, Mirko
More informationA scalable storage element and its usage in HEP
AstroGrid D Meeting at MPE 14 15. November 2006 Garching dcache A scalable storage element and its usage in HEP Martin Radicke Patrick Fuhrmann Introduction to dcache 2 Project overview joint venture between
More information高能物理分布式计算技术 张晓梅
高能物理分布式计算技术 张晓梅 2016.10.21 What is distributed computing in HEP?(1) Definition (Computing) A distributed system is a collection of autonomous hosts that are connected through a computer network. Each host
More informationData Management for Distributed Scientific Collaborations Using a Rule Engine
Data Management for Distributed Scientific Collaborations Using a Rule Engine Sara Alspaugh Department of Computer Science University of Virginia alspaugh@virginia.edu Ann Chervenak Information Sciences
More informationThe Grid: Processing the Data from the World s Largest Scientific Machine
The Grid: Processing the Data from the World s Largest Scientific Machine 10th Topical Seminar On Innovative Particle and Radiation Detectors Siena, 1-5 October 2006 Patricia Méndez Lorenzo (IT-PSS/ED),
More informationGrid Data Management
Grid Data Management Data Management Distributed community of users need to access and analyze large amounts of data Fusion community s International ITER project Requirement arises in both simulation
More informationPhilippe Charpentier PH Department CERN, Geneva
Philippe Charpentier PH Department CERN, Geneva Outline Disclaimer: These lectures are not meant at teaching you how to compute on the Grid! I hope it will give you a flavor on what Grid Computing is about
More informationUnified storage systems for distributed Tier-2 centres
Journal of Physics: Conference Series Unified storage systems for distributed Tier-2 centres To cite this article: G A Cowan et al 28 J. Phys.: Conf. Ser. 119 6227 View the article online for updates and
More informationUniversità degli Studi di Ferrara
Università degli Studi di Ferrara DOTTORATO DI RICERCA IN INFORMATICA Ciclo XXVI Coordinatore Prof. Massimiliano Mella Enabling parallel and interactive distributed computing data analysis for the ALICE
More informationForschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. Presented by Manfred Alef Contributions of Jos van Wezel, Andreas Heiss
Site Report Presented by Manfred Alef Contributions of Jos van Wezel, Andreas Heiss Grid Computing Centre Karlsruhe (GridKa) Forschungszentrum Karlsruhe Institute for Scientific Computing Hermann-von-Helmholtz-Platz
More informationWe are IntechOpen, the world s leading publisher of Open Access books Built by scientists, for scientists. International authors and editors
We are IntechOpen, the world s leading publisher of Open Access books Built by scientists, for scientists 4,100 116,000 120M Open access books available International authors and editors Downloads Our
More informationOverview. Grid vision Grid application domains The role of CERN in the Grid research Grid Architecture Standards and related activities Summary
Overview Grid vision Grid application domains The role of CERN in the Grid research Grid Architecture Standards and related activities Summary Laurence Field, Matti Heikkurinen CERN IT-department {firstname.lastname}@cern.ch
More informationCERN s Business Computing
CERN s Business Computing Where Accelerated the infinitely by Large Pentaho Meets the Infinitely small Jan Janke Deputy Group Leader CERN Administrative Information Systems Group CERN World s Leading Particle
More informationInteroperating AliEn and ARC for a distributed Tier1 in the Nordic countries.
for a distributed Tier1 in the Nordic countries. Philippe Gros Lund University, Div. of Experimental High Energy Physics, Box 118, 22100 Lund, Sweden philippe.gros@hep.lu.se Anders Rhod Gregersen NDGF
More informationUsing Hadoop File System and MapReduce in a small/medium Grid site
Journal of Physics: Conference Series Using Hadoop File System and MapReduce in a small/medium Grid site To cite this article: H Riahi et al 2012 J. Phys.: Conf. Ser. 396 042050 View the article online
More informationGrid Computing. Olivier Dadoun LAL, Orsay. Introduction & Parachute method. Socle 2006 Clermont-Ferrand Orsay)
virtual organization Grid Computing Introduction & Parachute method Socle 2006 Clermont-Ferrand (@lal Orsay) Olivier Dadoun LAL, Orsay dadoun@lal.in2p3.fr www.dadoun.net October 2006 1 Contents Preamble
More informationIntroduction to Grid Infrastructures
Introduction to Grid Infrastructures Stefano Cozzini 1 and Alessandro Costantini 2 1 CNR-INFM DEMOCRITOS National Simulation Center, Trieste, Italy 2 Department of Chemistry, Università di Perugia, Perugia,
More informationGrid Computing: dealing with GB/s dataflows
Grid Computing: dealing with GB/s dataflows Jan Just Keijser, Nikhef janjust@nikhef.nl David Groep, NIKHEF 21 March 2011 Graphics: Real Time Monitor, Gidon Moont, Imperial College London, see http://gridportal.hep.ph.ic.ac.uk/rtm/
More informationEvaluation of the Huawei UDS cloud storage system for CERN specific data
th International Conference on Computing in High Energy and Nuclear Physics (CHEP3) IOP Publishing Journal of Physics: Conference Series 53 (4) 44 doi:.88/74-6596/53/4/44 Evaluation of the Huawei UDS cloud
More informationOverview. About CERN 2 / 11
Overview CERN wanted to upgrade the data monitoring system of one of its Large Hadron Collider experiments called ALICE (A La rge Ion Collider Experiment) to ensure the experiment s high efficiency. They
More informationBeob Kyun KIM, Christophe BONNAUD {kyun, NSDC / KISTI
2010. 6. 17 Beob Kyun KIM, Christophe BONNAUD {kyun, cbonnaud}@kisti.re.kr NSDC / KISTI 1 1 2 Belle Data Transfer Metadata Extraction Scalability Test Metadata Replication Grid-awaring of Belle data Test
More informationEfficient HTTP based I/O on very large datasets for high performance computing with the Libdavix library
Efficient HTTP based I/O on very large datasets for high performance computing with the Libdavix library Authors Devresse Adrien (CERN) Fabrizio Furano (CERN) Typical HPC architecture Computing Cluster
More informationDESY. Andreas Gellrich DESY DESY,
Grid @ DESY Andreas Gellrich DESY DESY, Legacy Trivially, computing requirements must always be related to the technical abilities at a certain time Until not long ago: (at least in HEP ) Computing was
More informationOverview of HEP software & LCG from the openlab perspective
Overview of HEP software & LCG from the openlab perspective Andreas Unterkircher, CERN openlab February 2005 Andreas Unterkircher 1 Contents 1. Opencluster overview 2. High Energy Physics (HEP) software
More informationUK Tier-2 site evolution for ATLAS. Alastair Dewhurst
UK Tier-2 site evolution for ATLAS Alastair Dewhurst Introduction My understanding is that GridPP funding is only part of the story when it comes to paying for a Tier 2 site. Each site is unique. Aim to
More informationData Access and Data Management
Data Access and Data Management in grids Jos van Wezel Overview Background [KIT, GridKa] Practice [LHC, glite] Data storage systems [dcache a.o.] Data and meta data Intro KIT = FZK + Univ. of Karlsruhe
More informationCSCS CERN videoconference CFD applications
CSCS CERN videoconference CFD applications TS/CV/Detector Cooling - CFD Team CERN June 13 th 2006 Michele Battistin June 2006 CERN & CFD Presentation 1 TOPICS - Some feedback about already existing collaboration
More informationHEP Grid Activities in China
HEP Grid Activities in China Sun Gongxing Institute of High Energy Physics, Chinese Academy of Sciences CANS Nov. 1-2, 2005, Shen Zhen, China History of IHEP Computing Center Found in 1974 Computing Platform
More informationGrid Infrastructure For Collaborative High Performance Scientific Computing
Computing For Nation Development, February 08 09, 2008 Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi Grid Infrastructure For Collaborative High Performance Scientific
More informationData Management 1. Grid data management. Different sources of data. Sensors Analytic equipment Measurement tools and devices
Data Management 1 Grid data management Different sources of data Sensors Analytic equipment Measurement tools and devices Need to discover patterns in data to create information Need mechanisms to deal
More informationAndrea Sciabà CERN, Switzerland
Frascati Physics Series Vol. VVVVVV (xxxx), pp. 000-000 XX Conference Location, Date-start - Date-end, Year THE LHC COMPUTING GRID Andrea Sciabà CERN, Switzerland Abstract The LHC experiments will start
More information