Conference The Data Challenges of the LHC. Reda Tafirout, TRIUMF
|
|
- Job O’Brien’
- 1 years ago
- Views:
Transcription
1 Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF
2 Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment distributed computing Workload management system Data handling & distribution Future outlook 2
3 Science goals Explore the fundamental nature of matter and the basic forces that shape our universe Matter substructure One successful theory: the Standard Model basic building blocks and forces Perform precision measurements Theory build models Experiment observations test models Agree? Some of the important questions: Anything else beyond the Standard Model? Nature of dark matter in the universe? 3
4 Science Tools Powerful Particle Accelerator Sophisticated Detectors Swiss / France border 27 km circumference Collision modes: p-p, p-pb, Pb-Pb Large-scale Storage & Computing 4
5 Large Amount of data needed
6 Worldwide LHC Computing Grid Global collaboration of 167 computing centres in 42 countries MOU between parties participating in the project CERN (Tier-0) Tier-1 centres Tier-2 centres Description of roles: Baseline services & up-time Minimum Service Levels Response times Grid operations Pledged resources ~500k cores, ~350 PB disk, ~400 PB tape Additional capacity being deployed for 2017 LHC operations Tier-1: TRIUMF (Vancouver) Tier-2: Compute Canada centres (Victoria, SFU, Toronto, McGill) 6
7 Key Challenges Computing infrastructure is critical to the scientific output and success of the overall LHC experimental program: Middleware (baseline services) & Interoperability Availability & Reliability of each site Performance & Scalability of each site Distributed computing operations: workload and data management systems Site Usability Monitor (sample of 18 sites) Security Network performance Pledged resources Optimize physics output & costs 7
8 Network Connectivity (I) LHCOPN: LHC Optical Private Network, only for Tier-0 & Tier-1 centres 8
9 Network Connectivity (ll) LHCONE: LHC Open Network Environment (for Tier-2 centres + T0 / T1) Asia North America Europe South America 9
10 Global Network Traffic In 2016, more than 800 Petabytes moved across the computing grid (from all LHC experiments combined). ~35 GB/s peaks of global WAN transfers Shown for period 06 / 16 (from I. Bird, HSF Workshop, San Diego, 01/2017) 10
11 Network Monitoring (I) End-to-end network problems difficult to diagnose: multi-domain perfsonar-ps deployment (for bandwidth & latency): two instances at each site (from S. McKee, ATLAS TIM 2016) 11
12 Network Monitoring (II) perfsonar-ps configuration, data collection and dashboard (from S. McKee, ATLAS TIM 2016) 12
13 Network Monitoring (III) Data analytics tools using different sources: cost matrix and throughput predictions to optimize work flows (for ATLAS) (from S. McKee, ATLAS TIM 2016) 13
14 ATLAS Distributed Computing TDAQ Tier-0 - Mass storage - Raw data (2nd copy) - Simulation Tier-2 - User Analysis - First pass processing (10x)... Tier-1 - Mass storage - Calibration Tier-2 Tier-1 Tier-2 Tier-2 Tier-2 - Reprocessing - Group prod. Tier-2 Tier-2 chaotic scale - Raw data (1st copy) (70x) (ATLAS wide resources / MoU) - Code development - Grid UI Tier-3 Tier-3... (Private resources, Clouds, HPCs) Tier-3 - Small scale Analysis & Opportunistic production (~1000x) 14
15 Data Management System ATLAS DDM system (Rucio): datasets cataloging replication & lifetime policies primary vs secondary disk vs tape quotas & deletions transfer protocols hundreds of storage elements ATLAS 200 PB Tier-1 Tier-2 Tier-0 File Transfer Service (FTS) PB monthly (WAN) network saturation seen ATLAS Constant pressure on disk resources 15
16 Data Reduction & Derivation Reduction & derivation framework for more efficient use of computing resources; less chaotic and more coherent approach for physics analysis. ATLAS 16
17 Workload Management System Production And Distributed Analysis (PANDA) ATLAS - Priorities, resource allocations, and scheduling - Job goes to data* - Prod and analysis queues - Single core - Multi-core - Software validation (distributed via CVMFS) (factory) (Tier-3) 17
18 Production & Analysis Overall, smooth ATLAS distributed computing operations (ADC) ATLAS 200K cores (Tier-0,Tier-1,Tier-2) For ~10 months in 2016: 254 million jobs completed 57 million cpu-days consumed 89% production 11% user analysis ATLAS 200K cores ATLAS Opportunistic resources (Tier-3 clusters, Clouds, HPC) 50K cores 18
19 Future Outlook (I) Substantial increase due to excellent LHC performance in 2016 Reviewed by the WLCG Computing Resources Scrutiny Group and approved by CERN Resources Review Board. For 2018 (all LHC experiments): Disk: 520 PB Tape: 840 PB CPU: 6,200,000 HEP-SPEC06 (1 core ~ 14 HS06, benchmark unit) (HEP-SPEC06) Tier0 Tier0 CPU CPU 2017-A 2017-A Tier1 Tier2 Tier1 Tier2 (TB) Disk Disk Tier0 Tier %/yr 20%/yr 2017-S 2017-S Estimated Estimated % 20% (from I. Bird, HSF Workshop, San Diego,01/2017) Tape Tape (TB) 2017-S 2017-S Estimated Estimated 2017-A 2017-A Tier1 Tier2 Tier1 Tier A 2017-A Tier0 Tier1 Tier0 Tier S 2017-S 20% 20%
20 Future Outlook (II) LHC program has a very long time horizon Computing Resources requirements will increase by x10 for storage and by x50 for cpu (for HL-LHC run) K ATLAS ---Grid ---HPC ---Cloud Cost Drivers ATLAS (from I. Bird, HSF Workshop, San Diego,01/2017) 20
21 Extra / Additional Material 2/2/17 21
22 Canadian region / operations * Tier-1 ROC Tier-2 East Federation Tier-2 West Federation * Until Australian Tier-2 22
23 ATLAS experiment Explores the fundamental nature of matter and the forces that shape our universe by studying proton-proton collisions at very high energy at the Large Hadron Collider (LHC) in Geneva, Switzerland.(+heavy ions) Largest collaborative effort ever attempted in the physical sciences: 38 countries, 177 institutions (universities & labs) 3000 active scientists (1800 PhD's, 1200 students) Very broad, rich and high impact scientific program: Nobel Prize-winning Higgs boson discovery in 2012 (Phys. Lett. B 716 (2012) 1-29, with more than 4,300 citations) ATLAS-Canada: Alberta, British Columbia, Carleton, McGill, Montréal, Simon Fraser, Toronto, TRIUMF, Victoria, York (38 faculties, 26 postdoctoral researchers, 66 graduate students, 25 undergrads/year) The Canadian government has invested over $175 million 23
24 Streams & Data Quality 1380 bunches per beam 100 billion protons per bunch 24
25 Trigger & Data Acquisition (17,000 cores) 25
26 Simulation framework Several billion simulated proton-proton collisions are needed Essential for analysis: model comparison with data Monte Carlo generators: various physics processes resulting from protonproton collisions ATLAS detector response: Geant4 program Full vs Fast simulation: tracking, calorimetry Simulated data goes through same data processing chain as real data to produce derived data for analysis 26
27 Software performance Large development effort during LHC long shutdown phase 1 ( ) Improved code for event reconstruction tasks Move to Multi-threaded processing (memory reduction) 27
Big Data Analytics and the LHC
Big Data Analytics and the LHC Maria Girone CERN openlab CTO Computing Frontiers 2016, Como, May 2016 DOI: 10.5281/zenodo.45449, CC-BY-SA, images courtesy of CERN 2 3 xx 4 Big bang in the laboratory We
ATLAS Distributed Computing Experience and Performance During the LHC Run-2
ATLAS Distributed Computing Experience and Performance During the LHC Run-2 A Filipčič 1 for the ATLAS Collaboration 1 Jozef Stefan Institute, Jamova 39, 1000 Ljubljana, Slovenia E-mail: andrej.filipcic@ijs.si
Batch Services at CERN: Status and Future Evolution
Batch Services at CERN: Status and Future Evolution Helge Meinhard, CERN-IT Platform and Engineering Services Group Leader HTCondor Week 20 May 2015 20-May-2015 CERN batch status and evolution - Helge
Grid Computing a new tool for science
Grid Computing a new tool for science CERN, the European Organization for Nuclear Research Dr. Wolfgang von Rüden Wolfgang von Rüden, CERN, IT Department Grid Computing July 2006 CERN stands for over 50
Big Computing and the Mitchell Institute for Fundamental Physics and Astronomy. David Toback
Big Computing and the Mitchell Institute for Fundamental Physics and Astronomy Texas A&M Big Data Workshop October 2011 January 2015, Texas A&M University Research Topics Seminar 1 Outline Overview of
IEPSAS-Kosice: experiences in running LCG site
IEPSAS-Kosice: experiences in running LCG site Marian Babik 1, Dusan Bruncko 2, Tomas Daranyi 1, Ladislav Hluchy 1 and Pavol Strizenec 2 1 Department of Parallel and Distributed Computing, Institute of
The LHC Computing Grid
The LHC Computing Grid Visit of Finnish IT Centre for Science CSC Board Members Finland Tuesday 19 th May 2009 Frédéric Hemmer IT Department Head The LHC and Detectors Outline Computing Challenges Current
CERN openlab II. CERN openlab and. Sverre Jarp CERN openlab CTO 16 September 2008
CERN openlab II CERN openlab and Intel: Today and Tomorrow Sverre Jarp CERN openlab CTO 16 September 2008 Overview of CERN 2 CERN is the world's largest particle physics centre What is CERN? Particle physics
The Grid: Processing the Data from the World s Largest Scientific Machine
The Grid: Processing the Data from the World s Largest Scientific Machine 10th Topical Seminar On Innovative Particle and Radiation Detectors Siena, 1-5 October 2006 Patricia Méndez Lorenzo (IT-PSS/ED),
Towards Network Awareness in LHC Computing
Towards Network Awareness in LHC Computing CMS ALICE CERN Atlas LHCb LHC Run1: Discovery of a New Boson LHC Run2: Beyond the Standard Model Gateway to a New Era Artur Barczyk / Caltech Internet2 Technology
CouchDB-based system for data management in a Grid environment Implementation and Experience
CouchDB-based system for data management in a Grid environment Implementation and Experience Hassen Riahi IT/SDC, CERN Outline Context Problematic and strategy System architecture Integration and deployment
Grid Computing at the IIHE
BNC 2016 Grid Computing at the IIHE The Interuniversity Institute for High Energies S. Amary, F. Blekman, A. Boukil, O. Devroede, S. Gérard, A. Ouchene, R. Rougny, S. Rugovac, P. Vanlaer, R. Vandenbroucke
Data Transfers Between LHC Grid Sites Dorian Kcira
Data Transfers Between LHC Grid Sites Dorian Kcira dkcira@caltech.edu Caltech High Energy Physics Group hep.caltech.edu/cms CERN Site: LHC and the Experiments Large Hadron Collider 27 km circumference
Constant monitoring of multi-site network connectivity at the Tokyo Tier2 center
Constant monitoring of multi-site network connectivity at the Tokyo Tier2 center, T. Mashimo, N. Matsui, H. Matsunaga, H. Sakamoto, I. Ueda International Center for Elementary Particle Physics, The University
Grid Computing Activities at KIT
Grid Computing Activities at KIT Meeting between NCP and KIT, 21.09.2015 Manuel Giffels Karlsruhe Institute of Technology Institute of Experimental Nuclear Physics & Steinbuch Center for Computing Courtesy
The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland
The LCG 3D Project Maria Girone, CERN The rd Open Grid Forum - OGF 4th June 2008, Barcelona Outline Introduction The Distributed Database (3D) Project Streams Replication Technology and Performance Availability
ATLAS distributed computing: experience and evolution
Journal of Physics: Conference Series OPEN ACCESS ATLAS distributed computing: experience and evolution To cite this article: A Nairz and the Atlas Collaboration 2014 J. Phys.: Conf. Ser. 523 012020 View
AGIS: The ATLAS Grid Information System
AGIS: The ATLAS Grid Information System Alexey Anisenkov 1, Sergey Belov 2, Alessandro Di Girolamo 3, Stavro Gayazov 1, Alexei Klimentov 4, Danila Oleynik 2, Alexander Senchenko 1 on behalf of the ATLAS
CMS Computing Model with Focus on German Tier1 Activities
CMS Computing Model with Focus on German Tier1 Activities Seminar über Datenverarbeitung in der Hochenergiephysik DESY Hamburg, 24.11.2008 Overview The Large Hadron Collider The Compact Muon Solenoid CMS
CC-IN2P3: A High Performance Data Center for Research
April 15 th, 2011 CC-IN2P3: A High Performance Data Center for Research Toward a partnership with DELL Dominique Boutigny Agenda Welcome Introduction to CC-IN2P3 Visit of the computer room Lunch Discussion
Federated data storage system prototype for LHC experiments and data intensive science
Federated data storage system prototype for LHC experiments and data intensive science A. Kiryanov 1,2,a, A. Klimentov 1,3,b, D. Krasnopevtsev 1,4,c, E. Ryabinkin 1,d, A. Zarochentsev 1,5,e 1 National
The LHC Computing Grid. Slides mostly by: Dr Ian Bird LCG Project Leader 18 March 2008
The LHC Computing Grid Slides mostly by: Dr Ian Bird LCG Project Leader 18 March 2008 The LHC Computing Grid February 2008 Some precursors Computing for HEP means data handling Fixed-target experiments
Physics Computing at CERN. Helge Meinhard CERN, IT Department OpenLab Student Lecture 27 July 2010
Physics Computing at CERN Helge Meinhard CERN, IT Department OpenLab Student Lecture 27 July 2010 Location Building 513 (opposite of restaurant no. 2) Building Large building with 2700 m 2 surface for
High Throughput WAN Data Transfer with Hadoop-based Storage
High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San
The JINR Tier1 Site Simulation for Research and Development Purposes
EPJ Web of Conferences 108, 02033 (2016) DOI: 10.1051/ epjconf/ 201610802033 C Owned by the authors, published by EDP Sciences, 2016 The JINR Tier1 Site Simulation for Research and Development Purposes
Evaluation of the computing resources required for a Nordic research exploitation of the LHC
PROCEEDINGS Evaluation of the computing resources required for a Nordic research exploitation of the LHC and Sverker Almehed, Chafik Driouichi, Paula Eerola, Ulf Mjörnmark, Oxana Smirnova,TorstenÅkesson
CERN Lustre Evaluation
CERN Lustre Evaluation Arne Wiebalck Sun HPC Workshop, Open Storage Track Regensburg, Germany 8 th Sep 2009 www.cern.ch/it Agenda A Quick Guide to CERN Storage Use Cases Methodology & Initial Findings
First Experience with LCG. Board of Sponsors 3 rd April 2009
First Experience with LCG Operation and the future... CERN openlab Board of Sponsors 3 rd April 2009 Ian Bird LCG Project Leader The LHC Computing Challenge Signal/Noise: 10-9 Data volume High rate * large
Summary of the LHC Computing Review
Summary of the LHC Computing Review http://lhc-computing-review-public.web.cern.ch John Harvey CERN/EP May 10 th, 2001 LHCb Collaboration Meeting The Scale Data taking rate : 50,100, 200 Hz (ALICE, ATLAS-CMS,
Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science
Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science T. Maeno, K. De, A. Klimentov, P. Nilsson, D. Oleynik, S. Panitkin, A. Petrosyan, J. Schovancova, A. Vaniachine,
The Science DMZ: Evolution
The Science DMZ: Evolution Eli Dart, ESnet CC-NIE PI Meeting Washington, DC May 1, 2014 Why Are We Doing This? It s good to build high-quality infrastructure As network engineers, we like building networks
Overview of ATLAS PanDA Workload Management
Overview of ATLAS PanDA Workload Management T. Maeno 1, K. De 2, T. Wenaus 1, P. Nilsson 2, G. A. Stewart 3, R. Walker 4, A. Stradling 2, J. Caballero 1, M. Potekhin 1, D. Smith 5, for The ATLAS Collaboration
Data Management for the World s Largest Machine
Data Management for the World s Largest Machine Sigve Haug 1, Farid Ould-Saada 2, Katarina Pajchel 2, and Alexander L. Read 2 1 Laboratory for High Energy Physics, University of Bern, Sidlerstrasse 5,
Intercontinental Multi-Domain Monitoring for LHC with perfsonar
Journal of Physics: Conference Series Intercontinental Multi-Domain Monitoring for LHC with perfsonar To cite this article: D Vicinanza 2012 J. Phys.: Conf. Ser. 396 042060 View the article online for
The LHC computing model and its evolution. Dr Bob Jones CERN
The LHC computing model and its evolution Dr Bob Jones CERN Bob.Jones CERN.ch CERN was founded 1954: 12 European States Today: 20 Member States ~ 2300 staff ~ 790 other paid personnel > 10000 users
The extension of optical networks into the campus
The extension of optical networks into the campus Wade Hong Office of the Dean of Science Carleton University Outline Motivation CA*net 4 IGT From the Carleton U Perspective Issues Lessons learned Motivation
Unified System for Processing Real and Simulated Data in the ATLAS Experiment
Unified System for Processing Real and Simulated Data in the ATLAS Experiment Mikhail Borodin Big Data Laboratory, National Research Centre "Kurchatov Institute", Moscow, Russia National Research Nuclear
PanDA: Exascale Federation of Resources for the ATLAS Experiment
EPJ Web of Conferences will be set by the publisher DOI: will be set by the publisher c Owned by the authors, published by EDP Sciences, 2015 PanDA: Exascale Federation of Resources for the ATLAS Experiment
Improving Network Infrastructure to Enable Large Scale Scientific Data Flows and Collaboration (Award # ) Klara Jelinkova Joseph Ghobrial
Improving Network Infrastructure to Enable Large Scale Scientific Data Flows and Collaboration (Award # 1659348) Klara Jelinkova Joseph Ghobrial NSF Campus Cyberinfrastructure PI and Cybersecurity Innovation
Data Reconstruction in Modern Particle Physics
Data Reconstruction in Modern Particle Physics Daniel Saunders, University of Bristol 1 About me Particle Physics student, final year. CSC 2014, tcsc 2015, icsc 2016 Main research interests. Detector upgrades
The ATLAS Production System
The ATLAS MC and Data Rodney Walker Ludwig Maximilians Universität Munich 2nd Feb, 2009 / DESY Computing Seminar Outline 1 Monte Carlo Production Data 2 3 MC Production Data MC Production Data Group and
On-demand provisioning of HEP compute resources on cloud sites and shared HPC centers
On-demand provisioning of HEP compute resources on cloud sites and shared HPC centers CHEP 2016 - San Francisco, United States of America Gunther Erli, Frank Fischer, Georg Fleig, Manuel Giffels, Thomas
LHC Computing Grid today Did it work?
Did it work? Sept. 9th 2011, 1 KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association Institut www.kit.edu Abteilung Large Hadron Collider and Experiments
CMS High Level Trigger Timing Measurements
Journal of Physics: Conference Series PAPER OPEN ACCESS High Level Trigger Timing Measurements To cite this article: Clint Richardson 2015 J. Phys.: Conf. Ser. 664 082045 Related content - Recent Standard
Understanding the T2 traffic in CMS during Run-1
Journal of Physics: Conference Series PAPER OPEN ACCESS Understanding the T2 traffic in CMS during Run-1 To cite this article: Wildish T and 2015 J. Phys.: Conf. Ser. 664 032034 View the article online
Outline. March 5, 2012 CIRMMT - McGill University 2
Outline CLUMEQ, Calcul Quebec and Compute Canada Research Support Objectives and Focal Points CLUMEQ Site at McGill ETS Key Specifications and Status CLUMEQ HPC Support Staff at McGill Getting Started
Connectivity Services, Autobahn and New Services
Connectivity Services, Autobahn and New Services Domenico Vicinanza, DANTE EGEE 09, Barcelona, 21 st -25 th September 2009 Agenda Background GÉANT Connectivity services: GÉANT IP GÉANT Plus GÉANT Lambda
CHIPP Phoenix Cluster Inauguration
TheComputing Environment for LHC Data Analysis The LHC Computing Grid CHIPP Phoenix Cluster Inauguration Manno, Switzerland 30 May 2008 Les Robertson IT Department - CERN CH-1211 Genève 23 les.robertson@cern.ch
Data services for LHC computing
Data services for LHC computing SLAC 1 Xavier Espinal on behalf of IT/ST DAQ to CC 8GB/s+4xReco Hot files Reliable Fast Processing DAQ Feedback loop WAN aware Tier-1/2 replica, multi-site High throughout
The LHC Computing Grid
The LHC Computing Grid Gergely Debreczeni (CERN IT/Grid Deployment Group) The data factory of LHC 40 million collisions in each second After on-line triggers and selections, only 100 3-4 MB/event requires
Volunteer Computing at CERN
Volunteer Computing at CERN BOINC workshop Sep 2014, Budapest Tomi Asp & Pete Jones, on behalf the LHC@Home team Agenda Overview Status of the LHC@Home projects Additional BOINC projects Service consolidation
Towards a Strategy for Data Sciences at UW
Towards a Strategy for Data Sciences at UW Albrecht Karle Department of Physics June 2017 High performance compu0ng infrastructure: Perspec0ves from Physics Exis0ng infrastructure and projected future
Network Analytics. Hendrik Borras, Marian Babik IT-CM-MM
Network Analytics Hendrik Borras, Marian Babik IT-CM-MM perfsonar Infrastructure perfsonar has been widely deployed in WLCG 249 active instances, deployed at 120 sites including major network hubs at ESNet,
Operating the Distributed NDGF Tier-1
Operating the Distributed NDGF Tier-1 Michael Grønager Technical Coordinator, NDGF International Symposium on Grid Computing 08 Taipei, April 10th 2008 Talk Outline What is NDGF? Why a distributed Tier-1?
Presentation of the LHCONE Architecture document
Presentation of the LHCONE Architecture document Marco Marletta, GARR LHCONE Meeting Paris, Tuesday 5th April 2011 Agenda Background Design Definitions Architecture Services Policy Next steps 2 Background
30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy
Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of
GÉANT Mission and Services
GÉANT Mission and Services Vincenzo Capone Senior Technical Business Development Officer CREMLIN WP2 Workshop on Big Data Management 15 February 2017, Moscow GÉANT - Networks Manages research & education
Data Intensive Science Impact on Networks
Data Intensive Science Impact on Networks Eli Dart, Network Engineer ESnet Network Engineering g Group IEEE Bandwidth Assessment Ad Hoc December 13, 2011 Outline Data intensive science examples Collaboration
Spark and HPC for High Energy Physics Data Analyses
Spark and HPC for High Energy Physics Data Analyses Marc Paterno, Jim Kowalkowski, and Saba Sehrish 2017 IEEE International Workshop on High-Performance Big Data Computing Introduction High energy physics
Data Handling for LHC: Plans and Reality
Data Handling for LHC: Plans and Reality Tony Cass Leader, Database Services Group Information Technology Department 11 th July 2012 1 HEP, CERN, LHC and LHC Experiments LHC Computing Challenge The Technique
WLCG Network Throughput WG
WLCG Network Throughput WG Shawn McKee, Marian Babik for the Working Group HEPiX Tsukuba 16-20 October 2017 Working Group WLCG Network Throughput WG formed in the fall of 2014 within the scope of WLCG
Bringing ATLAS production to HPC resources - A use case with the Hydra supercomputer of the Max Planck Society
Journal of Physics: Conference Series PAPER OPEN ACCESS Bringing ATLAS production to HPC resources - A use case with the Hydra supercomputer of the Max Planck Society To cite this article: J A Kennedy
Physics Computing at CERN. Helge Meinhard CERN, IT Department OpenLab Student Lecture 21 July 2011
Physics Computing at CERN Helge Meinhard CERN, IT Department OpenLab Student Lecture 21 July 2011 Location (1) Building 513 (opposite of restaurant no. 2) Building 513 (1) Large building with 2700 m 2
An SQL-based approach to physics analysis
Journal of Physics: Conference Series OPEN ACCESS An SQL-based approach to physics analysis To cite this article: Dr Maaike Limper 2014 J. Phys.: Conf. Ser. 513 022022 View the article online for updates
SLATE. Services Layer at the Edge. First Meeting of the National Research Platform Montana State University August 7-8, 2017
SLATE Services Layer at the Edge Rob Gardner University of Chicago Shawn McKee University of Michigan Joe Breen University of Utah First Meeting of the National Research Platform Montana State University
Oracle and Tangosol Acquisition Announcement
Oracle and Tangosol Acquisition Announcement March 23, 2007 The following is intended to outline our general product direction. It is intended for information purposes only, and may
High Performance Computing from an EU perspective
High Performance Computing from an EU perspective DEISA PRACE Symposium 2010 Barcelona, 10 May 2010 Kostas Glinos European Commission - DG INFSO Head of Unit GÉANT & e-infrastructures 1 "The views expressed
RESEARCH DATA DEPOT AT PURDUE UNIVERSITY
Preston Smith Director of Research Services RESEARCH DATA DEPOT AT PURDUE UNIVERSITY May 18, 2016 HTCONDOR WEEK 2016 Ran into Miron at a workshop recently.. Talked about data and the challenges of providing
Evolution of Database Replication Technologies for WLCG
Journal of Physics: Conference Series PAPER OPEN ACCESS Evolution of Database Replication Technologies for WLCG To cite this article: Zbigniew Baranowski et al 2015 J. Phys.: Conf. Ser. 664 042032 View
Computing for LHC in Germany
1 Computing for LHC in Germany Günter Quast Universität Karlsruhe (TH) Meeting with RECFA Berlin, October 5th 2007 WLCG Tier1 & Tier2 Additional resources for data analysis - HGF ''Physics at the Terascale''
THOUGHTS ON SDN IN DATA INTENSIVE SCIENCE APPLICATIONS
THOUGHTS ON SDN IN DATA INTENSIVE SCIENCE APPLICATIONS Artur Barczyk/Caltech Internet2 Technology Exchange Indianapolis, October 30 th, 2014 October 29, 2014 Artur.Barczyk@cern.ch 1 HEP context - for this
The Canadian CyberSKA Project
The Canadian CyberSKA Project A. G. Willis (on behalf of the CyberSKA Project Team) National Research Council of Canada Herzberg Institute of Astrophysics Dominion Radio Astrophysical Observatory May 24,
Monte Carlo Production Management at CMS
Monte Carlo Production Management at CMS G Boudoul 1, G Franzoni 2, A Norkus 2,3, A Pol 2, P Srimanobhas 4 and J-R Vlimant 5 - for the Compact Muon Solenoid collaboration 1 U. C. Bernard-Lyon I, 43 boulevard
Use of the virtualized HPC infrastructure of the Novosibirsk Scientific Center for running production analysis for HEP experiments at BINP
Use of the virtualized HPC infrastructure of the Novosibirsk Scientific Center for running production analysis for HEP experiments at BINP S. Belov, V. Kaplin, K. Skovpen, A.Korol, A. Sukharev 1, A. Zaytsev
Data Grid Infrastructure for YBJ-ARGO Cosmic-Ray Project
Data Grid Infrastructure for YBJ-ARGO Cosmic-Ray Project Gang CHEN, Hongmei ZHANG - IHEP CODATA 06 24 October 2006, Beijing FP6 2004 Infrastructures 6-SSA-026634 http://www.euchinagrid.cn Extensive Air
ALICE Grid Activities in US
ALICE Grid Activities in US 1 ALICE-USA Computing Project ALICE-USA Collaboration formed to focus on the ALICE EMCal project Construction, installation, testing and integration participating institutions
RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP
RADU POPESCU IMPROVING THE WRITE SCALABILITY OF THE CERNVM FILE SYSTEM WITH ERLANG/OTP THE EUROPEAN ORGANISATION FOR PARTICLE PHYSICS RESEARCH (CERN) 2 THE LARGE HADRON COLLIDER THE LARGE HADRON COLLIDER
Evolution of Database Replication Technologies for WLCG
Evolution of Database Replication Technologies for WLCG Zbigniew Baranowski, Lorena Lobato Pardavila, Marcin Blaszczyk, Gancho Dimitrov, Luca Canali European Organisation for Nuclear Research (CERN), CH-1211
Geant4 Computing Performance Benchmarking and Monitoring
Journal of Physics: Conference Series PAPER OPEN ACCESS Geant4 Computing Performance Benchmarking and Monitoring To cite this article: Andrea Dotti et al 2015 J. Phys.: Conf. Ser. 664 062021 View the article
e-infrastructures in FP7 INFO DAY - Paris
e-infrastructures in FP7 INFO DAY - Paris Carlos Morais Pires European Commission DG INFSO GÉANT & e-infrastructure Unit 1 Global challenges with high societal impact Big Science and the role of empowered
Computing. DOE Program Review SLAC. Rainer Bartoldus. Breakout Session 3 June BaBar Deputy Computing Coordinator
Computing DOE Program Review SLAC Breakout Session 3 June 2004 Rainer Bartoldus BaBar Deputy Computing Coordinator 1 Outline The New Computing Model (CM2) New Kanga/ROOT event store, new Analysis Model,
The grid for LHC Data Analysis
The grid for LHC Data Analysis ICAP 2006 Conference Chamonix 5 October 2006 Les Robertson - CERN LHC Computing Grid Project Leader The LHC Computing Challenges 1. Data After reduction by triggers and data
Data preservation for the HERA experiments at DESY using dcache technology
Journal of Physics: Conference Series PAPER OPEN ACCESS Data preservation for the HERA experiments at DESY using dcache technology To cite this article: Dirk Krücker et al 2015 J. Phys.: Conf. Ser. 66
Computing grids, a tool for international collaboration and against digital divide Guy Wormser Director of CNRS Institut des Grilles (CNRS, France)
Computing grids, a tool for international collaboration and against digital divide Guy Wormser Director of CNRS Institut des Grilles (CNRS, France) www.eu-egee.org EGEE and glite are registered trademarks
How to discover the Higgs Boson in an Oracle database. Maaike Limper
How to discover the Higgs Boson in an Oracle database Maaike Limper 2 Introduction CERN openlab is a unique public-private partnership between CERN and leading ICT companies. Its mission is to accelerate
Early experience with the Run 2 ATLAS analysis model
Early experience with the Run 2 ATLAS analysis model Argonne National Laboratory E-mail: cranshaw@anl.gov During the long shutdown of the LHC, the ATLAS collaboration redesigned its analysis model based
Helix Nebula The Science Cloud
Helix Nebula The Science Cloud Deliverable Title: Data Management Plan Partner Responsible: CERN Work Package: 1 Submission Due Date: 26 January 2016 Actual Submission Date: 23 March 2016 Distribution:
Europe and its Open Science Cloud: the Italian perspective. Luciano Gaido Plan-E meeting, Poznan, April
Europe and its Open Science Cloud: the Italian perspective Luciano Gaido (gaido@to.infn.it) Plan-E meeting, Poznan, April 27 2017 Background Italy has a long-standing expertise and experience in the management
Grid Data Management
Grid Data Management Week #4 Hardi Teder hardi@eenet.ee University of Tartu March 6th 2013 Overview Grid Data Management Where the Data comes from? Grid Data Management tools 2/33 Grid foundations 3/33
CernVM-FS beyond LHC computing
CernVM-FS beyond LHC computing C Condurache, I Collier STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot, OX11 0QX, UK E-mail: catalin.condurache@stfc.ac.uk Abstract. In the last three years
THE ATLAS DATA ACQUISITION SYSTEM IN LHC RUN 2
THE ATLAS DATA ACQUISITION SYSTEM IN LHC RUN 2 M. E. Pozo Astigarraga, on behalf of the ATLAS Collaboration CERN, CH-1211 Geneva 23, Switzerland E-mail: eukeni.pozo@cern.ch The LHC has been providing proton-proton
MOHA: Many-Task Computing Framework on Hadoop
Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction
arxiv: v1 [cs.dc] 12 May 2017
GRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasets arxiv:1705.04513v1 [cs.dc] 12 May 2017 M Hushchyn 1,2, A Ustyuzhanin 1,3, P Charpentier 4 and C Haen 4 1 Yandex School of
DSpace Fedora. Eprints Greenstone. Handle System
Enabling Inter-repository repository Access Management between irods and Fedora Bing Zhu, Uni. of California: San Diego Richard Marciano Reagan Moore University of North Carolina at Chapel Hill May 18,
Introduction to Geant4
Introduction to Geant4 Release 10.4 Geant4 Collaboration Rev1.0: Dec 8th, 2017 CONTENTS: 1 Geant4 Scope of Application 3 2 History of Geant4 5 3 Overview of Geant4 Functionality 7 4 Geant4 User Support
Data Analysis in ATLAS. Graeme Stewart with thanks to Attila Krasznahorkay and Johannes Elmsheuser
Data Analysis in ATLAS Graeme Stewart with thanks to Attila Krasznahorkay and Johannes Elmsheuser 1 ATLAS Data Flow into Analysis RAW detector data and simulated RDO data are reconstructed into our xaod
Challenges of the LHC Computing Grid by the CMS experiment
2007 German e-science Available online at http://www.ges2007.de This document is under the terms of the CC-BY-NC-ND Creative Commons Attribution Challenges of the LHC Computing Grid by the CMS experiment
IBM Spectrum Scale vs EMC Isilon for IBM Spectrum Protect Workloads
89 Fifth Avenue, 7th Floor New York, NY 10003 www.theedison.com @EdisonGroupInc 212.367.7400 IBM Spectrum Scale vs EMC Isilon for IBM Spectrum Protect Workloads A Competitive Test and Evaluation Report
Deep Learning Photon Identification in a SuperGranular Calorimeter
Deep Learning Photon Identification in a SuperGranular Calorimeter Nikolaus Howe Maurizio Pierini Jean-Roch Vlimant @ Williams College @ CERN @ Caltech 1 Outline Introduction to the problem What is Machine