Not e: The new Babar Comput ing Model is st ill under development. Everything I present here today is still under discussion within Babar.

Transcription

1 1. Computer Model Working Group 2. Analysis Model J. Walsh 3. Event Store Technologies 4. Computing/Analysis Sites 5. Implementation/Migration Not e: The new Babar Comput ing Model is st ill under development. Everything I present here today is still under discussion within Babar. 1

2 Babar Comput ing Model est ablished end of 2000 changes in computing environment require review Babar Computing Review: April, 2002 large luminosity of PEP-II puts big burden on Babar computing resources Analysis Groups produce huge ntuples that essentially duplicate micro-dst information not scalable with luminosity two event store technologies, Root/IO (Kanga/Root) and Objectivity burden on Computing Group several other issues raised: role of remote sites, etc. Computer Model Working Group 2 - update Babar Computing Model. So far work concentrated in two main areas 1. New proposal for analysis model 2. Event store technology - Root vs. Objectivity debate Plus, will comment on Remote sites Tier A, Tier C, etc. 2

3 design luminosity (3x10 33 ) exceeded in 2001 already expect 3 times current data sample in 2005 where we are now 3

4 Comput ing Model Wor king Gr oup 2: Member s Members: David Brown/LBL Claudio Campagnari Andreas Hoecker Hassan Jawahery (co-chair) Yury Kolomensky (co-chair) David Lange Mike Roney Aaron Roodman Anders Ryd Bernhard Spaan John Walsh Fergus Wilson Technical Experts: Jacek Becla Fabrizio Bianchi Nicole Chevalier Nicolo De Groot Gregory Dubois-Felsmann Peter Elmer Steffen Luitz Mauro Morandin Ex-Officio: Stephen Gowdy CC Rainer Bartoldus DepCC Richard Mount SCS Dominique Boutigny Tier-A Rep. Livio Lanceri DepPAC 4

5 Analysis: performed exclusively on micro format Beta/Framework is analysis program, C++ based Central Skims produce subsets of data pertinent to various analyses Analysis Groups (AWG s) often produce ntuples (or rootuples) that are large and redundant (contain same info as micro) Fortran (or root scripts) analyze ntuples (rootuples) Event Store: Originally only in Objectivity Kanga/Root developed for micro format only, to enhance analysis performance, data distribution micro currently maintained in Objy and Kanga/Root Computing/Analysis Sites: Full copy of micro data available at SLAC, IN2P3, RAL Production reprocessing in place at INFN-Padova Numerous institutes produce MC at remote sites Tier A Sites 5

6 Data formats: Mini: new format with detailed detector information available. New Micro: upgraded form of current micro, allow faster access, customizable content Tag (nano-dst): no major changes, general cleanup Note: large RECO and RAW objects no longer written to event store Skims: New Micro will make centralized skims more useful and efficient Large ntuple production: Hopefully, rendered obsolete, freeing up resources (CPU, disk space and manpower) 6

7 New f or mat int r oduced over t he last year : contains essentially full detector information track hits, calorimeter crystals, DRC hits, etc. efficiently packed to optimize space about 8 kbytes per event (compare to micro: 2 kbytes/event) increased analysis capability w.r.t. micro, e.g.: track extrapolation through detector material follow changing conditions (e.g. SVT alignment) event display etc. 7

8 Additional characteristics: customization larger and slower than micro-dst: develop coherent staging system to retrieve events from tape. Target access times: < 1 hr for small (< 100 events) samples < 1 day for medium (< 1 k events) samples < 2-4 weeks for large samples exact use pattern of mini not really predictable need to remain flexible on implementation 8

9 Radical improvement w.r.t. current micro-dst 1. Dual usage: 1. regular framework/beta job (current Babar norm) 2. interactive use with Root 2. Customizable content option to store detector info or not additional user info can be added composite candidates different mass hypo track fits, etc. 3. High speed: the aim is to reach 1 khz with framework/beta will require Beta development current rate: few tens of Hz higher read rates envisioned with Root access (at cost of reduced functionality) 9

10 Impact on users: much analysis in Babar done at ntuple level ntuple analysis code will have to be adapted/converted to new-micro (use of paw/fortran discouraged) potentially disruptive Comment on Obj ect ivit y: since interactive Root access is a basic feature of the new-micro Objectivity event store is not an option new-micro will be in Root/IO format 10

11 Current system: hybrid wit h Obj ect ivit y at SLAC and I N2P3 and Kanga/Root at RAL, INFN-Padova Problems with Objectivity: lock collisions Prompt Reconstruction and Simulation Production performance issues poor record of scaling with luminosity: every jump in data sample has been accompanied by Objy problems distribution difficulties getting data samples to Tier C sites other HEP experiments have dropped Objy as an option concerns about viability of Objectivity Company we don t have source code how much expertise will be around in 2007? 11

12 Kanga/Root much easier maintenance easier to export smaller event size (although Objy event size is decreasing with deployment of compression and redesign of navigational info) more efficient CPU usage becoming HEP standard easy to attract manpower to support Kanga/Root 12

13 So, why not dr op Obj y and go wit h Kanga/ Root syst em? Cost of migration: effort and disruption: estimates ranged from 1 to 2 years to achieve migration most in Babar agree a switch that takes more than 2 years is probably not worth doing. Kanga/Root has some technical issues that need to be addressed: file server to handle huge number of files lack of transactions lack of cross-file associations (e.g. mini-to-micro navigation) bookkeeping staging system Political/human issues. Note: conditions database implemented in Objectivity too costly (estimate 2-3 years) to convert to Root-based DB Babar relationship with Objy will continue in any case Work to address these issues is ongoing (not just in Babar context) probably no showstoppers, but it is work. 13

14 Alt er nat ive t o Kanga/ Root -only system: a hybrid system where: new-micro in Kanga/Root format only everything else (event reconstruction, simulation production, mini format) in Objectivity Hybr id syst em has advant age of easier, lessdisr upt ive migr at ion, but we st ill need t o support 2 event store technologies Final decision/ r ecommendat ion on event st or e coming soon 14

15 The Working Group is just starting on this subject just present the issues Role of Tier A sites - large site that reduces significantly computing burden at SLAC Primarily analysis: IN2P3, RAL Production: INFN-Padova Issues: data replication at Tier A s data partitioning at Tier A s (micro, mini, beam data, MC) transparent access to data across Tier A s (BabarGrid) specialization of Tier A s: skimming, (re-)processing, etc. Role of Tier C sites smaller sites at remote institutes main contribution so far in MC production (majority of MC events produced away from SLAC) analysis at Tier C s has been difficult due to problems with data distribution need to resolve with new Computing Model 15

16 Mini Already implemented in Objectivity (minor fixes, improvements ongoing) Feasibility of Root implementation has been studied could be ready by early 2003 New -micro Dual usage (Beta and Root) prototype implementation has been achieved. Additional development needed: 1. customization 2. persistent composites 3. Beta/Framework optimization 16

17 Essential requirement: minimal disturbance to Babar capability to produce physics results Mini currently doing reprocessing in Padova of all data, producing mini format output the mini is new feature, so disruption of ongoing analyses is minimal New-micro exploit Babar s data replication to ease migration maintain old-micro at SLAC and IN2P3 sites introduce new-micro at RAL site users have choice of format during transition period Dependence on other parts of Computing Model use of Tier A sites choice of event store technology, etc. 17

18 Babar is currently updating its computing model, to be able to deal with large increase in data set in the coming years A new analysis model, based on the new-micro and mini data formats has been proposed and largely agreed to. the mini will permit more in-depth analysis the new-micro will eliminate largely wasteful/redundant ntuples The working group is also considering the future of event store technologies employed in Babar. Should Objectivity event store be dropped in favor of Root-based technology? Is Kanga/Root ready to be used as a full-scale event store? Does a hybrid system do enough to alleviate the problems of Objy? An important part of the new model will be how best to use remote computing/analysis sites: Tier A and Tier C Work starting on this subject within the Working Group 18

19 19

20 The customization features of the new-micro make it an attractive tool to use with Centralized Skims The idea is t hat each Analysis Wor king Gr oup (AWG) will provide the appropriate event selection and content customization to the Central Skim group Small skims will be encouraged: deep-copies, which provide fast access, will be possible for small skims (< few % selection rate) In addition to skims, a generic new-micro containing all physics events will be available important for new analyses Aiming for increased frequency for Central Skims every 3 months feasibility being evaluated 20

21 The Tag f or mat will cont inue t o be maint ained Opt imizat ion/ cleanup t o r emove unused or redundant information should get a f act or of 2 size reduction 21

22 Deep copy copy full event to new location faster read rate more disk space Pointer (shallow) copy write pointer only slower read rate less disk space Ev 1 Ev 2 Ev 3 Ev 4 deep copy Ev 2 Ev 4 shallow copy Ptr 2 Ptr 4 22

23 1. Mature analysis (like sin2 ) could create a Mini skim of a relatively small number of events and work from that 2. An analysis with loose skim cuts (2-body charmless) would customize a new micro skim, dropping unneeded info and saving B candidates. Mini could be used near end of analysis when number of events is sufficiently reduced. 3. A new analysis would use allevents generic new micro to explore concept and define cuts. Final analysis could require a customized new micro skim or a mini skim (if event sample is small enough). 4. An AWG could produce a skim that serves many analyses. Specific analyses could make pointer skims of the skim, or deep copy skims if small enough. 5. etc. 23

24 This document was created with Win2PDF available at The unregistered version of Win2PDF is for evaluation or non-commercial use only.