DØ Southern Analysis Region Workshop Goals and Organization

Transcription

1 DØ Southern Analysis Region Workshop Goals and Organization DØRAM Jae Yu SAR Workshop, UTA Apr , 2003 Present Status Workshop Goals Arrangements

2 Why Remote Analysis System? Total Run II data size reaches multiple PB (~4PB) DØ Experiment is global 2

3 650 Collaborators 78 Institutions 18 Countries 3

4 Why Remote Analysis System? Total Run II data size reaches multiple PB (~4PB) DØ Experiment is global Data should be readily available for transparent and expeditious analyses Preferably disk resident so that time for caching is minimized Analysis processing compute power should be available without having the users relying on CAC MC generation should be transparently done Should exploit compute resources at remote sites Should exploit human resources at remote sites Minimize resource needs at the CAC Different resources will be needed Preparation for the future experiments 4

5 What do we need? Remote DØ software development environment Allow remote participation for code development which might soon be a bottleneck in expediting physics results Allow remote analysis for histogram production Allow remote reconstruction or production environment Optimized resource management tools Allow to maximally utilize local resources Allow to tap into available computing resources in other locations Allow participation of remote resources for global reconstruction or production Efficient and transparent data delivery and sharing Allow location independent access to data Allow quicker access to sufficient statistics for initial analyses Allow data sharing throughout the entire network of collaboration Minimize central data storage dependence Alleviate load for central data storage and servers 5

6 DØRAM Architecture Normal Interaction Communication Path Occasional Interaction Communication Path Central Analysis Center (CAC) Regional Analysis Centers RAC. RAC Institutional Analysis Centers Desktop Analysis Stations IAC... IAC IAC IAC DAS. DAS DAS. DAS 6

7 Fermilab Centric Model Central vs Distributed Models DØRAM Fermilab UA UTA CINVESTAV UO Ric e LTU FS U Aachen Bonn Wuppertal Mainz GridKa (Karlsruhe) Munich 7

8 Regional Analysis Centers A few selected sites that satisfy certain requirements Provide almost the same level of service as FNAL to a few institutional analysis centers Analyses carried out within the regional center network Store 10~20% of statistically random enough data permanently for self-sustainable analysis Most the analyses performed on these samples with the regional network Refine the analyses using the smaller but unbiased data set Some policies necessary to determine approval for access to full data set When the entire data set is needed Underlying DØGrid architecture provides access to remaining data set 8

9 What services do we want a DØRAC do? Provide intermediary code distribution Generate and reconstruct MC data set Accept and execute analysis batch job requests Store data and deliver them upon requests Participate in re-reconstruction of data Provide database access Provide manpower support for the above activities Proposal at: 9

10 What has been done with RACs? Offline Analysis Task Force formed to define more clearly the RAC s, their requirements, roles, etc First RAC established at Karlsruhe, Germany, Fall 02 Transferred full data set (smallest tier ~ 5TB) Constantly transferring data Performed physics analyses within the region presented the results IN2P3, Lyon, France being prepared to the next UTA awarded with funds to construct the 1 st US RAC Need to form the Region 10

11 US DØRACE Deployment Map MRAC WRAC ERAC Processing Center Analysis Site w/ SAM Analysis Site w/o SAM No DØRACE SRAC 11

12 Institutional Analysis Centers The success of DØRAM is determined by not only the RAC s but also the interworking of RAC and IAC Institutions within the region of an RAC Provide first level resources to the users in the institution Routine DØ software updates Access to some commonly used data sets Work with the RAC to allow users to access resources Analyses carried out within the region between RAC and IAC Form a local group (just like states) making DØ a United Regions of DØ (URD??) 12

13 What has been done? In preparation of the workshop, many institutions has been working with us Setup for MC productions with McFarmC Dedicated farms: Tata Institute, India, & LTU Condor Desktop Machine farm: OU MC Job status monitoring, McFarmGraph 13

14 MC Job Status Monitoring: McFarmGraph 14

15 What has been done? In preparation of the workshop, many institutions has been working with us Setup for MC productions Dedicated farms: Tata Institute, India, & LTU Condor Desktop Machine farm: OU MC Job status monitoring Resource monitoring 15

16 Ganglia Resource Monitoring 16

17 What has been done? In preparation of the workshop, many institutions has been working with us Setup for MC productions Dedicated farms: Tata Institute, India, & LTU Condor Desktop Machine farm: OU MC Job status monitoring Resource monitoring Now the emphasis should be on IAC s and the Region 17

18 Workshop Goals Primary goals: Define the Southern Analysis Region (SAR) Kick-start the Southern RAC at UTA Prepare SAR sites for the "grid" environment (Globus, Condor, etc., along with SAM capability) via hands-on sessions Understand the software & hardware requirements Set clear goals and their corresponding time scales, including a road map for eventual integration of SAR into the global DØGrid Decide on work assignments to participating institutions Enable Monte Carlo production capabilities at the various sites, including those without dedicated resources Establish clear sets of goals and milestones Establish closer relationship between the institutions Form the Region 18

19 Some Workshop Arrangements Your package includes McFarmC Manuel Interim McFarmC operations instructions Wireless SSID: UTAwireless Coffee is in next door. Help yourself. If you haven t got parking permit, please do so now!!! The banquet is at 6:30pm tonight, Mercardo Juarez Restaurant. See the map and the directions included in your packet. Reservation under SAR workshop. Tomorrow morning s session begins at 9am here in room

20 Conclusions Let s have fun together!! Let s help each other to bring the institutions within the region as high a level as others are Let s set our goals high Let s push to get the goals accomplished in the coming future After all, it will be a constructive competition between regions 20