GRID TECHNOLOGY FOR CONTROLLED FUSION: CONCEPTION OF THE UNIFIED CYBERSPACE AND ITER DATA MANAGEMENT N. Marusov, I. Semenov Project Center ITER (ITER Russian Domestic Agency N.Marusov@ITERRF.RU)
Challenges of ITER data usage Data accumulation & distribution ~5 PB/year, ~100 PB for a lifetime Huge amount of data cannot be analyzed by humans Automated conversion of raw data to knowledge Scientific collaboration all over the world Diversity of end-user needs and tools Future integration of ITER scientific achievements with other TOKAMAKs 2
ITER research as control process Fusion community formulates the goals and gets the results Fusion Community Knowledge Information circulating in control loops represents experimental data and knowledge Goals Knowledge Management System Fusion Technology Results Impact to technology is done trough the Knowledge Management System Data ITER machine Fusion technology is viewed as the controlled object ITER machine work as sensors to assess the quality of technology 3
Domain knowledge allocation Slow data processing (10 min - hours) Fast data processing (< 10 min) 4
Unified cyberspace (UC) concept Knowledge Management System (KMS) is one of key components of ITER computing model Distributed nature of ITER collaboration Community-oriented research workflow Liaison standards for interoperation in heterogeneous computing environments Machine learning-enabled environment Distributed community-driven KMS semantically linking diverse computing resources, modeling codes and data storages on the base of existing standards 5
Most of such problems are already solved within existing projects Worldwide LHC Computing Grid (CERN + 42 countries) National Fusion Collaboratory Project (USA) European Grid Infrastructure European Middleware Initiative based on well-known concepts such as e-science, Science 2.0, Cyberinfrastructure and Semantic Web 6
General view of UC infrastructure Wiki-sites & Social Networks Virtual Laboratory User s Applications High-level Services SaaS Application Data Providers Computing Resources Experimen tal Facilities Applications Information Services Network VO Web Portal Storage Personal Environment Workflows Basic Grid Services Storage Element Computing Element End User Robotic Agents Other 7
Basic Grid middleware Security / Virtual organizations management service User Interface Computing element Storage element Information services Workload management 8
Data accumulation & distribution - LHC Worldwide LHC Computing Grid Tier 0 (CERN) Data recording Initial data reconstruction Data distribution Tier 1 (11 + KISTI, Korea in progress) Permanent storage Re-processing Analysis 10 Gbit/s links Tier 2 (~150 centres) Simulation End-user analysis Tier 3 (End-users) Customized processing Overall ~160 sites, 39 countries 300,000 cores 200 PB of storage 2 million jobs/day 9
Data accumulation & distribution - ITER ITER Site Tier 0 Full replication Domestic Agency Domestic Agency Domestic Agency Tier 1 National Institute National Institute Processed Sample or partial replica Tier 2 Processed Sample End User Tier 3 10
Unified Data providers Metadata update Information Service Facility-independent Metadata scheme Cache Storage Native data format On demand data selection Sampling processor Server-side data manipulation HTTP MDSPlus End User Unified Data Provider 11
Fusion research poses a lot of loosely-tied tasks Parametric optimization of fusion facilities and algorithms Data mining & scaling laws Empirical dynamic models generation 3D-animation of dynamic processes Monte-Carlo & Ray tracing 12
Job flow in the WLCG/EGEE Grid 13
Grid-enabled applications as Services Direct Grid API usage Direct Service API usage SaaS Application 1 Grid interface End User Workflow composer SaaS Application 2 Grid interface Grid Workflow sharing SaaS Application 3 Grid interface 14
Cooperation within ITER & ITER as a partner of cooperation JET ITER KSTAR T-15M Fusion community cyberspace DIII-D DEMO Step 1: Prove feasibility of Worldwide Fusion Laboratory on the base of ITER and existing solutions. Step 2: Integrate storages of other TOKAMAKs with the unified cyberspace on the base of proven technology. Step 3: New super-collaboration endeavors for DEMO optimized fusion power plant. 15
Thank you for your attention! Discussion 16