基礎架構之未來展望 林誠謙中央研究院物理所. 第八屆兩岸三院資訊技術交流與資源共享研討會 7 September 2010

Transcription

1 e-science 基礎架構之未來展望 林誠謙中央研究院物理所 第八屆兩岸三院資訊技術交流與資源共享研討會 7 September 2010

2 Emergence of a New Science Paradigm e-science or Data Centric Sciences Data deluge: from Petabyte to Exabyte scale. Effective way to store, manage, access, analyze distributed scientific data and their associated computing/processing e-science applications often require access to very large data collections, very large scale computing resources by very broadband dynamic network resources carried out through distributed global collaborations enabled by the Internet 2

3 Data-Driven Multiscale Collaborations for Complexity Great Challenges of 21st Century Multiscale Collaborations General Relativity, Particles, Geosciences, Bio, Social... And all combinations... Science and Society being transformed by CI and Data Completely new methodologies The End of Science (as we know it) CI plays central role No community can attack challenges Technical, CS, social issues to solve Places requirements on computing, software, networks, tools, etc *Small groups still important! Source: Ed Seidel 3 3

4 Summary of e Infrastructure Investment in US and Europe US NSF: Annual expenditure on cyberinfrastructure $865M EC Campus cyberinfrastructure ( ) OCI priority: software, service, education, computational sciences TeraGrid: $ 31M/year OSG: $ 6M/year FP7 INFRASTRUCTURE Call: 217 M Support to existing research infrastructure: 162 M Support to new research infrastructure: 45 M Support to policy development and program implementation: 10 M ESFRI (Roadmap 2008): Identified 7 areas, 44 projects, and invest ~20B in 10 years. Partnership for Advanced Computing in Europe (PRACE): 20M from Jan 2008 Dec Enabling Grid for E science (EGEE): 23.6M /year 4

5 5 5

6 e-science in Asia Diversity: Geographically large and culturally diverse in nature Level of scientific collaboration often reflected by the networking connectivity The region as a whole traditionally inexperienced in regional cooperation Grids in Asia Inhomogeneous Grids with limited operations experience, making collaboration difficult. Why e-science in Asia? The global infrastructure is establishing quickly Take advantage of sharing and collaboration to bridge the gap between Asia and the world To address the challenge of regional cooperation EGEE/EUAsiaGrid have helped building the unseen Regional Collaboration. One hopes many others will happen soon! 6

7 Standardise Grids and Clouds in OGF, glite and Globus AUTHZ : X509 proxy with VOMS extensions and X509 or SAML Information Service : GLUE 2.0 Job Description Language : JSDL, JDL Job Submission : BES, glite WMS API Job Logging & Bookkeeping : BES, UR, glite LB API Job Accounting : UR, RUS, glite APEL Job Acceptance : DRMAA, GRAM Job Manager Data Transfer : ByteIO, SRM, GridFTP, DMI Data Access : SRM, GridFTP, GFAL, LFC Open Cloud Computing Interface (OCCI): Interoperability, Portability, Integration cloud standards.org DMTF: Work registry SNIA: Cloud storage joint management SCRM: Cloud coordination Standards may create healthy competitive Business Model 11

8 ASGC is Facilitating Regional Collaboration and Bridging Asia with the World

9 Int l Collabora tion HEP BioInfo, BioMed Earth Science Env. Change ASGC Profile: Support Research Innovation and Innovate ICT WLCG TWGrid ATLAS, CMS EGEE, OSG WLCG T1 established CDF mpiblust-g2 Belle Drug Disc -AF EUAsiaGrid WLCG- CCRC Drug Disc - AF2 Seismic Data Center Carbon Flux WLCG- STEP09 GVSS- DengueF SWave Sim EGI-Inspire, CHAIN, EMI DEGISCO ehealth Seismogram Sim SGT Biodiversity, WRF4G Hum. SS Long-Term Preservation GISGrid Social Sim Grid Infra & Tech LCG-0 SRB ASGCCA APROC, GGUS, GStat Dissem ISGC and Training since 2003 Grid Interop - MS CCS GAP, SRM-SRB Grid Camp and Internship Program Gstat2, Wkf eng. VC, DesktopG Data Manager, VMM/Env, Remote Env. Smart Center 9

10 Building a Foundation for e-science International Collaborations ASGC, a WLCG Tier-1 Centre, has become an important partner in global e-infrastructure, Grid and Cloud technology The only team in Taiwan that has large-scale experience in building and negotiating for global R&E and grid backbone network Capable of supporting international collaboration and advanced scientific applications 10,000 CPU Cores compute capacity, Peta-Byte Storage 50 TB/day, 9.2 Gbps data transport, 100K jobs/day HTC system 1,900TB data In/Out of ASGC in 2009, ~1,800TB data transferred till end of July Resource and application integration with 200K+ CPU Cores in global grids Leads international advanced collaborative scientific applications The biggest Drug Discovery application in global grids Bridging SE Asia and global collaboration in Earthquake simulation and disaster mitigation Coordinate 14 countries, 30 institutes in AP in global grid collaboration 1 0

11 Enabling Grids for E-sciencE Computational Chemistry Social Science Bioinformatics and Biomedical High Energy Physics Mitigation of natural disasters 11

12 Collaborating Global e Infrastructure ~20,000+ Users 216,200 LCPUs (cores), 270+ sites, 60PB disk 39PB tape 0.7+ million Jobs/day TWGRID EUAsiaGrid linking ~80 countries

13 13

14 Clouds Interoperation Core Technology Operation HEP Life Science Earth Science E Science Applications Env Change HPC EGEE Asia Federation Volunteer Computing Social Science Dissemination & outreach Global & Regional e Infrastructure International e Science Network Global Grid Operation & Support 16

15 1 5 E-Science Collab Networking in Asia Pacific Region SINET SG I2 / GN2 WIDE JP-Tokyo- LCG2 AU-ATLAS JP-KEK2 JP-KEK1 PK-NCP HK-HKU KR-KNU KR-KISTI- GCRT CN-SDU- LCG2 CN-BEIJING- LCG2 AARNET PK- PAKGRID MY- MIMOS IN- VECC1 IN-TIFR TW-NIU TWAREN/ TANET IP Transit TW- NTCU TW-FTT TW- NCUHEP ASGC IN- VECC2 JP- HIROSHIMA- WLCG2 MY-UPM- BIRUNI-01 TH-HAII TH- NECTEC VN-IOIT- HN VN-IOIT- KEYLAB VN-IFI- PPS MY-UM- CRYSTAL PH-ASTI- LIKNAYA N KREONE T CSTNE T NL 2.5G 2.5G 2.5G 10G 622M 10G TW HK US JP CERNE T APAN- JP

16 Mission Asia Pacific Regional Operations Center Provide deployment support facilitating Grid expansion Maximize the availability of Grid services Supports EGEE sites in Asia Pacific since April production sites in 13 countries Over 14,000 CPU Cores and >4 PB in Jan Runs ASGCCA Certification Authority since 2003 chairs APGridPMA for the moment, part of IGTF (Int. Grid Trust Federation) Middleware installation support Production resource center certification Operations Support Monitoring Diagnosis and troubleshooting Problem tracking Security

17 Some Exemplar Applications

18 e-science Applications in ASGC High Energy Physics: WLCG, CDF, Belle Bioinformatics: mpiblast, BEAST, Gromacs, MUSCLE, LibSVM Biomedicine: Distributing AutoDock tasks on the Grid using DIANE Digital Archive: Data Grid for Digital Archive Long-term preservation Climate Change & Weather Simulation: Typhoon Simulation, Carbon Flux Earth Sciences: Seismic Wave Propagation Analysis, Seismogram-On-Demand, Hazard Map Humanity and Social Sciences: Social Simulation General HPC Services e-science Application Development Platform & Application Portal D4Sciene Gateway for Fish Resource and Ecosystem Management 31

19 CERN Mont Blanc Large Hadron Collider for High Energy Physics as Driver for Global e-infrastructure Geneva Basic physics research Asking and answering fundamental questions High Energy accelerator: Giant microscope (p=h/λ) Generate new particles (E=mc 2 ) Create conditions close to the Big Bang (1ns after) les.robertson@cern.ch 19

20 ATLAS Workflows Prompt Reconstruction Tier 0 Calibration & Alignment Express Stream Analysis CAF CASTOR Collab: Tokyo U, JP Melbourne U, AU IHEP, Beijing, CN IoP, AS, TW Complex Scientific Workflow for Event Reconstruction, Re-processing and Distributed Analysis MB/sec 650 MB/sec RAW Re processing HITS Reconstruction Tier 1 Tier 1 Tier MB/sec Simulation Analysis Tier 2 Tier 2 Tier 2 Tier 2 Tier 2 Tier 2 Tier 2 Tier 2 Tier 2 Tier 2 Tier 2 Tier 2 20

21 CMS Workflows Prompt Reconstruction Collab: KNU, KR NTU, NCU, TW IHEP, CN TIFR, IN; NCP, PK CASTOR TIER MB/s Simulation Analysis TIER-0 TIER-1 600MB/s MB/s TIER-1 TIER- TIER-2 2 TIER-2 TIER- 2 CAF ~20MB/s Calibration Express- Stream Analysis Re-Reco Skims Complex Scientific Workflow for Event Reconstruction, Re-processing and Distributed Analysis 21

22 Bio Portal and Virtual Screening Services Best Demo Award of EGEE 07 Conference x The 8 Avian Flu Mutations were proposed by GRC, AS Total computing power used 137 cpu-year 8 variants against 400k compounds 22

23 Dengue Fever Data Challenge in 2009 Total number of completed docking jobs Estimated needed computing power Duration of the experiment Cumulative computing results Total Computing Recourses in EUAsia VO Number of used Computing Elements 300,000 4,167 CPU*days 60 days 42.5 GB 268 Cores 6 Collaborators: UPM, MIMOS, MY IAMI, VN; HAII, TH Cesnet, CZ; GRC, TW 23

24 Hi-CLIMB CREATE-IES

25 Possible Sites Surrounding South China Sea 25

26 Numerical Simulation for Tsunami Hazards (Liu et al., 2007) 26

27 eismogram Simulation Services 1. Location and Tomography Model Selction 2. Epicenter Data Preparation 4. Seismogram Access & Visualization 3. Choose Position for Seismogram 28

28 Building Strain Green s Tensor Database 50 stations are chosen for SGT calculation, based on Taiwan 3D velocity model of Taiwan The simulation scope covers the main Taiwan island only 230 x 390 x 80 km 3 The side length of a grid is 200 m About 9 x 10 8 grids Challenges (for one-minute propagation running by a 64-thread MPI job) Total time: 50 (sites) x 3 (phases) x 1 (day) = 150 days CPU Time: 150 * 64 * 24 hours= 230,400 CPU Hours Total space: 50 x 3 x 15 = 2250 GB (about 2 TB) 29

29 Future Works Hazard Maps Achieving full process of quantitative seismic hazard assessment Collecting and analyzing event data Understanding fault characteristics in details Facilitating accurate simulation on seismic waves Assessing anticipated earthquake and potential damages by the correct seismic and engineering models Maps of disaster coverage, risk and also evacuation are pragmatic to better preparedness Seismologist EUAsiaGrid 30

30 Typhoon Morakot (2-11 August, 2009) Highest winds: 140 km/h (10-minute sustained) Fatalities Rainfall Damage : 789 total : 2777 mm (total) : $6.2 billion (2009 USD) Jhiben ( 知本 ), Taitung, Taiwan Xiaolin village ( 小林村 ), Taiwan 31

31 Isobaric Contour (Morakot) Simulation during ( ~ ) :00:00 CWB surface analysis WRF Simulation (at 0.25 km) Central presure deviation ( )/980=2.5% 32

32 Precipitation (Morakot) Simulation ( :00~ :00) WRF Simulation CWB history data 33 33

33 Computational Chemistry Characteristics of Complex Surface Reaction Formic acid molecule absorption on RuO 2 (110) surface VASP/MPI over glite is deployed Alternative Energy Non-Silicon Solar cells Dye Sensitized, Polymer, Organic solar cells Fuel Cell Gaussian is ported Catalysis n-hexane Cracking Electronic Structure Calculation QSAR analysis of cytotoxicity Collaborators: NECTEC TH, ITB ID EUAsiaGrid ASGC, Taiwan, CESNE 34

34 Social Simulation Project on population migration simulation from 2010 TW UK Collaboration Porting the UK based Migration Model to glite/euasiagrid Customization of the model for/of Taiwan Taking into account the birthrate, fertility, and mortality Deploy the local model based on regional researches Feedback cycle for model verification: Based on the real Census data of Taiwan Deployment of agent based modeling/simulation methods Further extension Financial model Social changes Collaborators: U. Manchester; U. St. Andrews; Survey Research Center, AS; EUAsiaGrid 35

35 EUAsiaGrid Application Portal Life Sciences Autodock 4 Beast Blast Gromacs MrBayes Muscle Prodist GVSS * Earthquake * WRF * R Other User Defined Applications * Will be available at the portal soon.

36 E-Science Collaboration in EUAsiaGrid Discipline Applications Partners Plan for DesktopGri d HEP ATLAS, CMS, ALICE TH, TW, CESNET, INFN X BioMedical Virtual Screening for Drug Discovery Avian Flu, Dengue Fever Pandemic disease analysis MY, TW, VN, CESNET, INFN VN, FR X Bioinformatics Grid enabling phylogenetic inference SVM Parameter optimization for prediction of Caspases Genome search to identify T3SS effect Autodock ligand-receptor docking Complex diseases studies SG, TW, VN, CESNET, INFN X X Earth Science Disaster Mitigation on Earthquake ID, MY, PH, TH, VN, TW, CESNET, INFN X Comp Chemistry Chemical compound property analysis TH, TW, CESNET X Climate Change Weather simulation, sea level rising ID, PH, TH, VN, TW 37

37 Future Perspectives

38 Core Component for e-science Core Component Objective Features/Requirement Internet High Performance Computing Connecting by Light Speed Dealing with larger scale problems Connection across administrative domain Tools, methodology, library, model & workflow Distributed Infrastructure Sharing & Integration Interoperation, standardization, security, scalability Scientific Data Knowledge Exploration Availability, accessibility, consistency, IPR, provenance Virtual Organization Research Innovation Collaboration model, resource sharing & provisioning mechanism

39 EGI-InSPIRE Integrated Sustainable Pan-European Infrastructure for Researchers in Europe A 4-year project with 25M EC contribution Project cost 70M Activity cost ~ 330M EGI European Grid Initiative Deploying Technology Innovation Distributed Computing continues to evolve Grids, Desktops, Virtualisation, Clouds Enabling Software Innovation Provide reliable persistent technology platforms Today: Tools built on glite/unicore/arc Supporting Research Innovation Infrastructure for data driven research Support for international research (e.g. ESFRI) 40

40 Usability Next Generation Grid Middleware Improve usability of DCI middleware by simplifying its architecture, configuration, deployment, management & functionality Security Harmonize security management across implementation and integrate new user-friendly security methods Standardization Interoperability Across Middlewares, Grid+Cloud, ServiceGrid+DG, HTC+HPC Innovation Driven by application pull and ICT push Commercial Relationships Target at sustainability & long-term improvement with minimized cost 41

41 Keynote & Plenary Speech 42

42 43

43 Grid + Cloud Both Grids and Clouds are evolutions of Virtualisation Virtulaisation will provide enormous flexibility in both Grids and Clouds management Grids and Clouds are opportunity to each other! The ICT is always changing, nothing is guaranteed to prevail Virtual execution environment is the way to reach interoperability, portability, scalability, taking advantage of the DCI 44

44 Integration of Service Grid and Desktop Grid Desktop Grid DG Client (PULL) Global File System or Easy Way to Share the Data GIS Service Grid Storage Application Server/Repository (VO-enabled) WMS CE SE Other Core Services DG Client (PULL) DG Client (PULL) DG Client (PULL) User Interface User 4 5

45 e Science for the Masses Grid-based global production e-infrastructure is already in operation Unique scientific values of e-science Application Data, e.g. LHC data, unprecedented energy frontier, new fundamental understanding of the Universe Earthquake data, first-principle simulation, archival and re-use Drug Discovery data, neglected diseases information, open access and generating more knowledge Regional collaborative data often related to Disaster Mitigation Must reduce the entry barriers for e-science in Asia 2-3 MW for top PetaFlop machine, how about ExaFlop machine? Application footprint must be reduced, cores for useful app e-science is for the masses instead of big science in Asia

46 CHEP