Data Driven Innova,on, High Performance Compu,ng and the Square Kilometre Array

Transcription

1 Data Driven Innova,on, High Performance Compu,ng and the Square Kilometre Array Accelerate Cape Town April

2 Background Big Data >> Data Driven Innova4on The Square Kilometre Array High Performance Compu4ng Facility 2

3 Data is Big 3

4 Defini4on Challenges Data Value Chain Genera,on Storage Distribu,on Collec,on Processing Analy,cs 4

5 Defini4on Challenges Big Data (OECD) Volume (size) Velocity (speed of crea4on, processing, analysis) Variety (diversity) Value (social and economic benefit) High Performance Compu4ng is the use of parallel and high processing power for running advanced applica4ons reliably and quickly. The term is osen used interchangeably with Supercompu4ng 5

6 Big Data involves all steps, but HPC is focused on Processing and Storage with some Analy4cs Genera,on Storage Distribu,on Collec,on Processing Analy,cs 6

7 Focus: Big Data Facility Genera,on Storage HPC Distribu,on Collec,on Processing Analy,cs 7

8 Data Driven Innova4on? Data volume Business Big Data Research Big Data Extreme Data Big Data Long Tail Number of data sets 8

9 Key aspects of a Data driven Innova4on strategy The following are common elements : Inves4ng in ideas collabora4on, public private partnerships, incubators, etc. Infrastructure for a data-driven economy HPC Cloud Compu4ng Infrastructure Broadband Policy issues Data protec4on, Data privacy, Standards, etc. Skills High Quality Datasets government data, open data, partner data, info marts, etc.

10 The Square Kilometre Array 10

11 Square Kilometre Array The Square Kilometre Array (SKA) - the world s largest radio telescope, with one million square metres of collec4ng area. The SKA will survey the Universe in unprecedented detail and survey the en4re sky much faster, and in more detail, than any system currently in existence. 11

12 Square Kilometre Array Mega-science project, which will push the limits of engineering and scien4fic endeavour over the coming decades. Will require the development of cufng edge technology and innova4on, including the design of some of the world's fastest supercomputers to process data at rates close to the limits of exis4ng technology. Collabora4on - 10 member countries, about 100 organisa4ons from 20 countries par4cipa4ng in the design & development 12

13 Terminology SKA and DIRISA use the term Tier to refer to the scope of par4cipa4on in a compu4ng facility, with Tier Zero being interna4onal, Tier One na4onal, Tier Two regional and Tier Three ins4tu4on-specific. This Tiering terminology should not to be confused with the Tier classifica4ons used by to Data Centres as set out by the Up4me Ins4tute. 13

14 User Requirement: SKA SKA Telescopes A SKA1 Interna4onal INTERNATIO HPC 5MW Tier Zero 100Gbps B Correlator Beam Former 7.4 Tbps Science Data Processor <1 Tbps Persistent Data Store KAROO A and B belong to the Telescope ( SKA Interna4onal) and C to SKA- SA for local use and 3rd par4es. B is HPC - type facility (~ 450sqm) C is also HPC - type facility (~ 150sqm) CAPE TOWN 115 RACKS Other Poten4al Users CHPC Universi4es Government Business Archive (Tape) 2MW Tier One 60 RACKS HPC Regional Science Centre for South Africa 30 RACKS C 14

15 User Requirements: SKA 7.4Tbps link from Carnarvon (4Tbps data transfer rate for telescope data; will be other occupancy from SKA in Carnarvon) 100Gbps interna4onal link/s from Tier 0 facility Space for 200 racks equivalent (~600m 2 = 450 T T1) Likely 19 inch rack size, custom build HPC 7MW power (5MW + 2MW), backup sufficient to shut down gracefully for both T0 and T1 facili4es. PUE <1.2 98% up4me both T0 and T1 facili4es Contents of T0 racks likely to change every 3 years or sooner (lower power consump4on, more pflops/faster) à flexible cooling methods, power densi4es (up to 50kW/rack) needed within facility Tier 0 to be located in Cape Town as a project requirement of SKA interna4onal Co-loca4on of T0 and T1 facili4es preferred to minimise costs of data transfer and to avoid opera4onal complexity for SKA-SA of managing disparate facili4es. SKA preference: Lease space in a suitable facility, own the racks and everything in them, outsource maintenance of the racks and the space Ready for first use 2020 latest, SKA would use both Tier 0 and Tier 1 in part if ready earlier. Ramp up planned, fully deployed and opera4onal by

16 User Requirements: Others Poten,al Users CHPC UCT Astronomy (IDIA) City of Cape Town + WCG CHEC SITA Private sector Major cloud service providers Comments Strapped for space now, much overlap of people, skills, data, research with SKA, IDIA. Similar requirements as for Tier 1 facility. Need staging area. Upgrades underway now will cover needs for next 3 years. Designing the SDP, have R20m of equip needs a home now in Tier 2 facility. Facility can be remote but scien4fic ops centre wanted at UCT. Governance to be scien4st-led. Poten4al user of DC space New T2 data centre to be built R30m funding sought from DST; IDIA/UCT lead partner. Universi4es need affordable off-site data centre services Poten4al user of DC space Low HPC usage at present (5%) but poten4ally double that ito demand, CHPC has capacity to offer services to assist: not being taken up much. Custom-built compu4ng and DCs (no Africa presence); no co-loca4on history, need % + up4me, N+n power 16

17 High Performance Compu4ng 17

18 Top Ten High Performance Computers RANK SITE SYSTEM CORES RPEAK (TFLOP/S) POWER (KW) 1 2 Na4onal Super Computer Center in Guangzhou DOE/SC/Oak Ridge Na4onal Laboratory Tianhe-2 (MilkyWay-2) 3,120,000 54, ,808 Titan - Cray XK7 560,64 27, ,209 3 DOE/NNSA/LLNL Sequoia - BlueGene/Q 1,572,864 20, , RIKEN Advanced Ins4tute for Computa4onal Science (AICS) DOE/SC/Argonne Na4onal Laboratory K computer, Fujitsu 705,024 11, ,66 Mira - BlueGene/Q 786,432 10, ,945 6 DOE/NNSA/LANL/SNL Trinity - Cray XC40 301,056 11, Swiss Na4onal Supercompu4ng Piz Daint - Cray XC30 115,984 7, ,325 Centre (CSCS) HLRS - Höchstleistungsrechenzentrum Hazel Hen - Cray XC40 185,088 7,403.5 Stu{gart 9 10 King Abdullah University of Science and Technology Texas Advanced Compu4ng Center/Univ. of Texas Shaheen II - Cray XC40 196,608 7, ,834 Stampede - PowerEdge C ,462 8, ,51 18

19 Centre For High Performance Compu4ng Hybrid cluster currently upgraded to 1 PFLOPS 4 Petabyte storage Achieved 92.4% uptime with utilisation of over 80%. Provide the services to over 700 researchers from academia and industry Facilities used as test-bed for SKA-SDP Provide fail-over service for South African Weather Services Support for CERN experiments (ALICE and ATLAS) Industry users at CHPC (SASOL, TOTAL, DeBeers, etc.) high fidelity simulation and modelling such as CFD, FED, etc.) to improve competiveness Comprehensive HCD and training initiatives in HPC and domain specific needs Info Courtesy CSIR 19

20 Student Cluster Championship The student cluster challenge is aimed at developing HPC skills at early age 8 10 teams (4 students each) enter the national competition A team of six selected to represent South Africa in International challenge 2013, 2014 champions and 2 nd place in Attracted Dell to invest R120 Million over 10 years Other companies such as Mellanox, CRAY, Eclipse, HP, BMW provide support Info Courtesy CSIR

21 Big Data - High Performance Compu4ng Facility 21

22 Analysis Interna4onal Karoo A Cape Town B SKA Array + CBF Tier Zero HPC filtering + Data Store C Tier One Regional Science Centre for South Africa 22

23 Analysis Interna4onal Karoo Cape Town B SKA Array + CBF Tier Zero HPC filtering + Data Store C A + B must be built, by SKA Int l Likely to be operated/managed by SKA-SA under contract Special-purpose, closed Cooling to the racks, high power density, non-standard racks? 98% up4me, power backup to shut down gracefully, restart quickly Tier One Regional Science Centre for South Africa 23

24 Analysis Interna4onal Karoo Cape Town B SKA Array + CBF Tier Zero HPC filtering + Data Store C to be established by SKA SA Mul4-user facility (Astro-related) Combina4on HPC + DC 2MW power 98% up4me, power backup to shut down gracefully This kind of facility needed now by partners in SKA SA (CHPC, MeerKAT) Tier One Regional Science Centre for South Africa C 24

25 Hypothe4cal Facility Modular and flexible Space Power Cooling Physical security Power: 10MW+ (15 max?) 98% up4me HPC facility SKA anchor tenant + CHPC? 600m 2 ini4al à 1200m 2? Connec4vity: 7.4Tbps for SKA, mul4ple 100Gbps connec4ons to WACS, SA backbone Services: run facility, can manage racks Reference: Cambridge HPC, not Pawsey DC Conven4onal public sector DC space poten4al SKA HPC spaces Other HPTC spaces HPBC spaces Conven4onal data centre spaces (white) (green) (orange) (blue) Tier Zero Tier One Other Sectors needing Tier One (Health, Safety and Security, etc.) University Off-site backup and applica4on hos4ng CHPC CHEC Tier 2 Government subsidised HPTC R&D and training facility Government Off-site backup and applica4on hos4ng Private Sector HPBC Cloud Service Providers Government subsidised HPBC R&D and training facility Commercial co-loca4on 25

26 Two Op4ons: Site Selec4on Criteria Exis4ng Facility Operator Site. Will be explored as part of the Expression of Interest phase New Facility Brownfield or Greenfield. The costs for brownfield and not much different to those for Greenfield. The only advantage of brownfield may be the elimina4on of a rezoning process 26

27 Site Selec4on Criteria Loca4on of Users, Partners, Specialists, Service Providers 50km radius of Cape Town CBD Available Electricity Capacity new build of capacity has long lead 4mes Electricity Authority Eskom vs CoCT (Costs) Suitable Erven size Percep4on of poten4al tenants. A facility based ONLY on SKA needs does not have the need to be able to a{ract private sector tenants 27

28 Site Selec4on Criteria Ownership: Those proper4es owned by the City of Cape Town or WCG may be available for development at a reduced cost. Zoning: Land with the appropriate development rights will have a shorter permifng process since only building plans will be required. Physical: Environmental, heritage studies etc may complicate the development of the property Pres4ge: High profile facility with Interna4onal status has presenta4on and visibility value 28

29 Opportuni4es Facility: Design, Construc4on, Project management Equipment and Infrastructure: HVAC, UPS, BMS, Fire suppression.. Facility Management 29

30 Opportuni4es Equipment and Systems The Science Data Processor (Supercomputer) built from scratch High Performance Compu4ng Resources for Businesses Skills Development: RSA have twice won the award for Student HPC Cluster at the Interna4onal Supercompu4ng Cluster compe44on 30

31 Highly scalable methods for modelling and simulation exploit massive parallelism and data visualisation. Heterogeneous computing environments require new programming models.. Emerging real-time, data-intensive applications require robust HPC capability. 31

32 Expression of Interest Opportunity to canvass the global private sector for perspec4ves Bridges strategy and business design Provide facility requirements and 4me frames Receive op4ons for mee4ng requirements, percep4ons of risk and opportunity, indica4ve costs Valuable reality-check 6 week exercise from mid-april (tbc) 32

33 Thank You 33