Fujitsu Petascale Supercomputer PRIMEHPC FX10 4x2 racks (768 compute nodes) configuration
PRIMEHPC FX10 Highlights Scales up to 23.2 PFLOPS Improves Fujitsu s supercomputer technology employed in the FX1 and K computer, the world s fastest supercomputer. Hybrid parallel (VISIMPACT) Collective SW FX1 SPARC64 TM VII 4C/40GF 121TFLOPS, CY2008~ Tofu interconnect ISA extension(hpc-ace) K computer * SPARC64 TM VIIIfx 8C/128GF 11PFLOPS, CY2010~ PRIMEHPC FX10 SPARC64 TM IXfx 16C/236.5GF ~ 23PFLOPS, CY2012~ * "K computer" is the name of a next-generation supercomputer developed by RIKEN in July 2010 1
PRIMEHPC FX10 Features High-speed and ultra-large-scale computing environment Up to 23.2 PFLOPS (98,304 nodes, 1,024 racks, 6 petabytes of memory) SPARC64 IXfx- high performance w/ low power consumption 236.5 GFLOPS, and over 2GFLOPS/W High execution performance with massively parallel apps Tofu interconnect, Technical Computing Suite, and VISIMPACT High reliability and high operability for a large-scale system 2
PRIMEHPC FX10 Configuration Job execution nodes (diskless) Tofu: 6D mesh/torus interconnect Compute nodes Disk and external comm. nodes IO nodes System Maximum configuration Node Number of nodes 98,304 (= 32x32x8x2x3x2) nodes Number of CPUs 1 CPU Peak performance 23.2 PFlops Peak performance 236.5 GFlops Total memory bandwidth 8.3 PB/s Memory bandwidth 85 GB/s Total interconnect bandwidth 3.1 PB/s Interconnect link bandwidth 5 GB/s x bidirectional Total IO bandwidth 49.1 TB/s IO bandwidth 8 GB/s per IO node 3
Implementation of PRIMEHPC FX10 Rack 96 compute nodes (24 SBs) 6 IO nodes (6 IOSBs) Maximum configuration 1,024 racks 98,304 compute nodes 6,144 IO nodes CPU 236.5 GFlops L2$ Core Core Core : Core : Node 1 CPU SX ctrl MC CPU Node DDR3 DIMM ICC 236.5 GFlops Memory of up to 64 GB System board (SB) 4 nodes (4 CPUs) 22.7 TFlops Memory of up to 6 TB 946 GFlops Memory of up to 256 GB 23.2 PFlops Memory of up to 6 PB 4
PRIMEHPC FX10 Packaging 12 System boards 12 PSUs 6 IO system boards 12 System boards Coolant inlet / outlet 5
Hybrid Cooling (Direct Water Cooling) Air vent valve Cooling plates for ICC ICC クーリングプレート Coolant couplers 液体コネクタ (Liquid Coupler) Intake hose for IOSBs Intake hose for IOSBs Motor valve Check valve Coolant inlet / outlet Temp. & dew sensors Filter Coolant pipes 配管 Piping Cooling plates for CPU SB with water cold plates 6
Hybrid Cooling (Slanted Mount System) Air vent valve Intake hose Exhaust air Apertures DIMMs for IOSBs Intake hose for IOSBs Check valve Motor valve DIMMs Temp. & dew sensors Apertures Filter Cabinet frame Coolant inlet / outlet Coolant pipes 7 Air velocity map of SB
K Computer and PRIMEHPC FX10 Fujitsu supercomputer w/ enhanced technology introduced for K computer K computer PRIMEHPC FX10 Note CPU SPARC64 VIIIfx SPARC64 IXfx SPARC V9 + HPC-ACE Peak perf. 128 GFLOPS 236.5 GFLOPS # of cores 8 16 Memory 16GiB 32GiB/64GiB 2GB/core~ BW 64GB/s 85GB/s Interconnect 6D mesh/torus Tofu interconnect System size X x Y x 17 X x Y x 9 Z=0 is I/O node link BW 5GB/s x bidirectional PRIMEHPC FX10 supports water cooling of ordinary chilling facility and resolves layout restrictions 8
Information Technology Center, The University of Tokyo Next Generation Supercomputer System Overview Compute nodes, Interactive nodes PRIMEHPC FX10 x 50 racks Local file system (4,800 comp. nodes, 300 I/O nodes) Peak Performance: 1.13 PFLOPS Memory capacity: 150 TB - aggregated memory bandwidth: 398 TB/sec Interconnect: 6D mesh/torus - Tofu - node-to-node: 5 GB/sec in both directions - network bi-section bandwidth: 6 TB/sec. PRIMERGY RX300 S6 x 2 (MDS) ETERNUS DX80 S2 x 150 (OST) Storage capacity: 1.1PB (RAID-5) Shared file system PRIMERGY RX300 S6 x 8 (MDS) PRIMERGY RX300 S6 x 40 (OSS) ETERNUS DX80 S2 x 4 (MDT) ETERNUS DX410 S2 x 80 (OST) Storage capacity: 2.1PB (RAID-6) InfiniBand network Log-in nodes Management servers Job management, operation management, authentication servers: PRIMERGY RX200S6 x 16 External file system PRIMERGY RX300 S6 x 8 Ethernet network External connection router LAN/WAN End users InfiniBand Ethernet FibreChannel
PRIMEHPC FX10 Summary Improves Fujitsu s supercomputer technology employed in the K computer, the world s fastest supercomputer Newly developed SPARC64 IXfx processor (236.5GFLOPS) Tofu interconnect (scales up to 98,304 nodes) Fujitsu provides all hardware and software Processor and interconnect Technical Computing Suite HPC middleware, as well as the FEFS highperformance distributed file system 10
11