Chapter 5b: top500 Top 500 Blades Google PC cluster Computer Architecture Summer 2005 5b.1
top500: top 10 Rank Site Country/Year Computer / Processors Manufacturer Computer Family Model Inst. type Installation Area R max R peak N max n half 1 DOE/NNSA/LLNL United States/2005 BlueGene/L eserver Blue Gene Solution / 65536 BlueGene/L eserver Blue Gene Solution Research 136800 183500 1277951 2 Thomas J. Watson Research Center United States/2005 BGW eserver Blue Gene Solution / 40960 BlueGene/L eserver Blue Gene Solution Research Information Processing Service 91290 114688 983039 3 NASA/Ames Research Center/NAS United States/2004 Columbia SGI Altix 1.5 GHz, Voltaire Infiniband / 10160 SGI SGI Altix SGI Altix 3700 Research 51870 60960 1290240 4 The Earth Simulator Center Japan/2002 Earth-Simulator / 5120 NEC NEC Vector SX6 Research 35860 40960 1075200 266240 5 Barcelona Supercomputer Center Spain/2005 MareNostrum JS20 Cluster, PPC 970, 2.2 GHz, Myrinet / 4800 Cluster JS20 CLuster Academic 27910 42144 977816 6 ASTRON/University Groningen Netherlands/2005 eserver Blue Gene Solution / 12288 BlueGene/L eserver Blue Gene Solution Academic 27450 34406.4 516095 7 Lawrence Livermore National Laboratory United States/2004 Thunder Intel Itanium2 Tiger4 1.4GHz - Quadrics / 4096 California Digital Corporation NOW - Intel Itanium Itanium2 Tiger4 Cluster Research 19940 22938 975000 110000 8 Computational Biology Research Center, AIST Japan/2005 eserver Blue Gene Solution / 8192 BlueGene/L eserver Blue Gene Solution Research 18200 22937.6 442367 9 Ecole Polytechnique Federale de Lausanne Switzerland/2005 eserver Blue Gene Solution / 8192 BlueGene/L eserver Blue Gene Solution Academic 18200 22937.6 442367 10 Sandia National Laboratories United States/2005 Red Storm, Cray XT3, 2.0 GHz / 5000 Cray Inc. Cray XT3 Cray XT3 Research 15250 20000 Computer Architecture Summer 2005 5b.2
top500: mare nostrum 5 Barcelona Supercomputer Center Spain/2005 MareNostrum JS20 Cluster, PPC 970, 2.2 GHz, Myrinet / 4800 Cluster JS20 CLuster Academic 27910 42144 977816 2282 blade servers in 27 racks, < 50 m 2 blades: very small, slim like a book minimalistic PC (processor, memory, hard drive) without power unit without USB, peripherals support without video support typically hot pluggable energy dissipation of a single blade much smaller than of a PC BUT: higher density in a rack Computer Architecture Summer 2005 5b.3
blades vs. regular server rack blades classical approach..\marenostrum\bsc_outside.mp4 Computer Architecture Summer 2005 5b.4
marenostrum installation Computer Architecture Summer 2005 5b.5
Germany Ran k Site Country/Year Computer / Processors Manufacturer Computer Family Model Inst. type Installation Area R max R peak N max n half 27 HWW/Universitaet Stuttgart Germany/2005 SX8/576M72 / 576 NEC NEC Vector SX8 Information Processing Service 8923 9216 829440 55296 576 NEC-SX8 processors binary compatibility to Earth Simulator Computer Architecture Summer 2005 5b.6
82 Bank United Kingdom/2004 BladeCenter HS20 Xeon 3.06 GHz, Gig- Ethernet / 3200 Cluster 19584 83 Westgrid Canada/2005 BladeCenter Xeon 3.06 GHz, Gig-Ethernet / 1680 Cluster Academic 10281.6 84 Petroleum Company (G) Saudia Arabia/2004 BladeCenter HS20 Xeon 3.2 GHz, Gig- Ethernet / 1536 Cluster Geophysics 9830.4 85 Credit Suisse/First Boston United Kingdom/2004 BladeCenter HS20 Xeon 3.06 GHz, Gig- Ethernet / 1500 Cluster Finance 9180 86 Animal Logic Australia/2005 BladeCenter HS20 Xeon 3.06 GHz, Gig- Ethernet / 1148 Cluster Digital Media 7025.76 87 Geoscience (B) India/2005 HS20 Cluster, Xeon EM64T 3.4 GHz - Gig- Ethernet / 1024 Cluster HS20 Cluster Research Geophysics 6963.2 88 Geoscience (C) United Kingdom/2004 xseries Xeon 3.06 GHz - Gig-E / 1128 Cluster Geophysics 6903.36 89 Geoscience (C) United Kingdom/2004 xseries Xeon 3.06 GHz - Gig-E / 1128 Cluster Geophysics 6903.36 90 Semiconductor Company (D) Israel/2004 xseries Xeon 3.06 GHz - Gig-E / 1120 Cluster Semiconductor 6854.4 91 Financial Services (H) United States/2005 BladeCenter HS20 Xeon 3.2 GHz, Gig- Ethernet / 1024 Cluster Finance 6553.6 92 Bank (H) Germany/2004 BladeCenter Xeon 3.06 GHz, Gig-Ethernet / 1064 Cluster Finance Computer Architecture Summer 2005 5b.7 6511.68
Germany 110 Adam Opel AG d Germany/2005 eserver pseries p5 575 1.9 GHz, GigEth / 720 SP SP Power5 Automotive 3068 5472 112 Max-Planck-Gesellschaft MPI/IPP Germany/2005 eserver pseries p5 575 1.9 GHz, GigEth / 720 SP SP Power5 Research 3068 5472 165 Deutscher Wetterdienst Germany/2003 SP Power3 375 MHz 16 way / 1920 SP SP Power3 375 MHz high node Research Weather and Climate Research 2106 2880 366 Robert Bosch Germany/2005 Cluster Platform 3000 DL360G4 Xeon 3.4 GHz, GigEthernet / 360 Hewlett-Packard HP Cluster HP DL360 Cluster Database 1395.4 2448 Computer Architecture Summer 2005 5b.8
HLRN: virtual supercomputing cluster 368 HLRN at Universitaet Hannover / RRZN Germany/2004 pseries 690 Turbo 1.3GHz / 512 SP SP Power4 Academic 1384 2662.4 369 HLRN at ZIB/Konrad Zuse- Zentrum fuer Informationstechnik Germany/2004 pseries 690 Turbo 1.3GHz / 512 SP SP Power4 Academic 1384 2662.4 Computer Architecture Summer 2005 5b.9
statistics Computer Architecture Summer 2005 5b.10
statistics Computer Architecture Summer 2005 5b.11
statistics Computer Architecture Summer 2005 5b.12
case study: google goal: no search takes longer than 0.5 seconds (incl. network delays) 1000 queries / second on average 6000 processors, 12.000 disks, 1 petabyte storage source: Patterson, Hennessy: Computer Organization and Design, Morgan Kaufmann, 2005 Computer Architecture Summer 2005 5b.13
google cluster site 40 racks, each 80 PCs fail-over Ethernet switch fail-over network connection (OC48: 2.4 Gbit/s, OC12: 600 Mbit/s) Computer Architecture Summer 2005 5b.14
google PC rack one 100 Mbit Ethernet switch in the center front and back 40 PCs each Computer Architecture Summer 2005 5b.15
google rack-unit PC PC contains: hard disks 256 MB RAM power supply Intel processor not a real blade energy: 55 Watt/PC, 4500 Watt/rack Computer Architecture Summer 2005 5b.16