Linux Compute Cluster in the German Automotive Industry

Transcription

1 Linux Compute Cluster in the German Automotive Industry Clusterworld, San Jose, June Dr. Karsten Gaier Altreia Solutions

2 Linux Compute Cluster are... Fast in Computation Cost-effective Perfect in Price/Performance-Ratio Failsafe Easy to expand Easy to administer Flexible to tailor

3 Linux Compute Cluster in the German Automotive Industry are... Based on PC-Hardware COTS-Products Usually rack-based Megware

4 Agenda 1. Statistics on Linux Compute Cluster in the German Automotive Industry 2. Usage and Implementation of Linux Compute Clusters in the German Automotive Ind. 3. Results of the Survey on Linux Compute Clusters conducted through a questionnaire

5 Comments on the Survey Includes data of Linux Compute Clusters (LCC) installed or ordered until May 2003 in the German Industry As complete as possible Of the 90 clusters identified 80 are used in Mechanical Design Automation (MDA)

6 Comments on the Survey LCC usage was analyzed by Market categories Companies Application Areas Hardware-Providers Service-Providers Market categories Companies Application Areas HW-Provider Service-Provider

7 Distribution of CPUs in different Market Categ. Market Categories Companies Application Areas HW-Provider Service-Provider Total of 5650 processors in 90 clusters # of CPUs

8 Total CPUs in LCC operated by different Companies Market Categories Companies Application Areas HW-Provider Service-Provider Total of 60 companies Altreia Solutions 2003, # of CPUs

9 Avg.. # of CPUs operated by different Companies Market Categories Companies Application Areas HW-Provider Service-Provider # of CPUs

10 LCC operated by selected MCAD-Companies Companies Market Categories Companies Application Areas HW-Provider Service-Provider

11 # of Companies using LCC in different market categ. Market Categories Companies Application Areas HW-Provider Service-Provider

12 Distribution of CPUs in different MDA-Appl Appl. Areas Market Categories Companies Application Areas HW-Provider Service-Provider Altreia Solutions 2003, # of CPUs

13 Cookbook: : A typical Linux Cluster processors installed Single- / dual-board depending on application Processors: : as fast as possible, right now 2.8 GHz 3.0 GHz Memory: : 1 2 Gbyte / node Fast-Ethernet Local harddisk: : IDE, 40 GByte

14 CPUs provided by different HW-Provider Market Categories Companies Application Areas HW-Provider Service-Provider # of CPUs

15 Avg.. Cluster-size size provided by different HW- Provider Market Categories Companies Application Areas HW-Provider Service-Provider # of CPUs

16 CPUs provided by different HW-Provider by Market # of CPUs Market Categories Companies Application Areas HW-Provider Service-Provider

17 External Service Providers for LCC About half of the companies has got their Linux Cluster up and running themself External Service Providers are also offering Hardware: NEC, Transtec,, s+c, SuSE, Teraport,, T-SystemsT Market Categories Companies Application Areas HW-Provider Service-Provider

18 Agenda 1. Statistics on Linux Compute Cluster in the German Automotive Industry 2. Usage and Implementation of Linux Compute Clusters in the German Automotive Ind. 3. Results of the Survey on Linux Compute Clusters conducted through a questionnaire

19 Day-by by-day - Operation of LCC: Administration Issues of Administrating a Linux Compute Cluster Zero-Admin in case of failures / new systems Monitoring / Triggering Software- and OS-Installation: de-/centralized Single Point of Administration Grouping / Sub-Clusters Parallelization

20 Administration Effort Day-by by-day - Operation of LCC: Administration 3 6 Hours / Node / Year using appropriate administration tools depending on complexity, so a cluster of 128 nodes (256 CPUs) needs from hours roughly ¼ to ½ man-year administration effort Remember! You save a lot on the HW!

21 Administration Effort Day-by by-day - Operation of LCC: Administration 3 6 Hours / Node / Year using appropriate administration tools depending on complexity, so a cluster of 128 nodes (256 CPUs) needs from hours roughly ¼ to ½ man-year administration effort

22 Day-by by-day - Operation of LCC: Failesaveness Failsaveness MTBF says: for a 128 node cluster you have a failure every 2 3 weeks; also because of high end usage of components: Spare Nodes Automatic switch-over using checkpointing and appropriate scheduling software mirrored disks (RAID 1), failsave network topology,,

23 Pentium-III III-Cluster Real world example FEKO-Cluster 16 nodes: : 32 Pentium-III 500 MHz orig. RedHat 6.0, updated to RedHat 7.0, Kernel per node: : 1 GB RAM, 8 GB SCSI-disk SCI interface (highspeed network interconnect) In productive use since 1999 Got big brother with 72 Xeon-2.4GHz 2.4GHz-Procs., 72 Gb Memory, in November 2002 Application: FEKO (electromagnetic compatibility) Installation: DHCP-Kickstart server Altreia Solutions 2003, K.Gaier@altreia.com Courtesy science + computing

24 Real world example FEKO-Cluster Transparent integration of all nodes into the corporate network Control-workstation also serves as installation server Central management of the whole workstation and server network User management Software management Configuration management Monitoring and inventory Altreia Solutions 2003, Courtesy science + computing

25 Real world example Crash-Cluster Cluster AMD-Cluster 192 nodes: : 384 Athlon RedHat 7.1, using Kernel per node: : 512 MB 2 GB RAM, 18(+36) GB-disk on 64 nodes: : SCI-Interface Interface Application: : LS-Dyna Altreia Solutions 2003, K.Gaier@altreia.com Courtesy science + computing

26 Real world example Crash-Cluster Cluster Dedicated cluster network Optimized installation: Efficient installation of the nodes: : Zero-Admin Dedicated administration domain Fast re-/installation in case of hardware failure Altreia Solutions 2003, K.Gaier@altreia.com Courtesy science + computing

27 Real world example Stochastic-Cluster Pentium-III/Pentium IV-Cluster 128 CPU: 52 Dual PIII 1GHz + 24 Single P4 1,5 GHz RedHat 7.1, Kernel per node: : 512 MB/1 GB RAM, 40 GB disk / 3x38 GB RAID 0 Fast Ethernet interconnect Applications: : Pamcrash/Nastran Altreia Solutions 2003, K.Gaier@altreia.com Courtesy science + computing

28 Day-by by-day - Operation of LCC: Portal Systems Web / Grid Portal Systems emerge to create interfaces to Linux Clusters from anywhere (inside) the company Automotive-OEM with space restrictions running jobs on remote systems through Web Portals Automotive Supplier with distributed sites creating seamless access to remote/all sites through a Grid Portal

29 Day-by by-day - Operation of LCC: Portal Systems

30 Agenda 1. Statistics on Linux Compute Cluster in the German Automotive Industry 2. Usage and Implementation of Linux Compute Clusters in the German Automotive Ind. 3. Results of the Survey on Linux Compute Clusters conducted through a questionnaire

31 Results of the Survey Participating Companies 20 returned questionnaires 35% Feedback-Rate Audi, Autoliv, Bertrandt, DaimlerChrysler, EDAG, Honda R&D, Leica Microsystems and many more companies of different size participated

32 Averages of Marks for different Aspects of LCC

33 Content with Application Availability for LCC Average of 0,72 Altreia Solutions 2003,

34 Content with Administration Effort for LCC Average of 0,86

35 Concerns about Instability of LCC Average of 0,21

36 Failing Disk Drives Instability Issues Other failing Hardware, sometimes overheating because of less careful handling of heating issues Pricing is competitive so HW-assembly is sometimes suboptimal Some companies let HW-pretesting be a system requirement,, Burn-In Phase Less: Linux-driver issues

37 Prob. that LCC will replace UNIX-Servers Average of 75%

38 Est.. Price/Perf. of LCC compared to UNIX- Sys. Average Factor of 4,4

39 Estimated Growth-Rates for LCC (1Year) Average Growth of 86%

40 TCO of Linux Compute Cluster Altreia Solutions 2003,

41 Main Issues to be resolved concerning LCC 1. Standardized MPI for Multi-Application usage (#1!) 2. Faster Networks cheaper, Easy wiring 3. Administration and Linux (Kernel,, Driver, Monitoring, ) 4. SMP-Multiproc Multiproc.-Systems with PC-Hardware TU Chemnitz 5. NFS faster / other free distributed Filesystems

42 Space!!! Blade Servers, Mixed Application Clusters Grid Portal Systems Distributed Sites Tighter Interaction of OEMs and Suppliers Linux Cluster as Application Server (also in combination with Compute Cluster) ERP: SAP, Outlook

43 Conclusion Linux Compute Clusters are established, productive and save a lot of money for the German Automotive Industry as well as in other market categories There are of course some issues but they don t weigh heavily

44 Thank you for your attention! Dr. Karsten Gaier Altreia Solutions altreia.com