CATbench. orkstation Benchmark

Transcription

1 CATbench 2005 Wor orkstation Benchmark Tips for Viewing This PDF article can be displayed and read on any computer which has Adobe Acrobat Reader software installed. It is best viewed using Adobe Acrobat Reader version 6.0 or newer. To download the latest version of this free software, visit the Adobe Website. You may print this article on standard letter-sized paper and read it anywhere, any time ConnectPress, Ltd. All rights reserved.

2 CATIA Community s 2005 CATIA Version 5 Workstation Benchmark By Phil Harrison Editor s Note: In 1999 CATIA Community (then CATIA Solutions Magazine) introduced the industry s first Version 5 CATIA workstation benchmark, CATbench. Now in our 7 th year, CATIA Community once again offers you a look at how well the various workstations on the market perform with CATIA Version 5. Return to the CATIA Community often for periodic CATbench2005 updates, which will rate new vendor systems as they become available. The mission of our benchmark remains unchanged: to provide CATIA users with an unbiased industry standard by which to measure workstation performance. There are many industry standard metrics to measure the various facets of computer performance; SPECint, SPECfp and Viewperf are used to measure integer and floating performance of the processor and graphics performance, respectively. Hardware manufacturers wax lyrical about bus speeds and disk speeds; however, it is difficult to know how these parameters interact to produce a system that will be fast and reliable in the hands of an end user. While any vendor can put a system together that is fast, CATIA Community feels that it is the ratio of price to performance that should be used to compare systems. Thus, when we rate systems we take price into consideration. Here is our price/performance rating system: ***** = Excellent **** = Good *** = Average ** = Below Average * = Poor Recently there have been several interesting developments in workstations: Intel has released updated versions of its Pentium 4 and Xeon processors, with speeds as high as 3.8GHz and Front Side bus speeds reaching 800 MHz. Competition remains cut-throat between Intel and AMD. Intel has just released its new families of dual-core processors removing the 32 bit limitation inherent in prior processors. Error correcting dual channel SDRAM has become the industry standard with speeds as high as 533 MHz; memory densities as high as 2 GB per card now allow even small form factor systems to support up to 8 GB of memory. NVIDIA has remained a cost effective graphics card provider with its Quadro FX family of cards for technical workstations. However, we are pleased to see fresh competition from ATI and others. Serial ATA disk drives have continued to narrow the gap to SCSI drive performance. Lastly, but importantly, prices have continued to drop. Dassault Systèmes released two more versions of CATIA V5, significantly improving process coverage, capability and stability. System Features and Results Table 1 shows prices and detailed system specifications for the systems tested for CATbench We decided to test an older Dell M60 with a 1.7 GHz processor to see how the M70 had improved over its predecessor. Dell Precision M60 Mobile Workstation Let s examine the unique features of each system and see how they rate. Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 2

3 Dell Precision Workstation 380 with NVIDIA FX1400 We were fortunate that Dell chose to submit their new Precision Workstation which features Intel s new Pentium EE840 dual core processor. First let s explain what dual core means. Prior processor families from Intel had a single processor per chip. To have two processors in the past we had to have two chips and use Intel s Xeon or AMD s Opteron 200 series processors. Now, with dual core, Intel is actually packaging two processors on a single chip. This allows us to run multi-threaded applications or multiple applications faster as the operating system can allocate tasks concurrently to the processors. The Precision 300 series of workstations is actually Dell s low end value offering. The 380 system is equipped with a Pentium Comparing Apples to Apples In order to give repeatable and comparable results, certain system parameters, test procedures and CATIA settings were standardized throughout the benchmark. System Setup CATIA code was installed locally on the system s hard disk. Systems were tested using CATIA Version 5 Release 14 P2 solutions. All systems were upgraded to Service Pack 5, the latest service pack available at the time of testing. The service pack was committed during installation. All data was stored on the local hard disk. If available, the second physical disk was used for data and cache. All systems were set to run at a Graphics resolution of 1280 x 1024 pixels with 32 bit (true color) with a refresh rate of 60Hz. No other applications were running on systems during testing (including such applications as anti-virus.) An AL2 license was used for all tests. This was either served from a local server or a nodelock license. We verified that serving the license from a server did not affect the CATIA start time. Wherever available, we configured the graphics card driver to use OpenGL settings for CATIA. CATIA Settings and Options CATIA options can have a marked effect on performance. To ensure consistent results, the following procedure was followed: Before each sub-set of tests (Part Design, Analysis, DMU etc) existing CATSettings were removed and standard CATSettings copied in from our CATbench directories. The CATIA application window was maximized. The document window was maximized within CATIA. CATLaunch was used to measure CATIA launch time. CATIA VBScript macros and batch files were used to automate actions and capture timing. All other settings for CATIA were left at the installation default values. After running one complete sequence of modeling or graphics tests, temporary files were deleted and each system was rebooted. Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 3

4 dual core 3.2 GHz processor with a 800 MHz front side bus, 1 GB of fast 533 MHz DDR2 memory, an 80 GB serial ATA disk drive and NVIDIA Quadro FX1400 graphics card. Dell has changed the chassis of the 380 from prior 300 series machines to a more traditional box with a slide-off cover, rather than their clamshell design. This new case is compact and has 3.5 and 5.25 drive bays available. The system comes with an integrated Gigabit Ethernet card and five free PCI card slots. As with several other vendors, the system uses a USB keyboard and mouse. System performance of the Dell 380 was satisfactory with a system score (CATbench2005S) of The system did well in our analysis scenario but lagged in our other design and DMU scenarios. We have seen this behavior from new processor families from Intel in the past, the first chip releases offering slightly slower performance than existing chip families. Watch this space! Graphics performance of the NVIDIA card was particularly good with a CATbench2005G score of 89.6 with strong performance on all tests and across all graphics modes. The NVIDIA Quadro FX1400 becomes the graphics card to beat. Combining the graphics and systems scores yields an overall CATbench2005 score of 100.9, while the system scored in our digital mockup benchmark. The new Dell Precision Workstation 380 offers reasonable performance for $3,298, plus we were particularly pleased with the performance of the NVIDIA FX1400 graphics card. Price/Performance 4 stars **** Performance 4 stars **** Acknowledgements Several individuals and organizations provided support for CATIA Community s CATbench2005 workstation benchmark. We thank them for improving the value of the benchmark: BMW AG provided one of the models used for V4 to V5 migration tests and provided a complete vehicle set of CATIA V4 models for our digital mockup (DMU) and graphics tests. Dell Precision M70 Mobile Workstation Dell s M70 workstation impressed us last year when it was released. Dell has refreshed it by upgrading the processor to a Pentium 4m (Centrino) at 2.13 GHz. As we noted in the past the performance of these mobile workstations approaches that of desktop/deskside systems so we ran it through CATbench2005. We also set the performance numbers as the norm to compare other mobile systems to mobile benchmark CATbench2005m. The General Dynamics Electric Boat (GDEB) provided extensive CATIA V4 models used to create the Holland graphics test. IBM Product Lifecycle Management provided CATIA code and licenses. Joe Costa of Costa Consulting LLC for automating our Analysis, Migration & Digital MockUp scenarios. Dassault Systèmes provided the test tool CATgtsPerformances and technical guidance. Thanks also to the hardware vendors who continue to provide technical feedback and help to improve the benchmark. Phil Harrison, author, is principal of LionHeart Solutions, Inc., a consulting firm specializing in CATIA and ENOVIA implementation and usage located in Cold Spring Harbor, NY. Harrison was President of the CATIA Operators Exchange (COE) for nine years. He also is the author of the CATIA Community s CATbench CATIA Version 5 Hardware Benchmark. Harrison has 15 years experience installing and using CATIA on Unix, mainframe, and Windows systems. He can be reached via at pph@lionheartsolutions.com. Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 4

5 Performance Measurement Tools Repeatability and accuracy are vital to any benchmark activity. We have taken extensive steps to ensure that our benchmark results are solid in both respects. To ensure meaningful results all systems were setup in the same manner (see Comparing Apples to Apples). Operations were timed thrice in the same sequence; results were later averaged, any spurious results were ignored, and the operation retimed. We used our CATLaunch tool to measure the time elapsed from triggering CATIA to having an active product document. The illustration below shows the user interface of CATLaunch. Dassault Systèmes provide a macro language that can be used on both UNIX and Windows systems. This language enables the user to record a series of interactions and later replay these actions. To automate result collection, the VBScript macros were modified by inserting timing operations and writing the results to text files. These macros were then triggered by batch files on the Windows machines. In addition to the macros recorded interactively, Joe Costa of Costa Consulting programmed our Analysis, DMU, Drafting and Migration scenarios so that 100 % of our interactive tests are automated. To accurately measure graphics performance, Dassault Systèmes has created a program called CATgts- Performances, which reports graphics card response time in the various modes and reads a graphics file exported from CATIA V4 or V5 data, allowing a user to graphically manipulate that data interactively. An automated mode allows the recording of a user s local graphic interactions such as zoom, pan and rotate; these interactions can then be replayed using a script and performance results in various modes output to a text file. Note that on large data sets the results are CPU dependent to an extent, since a significant proportion (up to 30% in our largest scenario) of the response time is spent culling data; that is, calculating which data needs to be drawn, normally done by the CPU. CATgtsPerformances measures the delay between a frame s regeneration and the screen refresh rate. Thus with a theoretically perfect card, CATgts- Performances would report a time of zero milliseconds. All the graphics cards submitted in systems for CATbench2005 performed well for small and medium part and assembly design tasks. main difference between the mobile and desktop benchmarks is the addition of a battery life test and we also take weight into consideration. For CATbench2005, we set the mobile workstation s screen resolution at 1280 x 1024 pixels as compared to the Dell M70 s native 1920 x 1200 display. Thanks to a magnesium case, the M70 weighs a reasonable 8.4 lbs (with its power supply) even with the extra wide aspect ratio 15.4 display. In the configuration tested, the M70 came with 1 GB of RAM and a 60GB 7,200 RPM Ultra ATA 100 hard disk. It also came with a CD-RW that will be useful for backups in the fixed bay. The system includes an integrated mini-pci wireless network card and a 56K modem. The configuration is feature packed for the mobile worker who needs access to CATIA. The display was stunning with a 15.4 screen with a resolution of 1920 x 1200 pixels. The M70 comes with touch pad and pointing device so we added a 3- button mouse to take advantage of CATIA s mouse manipulations. We were impressed with the battery life of the Dell machine, which ran our battery test for 138 minutes almost 2.5 hours of intensive activity. The system performance of the M70 was close to that of the 380 with a CATbench2005 score of The system performed extremely well in our engine block and analysis scenario but poorly in our engine assembly scenario. Graphics performance was better than average with a CATbench2005G score of Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 5

6 Combining system and graphics scores results in a CATbench2005 rating of 106.2, while the CATbench2005DMU score is a similar rating of 112.3; demonstrating that these workstations can now be used as replacements for desktop CAD workstations. To show how the M70 has improved over the M60 s performance we have included the M60 s test results for CATbench2005 and 2005m. At $3,137, the Dell M60 raises the bar for mobile computing again with its stunning display; it is a lightweight machine with excellent performance all around. It is the ideal machine for mobile workers and offers performance comparable to leading desktop workstations. Price/Performance 4 stars **** Performance 4 stars **** IBM Intellistation MPro z The machine submitted by IBM was a refresh of the popular MPro. A mid-range offering, it was equipped with a Pentium GHz processor with an 800 MHz front side bus, 1 GB of 400 MHz DDR memory, a fast 80 GB U320 SCSI disk drive and NVIDIA Quadro FX1400 graphics card. The case of this particular MPro is new to us and comes with an angled front. It has 3.5 and 5.25 drive bays available as well as two hard disk slots available for more storage. The system comes with an integrated Gigabit Ethernet card and two free PCI card slots. System performance of the MPro was similar to the Dell 380 with a CATbench2005S score of 113; performance was good in our design scenarios but poor in the migration and analysis scenarios. The graphics performance of the NVIDIA FX1400 graphics card was particularly good with a CATbench2005G score of 84.4, the best of all the systems and some 5.2 points ahead of the same card in the Dell 380. The NVIDIA Quadro FX1400 appears to offer good performance at a low price. Combining graphics and systems scores yields an overall CATbench score of The system scored 91.9 in our digital mockup benchmark, the best of all systems. Priced at a mere $2,897, the MPro provided superb performance in all areas and is highly recommended for entry level and mid-range use. It earns a 5-star rating for price/ performance. Price/Performance 5 stars ***** Performance 4 stars **** Sun Java Workstation W1100z We ve not seen a submission from Sun in five or six years, during which time their strategy has changed from proprietary hardware and Solaris operating system only to offering Windows systems running on AMD chips. The first system, the W1100z, is nicely packaged, offering plenty of room for expansion. The case is attractive, and we particularly liked the diagrams on the inside of the cover illustrating component layout. It seems Sun doesn t certify this system for running Windows, which is strange since their marketing people told us that many customers ran this system using Windows. None of Sun s AMD based systems are actually certified by Dassault Systèmes for CATIA; however, we found absolutely no issues in our extensive test suite. The W1100z came with a single AMD Opteron 100 Series 2.4 Hz processor, an NVIDIA FX3000 graphics card, 2 GB of memory, and an Ultra ATA disk drive. The system gave strong system performance Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 6

7 with a CATbench2005S score of 87.8; performance was good in all our design, analysis, and migration scenarios. Graphics performance of the NVIDIA FX3000 card was disappointing, with a CATbench- 2005G score of 110.8, the lowest score of the cards we tested. The NVIDIA Quadro FX3000 is now an old card, out-performed by the FX1400, which is also considerably less expensive. Combining graphics and systems scores yields an overall CATbench score of The system scored 98 in our digital mockup benchmark. Priced at a mere $4,018, it s good to see Sun back with a wellperforming workstation, even if it is not certified. Price/Performance 4 stars **** Performance 4 stars **** NVIDIA Graphics accelerator Sun Java Workstation W2100z The second system from Sun was their W2100z Java workstation. It came equipped with a pair of Opteron 200 series processors, as opposed to the W1100z s single 100 series processor. This system is certified to run Microsoft Windows XP, though it came loaded with Sun s Java Desktop. It was a challenge to load Windows XP on this system as it came with a SCSI controller but no floppy drive from which to load the SCSI drivers. The W2100z came with a pair of 2.4 Hz processors, an NVIDIA FX3000 graphics card, 4 GB of memory and a SCSI U320 disk drive. The Where Are the UNIX Machines? Deciding which platform to use when running CATIA (UNIX or Windows) isn t easy. Each choice has its strengths and weaknesses. Understanding them will help you make an informed decision. UNIX systems are rooted in proprietary hardware, and each vendor tends to have its own proprietary version of the operating system. Powerful scripting languages and a multiple-user operating system team are needed to enable automation of many system activities. Standards activity has led to use of standard peripheral devices (USB or PCI) and a common user interface (Common Desktop Environment). Windows systems originated with standardized generic hardware using Intel or AMD processors. The major advantage of the Windows platform is application support. We can give a designer or engineer a single system for both engineering and office tasks such as word processing, spreadsheet analysis, and . The choice to run CATIA Version 5 on UNIX or Windows largely depends on two issues: 1. Whether you need to operate Version 4 on the same hardware: today, any existing CATIA Version 4 installation is running UNIX hardware and will have to support Version 4 for some time. 2. How data is accessed: If you want to use CATIA Version 5 with file-based data, with Smarteam or Dassault s ENOVIA Lifecycle Applications (LCA), you can chose Windows platform. However, if your Product Data Management system is VPM, you are limited to a UNIX architecture unless you create your own VPM client software. Customers need to make informed decisions on whether to introduce a mixed environment of UNIX and Windows machines or stay with the UNIX platform. One reason for staying with CATIA Version 5 on UNIX is the immaturity of Dassault s ENOVIA product set on Wintel. We believe that there are still valid reasons for choosing UNIX as a platform for Version 5, such as a common platform to use with Version 4, data management, and support infrastructure. But as we have seen in the past, engineers and designers will have to live with significantly slower performance. Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 7

8 W2100z came in the same attractive chassis as the W1100z, and the motherboard came with integrated SCSI and SATA, network and sound. Surprisingly, given the twin processors and SCSI I/O system, the W2100z was actually 8.3 points slower on our system tests. It was faster in our design scenarios but considerably slower on our analysis scenario. Graphics performance of the NVIDIA FX3000 graphics card was identical to the card in the W1100z with a CATbench2005G score of This was surprising since we would have expected the second processor to be used for culling to improve the score. Combining graphics and systems scores yields an overall CATbench score of The system scored an impressive 86.8 in our digital mockup benchmark. Priced at $7,263, the Sun Java Workstation W2100z is relatively expensive and offers performance on a par with the much cheaper W1100z. Price/Performance 2 stars ** Performance 4 stars **** W2100z back Xi MTower with ATI FireGL 7100 Our final review covers an Xi system submitted by ATI so we could test their FireGL V7100 graphics card. Xi manufactures systems largely for the gaming and entry level CAD market using AMD processors. The case of the MTower 64 SLI is large with lots of room for expansion, with a free 3.5 drive bay, a free 5.25, two extra hard disk bays, and four free PCI card slots. Somewhat unusually, the Xi system came with a transparent window in the side of the case. In addition to the ATI V7100 graphics card, the Xi system was configured with a single AMD Fx GHz processor, 2 GB of memory, and an integrated SATA controller with two 72 GB disk drives. Like the Sun systems using AMD processors, the Xi system is not certified to run CATIA; however, we found absolutely no issues in our extensive test suite. Xi MTower with ATI FireGL 7100 The system performed superbly with a CATbench2005S score of 75.0; performance was good in all design, analysis and migration scenarios. Graphics performance of the ATI V7100 graphics card was on a par with the NVIDIA FX3000 with a CATbench2005G score of 108. Combining graphics and systems scores yielded an overall CAT- Bench2005 score of The system scored 92.1 in our digital mockup benchmark, just behind the IBM MPro. Priced at a mere $3,368, the Xi system was excellent; ATI has some work to do to overtake NVIDIA in performance, but competition is good for us as customers. Price/Performance 4 stars **** Performance 5 stars ***** Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 8

9 Conclusion Despite variations in performance, all systems submitted for the CATIA Community hardware benchmark performed well and would be acceptable for our design, analysis, and digital mock-up scenarios. The benchmark also demonstrates that: Intel Pentium 4 processorbased machines still offer the best price/performance. However, this family has reached the end of the line, and we await better performance from dual-core processors. AMD s Opteron 100 series processors seem to outperform Intel processors; however they are not certified by Dassault Systèmes with CATIA. The Serial ATA disk drive and controller seem to offer outstanding performance, on par with SCSI subsystems. NVIDIA is currently the leader in price/performance for technical workstations; however, ATI s V7100 showed good performance in the Xi system. Performance of mobile workstations has reached that of mid-range workstations and is suitable to replace those needing more flexibility. The march toward generic systems continues with hardware vendors producing fewer system components. Instead, they are purchasing major system components from a small number of component manufacturers. Computer hardware is developing at an incredibly rapid pace. Today s systems offer greater performance for less money than ever before. This benchmark demonstrates that systems costing as little as $3,000 are equipped to run CATIA in a highly productive manner. Certified and Derived Hardware Configurations CATIA is certified by Dassault Systèmes to run on selected combinations of UNIX and Windows operating systems and patches, processors, graphics cards, drivers, network cards, and motherboards. Note that Dassault Systèmes certifies a particular system, not a vendor. Dassault has a validation process in place to test systems, ensuring that those selected from the certified configuration list can be used reliably. Dassault Systèmes claims the following benefits for customers using certified or derived hardware: Best optimized performances Complete UNIX and Windows interoperability Clearly identified hardware configs for worldwide availability Support from hardware vendors on the certified configurations Support for CATIA Version 5 and ENOVIA DMU Version 5, on the certified configurations To view information about certified systems, go to: implementation/technology/windows/ certified-workstations-list for Windows systems. implementation/technology/unix/ unix-based-workstations for UNIX systems. Dassault Systèmes publishes a list of derived configurations; these are supported systems with a change in processor speed, different storage, network systems, or texture memory on the graphics card from certified systems. No other changes are allowed in processor, chipset or graphic adapter. In our experience, change in a single component such as a graphic device driver can have a dramatic effect on both system stability and performance, so we applaud Dassault Systèmes activity in this regard. We have noted in the system specification chart whether a system has been, is being or will not be certified by Dassault Systèmes. But note at this point in time that there are no certified systems using AMD processors or the new dual-core processors from Intel, though we expect the latter to be certified by the summer. Non-Certified Systems What do you risk by purchasing a configuration not certified by Dassault Systèmes to run V5? Basically, you have no guarantee that the application will be stable. If you experience a problem (a defect in IBM/Dassault parlance), your local defect support structure will attempt to reproduce the problem on a certified system. If the problem cannot be reproduced, you will be asked to refer the problem to your hardware or software driver supplier. Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 9

10 Benchmark Scenarios CATIA Community has created several extensive modeling scenarios that are typical of the processes used within engineering organizations running CATIA. These include processes used to model components and assemblies, make extensive changes, and export data. Graphics tests used previously in CATbench2004 were not changed; however, the graphics (cgr) files were recreated in line with current tessellation routines. The design scenario, unchanged from last year, models the conceptual design phase of a gasoline internal combustion engine. Within our simulation, we want to be able to modify the compression ratio of our engine, which we can do in a number of ways, by changing: The crown height of the piston The crankshaft throw The deck height of the block We modeled these components in such a way that design intent is captured and extensive modifications can be made quickly. This data can then be passed to downstream applications such as drafting and stress analysis. These three modeling scenarios were designed to capitalize functionality in the part design, assembly design, generative part stress (GPS), generative drafting, and interface products. Modeling Scenario 1 Simple Part Design: Piston The first scenario models the design, analysis, and documentation of a relatively simple part a piston. Illustrated in Figure 1, it is a single part, approximately 1.5MB in size, and uses parameterization, filleting, symmetry, and patterning throughout to capture design intent. We drafted the part using CATIA s powerful generative drafting and dimensioning capabilities and carried out a stress analysis on the part. Stress analysis of the part used 49MB of external storage, so the analysis taxes the computer s I/O system. The results of the stress analysis are shown in Figure 2. We then made a major design change to the part, changing the height of the piston and lightening the piston by reducing the wall thickness. Now that we were satisfied with the part design, we produced a stereolithography file, Figure 1: Piston Model used to produce a rapid prototype. Having produced an acceptable three-dimensional definition of our part, we automatically updated the drawing of the part using the generative drafting product and exported both DXF and STEP files of our design. Modeling Scenario 2 Complex Part Design: Crankshaft The second scenario modeled the design, analysis, and documentation of the crankshaft as illustrated in Figure 3. This also is a single V5 Part, but this model is 4 MB in size and takes maximum advantage of design intent capture by using copied and transformed bodies throughout. A change to the master bearing or web is reflected throughout the design. As with the first scenario, we drafted the part Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 10

11 and then performed a stress analysis (see Figure 4). From the results, we could tell that changes needed to be made. Stress analysis is demanding and requires more than 110MB of external storage to tax both the I/O system and processor. Following our scenario, we changed the throw of the crankshaft, reducing the compression ratio of the combustion chamber. We also changed the fillet on a web of the crankshaft as shown in Figure 5 and propagated our design change throughout the part. When we were happy with our design, and as with the piston scenario, we updated our drawing and exported both DXF and STEP files of our design. Figure 2: Stress Analysis Figure 3: Crankshaft Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 11

12 Figure 4: Crankshaft Stress analysis Figure 5: Crankshaft fillet change Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 12

13 Modeling Scenario 3 Complex Part Design: Engine Block Our third modeling scenario modeled the preliminary design of the cylinder block itself, as shown in Figure 6. We paid extra attention in modeling this part to capture full design intent; extensive changes could be made later by changing a single parameter such as the block height. Again, we used generative drafting and dimensioning to create a print of the part (see Figure 7). Figure 6: Cylinder block model To change the compression ratio of the engine, we changed the Figure 7: Print of the crankshaft Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 13

14 height of the block. Since we captured design intent in creating the part, CATIA itself changed affected features throughout the design, such as the cooling waterways and cylinder head bolt holes. As with the other part design scenarios, we updated the print and then exported the 2D and 3D definitions. Modeling Scenario 4 Assembly Design We carried on modeling engine components to create a short engine assembly including pistons, piston rings, connecting rods, bearings, crankshaft, and engine block. Once we assembled the design, we fully constrained the assembly to create an operational assembly, so that rotating the crankshaft would cause the pistons to reciprocate in their bores. We added the front drive assemblies as well as the flywheel assembly, as shown in Figure 8. Loading this assembly was a processor-intensive activity as well as a significant I/O task. Since we planned on doing more design in context, we did not use the cache system in this instance. CATIA Release 14 is now significantly more robust than earlier releases, allowing us to export STEP data and draft large assemblies. Thus after reading in our engine assembly in design mode, we exported our design to STEP AP203. Once exported, we opened an existing drawing of the engine, updated all views, and saved the drawing. Figure 8: Shaft engine assembly Figure 9: V4 model browser Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 14

15 Migration Scenarios The migration of existing CATIA V4 models into V5 is a concern to all existing CATIA users. For the interactive tests, we opened up the V4 parts in the model browser, copied the geometry and then pasted the geometry into V5, bringing history with the parts (with the Paste Special option). Once the geometry was pasted, we updated the model to create a V5 part. The first of our parts was a solid model as shown in the V4 model browser (Figure 9). The second of these scenarios (a heating duct) was a surface-based design. We next tested systems manually using the batch-mode converter, selecting both models. Digital Mockup Scenarios Many CATIA installations model large scale assemblies in a process called Digital Mockup. CATIA V4 introduced the 4D Navigator tool that created tessellated representations of all components and allowed the user to analyze assembles, such as for clash and clearance. In V5, Dassault has introduced a suite of products, including DMU Navigator and DMU Space Analysis, that build upon the prior products. The first test consisted of loading 57 automotive body panels (55MB of data), reloading the data, and then performing a clash analysis (see Figure 10). The first time the data loaded each CATIA file had to be tessellated (note: cache system must be turned on); this was a system-intensive action, with the I/ O system having to read the original CATIA file and write tessellated files, and the CPU was stressed while performing the tessellation. The data loads much faster the second time since the data already was tessellated and also was approximately one-sixth the size of the original data. Once the data loaded, we switched to DMU Space Analysis and performed a clash analysis, again a very CPUintensive action. Our second DMU scenario was similar but used a data set that was more solids-based and significantly larger, with 66 files and 220 MB of data. Figure 10: Clash analysis Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 15

16 Figure 11: Body-in-white structure Graphics Testing We tested systems using seven different sets of data, and by using Dassault Systèmes graphics test tool, CATgtsPerformances. Since CATbench 2000, we also have standardized the settings for creating the files with Dassault Systèmes so our results can be correlated. Note that all cgr files were re-tesselated for CATbench since Dassault Systèmes has significantly improved their tessellation routines, reducing the size of the computed cgr files. The first and second tests used data derived from the piston model and engine block scenarios. Our third graphics test used our engine assembly; note we made the block transparent so that the internal components were visible. This graphics file was 3.7 MB in size. For the fourth graphics test we used the models from the first of our digital mockup tests to create a surface-based, 13 MB CGR file. Our fifth graphics test used data supplied by General Dynamics Electric Boat (GDEB) of the Holland Nuclear Submarine and was a mixture of surfaces and solids. Once extracted, the graphics test file was a substantial 21 MB. For the sixth test scenario we extracted a 65 MB graphics file from the BMW digital mockup data, which included all the main components of the drive train. Our final graphics test used a complete body-in-white structure, 89 MB in size, which is shown in Figure 11. RSVP Once we had modeled our design and digital mockup process, we invited hardware vendors to submit systems for CATbench Each of the vendors was given an outline of our scenarios in terms of model size and complexity but not given any models themselves. Vendors configured systems themselves based on basic guidelines from Dassault Systèmes and from information within our invitation. Vendors also were allowed to select the operating system to load on their machine. Hardware vendors submitted the six systems listed in Table 1. We have listed whether or not the vendor is certifying the system at Dassault Systèmes. We have also included our two reference machines in this table, our trusty Dell Precision 340 with a 2.4 GHz P4 processor used to develop the workbench and a Dell M60 mobile workstation. These systems provide a useful comparison to the latest state of the art machines. We invited both Windows and UNIX systems vendors to take part in the benchmark, but we had no entries from the UNIX vendors. There are plenty of legitimate reasons to use V5 on UNIX, especially if you need to modify Version 4 models. UNIX systems have come down remarkably in price thanks largely to using graphics engines and PCI-based buses. However, UNIX systems still Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 16

17 cannot compete with the low prices in the Windows hardware market due to much higher volumes. Each system was tested using the same data in an identical manner to produce the results table shown on page 23 in Table 2. This data then was weighted based on our estimate of how often a user might do each operation. This was not intended to be an accurate statistical representation of the CATIA Community members, and indeed you might decide to use different weighting factors based on your company s usage. The results of each system were then normalized to produce four benchmark ratings: CATbench2005S measures the performance of the CPU, bus and I/O subsystems. CATbench2005G measures the graphics performance of each system. CATbench2005 is the combination of the CATbench2005G and CATbench2005S, giving each equal weighting to both system and graphics performance. CATbench2005DMU is a new measurement of performance in the Digital Mockup domain. A summary of the results for all systems is shown on page 24 in Table 3. Normalizing the data makes it easier to understand since a value of 100 represents the average of all systems. Thus, for example, a value of 200 means that a system took twice the amount of time to accomplish a task as the average of all the systems tested. As always with CATbench, a lower score is better! Results also are shown on pages in Graphs 1, 2, 3 and 4, for overall (CATbench-2005), graphics (CATbench2005G), system (CATbench2005S), and Digital Mockup (CATbench-2005DMU) performance respectively, after tabulation systems were rated based on our rating system for price/ performance and performance only. Multi-Processor Support In CATIA Version 5 Release 11, Dassault Systèmes added multiprocessor capability to CATIA Version 5 in the Digital Mock Up (DMU) and analysis domains. Therefore, as Intel has just released their dual core processors, we thought it timely to discuss the issue again. CATbench 2000 (Page 35, March/ April, CATIA Solutions Magazine) evaluated the cost effectiveness of adding a second processor using CATIA Version 5 Release 3. We concluded that significant performance improvements with two or more processors with a single application generally only will be seen in a multi-threaded application. The application starts child processes which the operating system allocates to the different processors. This is difficult to do with computer aided design and appli-cations such as CATIA, in which geometry is usually dependent on prior geometric definitions. We have the opportunity to see the effects of Intel s dual core technology as we have a IBM M- Pro with a single 3.8 GHz Pentium 4 processor and a Dell 380 system with a dual core 3.2 GHz processor; the systems even have the same NVIDIA Quadro FX 1400 graphics card. You can see from our results in the CATbench2005 System scores that the dual core processor offers similar perform-ance to that of the faster single core processor. The dual core processor machine is 50 points ahead of the single core processor. However, to our surprise, the DMU and graphics scores were actually better for the single processor machine. Consideration should be given to adding a second processor or dual core processor if the user also runs another processor-intensive application simul-taneously with CATIA. However, be advised to also add memory so that the applications do not become memory starved and start paging. Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 17

18 Table 1: System Specifications $2,897 Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 18

19 Table 1: System Specifications (cont.) $3,368 Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 19

20 Most of us operate on a limited budget. Given that constraint, how do we maximize our system s performance and how do we decide where tradeoffs should be made? Use the following as a guide: system components are listed in order of their effect on overall system performance. CPU speed: Given no other bottlenecks (such as I/O, memory or graphics), system performance is proportional to processor speed. Therefore, the first thing we need to do is select our system s processor. Most technical workstations either use Intel s 2.8 GHz to 3.8 GHz Pentium 4 or Xeon processors. None of the systems certified by Dassault Systèmes are yet equipped with AMD processors; although we expected AMD s Opteron systems to be certified by IBM for their APro systems, it did not happen. The large financial premium that Intel used to be able to charge for its fastest CPUs has largely disappeared, mostly due to fierce competition between Intel and AMD. For a high-end system used for DMU we recommend selecting the 3.2 GHz Xeon processor. For a value-based system we recommend the 3.8 GHz Pentium 4 processor. The main reason is because Xeon systems usually come in a larger chassis with more upgrade options for storage and memory and also provide dual processor capability that is supported by Dassault Systèmes for digital mockup applications. Larger caches (L2 or Getting the Most for Your Money L3) on the CPU have shown only marginal performance boosts in our testing, so we would advise to not opt for larger cache CPUs for which Intel charges a significant premium. Graphics card: The second most important component is the graphics card. An inadequate graphics card leads to slow apparent refresh rates with jumpy screen behavior, which leads to distractions for the operator. CATIA demands that the card support OpenGL. Since the NVIDIA Quadro FX graphics range gave good performance in all scenarios, we recommend the FX1400 for entry and mid-level use, and the FX 3400 for high-end DMU use. Random access memory (RAM): CATIA is a memory-intensive application; there is nothing worse for CATIA than to run out of real memory and start to page. If this occurs, users notice a dramatic drop-off in performance as hard disk access times are orders of magnitude worse than memory access times. The minimum recommended memory for CATIA from Dassault Systèmes is 256MB, but the amount required is strongly dependent on the data sets to be loaded and what other programs are run concurrently. Data sets used in CATbench2005 dictated a minimum of 1024MB and we advised each hardware vendor to submit systems with at least this amount of memory. Summary Two distinct types of memory are available for technical workstations: synchronous DRAM and Rambus DRAM, both of which are error correcting. Pentium 4 systems once required the use of expensive RDRAM memory. However, it has all but been supplanted in the marketplace by cheaper SDRAM memory. Choose the faster 400MHz Double Data Rate (DDR2) RAM for ultimate performance. Another word on configuring your memory: larger memory chips tend to be more expensive than the equivalent price of smaller memory chips. Thus, there is a temptation to order, for example, four 512 MB memory boards rather than two 1 GB chips. Be cautioned that the majority of systems only allow the installation of four memory cards. Thus by selecting the smaller chips, you have limited your upgrade options. In fact, you may have to throw the smaller cards away if you need more memory. Hard disk/controller: Fast read and write times are desirable to improve the speed at which CATIA is read into memory and to optimize the read and write of data. While SCSI disks and controllers offered a significant performance advantage over ATA disk drives and controllers in the past, the new Serial ATA disks and controllers seem to offer similar performance to SCSI at a lower cost. Therefore we d recommend the use of SATA in all but server machines. Design System High-End DMU System 3.8GHz Intel Pentium 4 w/ 800MHz FSB 2x 3.2 GHz Intel Pentium Xeon w/ 533MHz FSB 1.0GB 400MHz ECC DDR SDRAM 2.0GB 400MHz ECC DDR SDRAM NVIDIA Quadro FX 1400 NVIDIA Quadro FX GB Serial ATA 2x 80 GB Serial ATA 17 Flat Panel minimum resolution 1280 x Flat Panel minimum resolution 1280 x 1024 $2,800 $7,000 Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 20

21 Once all systems were tested, the results required analysis to create our system, graphics, overall, and digital mockup CATbench2005 scores. This analysis applied weighting factors in line with CATIA usage and normalizing, such that a score of 100 represents the average of the submitted systems. Results Analysis System Performance Table 4 shows the weighting factors that we applied to each operation from the design, analysis, and digital mockup (DMU) scenarios. Note that some operations are given a higher rating than others, since a typical operator would do these operations more frequently than Table 5: Graphics Summary Graphics Performance Our graphics test tool reports milliseconds delay results beyond the time for a screen refresh. We sum these delays for all seven of our graphics tests as shown in Table 5 for the IBM MPro system with the FX 1400 graphics card. Table 6: Weighting Factors for Graphics Table 4: Weighting Factors for Individual Tasks others. For example, reading a part or updating a part is more frequently performed than exporting a STEP file. Table 6 shows how we weighted each mode of the graphics test results from CATgtsPerformances. Because some graphics modes are particularly sensitive to CPU performance (e.g., outlines) we chose only Shading, Edges and Shading+Edges results to calculate our graphics performance measure (CATbench2005G). Table 7: Weighted Test Times Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 21

22 Normalizing the Data Because we need our data to be understood easily, we chose a value of 100 to represent the average score of all test systems for graphics, system, and overall performance. A score of 50 represents a system that would take half the time than the average system to accomplish the same task. Conversely, a score of 200 indicates a system that would take twice as long as the average system. Thus: Lower scores are better in all results! For the system data, we applied the weight factors by multiplying each system s task time by the weight factors; as shown in Table 7 (on the previous page for the IBM MPro system). We then added the total time for each scenario and calculated an average value for all test systems for each scenario for the system and DMU tests (Table 8) and for each graphics mode (Table 9). Note that we ignored the results from the two reference systems (Dell M GHz system and the Dell 340 system. Table 10 shows how we weighted the various domains to create our systems rating. We weighted the results toward the design scenario, but if you used the system mainly for engineering analysis, you might alter these factors. Note that a fast design machine is also generally a fast analysis or DMU machine. For the graphics rating we rated each graphics mode equally. We divided each system s test result for each scenario by the average value to get a rating for each scenario. We then calculated the system rating by taking the average score within a domain and applying our domain weight factors to the results. This created summarized results, as shown in Table 11 for the IBM system. Table 8: Average System Test Results Overall Performance Finally, we weighted the graphics and system scores (Table 12) to calculate the systems overall performance CATbench2005. We chose to weight the system and graphics scores equally (50:50), but as with the weighting factors shown in Table 4, 6 or 10, you can change the factors to be more representative of your site s usage. Perhaps you would use a higher graphics weighting if you do more digital mockup visualization than modeling. However, changing the weighting factors does not seem to change the relative standing of the machines significantly. Digital Mockup (DMU) Performance DMU results are not included in the system performance measurement but are used to create a separate measurement specific to DMU. We applied the same relative graphics/ DMU weight factor as applied to our overall results to create our DMU performance measure CATbench2005DMU. Table 9: Average Graphics Test Results Table 10: Systems Domain Weighting Table 12: Overall Weighting Factors Table 11: Normalized Performance Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 22

23 Table 2: Design Scenarios Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 23

24 Table 3: Results Summary Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 24

25 Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 25

26 Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 26

27 Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 27

28 Copyright 2005 ConnectPress, Ltd. All Rights Reserved. Page 28