7th JT Application Benchmark SHORT REPORT

Transcription

1 ProSTEP ivip/vda 7th SHORT REPORT

2 TABLE OF CONTENTS Contents Table of Contents 3 Disclaimer 4 Copyright 4 1 Introduction 5 2 Approach Four Steps Building Blocks Documentation Criteria Translation Quality Benchmark Structural Criteria Geometry Criteria Tessellation Criteria Attribute Criteria Application Performance Benchmark Translator Performance Viewer Performance 7 3 Testing Configuration and Settings Translation Quality Benchmark Tested Translators Test Models Testing Procedure Application Performance Benchmark Tested Translators Tested Viewing Applications Siemens PLM JT2Go TechniaTranscat LiteBox3D Siemens PLM Teamcenter Vis Base & Professional Test Models Testing Procedure 11 4 Results Translation Quality Benchmark Results CAD to JT and STEP AP242 XML Test Results JT and STEP AP242 XML to CAD Test Results Application Performance Benchmark Results 13 5 Summary and Outlook 14 6 Acknowledgements 14 3

3 SHORT REPORT Disclaimer This document is a ProSTEP ivip / VDA Documentation. It is freely available for all ProSTEP ivip e. V. members and those of the VDA AK PLM. Anyone using these recommendations is responsible for ensuring that they are used correctly. This Documentation gives due consideration to the prevailing state-of-the-art at the time of publication. Anyone using PSI Documentations must assume responsibility for his or her actions and acts at their own risk. The ProSTEP ivip Association and the parties involved in drawing up the Documentation assume no liability whatsoever. We request that anyone encountering an error or the possibility of an incorrect interpretation when using the Documentations contact the ProSTEP ivip Association (psi-issues@prostep.com) immediately so that any errors can be rectified. Full or partial reproduction of tables or pictures is only allowed, using the complete original underline, including a description of symbols. For all full or partial reproductions a deposit copy to ProSTEP ivip is required. Copyright I. All rights on this PSI Documentation, in particular the copyright rights of use and sale such as the right to duplicate, distribute or publish the Documentation remain exclusively with the ProSTEP ivip Association and its members. II. The PSI Documentation may be duplicated and distri - buted unchanged, for instance for use in the context of creating software or services. III. It is not permitted to change or edit this PSI Documentation. IV. A suitable notice indicating the copyright owner and the restrictions on use must always appear. 4

4 SHORT REPORT 1 Introduction 2 Approach JT has become a widely used format for product visualization during the product development process. The ProSTEP ivip Association and the German Association of the Automotive Industry (VDA) have launched three JT-related projects in succession, which are being coordinated with each other: The ProSTEP ivip / VDA JT Workflow Forum, the ProSTEP ivip / VDA and the ProSTEP ivip / VDA JT Implementor Forum. In August 2010, the ProSTEP ivip Association submitted the latest JT specification (version 9.5) to the ISO for standardization. JT 9.5 was published by ISO as ISO 14306:2012 in December, Since then, ProSTEP ivip and VDA have developed implementation guidelines for JT and discussed issues and further requirements. This eventually led to the publishing of the JT Industrial Application Package (PSI 14), combining the guidelines and enhancements with the latest JT specification. This specification is compatible with the ISO standard released in 2012 as well as with the latest proprietary versions of JT. As the latest in a row of seven benchmarks, the JT Application Benchmark was carried out in 2016 to achieve an independent evaluation of the progress being made with regard to the development of JT translators and viewing applications. The object of testing was ISO 14306:2012 (JT 9.5). Additionally, the interoperability between JT and the STEP AP242 Business Object (BO) Model XML schema (published as ISO Standard ISO :2014) was also part of the benchmark. The is a common activity of the ProSTEP ivip Association and the VDA. Goal of the benchmark is a neutral evaluation of actual JT applications. Focal points of this 7th benchmark were the testing of CAD-JT and JT-CAD translators as well as testing the state of the art regarding the interaction of JT and STEP AP242 XML. Also, the performance of viewing applications was tested. 2.1 Four steps Based on the Lessons Learned from previous benchmarks, the JT Workflow and JT Implementor Forum agreed on the following four-step approach: 1. The JT Workflow Forum clarified the target intent for the benchmark and provided details on the expected outcome. 2. The vendors made proposals for the JT file scope, configuration settings and evaluation approach which in their eyes would best fit the requirements. 3. A proof of concept / test run for the benchmark was conducted, using the agreed-on settings and test models, with close involvement of the vendors. 4. After the successful test run, the actual benchmark was conducted. The benchmark was managed by the JT Workflow Forum and JT Implementor Forum. Because the benchmark is an independent activity, it was financed directly by the two organizations, the ProSTEP ivip Association and the VDA, and not by the participating companies whose products were tested. It is a neutral test of trendsetting JT applications with regard to the selected criteria. Therefore, the results of the benchmark cannot only be used to evaluate the application of JT in PLM environments, but also for improvement of the interoperability of the applications. As such applications are undergoing a permanent development; the benchmark can only give a snapshot of the functions and qualities of the applications. 5

5 SHORT REPORT 2.2 Building blocks This benchmark was composed of two independent building blocks, the translation quality benchmark and the application performance benchmark: Figure 1: Benchmark building blocks In the translation quality benchmark, the results of CAD to JT and STEP AP242 XML translations and the results of JT and STEP AP242 XML to CAD translations were evaluated in two steps. CAD to JT translations: Translation of CATIA V5-6R2014, Creo2 and NX9 CAD models to JT and STEP AP242 XML with quality checks regarding geometry (XT- BREP), tessellation, assembly structure and attributes. JT to CAD translations: Translation of the JT and STEP AP242 XML models that were created during the first translation step back to CATIA V5-6R2014, Creo2 and NX9 CAD models with quality checks regarding geometry (XT-BREP), assembly structure and attributes. In the application performance benchmark, the time needed for several actions was measured: Converting complex CAD models to JT. Loading and viewing large JT models. 2.3 Documentation This short report is made publicly available; a long report with more detailed information is provided to the project members. 2.4 Criteria 2.5 Translation Quality Benchmark The translation quality benchmark focused on the application of different options for converting assembly structure. The investigated options were one JT file for the entire assembly; one JT file containing the assembly structure with references to JT files holding the parts geometry and a STEP AP242 XML file containing the assembly structure with references to JT files holding the parts geometry. For the translation to CAD systems, a STEP AP242 XML file also referencing a JT assembly file was introduced. The used CAD source formats for the CAD to JT translations and the target CAD formats for the JT to CAD translations were CATIA V5-6R2014, Creo 2 and NX 9. The used JT format for all translations was JT ISO 14306: Structural Criteria Independent from the used option for the file structure, the resulting models should fulfill the following criteria: The assembly structure in the target system is equivalent to source assembly structure. The positions of all instances are correct in target application. The instantiation of components in assembly are equivalent to source definition Geometry Criteria Regarding the correctness of geometry it was checked if the center of gravity, volume and surface of the resulting geometry were the same as in the source geometry. The BREP data had to be stored as XT-BREP Tessellation Criteria Regarding the correctness of the created tessellations, it was checked if the settings for Level of Detail (LOD) creation were conforming to the content harmonization guidelines of the JT Workflow Forum. The criteria were defined by the JT Workflow Forum. Details, especially the validation methods, were elaborated in collaboration with the JT Implementor Forum. 6

6 SHORT REPORT 3 Testing Attribute Criteria Regarding the correct translation of metadata from CAD to JT, three categories of attributes were checked: The CAD properties, defined in the JT ISO specification. The translator information, defined in the JT-IF Implementation Guideline. User-defined properties created on different nodes of the assembly structure. 2.6 Application Performance Benchmark In the second part of this benchmark the performance of translators and viewing application was tested Translator Performance For the translator performance tests the time that elapsed during the translation of a complex CAD model to JT was measured Viewer Performance The performance of viewing applications was measured in three steps: First, the time for the start of the application itself was measured. Second, the elapsed time between giving the load/open command and the full visualization of the model was measured. Third, the model was rotated and zoomed to see if there is any recognizable delay between the end of the user s action and re-visualization of the model. The benchmark tests were executed at PROSTEP to assure neutral testing and documentation. The vendors provided the software to be benchmarked and licenses for the runtime of the benchmark testing and evaluation. 3.1 Configuration and settings The vendors of the translators were asked to provide the configuration and settings that would fit best to the benchmark criteria. The viewing applications were tested as installed, without any changes of the default settings. 3.2 Translation Quality Benchmark Table 1 gives an overview of the participating vendors in the translation quality benchmark. It also shows who participated in the CAD-JT tests or in the JT-CAD tests Tested Translators Table 2 gives an overview of the translators tested in the CAD to JT test of the translation quality benchmark. It also shows which of the tested CAD systems were supported by each translator. Table 3 gives an overview of the translators tested in the JT to CAD tests of the translation quality benchmark. It also shows which of the tested CAD systems were supported by each translator. 7

7 SHORT REPORT Vendor CAD-JT/STEP AP242 XML JT/STEP AP242 XML-CAD CoreTechnologie Yes No Elysium Yes Yes Siemens PLM Yes Yes Theorem Yes Yes T-Systems Yes Yes Table 1: Vendor participation in the translation quality benchmark Vendor Translator Version CATIA V5-6R2014 Creo 2 NX 9 CT CoreTechnologie 3D_Evolution 4 x x x Elysium Asfalis EX 7.0 x x x Siemens PLM JT bi-directional translator 11 x for CatiaV5 Siemens PLM JT translator for ProE/Creo 12.1 x Siemens PLM NX 11.0 x Theorem CADverter 19.3 x - - T-Systems COM/FOX x - - Table 2: Benchmarked JT translators and supported CAD formats in the CAD to JT/AP242 XML tests Vendor Translator Version CATIA V5-6R2014 Creo 2 NX 9 Elysium Asfalis EX 7.0 x x x Siemens PLM JT bi-directional translator 11.0 x - - for CatiaV5 Siemens PLM NX 11.0 x Theorem CADverter 19.3 x - T-Systems COM/FOX x - Table 3: Benchmarked JT translators and supported CAD formats in the JT/STEP AP242 XML to CAD tests 8

8 3.2.2 Test Models The test models were the torque converter assembly which was already used in previous JT and STEP benchmarks. It is available in all tested source formats. It contains several subassemblies and multiple instances of several parts. Therefore, it fits the criteria of the translation quality benchmark. Userdefined attributes where added to the model on different assembly nodes. Figure 2 shows the NX 9 test model and its assembly structure Testing Procedure Figure 3 gives an overview of the translation quality benchmark testing procedure. The quality of each individual step was checked and the errors that had occurred in the first step did not affect the results of the second step. SHORT REPORT 3.3 Application Performance Benchmark The application performance benchmark consisted of two separate sections: The translator performance tests and the viewing performance tests Tested Translators For the translator performance tests, the same translators as in the translation quality benchmark were tested Tested Viewing Applications The tested viewing applications include basic viewers as well as more advanced engineering tools with functionalities for collaboration, simulation, planning and analysis. Figure 2: NX9 test model 9

9 SHORT REPORT Siemens PLM JT2Go Tested Version: JT2Go is a basic viewer, which is available for free TechniaTranscat LiteBox3D Tested Version: LiteBox3D is an easy and free to use JT viewer, based on the specification of the JT ISO standard. It can be enhanced for multiple use cases Siemens PLM Teamcenter Vis Base & Professional Tested Version: Teamcenter Vis is a scalable solution for different engineering visualization use cases. For the application performance benchmark the basic level Base and the high-level Professional versions were tested. Figure 4: Translator performance test model Figure 3: Testing procedure for translation quality benchmark 10

10 SHORT REPORT Test Models Testing Procedure For the translator tests, a large pressure cast part with complex geometry was used (see Figure 4). This model was kindly provided by Daimler. The translator performance tests were run in batch mode. The time elapsed between the start of the batch command and the finished translation was measured. For the viewing tests, two models where used: A generic large model created especially for stress tests, also kindly provided by Daimler, and a model of a nuclear powered submarine, kindly provided by Siemens PLM and BAE (see Figure 5). The viewing performance was measured in three steps: Measuring of duration of application start. The elapsed time from start command until menus and buttons are usable was measured. Measuring of duration of part load. The elapsed time between load/open command and full visualization of the model was measured. Measuring of duration of part rotate / zoom. The fully visualized model was rotated and zoomed in on. The time elapsed until visualization of the model from the end of user interaction was measured. All tests were repeated several times to assure the measured times are representative. Figure 5: Viewing performance test models 11

11 SHORT REPORT 4 Results In the following chapters, an overview of the test results for the different Building Blocks is given. 4.1 Translation Quality Benchmark Results In the following, the results for the two steps of the translation quality benchmark are summarized CAD to JT and STEP AP242 XML Test Results In the first step of the translation quality benchmark, the quality of the CAD to JT and STEP AP242 XML translations was evaluated. The tests showed very good results for the translation of geometry, both for tessellated and exact representation. With the file structure options using only JT files, the assembly structure was always translated correctly. Also, the results for the attributes are very good. The use of STEP AP242 XML as assembly format was not yet fully supported by all translators. This regarded mostly the support of attributes; the assembly structure was always translated correctly JT and STEP AP242 XML to CAD Test Results In the second step of the translation quality benchmark, the quality of the JT and STEP AP242 XML to CAD translations was evaluated. Again, the test showed very good results for the translation of geometry. It also shows that influences of the file structure have little impact on the assembly structure or the translation of user-defined attributes. Only with one tool monolithic JT files were not converted to an assembly but to a part with multiple bodies. The import of STEP AP242 XML files is not yet supported by all tested translators. Also, there were interoperability issues with some of the STEP AP242 XML files, which will be solved with future versions of the translators. Figure 6: Overview of CAD to JT and STEP AP242 XML test results 12

12 SHORT REPORT 4.2 Application Performance Benchmark Results For the translator performance, measured times were between a few seconds and several minutes. The tests showed that the time needed is more dependent on the used source format than on the used translator. For the same source format, translation times of different translators were close together. The viewing performance benchmark showed that all tested viewers were capable of handling the test models. There was no measurable delay between user interaction like zooming and rotating and the visualization of the model with the new view point. For the loading times, the test showed that it is depending on the details that are loaded. Some of the tested viewers load a coarse and simplified tessellation first, while another always loads the finest tessellation available by default. Figure 7: Overview of JT and STEP AP242 XML to CAD test results 13

13 SHORT REPORT 5 Summary and outlook 6 Acknowledgements The translation quality benchmark has proven the capabilities of JT translators. It has shown that the quality and completeness of JT translations from and to other CAD formats is very high. This regards geometrical data as well as assembly structure and metadata. Compared to previous benchmarks, the tests showed a massive improvement of the tested translators. We would like to thank the software vendors who provided their software for the benchmark testing and supported the installation and configuration of their software. As STEP AP242 XML is a relatively new format, it is not yet supported by all translators and there are still some interoperability issues. Due to the close cooperation between the software vendors and testers, issues found during the testing could be directly communicated. This allows the vendors to consider the benchmark results in the current development. With issues discussed in the JT Workflow Forum and the JT Implementor Forum, a common understanding of the users requirements is achieved. Also, the discussion of issues and results among the vendors will lead to improved interoperability between the various tools. 14

14 ProSTEP ivip Association Dolivostraße Darmstadt Germany Phone: Fax: February 2017