JAVA COMPONENT DEVELOPMENT GUIDE PIPELINE PILOT INTEGRATION COLLECTION 2016

Transcription

1 JAVA COMPONENT DEVELOPMENT GUIDE PIPELINE PILOT INTEGRATION COLLECTION 2016

2 Copyright Notice 2015 Dassault Systèmes. All rights reserved. 3DEXPERIENCE, the Compass icon and the 3DS logo, CATIA, SOLIDWORKS, ENOVIA, DELMIA, SIMULIA, GEOVIA, EXALEAD, 3D VIA, BIOVIA and NETVIBES are commercial trademarks or registered trademarks of Dassault Systèmes or its subsidiaries in the U.S. and/or other countries. All other trademarks are owned by their respective owners. Use of any Dassault Systèmes or its subsidiaries trademarks is subject to their express written approval. Acknowledgments and References To print photographs or files of computational results (figures and/or data) obtained using BIOVIA software, acknowledge the source in an appropriate format. For example: "Computational results obtained using software programs from Dassault Systèmes BIOVIA. The ab initio calculations were performed with the DMol 3 program, and graphical displays generated with Pipeline Pilot." BIOVIA may grant permission to republish or reprint its copyrighted materials. Requests should be submitted to BIOVIA Support, either through electronic mail to biovia.support@3ds.com, or in writing to: BIOVIA Support 5005 Wateridge Vista Drive, San Diego, CA USA

3 Contents Chapter 1: Java Component Development Overview 1 Who Should Read this Guide 1 Requirements 1 Getting Started with Java Component Development 1 Java Component API 1 Component Interface 2 Supported Objects 2 Additional Information 2 Chapter 2: Creating Java Components 3 Application Package 3 Anatomy of a Java Component 3 Development Requirements 4 Creating a Java Component with Eclipse 4 Creating a Package 4 Creating a Java Project in Eclipse 5 Creating a Project Named "pilotcomponent" 5 Adding the External jar reference to the Build Path 9 Creating a Pilot Component Class 9 Using the Java API to Implement the Component Interface 10 Component Interface 11 Component Categories 11 Saving the Java Code 12 Activating the Package 12 Chapter 3: Testing a Java Component in Pipeline Pilot Client 14 Chapter 4: Adding New Java Components to a Package 16 Saving Components in Package Destinations 16 Chapter 5: Distributing the Package 18 Chapter 6: Debugging a Java Component 19 Writing Additional Properties to the Output 19 Printing Debug Messages 21 Debugging with Eclipse IDE 22 Notes on Debugging 24 Appendices 26 Appendix A: Java VM Environment 26 Appendix B: Using JNI within a Java Component 26 Appendix C: Hash Table Values 26 Creation from a Java Map 27 Values as Java Maps 28 Comment on null values 30 Appendix D: Memory Management 30 Appendix E: Testing 31

4 Chapter 1: Java Component Development Overview The Pipeline Pilot Integration collection includes tools for developing components in the Java programming language for use with protocols. Java is a powerful environment for experienced programmers who need to develop and deliver high-performance capabilities. Java is known for its speed and ability to integrate the capabilities of a pre-existing Java library. If speed is not an issue, it may be easier to use a scripting language such as Perl, Python, or VB. Who Should Read this Guide This guide provides information about how to use the Java component application programming interface (API) to access and modify data objects for developing your own components. It offers the necessary architectural background and technical instructions for creating, testing, and deploying your customized Java components. Requirements To develop components in Java, you need the following: Experience writing code in Java A Java-integrated development environment (IDE), such as Eclipse 3.1 You should also have a basic understanding of how to use Pipeline Pilot Client to design and run protocols Note: This document assumes you are familiar with the Protocol Development Quick Start guide, which explains Component Lifetime Management. Getting Started with Java Component Development The tools available for developing components in Java include: Java-based components, such as Java (on Server) Java API and accompanying API documentation API examples that demonstrate how to design Java components that access the Pipeline Pilot data model to carry out a variety of tasks. A set of template Java Classes to use as a starting point for creating different types of Java components that call the API, such as calculators, manipulators, filters, readers, and writers. This guide includes instructions that walk you through the steps of creating a new component using the Java API, based on the Java (on Server) component that acts as a common "empty vessel" for all Java components. Java Component API The Java Component API provides a convenient way to access and modify the data structures, including the data records that flow through the components, the various parameter settings, and the global properties of the enclosing protocol. Java Component Development Overview Page 1

5 The API is implemented in an object-oriented fashion that reflects the natural syntax and features of the Java programming language. For the API definition, refer to the published Javadocs. Component Interface The Java component interface is com.scitegic.pilot.component. Every Java component implements the following methods, which are detailed in this guide: Component.onInitialize: Executed once before processing the first data record. Component.onProcess: Executed for each data record passed as input or generated by the component. Component.onFinalize: Executed after processing the last data record. The support of this interface is what allows the Java component to plug and play in protocol. Supported Objects The objects implemented by the Java API include: com.scitegic.pilot.datarecord: The data upon which the component operates to implement its function. com.scitegic.pilot.node: An element of the data record and global data hierarchies. com.scitegic.pilot.propertycollection: An ordered set of name/value pairs used to hold the properties assigned to each node and to represent parameter settings. com.scitegic.pilot.property: One of the name/value pairs in a property collection. com.scitegic.pilot.value: A generic value object used to define name/value pairs in a property collection. com.scitegic.pilot.context: Passed to the Component interface methods above, to allow access to component parameters and global properties. Additional Information For more information about the Pipeline Pilot Integration collection and other BIOVIA software products, visit Page 2 Pipeline Pilot Java Component Development Guide

6 Chapter 2: Creating Java Components The following topics are covered: Application Package Anatomy of a Java Component Development Requirements Creating a Java Component with Eclipse Activating the Package Application Package To deploy and run a Java component, you need to work within a named package. A package is a directory structure that holds all files required to deploy an application binaries, component XML, example protocols, example data files, and documentation. All components in Pipeline Pilot are deployed in packages, representing licensed modules. The package approach is particularly important for components that include a separately compiled binary content, such as Java components. IMPORTANT! We recommend that you become familiar with the packaging concept described in the Application Packaging Guide. The "package.conf" file at the top of the package directory structure provides details about how everything is organized in this package and other configuration information. To make a Java component accessible to Pipeline Pilot, define CLASSPATH within java directive in "package.conf", pointing to the location of your classes. <java> CLASSPATH$(package)/ /classes </java> or.jar <java> CLASSPATH$(package)/ /jars/your_component.jar </java> Anatomy of a Java Component When completed, a Java component consists of two parts that reside within a package file structure: A Java class that implements the interface com.scitegic.pilot.component. The jar file (included in the Integration collection) required to build such classes is "<install_ root>/apps/scitegic/integration/bin/lang/java/jars/pilot.jar" An instance of the Java (on Server) component with its JavaClass and Use Package parameters set to refer to the appropriate class and package. This is the XML definition of the interface that is exposed to a protocol designer in the Pipeline Pilot Client. It defines the component parameters, default parameter values, the component icon, and configuration of input and output ports. Creating Java Components Page 3

7 Development Requirements To implement Java components, be sure you have the following resources available: Develop the Java components on a server and write the compiled class files on the same server for testing. We recommend using a dedicated server for developing and debugging Java components. Creating a Java Component with Eclipse This example provides step-by-step instructions on how to create a Java component using Eclipse 3.1 as the Java development environment. IMPORTANT! Before you start, ensure that Eclipse is configured to use Java 5.0 features. From Eclipse, select Window > Preferences > Java > Compiler. Set the Compiler compliance level to "5.0", and enable the flag for Use default compliance settings. To work with this example, use a command window on your server. The example includes the following steps: Creating a package Creating a Java project in Eclipse that implements the Pipeline Pilot component interface and links to the pilot and chemistry jars Implementing a Java class to create a component to change atom element types Activating the package Testing the new Java component Creating a Package 1. In a command prompt window, change directories to "<install_root>/bin" (or "linux_bin" on Linux). 2. Use the "pkgutil -n" command to create a package for product "coolstuff" by company "acme". The command line below will pre-configure the new package with a template file structure for java component development. pkgutil n acme/coolstuff -directive xml directive java Under the new "<install_root>/apps/acme/coolstuff" folder, an appropriate directory structure is created with a specific folder to hold the Java classes and a folder for new components. A "package.conf" file is also created in the new package folder to define the component XML subfolder and to define the package folder location for inclusion in a runtime Java CLASSPATH. This line is initially commented out; uncomment it and fill in the correct jar file name when you have a jar file ready to be used. These package items are located as follows: "<install_root>/apps/acme/coolstuff/bin/lang/java/jars" "<install_root >/apps/acme/coolstuff/xml/components" "<install_root >/apps/acme/coolstuff/package.conf" Page 4 Pipeline Pilot Java Component Development Guide

8 Contents of the package configuration file "package.conf" Creating a Java Project in Eclipse This task involves the following: 1. Creating a project named "pilotcomponent". The source folder is named "pilotcomponent/coolstuff_src" and is linked to the folder "<install_ root>/apps/acme/coolstuff/src/lang/java". For this example we will work with source code within the package file structure, for the sake of simplicity. The default output folder is named "pilotcomponent/coolstuff_build" and is linked to "<install_ root>/apps/acme/coolstuff/bin/lang/java/jars". 2. Adding a reference to the external jar "<install_ root>/apps/scitegic/integration/bin/lang/java/jars/pilot.jar" to the build path. 3. Creating a class com.acme.coolstuff.elementchanger that implements com.scitegic.pilot.component. 4. Adding the Java Component API methods to the class to implement a component to change atom element types. Creating a Project Named "pilotcomponent" To create a project named "pilotcomponent": 1. Start Eclipse and select File > New > Project. The New Java Project dialog opens. 2. Select Java Project, and click Next. 3. At the next dialog, do the following: In Project name, enter "pilotcomponent". Select Create new project in workspace. Select Create separate source and output folders. Click Next. Creating Java Components Page 5

9 4. From the Source tab, select Link Additional Source to Project. Java build settings for pilotcomponent 5. Set up the additional source by doing the following: Click Browse and select the folder "<install_root>/apps/acme/coolstuff/src/lang/java". In Folder name, type "coolstuff_src" and click OK. Set the output folder for the classes. Click Browse. Page 6 Pipeline Pilot Java Component Development Guide

10 Java settings for coolstuff_src Creating Java Components Page 7

11 6. Select the top folder "pilotcomponent" and click Create New Folder. Folder Selection dialog Create New Folder option 7. Click Advanced. It s necessary to create an eclipse folder for the output, which is a link to the file system folder "<install_root>/apps/acme/coolstuff/bin/lang/java/jars". Select Link to folder in the file system. Click Browse and select the folder "<install_root>/apps/acme/coolstuff/bin/lang/java/jars". In Folder name, type "coolstuff_build" and click OK. Select the newly defined coolstuff_build folder and then click OK. You return to the New Java Project window. Page 8 Pipeline Pilot Java Component Development Guide

12 Adding the External jar reference to the Build Path 1. From the Libraries tab, select Add External JARs. 2. Add the JAR file "<install_root>/apps/scitegic/integration/bin/lang/java/jars/pilot.jar". Java settings for coolstuff_build 3. Click Finish. A new project, pilotcomponent, is displayed in the Package Explorer window in Eclipse. 4. If Eclipse prompts you to switch to Java Perspective, select Yes. 5. Remove the folders "bin" and "src" (created by default under pilotcomponent). These folders are not needed. Creating a Pilot Component Class 1. From the Package Explorer window in Eclipse, select the pilotcomponent package. 2. Right-click the folder "coolstuff_src" and select New > Class. 3. Ensure that the Source folder displays the value "pilotcomponent/coolstuff_src". Creating Java Components Page 9

13 4. In the Package field, type "com.acme.coolstuff". 5. In the Name field, type "ElementChanger". 6. To add the interface com.scitegic.pilot.component to the class, click Add. 7. Click Finish. New pilot component class Using the Java API to Implement the Component Interface You now have a class that implements the interface com.scitegic.pilot.component (oninitialize, onprocess and onfinalize methods). Now you need to bulk up the code and add the Java API methods to the class, implementing the behavior that you want. Note: To write Java code that works with the component parameters and data records (and global properties if necessary), refer to the Java API documentation. Page 10 Pipeline Pilot Java Component Development Guide

14 Component Interface The three interface methods to populate are: Component.onInitialize(): Invoked once before any data records arrive. Receives one argument, a com.scitegic.pilot.context object. Must return one of these Component.State values: ReadyForInputData, DoneProcessingData, ReadyForNewData, or ReadyForInputThenNewData. Component.onProcess(): Called once per data record. Receives two arguments (a com.scitegic.pilot.context object and a com.scitegic.pilot.datarecord object) and returns a Component.State value that indicates the component s state after processing the data. Component.onFinalize(): Called when no more data records are available. Receives one argument, a com.scitegic.pilot.context object. Component Categories Different categories of component behavior can be defined (a reader, calculator, manipulator, etc.) based on the component state returned by the oninitialize and onprocess methods. With a few simple states, you can define a wide range of component behaviors. The list below names each of the available component states (the values of the enum com.scitegic.pilot.component.state) and explains their meaning: DoneProcessingData: The component s task is complete. The component requests that the framework does not invoke its Process event again. ReadyForInputData: The component requests that the framework invoke its Process event with any data record that arrives at its input port. ReadyForNewData: The component requests that the framework repeatedly invoke its Process event with a new, empty data record as long as this component state is in force. ReadyForInputThenNewData: The component requests that the framework invoke its Process event with any data record that arrives at its input port. When there are no more input records to process, the framework should repeatedly pass to the component a new, empty data record as long as this component state is in force. Below are some broad categories of components and how they make use of component state. Component Category Calculator Filter Reader or Generator Writer Aggregator Management of Component State Component state is always set to "ReadyForInputData" Component state is always set to "ReadyForInputData" Component state is set to "ReadyForNewData". When the operation is complete, the state is set to "DoneProcessingData". Component state is set to "ReadyForInputData". If a maximum output limit is reached, the state is changed to "DoneProcessingData". Component state is set to "ReadyForInputThenNewData". For each input data record, the component caches the necessary data. When there is no further data on the input port and a new data record is processed, the component starts to output records of aggregated data. Creating Java Components Page 11

15 Saving the Java Code To save the class, enter CTRL-S. Eclipse compiles the code automatically, creating the output in "<install_ root>/apps/acme/coolstuff/bin/lang/java/jars". Eclipse editor Activating the Package The "package.conf" file contains the classpath and library information needed to run the Java component. The "pkgutil n" command with the java directive option generates a "package.conf" file Page 12 Pipeline Pilot Java Component Development Guide

16 that already includes a sample classpath for the components which references the output jars directory, so simply ensure you remove the line comment marker and change the name of the jar file. To active the pacmanagement of Component Statekage: 1. In a command prompt window, change directories to "<install_root>/bin" ("linux_bin" on Linux). 2. Use pkgutil on the command line to activate the "acme/coolstuff" package: pkgutil -i acme/coolstuff Notes: On Linux, first source "ppvars.sh" to set up the environment for loading the binaries ("ppvars.sh"). For details on the pkgutil command, see the Application Packaging Guide. Creating Java Components Page 13

17 Chapter 3: Testing a Java Component in Pipeline Pilot Client 1. In Pipeline Pilot Client, add the Java (on Server) component to a new blank workspace and fill in the parameter values as follows: Parameter Value Notes JavaClass com.acme.coolstuff.elementchanger This is the class you created and built in Eclipse. Use Package acme/coolstuff This is how the component knows what CLASSPATH to use; it gets it from the package's "package.conf" file. This is an array type parameter, so more than one package name can be specified. 2. Parameters required by your Java code must now be added to the component itself: Right-click the component and select Edit. The Edit dialog opens. Use the Interface tab to add new parameters. Click Add Parameter. The Define Parameter dialog opens. Use the Parameter tab to define a parameter name and type. It is good practice to author brief information for each new parameter. Enter the text in the Parameter Help field. You can also set the Required to run protocol flag, if needed. Testing a Java Component in Pipeline Pilot Client Page 14

18 Create new parameters for a component The Parameters tab in the lower-right corner of the Pipeline Pilot Client window exposes your new parameters. From here, you can set values for them before running the protocol. In the example below, the component author added two new parameters (ToElement and ChangeElement) and set their values. New parameters exposed in Parameters tab 3. To test out the component, add other components that flow and view the data when you run the protocol. 4. Run the protocol and see if the behavior is as expected. Page 15 Pipeline Pilot Java Component Development Guide

19 Chapter 4: Adding New Java Components to a Package Before saving a new Java component to its package, complete your component interface to fine-tune its design. To fine-tune your design: 1. In the Pipeline Pilot Client, edit the caption displayed beneath the component icon. Assign meaningful information that conveys what the component is designed to do when used in a protocol. 2. Define the icon, ports and help text to use with your component. Right-click the component and select Edit. The Edit dialog opens. To assign an icon, click the Ports and Icon tab, and select an appropriate icon from the list. The drop-down list displays the icon types you can select and provides a description of its purpose. An enlarged preview of the icon is also displayed on the right. From Input Port, Output Port Pass, and Output Port Fail, set the appropriate port configuration for the function and design of your component. To define help text, click the Help tab. In Summary, enter brief text to use as the flyover help. In Description, enter more details about how to use the component. Your help should describe the component s purpose (what it does and how it can be used). Component-level help provides detailed information about what the component does and how it can be used. Details for setting individual parameters are provided by parameter-level help. Saving Components in Package Destinations Configuring the interface of a component setting parameters, adding or removing ports, adding or removing parameters does not immediately create a new component or establish ownership of the new configuration. A copy of the component created after you save the component. When you save a copy, a new global identifier (GUID) is assigned and registered as a separate element of the protocol database (XMLDB). To create a copy of your Java component that is distinct from the generic Java (on Server) component and is bundled with the underlying Java class: 1. Save the component to your "User Name" tab, assigning it an appropriate name. This is the name it will be known as from now on, so it should be descriptive and as unique as you can make it without being too verbose. Consider that the final location of the component in the Component hierarchy will provide information on the component s type. For example, if it resides in a folder named "Calculators", it is not necessary to repeat the fact that it is a calculator component in the component name. See Component Development Guide. Note: Be sure to save only the component and not the entire protocol. To save a component, right-click the component icon and select Save. 2. Prepare a location in your package to store the Component XML definition. This is within the XML directory (refer back to the <xml> directive in the "package.conf" file). The folder structure that you create determines where the component is exposed in the Components tab in the Explorer hierarchy. For correct tab placement, create a folder named "xml/components", and then build an appropriate folder structure with subfolder names (e.g., "xml/components/coolstuff/manipulators"). Adding New Java Components to a Package Page 16

20 3. To store the XML file in the desired location within your package, export the component from your "User Name" tab. Right-click the component icon in the Explorer window, select Export, and save it to the appropriate path. 4. Delete the component from your "User Name" tab. You do not need it anymore since the component is now a packaged item. Note: It is necessary to export the component (in addition to saving it), to ensure that the component is assigned a unique GUID (global identifier). A simple export of the Java (on Server) component would result in a component with the same GUID as the original Java (on Server) component; which would cause confusion for future upgrades and would not let you track and manage the component effectively. 5. Insert the package component into the XMLDB by reactivating the package. pkgutil -r acme/coolstuff 6. Refresh the Explorer tab to check that the component is now installed where you expect it to be exposed in the Component tab folder hierarchy. 7. Build an example protocol to show the use of the component and store this in your package in the same way. In the case of Protocols, export the protocol XML to a suitable location within the "xml/protocols" folder. Tip: You can add multiple components and example protocols to the package in this way. Page 17 Pipeline Pilot Java Component Development Guide

21 Chapter 5: Distributing the Package It is advantageous to have a package that includes your binaries, components, documentation, and examples. It is easy to compress it in ZIP format and reinstall it on another server. To distribute the package for installation: Place the folder in the same location under the "<install_root>/apps" folder and then activate the package with the pkgutil commands (which should exist on each server). This installs the components and example protocols, and adds your documents and data files to the server infrastructure, exposing it to end users and making it available in the appropriate locations. Distributing the Package Page 18

22 Chapter 6: Debugging a Java Component Various options are available for debugging components, including: Writing additional properties to the component output Using add-in debug messages Connecting with a debugger This chapter takes a closer look at debugging Java components using the Element Changer component. Writing Additional Properties to the Output The Element Changer component writes out the extra "NumAtomsChanged" property. However, it is useful to write out the indexes of each replaced atom to highlight them in the output. This can help for debugging issues with the algorithm. To output the index value of each affected atom: 1. Access your ElementChanger.java source file and make the following additions: 2. Compile your source code. 3. In the Pipeline Pilot Client, update the ChangeElement parameter value from "Cl" to "C" and ToElement from "F" to "S" (to ensure that the component picks up new values). Debugging a Java Component Page 19

23 Element Changer component parameters 4. Add the Highlight Atoms component between the Element Changer and the HTML Molecular Table Viewer, and split the SD Reader output so that two pipes enter the viewer like this: Protocol with newly added Highlight Atoms component 5. Set the component's PropertyName parameter to "ReplacedAtoms" (so your new ReplacedAtoms property is used to select atoms for highlighting), and set Highlight Color to "Green": Highlight Atoms component parameters 6. Run the newly updated protocol. The output should look something like this: Protocol output. The top row in the HTML output shows the original input molecule and the second row shows the changes made by our Element Changer component. Tip: Adding extra parameter values is useful when you need to re-use additional parameters downstream. You can also output debug messages that are only exposed in the Debug Messages console. Page 20 Pipeline Pilot Java Component Development Guide

24 Printing Debug Messages Most Java developers are familiar with debugging by System.out.println() statements. You can print debug messages from Java components directly to Pipeline Pilot. To print debug messages: 1. In the Pipeline Pilot Client, select View > Debug Windows > Debug Messages, and ensure that the Debug Messages option is selected. The debug messages are printed in the Debug Messages window. 2. Return to your ElementChanger.java source file and make the following additions: 3. Compile your source code. 4. To run the original protocols in debug mode in Pipeline Pilot Client, select Tools > Run Protocol with Debugging. 5. When the protocol job runs, the Debug Messages window should display debugging messages like the following: Debugging messages for Java components in Pipeline Pilot Tip: Most debugging problems can be solved with these simple techniques. Occasionally, you might need a full debugger, so the following section explains how to attach a debugger to your component. Debugging a Java Component Page 21

25 Debugging with Eclipse IDE Pipeline Pilot is a multi-process system. Each protocol job executes in a separate process and loads a new JVM. This makes it easy to attach a debugger to your Java components. Note: The following script assumes you are using Eclipse IDE 3.1 or later with the Java perspective open. IMPORTANT! Always develop, test, and debug Java plug-in components on a separate development server. Do not change the JVM settings on a production server. To debug Java components in Eclipse: 1. Open the Element Changer protocol. 2. In Eclipse, select Run > Open Debug. The Debug dialog opens. 3. From the list on the left, right-click Remote Java Application and click New. Debug dialog for Java applications 4. Specify the name of your debug configuration (for example, "pilotcomponent") and browse to the project you want to debug. Page 22 Pipeline Pilot Java Component Development Guide

26 Debug configuration 5. Leave the rest of the settings set to Eclipse's defaults (localhost, 8000) and click Close. 6. Open "$(install_root)\apps\scitegic\core\xml\objects\javaenvironement.xml". Comment out the first "sci:proplist" element, and un-comment the second one, which defines the debug setup. (For more details on the JVM settings, see Appendix A: Java VM Environment.) JavaEnvironement.xml 7. To have a running protocol pause when Java is loaded and wait for you to attach the debugger, change "suspend=n" to "suspend=y". If you are the only person running protocols, this is recommended, as it means you do not have to put a pause construct in your code. 8. In the Pipeline Pilot Client, run the Element Changer protocol (or any protocol containing at least one Java (on Server) component). This starts the Java VM inside the protocol job process. 9. If you set "suspend=y", run the target protocol to debug. Debugging a Java Component Page 23

27 10. In Eclipse, toggle a breakpoint somewhere in the onprocess method body of the ElementChanger.java source file or anywhere in the file to debug. 11. In Eclipse, select Debug <your debug configuration> where "<your debug configuration>" is the name you supplied above (step 4). In this example, use "pilotcomponent" as your configuration name. This attaches the debugger to the running Java VM. If you select "suspend=y", then the debugger should break in the onprocess method and open the Eclipse debug perspective. It should look something like this: Eclipse debug perspective IMPORTANT! Make sure the first line in your component Classpath directory is an open "classes" directory, not a jar directory. This way, you will not have to rebuild and deploy your jars every time you make a code change. Notes on Debugging The process, "scisvr.exe", runs in the background for processes pools, scheduled tasks, and background tasks that keep the server warm. If you configure the "JavaEnvironment.xml" file to use a debug port (such as port 8000), a different process might use the debug port before you launch your protocol for debugging. If you see either of these conditions: "No Server Process" error displays in the Pipeline Pilot Client status bar The Java debugger attaches but does not stop at your breakpoints Page 24 Pipeline Pilot Java Component Development Guide

28 Disable process pooling and warm-up tasks in the Pipeline Pilot Admin Portal: 1. Log onto the Admin Portal. 2. Go to Jobs > Settings. 3. Set Maximum Number of Persistent Daemons per Job Pool to "0". 4. Set Job Readiness Refresh Rate (seconds) "0". Save the job settings configuration. You do not need to re-start the Apache web server. All processes share one configuration, so changing the port in "JavaEnvironment.xml" does not prevent the issue. Debugging a Java Component Page 25

29 Appendices Appendix A: Java VM Environment The file "<install_root>/apps/scitegic/core/xml/objects/javaenvironment.xml" controls the JVM than runs within a protocol. The JVM is a singleton per running protocol. These settings are universal on your server. Therefore use care when developing on a production server. You can adjust the following settings: Setting JVMType Description Windows only: Select client or server JVM. The client JVM loads faster and is best suited for quick running protocols. JVMArgs Parameters that are passed to the JVM when it is loaded; useful for adjusting memory size, debugging, and profiling Java code. For example: <sci:mpropval name="jvmargs" type="stringtype"> <sci:value>-xmx256m</sci:value> <sci:value>-xss2m</sci:value> </sci:mpropval> Appendix B: Using JNI within a Java Component If a Java component needs to load and use a native library, Pipeline Pilot needs to know where to find it. To make a native library accessible to Pipeline Pilot, define a nested jni directive within java directive in "package.conf". Within the jni directive, define CLASSPATH key so it points to the Java class file or jar file that loads the native library, and LIBRARYPATH key so it identifies the location of the native library. <java> CLASSPATH $(package)/ /jars/your_component.jar <jni> CLASSPATH $(package)/ /jars/jni_wrapper.jar LIBRARYPATH $(package)/ /libs </jni> </java> CLASSPATH and LIBRARYPATH keys within the jni directive are used to initialize the JVM. Java classes pointed by CLASSPATH within the jni directive are loaded by the system class loader, whereas Java classes pointed by CLASSPATH within the parent java directive are loaded by a child class loader. This implies that classes defined within the jni directive cannot use classes from the java directive. Hence, it is recommended that classes within the jni directive contain only native method declarations and the code to load native libraries. Appendix C: Hash Table Values Hash table values are exposed in Java via a Value object. Appendices Page 26

30 Creation from a Java Map An object implementing the Java Map interface can be used to define the value of a property. At the time of definition, a copy of the Map is made into a native hash table value contained by a property on the property collection. Native Java data types are reinterpreted into native Pipeline Pilot value types. For example, we can create a HashMap in Java, storing entries using a number of different Java native types. Assigning that Map to a property causes the creation of a HashTableValue with the correct data types: public Component.State onprocess(context context, DataRecord data) Exception { PropertyCollection props = data.getproperties(); HashMap map = new HashMap(); map.put("a", false); map.put("b", 10); map.put("c", 11.11); map.put("d", "DD"); boolean[] e = {false, true}; map.put("e", e); int[] f = {11, 12}; map.put("f", f); double[] g = {11.11, 12.34}; map.put("g", g); String[] i = {"a","b"}; map.put("i", i); props.define("h", map); return Component.State.ReadyForInputData; } Viewing the results in the Data Record Tree Viewer: throws You could now inspect the created HashTableValue downstream to confirm that the item types are indeed as expected from the inputs. (Note that both the simple data types (Boolean, int, double, and String) are supported, as are arrays of those types. To support this flexible use of Object, the Java PropertyCollection "define" method has been extended to accept a generic Object, so you can call "define" with a native Java data type (Boolean, String, String[], Map, etc) typed as an Object, and it will define the appropriate Value type.) Use the HashTableValue "H" as the input object for some of the examples below. Page 27 Pipeline Pilot (Undefined variable: ProductVariables.Product Version) Java Component Development Guide

31 Values as Java Maps A Value created from a property containing a hash table value can be cast to a Java Map, and manipulated with standard Java commands as a native Java object. Note that the Java hash table object is a copy of the HashTableValue you can make changes to it, but it does not change the original Pipeline Pilot object. You will need to use a "define" or "setvalue" call to transfer any of your changes back to Pipeline Pilot. This is analogous to how other Value types are exposed in Java changing them does not change the original value. Since a Value created from a HashTableValue can be cast to the Java Map interface, you can call all the methods of a native Java Map on such a Value. For example, you can call the Map "keyset()" method to iterate over all the keys in the hash table "H" we created in the previous example: public Component.State onprocess(context context, DataRecord data) throws Exception { PropertyCollection props = data.getproperties(); Property prop = props.getbyname("h"); Value hashval = prop.getvalue(); // If the value is a HashTable type, then the return object // from get() can be cast to a Map if (hashval.gettype()!= Value.Type.HashTable) { throw new Exception( Expected a HashTable ); } Map<Object, Object> valmap = (Map<Object, Object>)hashval.get (Value.Type.HashTable); Set keys = valmap.keyset(); Iterator it = keys.iterator(); int count = 1; while (it.hasnext()) { String key = it.next().tostring(); String name = "KEY"; name = name.concat(integer.tostring(count)); props.define(name, key); count += 1; } return Component.State.ReadyForInputData; } The results viewed in the Data Record Tree Viewer looks like: Appendices Page 28

32 Similarly, you could call "entryset()" to iterate over all the entries: public Component.State onprocess(context context, DataRecord data) throws Exception { PropertyCollection props = data.getproperties(); Property prop = props.getbyname("h"); Value hashval = prop.getvalue(); // If the value is a HashTable type, then the return object // from get() can be cast to a Map if (hashval.gettype()!= Value.Type.HashTable) { throw new Exception( Expected a HashTable ); } Map<Object, Object> valmap = (Map<Object, Object>)hashval.get (Value.Type.HashTable); Iterator it = valmap.entryset().iterator(); int count = 0; while (it.hasnext()) { Map.Entry entry = (Map.Entry)it.next(); Object key = entry.getkey(); Object item = entry.getvalue(); } } count += 1; props.define("key" + Integer.toString(count), key); props.define("item" + Integer.toString(count), item); return Component.State.ReadyForInputData; The output looks like: Page 29 Pipeline Pilot (Undefined variable: ProductVariables.Product Version) Java Component Development Guide

33 Both of these examples demonstrate how you can test whether a given value "val" contains a hash table. This is done by testing the return value of gettype(): If (val.gettype() == Value.Type.HashTableValue) { // (perform hash table action on val ) //... } Comment on null values Null values are allowed for either the key or item, but you should be careful in the definition of some of the Map methods, a 'null' return means the entry was not found. If you store nulls in a hash table, you'll have to test for existence of the property using "containskey()" rather than simply testing whether the Value returned by "findkeyitem()" is null. Appendix D: Memory Management By default, Pipeline Pilot manages the memory associated with DataRecords and Nodes in the majority of situations. This includes: Appendices Page 30

34 Existing DataRecords and their Nodes passed in to the onprocess Method New DataRecords passed into the onprocess and when the component has the ReadyForNewData or ReadyForInputThenNewData state You can also create new Nodes or DataRecords within your component by using the instance methods createnode() and createdatarecord() on the Context. You can also clone Nodes using the clone () method on the Node. Note: It is best practice is to relinquish the native memory associated with the returned object as soon as possible by calling the.relinquish() method on the instance. The common places to relinquish memory are: In the oninitialize or onprocess method in which you created the Node or DataRecord if it is only required for the duration of that method. In the onfinalize method once the component has finished its work. Note: The NodeCache class is available if you need to manage a collection of Nodes. There is no need to use a ProtocolShutdownListener registered with the Context to relinquish memory as the framework automatically cleans up memory when the protocol finishes. What happens if I forget to call relinquish? The framework will clean-up at the end of the run. If the protocol is big or memory intensive, this can become a problem, thus it is best practice to use relinquish() within the narrowest scope possible. What happens if I call relinquish on an object I didn't create? The object will continue to exist until the framework releases it. What happens if I cache Nodes? You should not cache nodes yourself. Nodes, Data Records, Property Collections, Properties, and Values are all pointers to native memory owned by the framework. These objects use reference counting and can be freed without your notification unless your code created them. If you need to hold onto Nodes or data for a longer time, either copy it to your memory or use NodeCache to have the framework cache the data. Appendix E: Testing Testing your Java components is best achieved with two complimentary approaches: 1. Test the component using Regressions. 2. Test the logic within your component using JUnit (or Regressions). This approach allows you to take advantage of the testing tools built into Java IDEs while you are developing your logic, but also ensure that your component is going to function correctly in real world protocols across different versions of Pipeline Pilot and Pipeline Pilot. To ease testing, it is recommended that you architect your components to separate the bulk of your logic from the component wrapper needed to interact with Pipeline Pilot. Your classes implementing the logic should know nothing about Pipeline Pilot, but expose an API that can be used by the component wrapper. For example, if your component looks up location details from a zip code, write a class like this: package com.acme.coolstuff; public class Location { public Location(String zipcode) {... Page 31 Pipeline Pilot (Undefined variable: ProductVariables.Product Version) Java Component Development Guide

35 } } public String getcity() {... } public String getstate() {... } This class does not import any com.scitegic.pilot.* classes, which makes it easy to test using standard Java unit testing frameworks such as JUnit. To expose the functionality of the Location class as a component it needs to be wrapped in a class that implements the Component. This is relatively simple with the class extracting the zip code from the DataRecord, creating an instance of the Location class and adding two new properties back onto the DataRecord. To integration test this wrapper class, use a regression (see Component Development Regression Test Guide.) Appendices Page 32