All About Integration - PDF Free Download

All About Integration XOG XML Open Gateway Lars Seibert, Engineering Services Architect July 2009

Terms of This Presentation This presentation was based on current information and resource allocations as of October 2009 and is subject to change or withdrawal by CA at any time without notice. Notwithstanding anything in this presentation to the contrary, this presentation shall not serve to (i) affect the rights and/or obligations of CA or its licensees under any existing or future written license agreement or services agreement relating to any CA software product; or (ii) amend any product documentation or specifications for any CA software product. The development, release and timing of any features or functionality described in this presentation remain at CA s sole discretion. Notwithstanding anything in this presentation to the contrary, upon the general availability of any future CA product release referenced in this presentation, CA will make such release available (i) for sale to new licensees of such product; and (ii) to existing licensees of such product on a when and if-available basis as part of CA maintenance and support, and in the form of a regularly scheduled major product release. Such releases may be made available to current licensees of such product who are current subscribers to CA maintenance and support on a when and if-available basis. In the event of a conflict between the terms of this paragraph and any other information contained in this presentation, the terms of this paragraph shall govern. 2 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

For Informational Purposes Only Certain information in this presentation may outline CA s general product direction. All information in this presentation is for your informational purposes only and may not be incorporated into any contract. CA assumes no responsibility for the accuracy or completeness of the information. To the extent permitted by applicable law, CA provides this document as is without warranty of any kind, including without limitation, any implied warranties or merchantability, fitness for a particular purpose, or non-infringement. In no event will CA be liable for any loss or damage, direct or indirect, from the use of this document, including, without limitation, lost profits, lost investment, business interruption, goodwill, or lost data, even if CA is expressly advised of the possibility of such damages. 3 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Session Abstract > This session is dedicated to the technology available, be it Web Services (XOG), Business Processes or GEL scripting, to get your data moving in and out of Clarity as your business demands.

Agenda > XOG Overview / XOG usage > XOG Client > XOG Example > XOG Via GEL > Error trapping with GEL > XOG Via WSDL and Java > XOG Tips / Common Issues > Documentation / Community Assistance

XOG Overview > XOG XML Open Gateway > XOG is the SOAP based CA Clarity PPM web service interface that can be used to: Import/Export data Move configuration data between Clarity environments Initiate processes in Clarity Execute NSQL queries via the Query API via XML files 6 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Overview > SOAP (Simple Object Access Protocol) SOAP is used for data exchange in distributed environments SOAP is a industry standard, describing the exchange of XML based messages in heterogeneous environments SOAP is a remote procedure call (RPC) protocol based on HTTP and XML 7 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Overview XML (Extensible Markup Language) XML is an open standard produced by the W3C committee XML is used for the representation of arbitrary data structures in a textual format XML contains no binary data therefore it is human readable XML supports all the languages of the world via Unicode 8 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Overview > Conclusion XOG is based on standards produced by the W3C committee XOG can be used to import or export data from any platform supporting these standards Therefore we call it the Clarity XML Open Gateway 9 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Typical XOG usage > What kind of data can be Xogged? Content Data such as objects, views, filters, lookups etc Instance Data such as specific projects, resources, timesheets, etc 10 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Typical XOG usage > How do our customers typically use XOG? Here are a few examples: Importing resources out of a resource management system such as PeopleSoft and into Clarity Exporting financial transactions out of Clarity and into an external billing system Content pack XOG between Clarity Environments 11 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Content XOG > Migration of object customisations between Clarity environments, for example: DEV Develop Custom Portlet XOG UAT User Testing XOG PROD Deploy into Production > See KB TEC482079 - General Migration XOG Guidelines 12 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Different ways to use XOG > From the command line > In the XOG Shell > In a process with a custom GEL Script > In a Web service call via WSDL > In a high level programming language like Java 13 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Client > The XOG Client communicates with the Clarity PPM server via HTTP SOAP protocol to: Log in Start an authenticated session Execute requests to read / write data Log out End session > Can be run via command line 14 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Client > Where do we get the XOG Client for the command line or the XOG shell from and how is it installed? 1. In the Clarity Admin Tool, select Client Downloads 2. Download and install the XML Open Gateway Client directly to your workstation. You ll want to make sure you have your Clarity user configured with the XOG Administration global right as well as the rights required for your XOG activity. 15 July 28, 2009 [XOG - The Basics] Copyright 2009 CA

XOG Client > Folder Structure In the folder structure created by the XOG install, the two main folders you will need are \bin and \xml. The \bin folder contains the properties file needed for modifying your XOG settings. The \xml folder contains the xml files used to tell XOG what data to read as well as example files for XOG writes. For example, prj_projects_read.xml reads project data from Clarity. 16 July 28, 2009 [XOG - The Basics] Copyright 2009 CA

XOG Example > Getting started For example, you ve added a new project in your DEV environment that now needs to be moved into your PROD environment 1. In your XOG installation XML folder, locate the correct project instance read file to use 2. For our example, we ll be using the prj_projects_read.xml file. Open this file and save it as a new copy for your test (prj_projects_readtest.xml) 17 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Example > Setting up the XOG Read File The xml read files provided in the installation XML folder, give examples of how to set the filter section per item. Instance related data could be Xogged out by filtering on the ID of the instance or by applying filters to read multiple instances. Arguments can also be utilised in some read xml files to specify which items associated with an instance you would like to XOG (a project s tasks but not resources, for example). Once you have modified the XML to suit your needs, you are ready for XOGing. 18 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Example > Setting up the XOG Read File In our example, since we know exactly which project we want to read out, you ll want to use the projectid filter in order to indicate you are filtering your data based upon a specific projectid <Filter name="projectid criteria="equals >PJ0000043</Filter> 19 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Example > Filter section  This is what the filter section of our read file looks like before we perform our modifications: <Filter name="projectid" criteria="equals">test</filter> <Filter name="active" criteria="equals">true</filter> <Filter name="approved" criteria="equals">true</filter> <Filter name="closed" criteria="equals">false</filter> <Filter name="approvedforbilling" criteria="equals">false</filter> <Filter name="start" criteria="between >2004-01-07,2001-01-15</Filter> <Filter name="finish" criteria="equals">customer</filter> <Filter name="lastupdateddate" criteria="equals">2009-01-26t11:42:03</filter> <Filter name="resourceid" criteria="equals >SSquarepants</Filter> <Filter name="trackmode" criteria="equals">2</filter> Note the open and close XML comment tags  20 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Example > Example of our completed XOG read XML: <?xml version="1.0" encoding="utf-8"?> <NikuDataBus xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:nonamespaceschemalocation="../xsd/nikuxog_read.xsd"> <Header version="6.0.11" action="read" objecttype="project" externalsource="niku"> </Header> <Query> </Query> </NikuDataBus> <args name="include_tasks" value="true"/> <args name="include_dependencies" value="true"/> <args name="include_subprojects" value="true"/> <args name="include_resources" value="true"/> <args name="include_custom" value="true"/> <Filter name="projectid" criteria="equals">pj00000043</filter> 21 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Example Invoking XOG > Now that we have a modified read file, how do you run the XOG Client on the command line? You can invoke XOG in the following ways: a) From the command line you can type in the parameters required to import and export data directly on the command line b) You can store the parameters in a.properties file and call the file from the XOG Client. 22 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Example Invoking XOG > Running XOG directly from the command line xog servername=clarityserver input=../xml/prj_projects_readtest.xml -output =../xml/prj_projects_writetest.xml user=admin password=nimda This method is normally only used in batch scripts > Running XOG from the command line with a propertyfile Modify test.properties file in the \bin folder servername=clarityserver input=../xml/prj_projects_readtest.xml output=../xml/prj_projects_writetest.xml username=admin password=nimda Execute XOG with the following command in the /bin folder xog propertyfile test.properties 23 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Example Invoking XOG > Running XOG from the command line with a propertyfile Some things to remember when using the test.properties file Set the servername = the server you are reading from or writing to Check the settings for portnumber and sslnumber. They are set to default and commented out. Set the input = the read or write xml file you want to use Set the output = whatever you want to name the output file. By default, the properties file sets the name to out.xml and the location to the bin folder. Set the username and password to a user with appropriate rights (be sure there are not spaces at the end of the username or password) 24 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Example Invoking XOG > The XOG shell Enter the following command in the /bin folder xog Login > login admin/nimda@clarityserver Set output > output../xml/prj_projects_writetest.xml Invoke XOG > call../xml/prj_projects_readtest.xml 25 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via GEL > Using XOG in a custom GEL script Load the needed libraries <gel:script xmlns:soap- ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:core="jelly:core" xmlns:gel="jelly:com.niku.union.gel.geltaglibrary" xmlns:soap="jelly:com.niku.union.gel.soaptaglibrary" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope" xmlns:xog="http://www.niku.com/xog" xmlns:xsd="http://www.w3.org/2001/xmlschema" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance"> 27 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via GEL > Using XOG in a custom GEL script (continued) Login <soap:invoke endpoint="http://clarityserver/niku/xog" var="auth" > <soap:message> <soapenv:envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope" xmlns:xog="http://www.niku.com/xog"> <soapenv:header /> <soapenv:body> <xog:login> <xog:username>admin</xog:username> <xog:password>nimda</xog:password> </xog:login> </soapenv:body> </soapenv:envelope> </soap:message> </soap:invoke> 28 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via GEL > Using XOG in a custom GEL script (continued) Get the session id <gel:set asstring="true" select="$auth/soap-env:envelope/soap- ENV:Body/xog:SessionID/text()" var="sessionid" /> 29 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via GEL > Using XOG in a custom GEL script (continued) Set XML <gel:parse var="xogin"> <NikuDataBus xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:nonamespaceschemalocation="../xsd/nikuxog_read.xsd"> <Header version="6.0.11" action="read" objecttype="project" externalsource="niku"> <args name="order_by_1" value="name"/> <args name="order_by_2" value="projectid"/> <args name="include_tasks" value="true"/> <args name="include_dependencies" value="true"/> <args name="include_subprojects" value="true"/> <args name="include_resources" value="true"/> <args name="include_custom" value="true"/> </Header> <Query> </Query> </NikuDataBus> </gel:parse> <Filter name="projectid" criteria="equals">pj00000043</filter> 30 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via GEL > Using XOG in a custom GEL script (continued) Invoke XOG <soap:invoke endpoint="http://clarityserver/niku/xog" var="xogout"> <soap:message> <soapenv:envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xog="http://www.niku.com/xog"> <soapenv:header> <xog:auth> <xog:sessionid>${sessionid}</xog:sessionid> </xog:auth> </soapenv:header> <soapenv:body> <gel:include select="$xogin"/> </soapenv:body> </soapenv:envelope> </soap:message> </soap:invoke> 31 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via GEL > Using XOG in a custom GEL script (continued) Close XOG connection <soap:invoke endpoint="http://clarityserver/niku/xog" var="result"> <soap:message> <soapenv:envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xog="http://www.niku.com/xog"> <soapenv:header> <xog:auth> <xog:sessionid>${sessionid}</xog:sessionid> </xog:auth> </soapenv:header> <soapenv:body> <xog:logout/> </soapenv:body> </soapenv:envelope> </soap:message> </soap:invoke> 32 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via GEL Via Process > GEL Scripts can be run in a process custom script action Use a process with Start Conditions rather than a trigger Stay in support compliance Insure business rules are maintained Automate manual data entry processes 33 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Capturing Errors in GEL > Be sure to add check points to capture errors Use a Choose function to test for a successful response Log response of success or failure > Example 1 Login Test for a valid session ID 34 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Capturing Errors in GEL <soap:invoke endpoint="${xogurl}/niku/xog" var="auth"> <soap:message> <soap-env:envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xog="http://www.niku.com/xog"> <soap-env:body> <xog:login> <xog:username>${xoguser}</xog:username> <xog:password>${xoguserpassword}</xog:password> </xog:login> </soap-env:body> </soap-env:envelope> </soap:message> </soap:invoke> <gel:set asstring="true select="$auth/soap-env:envelope/soap-env:body/xog:sessionid/text() "var="sessionid"/> <core:choose> <core:when test="${sessionid == null}"> <gel:log level="error">error: Login to Clarity XOG failed. Check username and password. </gel:log> </core:when> <core:otherwise> <gel:log>success: Login to Clarity XOG successful.</gel:log> </core:otherwise> </core:choose> 35 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Capturing Errors in GEL > Example 2: Use XOG output to log success and failures response Scenario: XOG is initiated from GEL script within a process Process starts on update to project XOG closes milestones if all previous tasks are complete Record XOG output and number of records affected 36 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Capturing Errors in GEL <gel:set asstring="true select="$result/soap-env:envelope/soap-env:body/xogoutput/status/@state" var="xogoutcome"/> <gel:set asstring="false select="$result/soap-env:envelope/soap-env:body/xogoutput/statistics" var="stats"/> <gel:set asstring="true select="$result/soap-env:envelope/soap-env:body/xogoutput/records" var="vrecords"/> <core:switch on="${xogoutcome}"> <core:case value="success > <gel:log>success: XOG to update Clarity milestones is successful. </gel:log> </core:case> Total = <gel:expr select="$stats/@totalnumberofrecords"/>. Inserted Records = <gel:expr select="$stats/@insertedrecords"/>. Updated Records = <gel:expr select="$stats/@updatedrecords"/>. Failure Records = <gel:expr select="$stats/@failurerecords"/> <core:case value="failure"> <gel:log level="error">failed:**${proj_unique_name}** XOG to update milestones failed. </gel:log> </core:case> <core:default> Total = <gel:expr select="$stats/@totalnumberofrecords"/>. Inserted Records = <gel:expr select="$stats/@insertedrecords"/>. Updated Records = <gel:expr select="$stats/@updatedrecords"/>. Failure Records = <gel:expr select="$stats/@failurerecords"/> <gel:log level="error">failed: to find the Clarity Project information! </gel:log> </core:default> </core:switch> 37 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via WSDL > WSDL? Web Service Descriptor Language > Third-party SOAP Integration Toolkits Apache AXIS 1.3 MS Visual Studio (.NET Framework 2.0) SOAP Sonar (For educational purposes) Download from www.crosscheck.com 38 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via WSDL > How to access WSDL Definitions? http://clarityserver/niku/wsdl > Types of WSDL Object - XOG Invoke action (Can invoke a process definition) Query (NSQL to read data) 39 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Via Java > Using XOG in a high level language like Java It would exceed this presentation to show the necessary coding for Java, therefore we will only describe the steps which have to be implemented: A SOAP connection has to be established With a XOG login you get a session id Create a XML document with a tool like JDOM Invoke XOG with the session id and the XML document Close the XOG connection 41 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Tips > Some things to remember: Not everything in Clarity is XOGable, so when migrating data it is important to design a migration plan. Dependencies will read out and appear in the output write file. For example, if you are XOGing out an object and in that object is an active attribute that uses a certain lookup, both the active attribute and the lookup will be Xogged out and appear in the write output file. 42 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

XOG Tips > In addition Always XOG between like-to-like Clarity environments You cannot XOG read out of Clarity 8.1 FP03 and write into Clarity 12.0.1 Keep your XOG Client current even after SP upgrades When you perform a read using XOG, the output file is formatted to be a write xml file. This means that if you XOG out a project, the output xml file will be XOGready as a write xml file. 43 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Common Issues > Special Characters Users will frequently copy/paste from text editors which allow special formatting (such as MSWord) into Clarity text fields. If any of these characters are outside of the basic ASCII character set used by XML, the XOG read and/or write will fail with an invalid XML error message. But unfortunately will not tell you where your invalid XML is > Wrong XOG permissions > Inconsistent XOG and/or Clarity versions 44 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Documentation > For further and more detailed reading enjoyment Clarity Project & Portfolio Manager Integration Guide Review KB articles by searching CA support site. Keywords = Clarity XOG TEC482079 (General Migration Guidelines) Example Script: 45 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Community Assistance > http://caforums.ca.com/ca/ Clarity is the MOST ACTIVE CA board > XOG/GEL/WSDL is the MOST ACTIVE topic on CA Clarity forums > How to questions are answered by community experts and Clarity Support/Services/Development > Lots of sample code shared 46 October 11th, 2009 [XOG - Integrations] Copyright 2009 CA

Questions?