EMC InfoArchive Documentum Connector

Transcription

1 EMC InfoArchive Documentum Connector Version 3.2 User Guide EMC Corporation Corporate Headquarters Hopkinton, MA

2 Legal Notice Copyright 2015 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. Adobe and Adobe PDF Library are trademarks or registered trademarks of Adobe Systems Inc. in the U.S. and other countries. All other trademarks used herein are the property of their respective owners. Documentation Feedback Your opinion matters. We want to hear from you regarding our product documentation. If you have feedback about how we can make our documentation better or easier to use, please send us your feedback directly at IIGDocumentationFeedback@emc.com

3 Table of Contents Chapter 1 Overview... 5 Chapter 2 Exporting Registered Tables... 7 Processing with XSLT... 8 Understanding Documentum Extension... 8 Example: Enriching the Registered Table Data using XSL... 9 Chapter 3 Extracting ACL and Virtual Documents Chapter 4 Generating SIPs

4 Table of Contents 4

5 Overview Chapter 1 EMC InfoArchive Documentum Connector is a command-line data extraction and transformation utility that lets you export content to be archived directly from the EMC Documentum repository and generate Submission Information Packages (SIPs) to be ingested into InfoArchive. InfoArchive Documentum Connector extracts persistent objects from the repository by executing a DQL query statement defined in a configuration file. You can also specify what type of information about the objects (attributes, contents, object types, folders, and relations) to extract by configuring the settings in the configuration file. All extracted information is stored in the resulting PDI file (eas_pdi.xml) as a part of the SIP generated by InfoArchive Documentum Connector. InfoArchive Documentum Connector can run against any version of the EMC Documentum repository. Since the SIP is repository version-neutral, you can extract content from an earlier version of repository but archive and access it through a later version. The complete distribution package of InfoArchive Documentum Connector comprises these files: Script files for supported platforms: eas-launch-documentum-extractor.bat (Windows) and eas-launch-documentum-extractor.sh (Linux) Configuration file eas_documentum_extractor.properties PDI schema file DctmDocbase.xsd The DctmExtractor.jar file and its dependency.jar files in the lib sub-directory 5

6 Overview 6

7 Chapter 2 Exporting Registered Tables InfoArchive Documentum Connector supports the exporting of registered tables. You can archive the data from the Documentum repository that uses registered tables to store support custom metadata. Registered tables are RDBMS tables that are not part of the repository but are known to Content Server. They are created by the DQL REGISTER statement and automatically linked to the System cabinet in the repository. They are represented in the repository by objects of type dm_registered. After an RDBMS table is registered with the server, you can use DQL statements to query the information in the table or to add information to the table. A number of views are created in a Content Server repository automatically. Some of these views are for internal use only, but some are available to provide information to users. The views that are available for viewing by users are defined as registered tables. To obtain a list of these views, you can run the following DQL query as a user having the privileges of a system administrator. SELECT "object_name", "table_name", "r_object_id" FROM "dm_registered" The following configuration parameters have been introduced for registered tables: registeredtables: This parameter is used for configuring extracting the data. maxtablespersip: This parameter is used to split the data of the table that is extracted by specifying a special breaking rule. You can extract a sequence of registered table names by using the configuration parameters. For example: registeredtables=dm_dbo.eas_history_volume WHERE some_attribute = '...'; dm_dbo.some_reg_table WHERE some_attribute1 = '...'; By using ;, the parameter value is split so that you can extract multiple tables per launch. The maxtablespersip parameter can be used to split the data of each table that is extracted to apply a rule such as one table per SIP file. Sometimes if the object type of the registered table data is associated with the regular object type of Documentum, you need to enrich a PDI file that is extracted from the Documentum repository. This task is achieved by using XSLT transformation with a special Documentum extension. See EMC Documentum Content Server DQL Reference for more information on querying the registered tables. 7

8 Exporting Registered Tables Processing with XSLT If the regular DCTM object type data is associated with the registered table data, you might need to enrich a PDI file that is extracted from the Documentum repository. This task is achieved by using XSLT with a customized Documentum extension. Every PDI file that is generated by Documentum Connector can be transformed into another XML file. The following configuration parameters have been introduced for registered tables: xslt: This parameter is used to set the path to the XSLT file. enablelargexmlprocessing: This parameter is set to true only if the PDI files are very large. If enablelargexmlprocessing is true, then the chunks of the PDI file are processed separately. Note: Use the maxobjectspersip parameter to limit the size of the PDI file. Understanding Documentum Extension The customized Documentum extension involves the DctmExtension class which extends the net.sf.saxon.lib.extensionfunctiondefinition. This class defines the extension function that takes a DQL query as a parameter. The extension function has a structured name that consists of the following: prefix:dctm URI: urn:emc:eas:connector:xslt-ext localname: dql The extension function produces output as follows: <row> <column1>...</column1> <columnn>...</columnn> </row> Usually, it is assumed that the only row will be returned by a query. If there are more than two rows, the output will be as follows: <rows> <row> <column1>...</column1> <columnn>...</columnn> </row> </rows> Note: There can be only one root node. The section within the row element is defined in the schema file DctmDocbase_1.0.xsd (part of the Documentum Connector package). This section is same as the attributes section. Any XPath query expressions can be applied to the function result. In the following example, the XPath expression is applied to the function result: 8

9 Exporting Registered Tables <?xml version="1.0" encoding="utf-8"?> <xsl:stylesheet version= 2.0 xmlns:xsl= xmlns:xs=" xmlns:emc="urn:x-emc:eas:schema:documentum:1.0" xmlns:dctm="urn:emc:eas:connector:xslt-ext" xmlns:exsl=" extension-element-prefixes="exsl" exclude-result-prefixes="dctm"> <xsl:template match= //emc:object > <output> <count> <xsl:value-of select="exsl:node-set(dctm:dql ('select... from... where...'))/row/r_object_id"> </xsl:value-of> </count> </output> </xsl:template> </xsl:stylesheet> Example: Enriching the Registered Table Data using XSL 1. Create a registered table. CREATE TABLE docubase.custom_data ( id numeric(20) primary key, description varchar2(50)); --SQL REGISTER TABLE dm_dbo.custom_data ("id" INT, "description" CHAR(50) KEY ("id") insert into insert into insert into insert into dm_dbo.custom_data values (1, 'abc'); dm_dbo.custom_data values (2, 'def'); dm_dbo.custom_data values (3, 'ghi'); dm_dbo.custom_data values (4, 'jkl'); 2. If you try to export the registered table data separately, it is tough to search data after ingesting it. So use XSL to enrich the attributes in the PDI files <?xml version="1.0" encoding="utf-8"?> <xsl:stylesheet xmlns:xsl=" version="1.0" xmlns:emc="urn:x-emc:eas:schema:documentum:1.0" xmlns:dctm="urn:emc:eas:connector:xslt-ext" exclude-result-prefixes="dctm" <xsl:param name="production_date"/> <xsl:template node() > <xsl:copy> <xsl:apply-templates select="@* node()"/> </xsl:copy> 9

10 Exporting Registered Tables </xsl:template> <xsl:template match="emc:objects"> <objects> <xsl:attribute name="production_date"> <xsl:value-of select="$production_date"/></xsl:attribute> <xsl:apply-templates node()"/> </objects> </xsl:template> <xsl:template match="emc:attributes"> <xsl:copy> <xsl:apply-templates node()"/> <letters> <xsl:value-of select="dctm:dql(concat ('SELECT rt.description AS city FROM dm_dbo.custom_data rt WHERE rt."id" = ',emc:custom_table_id))"/> </letters> </xsl:copy> </xsl:template> </xsl:stylesheet> 10

11 Chapter 3 Extracting ACL and Virtual Documents From this release onwards, the extraction of Access Control List (ACL) and virtual documents is supported for Documentum Connector. An ACL is the mechanism that Content Server uses to impose object-level permissions on SysObjects. An ACL is a set of permissions that apply to an object. Every object in the repository has an ACL. The information about ACLs related to an object can be extracted by using Documentum Connector. A virtual document is a hierarchically organized structure composed of component documents. The child components of a virtual document can be simple documents (that is, nonvirtual documents), or they can themselves be virtual documents. The information regarding these virtual documents can be extracted by using Documentum Connector. The following parameters have been introduced for the support of ACLs and Virtual documents: extractacls: Set this parameter to true if you need the extraction of ACL. extractcontainments: Set this parameter to true if you need the extraction of virtual documents. By configuring these parameters in the eas_documentum_extractor.properties file and running the Documentum Connector, you can extract virtual documents and ACLs related to an object. Here is an example of the section for ACL structure in the eas_pdi.xml file: <acl> <!--Section describing the ACL applied on the object. The usage of the DFC computed attributes _accessor_app_permit, _accessor_name, _accessor_permit, _accessor_permit_type, _accessor_xpermit, _accessor_xpermit_names allows to implement a simple and generic extraction. NOT REQUIRED FOR THE PoC--> <acl_class>...</acl_class> <description>...</description> <object_name>...</object_name> <owner_name>...<owner_name> <entries> <entry> <accessor_name>...</accessor_name> <accessor_permit>...</accessor_permit> <accessor_xpermit>...</accessor_xpermit> 11

12 Extracting ACL and Virtual Documents <application_permit>...</application_permit> <permit_type>...</permit_type> </entry> <entry>...</entry> </entries> </acl> Here is an example of the section for the virtual document structure in the eas_pdi.xml file: <containment> <object_id>...</object_id> <component_chronicle_id>component chronicle id</component_chronicle_id> <order_no>...</order_no> <use_node_ver_label>...</use_node_ver_label> <version_label>...</version_label> <component_id>component object id<component_id> <component_object_name>object name of the component object for being able to display a meaningful value</component_object_name> <component_contents> <original full_format="..." mime_type="..."> <!-- Section describing the original contents of the object --> <page number="..." full_content_size="...">file name containing this content</page number> </original> <rendition full_format="..." mime_type="..." page_modifier="..."> <!-- Section describing the first rendition contents of the object --> <page number="..." full_content_size="...">file name containing this content</page> </rendition> <rendition full_format="..." mime_type="..." page_modifier="..."> <!-- Section describing the other rendition contents of the object --> <page number="..." full_content_size="...">file name containing this content</page> </rendition> </component_contents> </containment> <containment>...</containment> <containments> 12

13 Chapter 4 Generating SIPs 1. Make sure the computer from which you run InfoArchive Documentum Connector meets the following prerequisites: You can connect to the Content Server from the computer and the Documentum repository from which you want to extract content for archiving is up and running. Documentum Foundation Classes (DFC) along with the compatible JRE version is installed locally. DFC is the published and supported programming interface for accessing the functionality of the Documentum platform. Content Server and any EMC Documentum client product that uses DFC install DFC automatically. If you have not installed these products on the computer, you must install DFC as a standalone product. The EMC Documentum Foundation Classes Installation Guide contains information about how to install DFC individually. EMC recommends that you run InfoArchive Documentum Connector from the Content Server host where you can perform data extraction faster and more conveniently. Note: Ensure that there is enough space to store the SIP packages. 2. Open eas-launch-documentum-extractor.bat (Windows) or eas-launch-documentum -extractor.sh (Linux) in a text editor and set the following environment variables: JAVA_HOME: The path to your JRE/JDK installation BASEDIR: The path to the directory where the InfoArchive Documentum Connector files are located For example: SET BASEDIR=C:\eia\DctmExtractor SET JAVA_HOME=C:\Program Files\Java\jdk1.7.0_17 3. Edit eas_documentum_extractor.properties and configure data extraction and transformation settings. a. Configure how to extract content from the repository by specifying the following information through parameters: The required information to access the Documentum repository The DQL query statement for selecting objects to be extracted for archiving What type of information about the objects will be extracted (object properties and attributes are always extracted) Parameter Docbase username Description Specifies the name of the Documentum repository. The rest of the Documentum connection parameters must be specified in dfc.properties found in the CLASSPATH variable Specifies the user name for logging in to the Documentum repository. 13

14 Generating SIPs Parameter password dqlpredicate Description Specifies the password for logging in to the Documentum repository. In CLI, use password command to specify this value. the password will be encrypted and will be added to the configuration file. This field has to be blank only when Documentum Connector is running on Content Server. Specifies the partial DQL query string after the FROM clause. The string you specify here is automatically appended to the preset string "select r_object_id from" to form the complete DQL query statement when you execute the utility. For example, if you set dqlpredicate as follows: dqlpredicate=dm_sysobject where folder('/phonecalls' ) The complete DQL query statement that will be constructed and executed is: select * from dm_sysobject where folder(' /PhoneCalls') extractcontents extracttypes extractfolders extractrelations Only objects returned by the query statement will be extracted for archiving. Set this to true or false to specify whether to extract information of content files associated with the selected objects. When extracted, content file information is stored in the contents element in eas_pdi.xm as part of the generated SIP. The default value is true. Set this to true or false to specify whether to extract object type information of the selected objects. If the objects belong to multiple object types (a subtype with its supertypes; for example, dm_document and dm_sysobject), all these types will be extracted. When extracted, object type information is stored in the types element in eas_pdi.xml as part of the generated SIP. The default value is true. Set this to true or false to specify whether to extract information of the folder (and all its parent folders, if any) of the selected objects. When extracted, folder information is nested in the folders element in eas_pdi.xml as part of the generated SIP. The default value is true. Set this to true or false to specify whether to extract information of relationship (represented by dm_relation_type and dm_relation objects) associated with the selected objects. When extracted, relationship information is stored in the relations element in eas_pdi.xml as part of the generated SIP. The default value is true. 14

15 Generating SIPs Parameter extractcontainments extractacls Description Set this to true or false to specify whether to extract virtual documents. Set this to true or false to specify whether to extract ACLs. b. Configure how to generate SIPs by setting the following parameters. Parameter xsdfilevalidation xslt validationmode Description Specify an XSD schema file, for example DctmDocbase.xsd (default schema) as the schema (.xsd) file against which the PDI file generated by the utility will be validated. If you don t set a value, there will not be any validation. If you use the XSL transformation to transform PDI files to your own schema you could specify this schema here to validate the result of the XSL transformation. (Optional) Specifies the path of the XSLT file. It is applied for each PDF file. Whether you want the utility to stop running when errors occur: false: Once an error occurs (for example, when the content files associated with an object being extracted is not available), the extraction process stops, and you must fix the issue and run the utility again. This can be time-consuming when there many issues during the extraction process. true: When errors occur, the utility displays error messages but do not stop the extraction process. This way, you can catch all the issues and address them before running the utility again. holding pdischema pdischemaversion application workingdir Note: Do not ingest SIPs generated in this mode. If errors occur during the extraction process, generated SIPs are invalid. Specifies the target holding into which the extracted objects will be archived. Specifies the PDI schema that is written to the SIP files. Specifies the PDI schema version that is written to the SIP files. Specifies the business application that produced the data to be archived. Specifies the complete path of the working directory in which to generate SIPs. If the directory does not exist, the utility will create it when executed. For example: workingdir=c:\\app\\eia\\dctmextractor\\data\\ maxobjectspersip maxcisizepersip Note: Use double slashes as the delimiter in the path. Specifies the maximum number of objects in a SIP. Specifies the maximum size in bytes for content files per SIP. 15

16 Generating SIPs Parameter Description enablelargexmlprocessing (Optional) Specifies that it is possible to process large XML files split in chunks when set to true. priority Specifies the ingestion priority of the SIP. The greater the value, the higher the priority. producer entity archiveid The order by which SIPs are ingested is determined first by ingestion priority (higher-priority SIPs are ingested first), and then by ingestion deadline date (SIPs with earlier deadlines are ingested first). Specifies the application that generates the SIP. Specifies the business entity that owns the data to be archived. Specifies a string that is used to generate a unique archive ID for each SIP. This string may contain a pattern that is a combination of static text, date value pattern, and UUID part for generating ID. In the pattern, a DateTime value is applied and it uses the same date/time format as it is specified in the java.lang.string# format in the Java method. (See the official Oracle JSE JavaDoc for more details.) If the pattern has the %%UUID%% string, then it indicates that some unique number has to be added to the ID. The maximum length of the generated ID value must not exceed 32 characters, otherwise the utility will throw an exception and will stop executing. An example of the configuration pattern is as follows: TestArch-%1$tY-%1$tm-%1$td-%%UUID%% The SDK generates such IDs for all the ZIP files in an extraction session: retention_class TestArch uakK99PX1oOX Specifies the retention class policy parameter. See InfoArchive Configuration and Administration Guide for the possible values for this parameter. c. Configure extracting information from the registered tables by setting the following parameters. Parameter registeredtables maxtablespersip Description Specifies the DQL query parts after the FROM clause related to the registered tables that have to be extracted. Specifies the maximum number of registered tables objects per SIP. d. Configure file hash computing for the PDI file by setting the following parameters. 16

17 Generating SIPs Parameter includepdihashinsip checksum_algorithm Description Specifies that a hash value of each PDI file is written to a SIP when set to true. Specifies the algorithm for calculating hash value. The algorithm can be either MD5, SHA-1, or SHA Open a command prompt window and execute eas-launch-documentum-extractor.bat (Windows) or eas-launch-documentum-extractor.sh (Linux). Note: On Windows, the dfcpath system variables cannot contain spaces; otherwise, an error message appears: dfc.jar not found. 5. The data extraction and transformation process begins. You can see tracing information in the command prompt window similar to the following: >>> START... Reader instantiated. >>> DQL query : select r_object_id from dm_sysobject New PDI File (count:0) > 0: Writing 0900c d9b >>> DQL query : select relation_name, description, child_id from dm_relation where parent_id = 0900c d9b > 1: Writing 0900c d9f >>> DQL query : select relation_name, description, child_id from dm_relation where parent_id = 0900c d9f [...] > 539: Writing 4c00c >>> DQL query : select relation_name, description, child_id from dm_relation where parent_id = 4c00c Deleting c:\eia\dctmextractor\data\archive1 >>> END! 6. When the data extraction and transformation process is complete, you can see SIP files generated in the working directory (specified by the workingdir parameter in the configuration file). 17

18 Generating SIPs 18