EMC ApplicationXtender Index Agent

Transcription

1 EMC ApplicationXtender Index Agent Version 7.0 Administration Guide EMC Corporation Corporate Headquarters Hopkinton, MA

2 Legal Notice Copyright EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. Adobe and Adobe PDF Library are trademarks or registered trademarks of Adobe Systems Inc. in the U.S. and other countries. All other trademarks used herein are the property of their respective owners. Documentation Feedback Your opinion matters. We want to hear from you regarding our product documentation. If you have feedback about how we can make our documentation better or easier to use, please send us your feedback directly at IIGDocumentationFeedback@emc.com

3 Table of Contents Preface... 5 Chapter 1 Starting AppXtender Index Agent Implementation... 7 Introduction to ApplicationXtender Index Agent... 7 Configuring ApplicationXtender to Use the Index Agent... 8 xplore Full-Text Engine... 8 Full-Text Indexing... 8 Full-Text Searching... 9 Building Full-Text Search Expressions... 9 Using Multiple Processors Creating the Impersonation Account Creating the AppXtender Index Agent Impersonation Account Adding Advanced Rights to the AppXtender Index Agent Impersonation Account Chapter 2 Installing, Configuring, and Using xplore for AppXtender Installing xplore and Configuring the Primary Server Instance Installing xplore Installing xplore 1.3 Patch 13 package Configure Indexserverconfig.xml for AppXtender Full-Text Creating xplore Domains and Collections Configuring Applications to use xplore Full-text Engine in Application Generator Chapter 3 Configuring AppXtender Index Agent Configuration Overview Data Sources Tab Documents Submitted from Other Data Sources Setup Tab Specifying an AppXtender Login Account Specifying an AppXtender Index Agent Impersonation Account Specifying an Index Agent Role Queues Tab Adding a New Queue Adding Queues to be Processed Removing Queues from the Processing Queues List Legacy Full-Text Tab xploretab OCR Tab Log File Tab Misc Tab Configuring AppXtender Index Agent Error Behavior

4 Table of Contents Remove Job from Queue when Finished Chapter 4 Managing the AppXtender Index Agent Resubmitting Documents to the AppXtender Index Agent To use the AppXtender Full-Text Index Wizard: Managing Legacy AppXtender Full-Text Index Data Migrating to xplore Engine Using Old (Verity K2 or ProIndex) Full-Text Engine Controlling the AppXtender Index Agent Starting the AppXtender Index Agent Viewing AppXtender Index Agent Information Glossary

5 Preface This document provides instructions on how to install, configure, and manage the EMC ApplicationXtender Index Agent (AppXtender Index Agent) software. As part of an effort to improve and enhance the performance and capabilities of its product lines, EMC periodically releases revisions of its hardware and software. Therefore, some functions described in this document may not be supported by all versions of the software or hardware currently in use. For the most up-to-date information on product features, refer to your product release notes. If a product does not function properly or does not function as described in this document, please contact your EMC representative. Audience This document is intended for system administrators who are responsible for installing software and maintaining the servers and clients on a network. Operators who monitor the daily backups may also find this manual useful. 5

6 Preface 6

7 Starting AppXtender Index Agent Implementation Chapter 1 Previously known as ApplicationXtender Index Server, it has been renamed as ApplicationXtender Index Agent (AppXtender Index Agent) with this release of ApplicationXtender. Though the product name has changed, its functions remain the same as before. AppXtender Index Agent is still an optional module that adds full-text indexing and OCR functionality to AppXtender systems. This chapter describes the implementation of AppXtender Index Agent. It covers the following topics: Introduction to ApplicationXtender Index Agent, page 7 Configuring ApplicationXtender to Use the Index Agent, page 8 Creating the Impersonation Account, page 11 Introduction to ApplicationXtender Index Agent Users can submit text to AppXtender Index Agent for full-text indexing. With full-text indexing, users can search for documents in AppXtender Document Manager or Web Access.NET by index values and/or keyword content within the document. To submit text to the AppXtender Index Agent, users can submit entire AppXtender documents to the server for indexing and they can also submit scanned documents and pages to the server to be processed using OCR and have the resultant text added to Index Agent in one step. Users are also able to submit documents to AppXtender Index Agent for just OCR processing. The mechanism in which documents are submitted is in the form of job queuing. This is the method for both full-text indexing and OCR processing. The user either uses AppXtender Document Manager or Web Access.NET to submit the full-text queues and OCR queues. The AppXtender Index Agent polls the queues for the next job (document or page) to process. Queues are set up to accommodate jobs from multiple workstations, and can be configured at any AppXtender Index Agent computer. The queues are database table rows (in the AppXtender database) that track which documents are processed by AppXtender Index Agent. Note: Full-text queues and OCR queues are created and monitored using either the AppXtender Index Agent Configuration dialog box or in the AppXtender Queue Manager. Once you create a queue, you must designate the queue as a processing queue. This can only be done using the AppXtender Index Agent Configuration dialog box. Adding a New Queue, page 27 and Adding Queues to be Processed, page 28 provide more information about designating a queue. 7

8 Starting AppXtender Index Agent Implementation Configuring ApplicationXtender to Use the Index Agent The following instructions describes how to configure the AppXtender system to use AppXtender Index Agent. AppXtender Admin allows you to configure connectivity between AppXtender Index Agent and the DiskXtender software and to specify mapped drives. The EMC ApplicationXtender Core Components Administrator s Guide provides more information about using DiskXtender. xplore Full-Text Engine EMC Documentum xplore is a multi-instance, scalable, high-performance, full-text index server that can be configured for high availability and disaster recovery. xplore is the new full-text engine for ApplicationXtender. The following tables list out full-text indexing, searching, and using multiple processors in xplore: Full-Text Indexing, page 8 Full-Text Searching, page 9 Building Full-Text Search Expressions, page 9 Using Multiple Processors, page 11 Full-Text Indexing The following table describes how xplore full-text processing engine processes each page type: Page Type Image COLD/Text PDF RTF HTML xplore AppXtender Index Agent uses OCR on each image file to generate a text file for xplore to index. xplore indexes the COLD/Text file. Areas normally covered by redactions and non-transparent text annotations are replaced with blank spaces before indexing. AppXtender Index Agent does not convert PDF files into text files. xplore indexes each PDF file directly. If a PDF file is an image-only PDF file, AppXtender Index Agent uses OCR to generate a text file for xplore to index. AppXtender Index Agent does not convert RTF files to text files. xplore indexes each RTF file directly. AppXtender Index Agent does not convert HTML files to text files. xplore indexes each HTML file directly. 8

9 Starting AppXtender Index Agent Implementation Page Type Foreign OLE Embedded files xplore AppXtender Index Agent does not convert foreign files to text files. xplore indexes each foreign file directly. AppXtender Index Agent does not process OLE files. Full-Text Searching The following table describes the appearance of full-text search results in AppXtender Document Manager or Web Access.NET: Page Type Image COLD/Text PDF RTF HTML Foreign xplore AppXtender highlights the hit results using the text view of the image in KeyView. AppXtender highlights the hit results in KeyView. Areas normally covered by redactions and non-transparent text annotations are replaced with blank spaces. Annotations and form overlay do not apply. You can use the text view toolbar button or menu item to switch to normal text display mode with annotations and form overlays. AppXtender highlights the hit results in the native AppXtender viewer. AppXtender highlights the hit results in KeyView. AppXtender highlights the hit results in KeyView. AppXtender highlights the hit results in KeyView. Building Full-Text Search Expressions You can select one of the full-text search options for your query. The options include: All Words, Any Words, Exact Phrase, or Expression. This section describes how to build full-text search expressions with xplore full-text engine. Note: You must enclose the words and text strings in single quotation marks. The following is a list of expression operators supported by the xplore full-text engine: Operator Example Within the Documents Highlighted Occurrences Detail ftand AEX ftand Simon Both AEX and Simon AEX and Simon When you use the ftand operator between two words, AppXtender searches for documents that contain both words. 9

10 Starting AppXtender Index Agent Implementation Operator Example Within the Documents Highlighted Occurrences Detail ftor AEX ftor Simon Either AEX or Simon AEX and Simon When you use the ftor operator between two words, AppXtender searches for documents that contain at least one of the words. ftnot AEX ftand ftnot Simon AEX but not Simon AEX When you use the ftnot operator before a word, AppXtender searches for documents that do not contain the word.? A?X with wildcards A string of 3 consecutive characters, in which the first must be A, the second may be any character, and the last must be X Matching words When you use a question mark (?) in a string of text with wildcards, AppXtender searches for documents that contain that string of text, except when other any single character replaces the question mark. (If you are searching for multiple words, make sure that another operator is used between each word.) * S* with wildcards Any string that begins with S Matching words When you use an asterisk (*) in a string of text with wildcards, AppXtender searches for documents that contain that string of text, except when any other characters or even lack of a character replaces the asterisk. (If you are searching for multiple words, make sure that another operator is used between each word.) () AEX ftand ( Simon ftor Schiff ) Either Simon or Schiff first, and of those documents, AppXtender finds ones that also have AEX AEX, Simon and Schiff By default, AppXtender interprets a search expression from left to right, interpreting each search operator one at a time, and narrowing the results with each subsequent search. You can use parentheses to 10

11 Starting AppXtender Index Agent Implementation Operator Example Within the Documents Highlighted Occurrences Detail group search expressions to control the order in which AppXtender performs these searches. When you use parentheses in an expression, AppXtender searches first for documents that contain the expression in parentheses. Using Multiple Processors AppXtender Index Agent full-text indexes documents with EMC Documentum xplore. AppXtender Index Agent retrieves the document content and saves it in an index job temporary file folder. It then sends an index request to the xplore engine. After receiving the index request, the xplore engine generates a request ID for Index Agent to continuously process documents. During this time, the xplore engine asynchronously processes the index request. Once the xplore engine finishes processing the index request, AppXtender Index Agent deletes all the files from the index job temporary folder, and finishes indexing the document. Note: xploretab, page 30 provides more information about configuring index job temporary file folders. If AppXtender Index Agent is installed on a computer with multiple processors, you can increase the AppXtender Index Agent processing speed by configuring it to use all available processors to perform full-text index operations on documents. You can configure up to 16 full-text processors (16 threads) for each AppXtender Index Agent. Each processor processes documents synchronously. Multiple Processors can be configured under the Legacy Full-Text tab for xplore full-text engine, and multiple processors can be specified for the OCR engine under the OCR tab. Legacy Full-Text Tab, page 28 and OCR Tab, page 31 provide more information about configuring multiple processors. Creating the Impersonation Account The AppXtender Index Agent impersonation account grants security privileges to AppXtender Index Agent and is essential for AppXtender Index Agent to operate properly. AppXtender Index Agent uses the impersonation account to access remote and server resources, including AppXtender document paths, the License Server, the registry, and the image repository (in some cases, the DiskXtender Document Server). Because no login is required for the AppXtender Index Agent service to run, some security context needs to be established, and this is done with the AppXtender Index Agent impersonation account. The AppXtender Index Agent uses the same security provider as AppXtender to log in and access 11

12 Starting AppXtender Index Agent Implementation images and documents. The impersonation account must have the Log on as a service advanced user right. If any of the components used by AppXtender Index Agent will be located remotely, the AppXtender Index Agent impersonation account must be added as a domain account on a domain that is trusted by the AppXtender Index Agent, License Server, and Document Server (in many cases, the DiskXtender Server). For example, you may install AppXtender Index Agent on one computer and install SQL Server and AppXtender Desktop on another computer to allow for faster processing. The domain account and the local accounts must be the same Windows account in order for account rights to translate across the domain to the local machines. Also, in a native Active Directory environment, when domain-level permissions are active, they override local permissions, so it may be necessary to grant the two advanced rights at the domain level. It is recommended that the AppXtender Index Agent impersonation account be created as a domain account and then added locally. By adding the domain account to the local AppXtender Index Agent, you ensure that only one account is used. The Test button can be used to quickly verify information regarding the AppXtender Index Agent impersonation account and the current active account. You must create and correctly configure an impersonation account to be used to run the AppXtender Index Agent. Creating the AppXtender Index Agent Impersonation Account 1. Log on to the computer on which AppXtender Index Agent has been installed. Make sure that you are logged on as a member of the local Administrators group. 2. From the Windows Start menu, select Programs and then Administrative Tools. From the Administrative Tools menu, select Computer Management. The Computer Management console appears. 3. Expand the System Tools tree item, and then expand Local Users and Groups. Right-click Users, and select New User. The New User dialog box appears. 4. Enter a User name, Full name, Description, Password, and Password confirmation in the available text boxes. 5. Disable the User must change password at next logon checkbox, and enable the Password never expires checkbox. 6. Click Create. The new account is created. Click Close to exit the New User dialog box. When you return to the Computer Management console, the list of local users should appear in the console s right-side pane (if it does not appear, click the Users folder, located beneath Local Users and Groups). 7. Right-click the account you just created and select Properties. The User Properties pages appear. 8. Click the Member Of tab. 9. Click Add. The Select Groups dialog box appears. 12

13 Starting AppXtender Index Agent Implementation 10. Select Administrators from the top Groups pane, and click Add. DOMAINNAME \Administrators should appear in the bottom pane. Click OK to return to the User Properties page. Click OK again to exit the User Properties page and return to Computer Management. 11. Close Computer Management. You now need to configure the new account to have the following advanced right: Log on as a service. Adding Advanced Rights to the AppXtender Index Agent Impersonation Account The AppXtender Index Agent impersonation account requires the advanced right of Log on as a service. 1. Open the Local Security Settings utility. From the Start menu, select Programs, then Administrative Tools. From the Administrative Tools menu, choose Local Security Policy. The Local Security Settings Console appears. Note: This method may differ depending on your operating system. 2. Expand the Local Policies folder, and then select User Rights Assignment. All user rights should appear in the right-side pane of the Local Security Settings Console. 3. Right-click Log on as a Service and choose Security. The Local Security Policy Setting dialog box appears. 4. Click Add. The Select Users or Groups dialog box appears. 5. Select the account you created to use as the Index Agent impersonation account from the top pane, and click Add. It should appear in the bottom pane as DOMAINNAME\Account Name. Click OK. Click OK again to return to the Local Security Settings Console. 6. Close the Local Security Settings Console. Your impersonation account for AppXtender Index Agent can now be entered onto the Setup tab of the AppXtender Index Agent Configuration dialog box. 13

14 Starting AppXtender Index Agent Implementation 14

15 Installing, Configuring, and Using xplore for AppXtender Chapter 2 EMC Documentum xplore for Windows is the new full-text engine for the current version of AppXtender. This chapter describes the following: Installing xplore and Configuring the Primary Server Instance, page 15 Configure Indexserverconfig.xml for AppXtender Full-Text, page 17 Creating xplore Domains and Collections, page 18 Configuring Applications to use xplore Full-text Engine in Application Generator, page 18 Installing xplore and Configuring the Primary Server Instance This section describes how to install xplore and configure the primary server instance. Installing xplore Download EMC Documentum xplore for Windows x64 from the EMC Support website (registration required) at 2. Unzip the xplore_1.3_windows-x64.zip file to a temporary directory. Note: xplore 1.3 only supports Windows 2008 R2 SP1. 3. Run setup.exe to launch Installer and follow online instructions to complete the installation process. a. Where would you like to install: Install xplore and its components to a root directory and choose a path that does not contain spaces. The default installation directory is C:\xPlore on Windows. On Windows, use a local path during installation. For a multi-instance configuration, this drive must be accessible to other xplore instances by UNC path. For example, you install to local path F:\xPlore. When you configure the primary instance, you specify the xplore 15

16 Installing, Configuring, and Using xplore for AppXtender data and config paths as \\hostname\f$\xplore\data and \\hostname\f$\xplore\config respectively. b. Watchdog Administrator Information: Enter an SMTP server and address. The SMTP server cannot be a Microsoft Exchange server. This step is optional. You can click Next to proceed and safely ignore the error prompt. You can set the SMTP server and address at a later time by setting the properties of the SendMailTask task in the file dsearch-watchdog-config.xml located in xplore_home\watchdog\config. The xplore watchdog service is a Windows service or daemon process (a standalone Java process) that is installed on each xplore host. The watchdog service monitors and checks the status of various xplore processes and sends an notification to the administrator 4. Create the primary xplore instance. Run xplore_home\setup\dsearch\configdsearch.bat and follow the online instructions to complete the installation process. a. Select Configuration Mode: Choose Create Primary Server Instance. b. Installation Owner Password: Installation owner password is required for setting up this instance. Installation Owner Domain is the domain for the repository with which you want to associate this xplore instance. Default: The current host name. c. Enter the fully qualified domain name of the xplore primary instance host. d. Server Instance Information: Server Name must be unique in your xplore federation of instances (default PrimaryDsearch). Base Port default is The next 100 consecutive ports must be available. Password for Admin User is also the xdb Administrator password. Note: The administrator password can be up to 127 characters long and must follow these rules: Cannot begin with # (Sharp) Cannot contain the following special characters: > < % ^ & ( ) " e. Dsearch Data and Config Directories. For a single-instance environment, specify a local directory on this host for better performance. For multiple instances, these directories must be accessible and writeable by all xplore instances. For NAS-based storage, you can map the same storage to the same path for all hosts. If you want to deploy multiple xplore instances or there might be a future need to do so, you must specify UNC paths accessible to other xplore instances; for example, \\hostname\d$\xplore\data and \\hostname\d$\xplore\config (when xplore_home is D:\xPlore). Note: It is highly recommended that you use UNC paths for these directories even if you are not deploying multiple xplore instances at the moment. Data directory stores the xdb transaction log of the primary instance and the default collection storage area. (The storage area is different from the index agent temporary storage location.) Default: xplore_home/data. Configuration Directory stores configuration information. Default: xplore_home/config. 16

17 Installing, Configuring, and Using xplore for AppXtender 5. Start the primary xplore instance: In xplore_home/jboss_version/server/, run the script startprimarydsearch.cmd. On Windows, you can also start the service Documentum xplore PrimaryDsearch. Note: On Windows, if you start an xplore instance using the command, the status of the xplore service may not be accurately reflected in the Windows Management Console. 6. Test whether the primary instance is running. Open your web browser and enter the following URL. host is the DNS name of the primary instance host and port is the xplore port (default: 9300). For example: If the instance is running, you see a message like the following: The DSS instance PrimaryDsearch [version= ] normal After installing the primary instance, you can optionally proceed to install secondary and spare instances. Installing xplore 1.3 Patch 13 package Note: xplore 1.3 Patch 13, or a latest patch, is required for AppXtender. 1. Download Documentum xplore for Windows x64 Patch 13 from the EMC Support website (registration required) at 2. Make sure you stop xplore before proceeding with the Patch installation. 3. Unzip the xplore 1.3 Patch 13 package to a temporary directory. 4. Refer to readme.txt for more information about installing the xplore 1.3 Patch 13 package. 5. Restart xplore when the installation is completed. Configure Indexserverconfig.xml for AppXtender Full-Text 1. Verify that the xplore server is stopped before modifying the configuration. To stop the xplore server, either go to Windows Service and right-click Stop or run stopprimarydsearch.cmd from C:\xPlore\jboss5.1.0\server. 2. From C:\xPlore\config, open indexserverconfig.xml and replace DefaultSummary with AXSummaryEntryPoint in the following line: <property name="query-summary-default-highlighter" value= "com.emc.documentum.core.fulltext.indexserver.services.summary.defaultsummary"/> to specify an AppXtender highlighter. 3. Add the following sub paths to indexserverconfig.xml: <sub-path sortable="true" leading-wildcard="false" compress="false" boost-value="1.0" description="used to support AX FullText Search" include-descendants="false" returning-contents="true" value-comparison="true" 17

18 Installing, Configuring, and Using xplore for AppXtender full-text-search="false" enumerate-repeating-elements="false" type="integer" path="dmftmetadata/dm_document/ax_page_num"/> <sub-path sortable="true" leading-wildcard="false" compress="false" boost-value="1.0" description="used to support AX FullText Search" include-descendants="false" returning-contents="true" value-comparison= "true" full-text-search="false" enumerate-repeating-elements="false" type="integer" path="dmftmetadata/dm_document/ax_doc_id"/> 4. Run startprimarydsearch.cmd located in C:\xPlore\jboss5.1.0\server to start the primary xplore instance, or start it from Windows Services. 5. Enter the on your web browser to test whether the primary instance is running. Note: You may get an HTTP 500 Page unavailable error message if you load the dsearch page right after starting the xplore PrimaryDsearch service. Wait for a couple of minutes before reloading the page. DSS instance PrimaryDsearch[version=1.3] normal appears if the primary instance is running. Creating xplore Domains and Collections xplore domains are separate, independent, logical or structural grouping of collections that are managed through Data Management in the xplore Administrator. Collections are a logical groups of XML documents that are physically stored in an xdb detachable library. All documents submitted for indexing are assigned to a collection. A collection generally contains one category of documents. When you configure xplore full text engine in Application Generator, you must specify xplore Domains and Collections. 1. From the xplore Administrator site ( select Data Management in the left panel and click New Domain in the right panel. 2. Enter the domain name in the Name field. 3. Set the Default Document Category and Storage Location. 4. Click Save, a new domain is created to the Data Management tree list on the left-pane. 5. Click on the newly created domain to assign the collection(s) for AppXtender applications. 6. Click New Collection and enter the collection name in Name field. Click Save. 7. Click Save. Configuring Applications to use xplore Full-text Engine in Application Generator 1. Log into Application Generator. 2. Open the AppXtender application that you want to setup the xplore full-text configuration. 18

19 Installing, Configuring, and Using xplore for AppXtender Note: Make sure you have created and configured the AppXtender Index Agent impersonation account before proceeding with the AppXtender Generator configuration. Creating the Impersonation Account, page 11 provides more information about creating the impersonation account. 3. Enter the xplore Server URL, xplore Domain Name, and the Collection Name for this application. Note: Include the port number (default is 9300) at the end of the URL. You are recommended to use a static IP for your xplore Server URL. 4. Click Apply to close out of Application Generator. 19

20 Installing, Configuring, and Using xplore for AppXtender 20

21 Chapter 3 Configuring AppXtender Index Agent This chapter describes how to configure the AppXtender Index Agent. Configuration Overview, page 21 Data Sources Tab, page 23 Setup Tab, page 24 Queues Tab, page 26 Legacy Full-Text Tab, page 28 xploretab, page 30 OCR Tab, page 31 Log File Tab, page 32 Misc Tab, page 33 Configuration Overview The following configurations are required before you use the AppXtender Index Agent. Note: The service continues running while you configure the AppXtender Index Agent. When you click OK in the AppXtender Index Agent Configuration dialog box, the AppXtender Index Agent service is restarted automatically. To configure the AppXtender Index Agent: 1. Open the AppXtender Index Agent Configuration dialog box: If the AppXtender Index Agent has not been started, from the Windows Start menu, select Programs > ApplicationXtender Desktop > Index Agent. If the AppXtender Index Agent has been started, right-click the AppXtender Index Agent icon in the status area of the Windows task bar. A shortcut menu appears. From the shortcut menu, select Configure AppXtender Index Agent. The AppXtender Index Agent Configuration dialog box appears. 2. Verify that each of the configuration requirements in the following table has been configured for the AppXtender Index Agent. This step is necessary for you to use the AppXtender Index Agent. The following table refers you to the appropriate sections topics for more information: 21

22 Configuring AppXtender Index Agent Configuration Requirement Locate and refresh the list of data sources, and set one data source as default. Specify an AppXtender login account. Specify an impersonation account. For More Information, See Data Sources Tab, page 23 Specifying an AppXtender Login Account, page 24 Specifying an AppXtender Index Agent Impersonation Account, page 25 Create at least one available queue. Queues Tab, page 26 Specify a temporary folder path for xplore engine to search and process the job. If AppXtender submit documents to the AppXtender Index Agent for OCR processing, select an OCR engine. xploretab, page 30 OCR Tab, page If necessary, the AppXtender Index Agent Configuration dialog box allows you to perform the following additional configuration functions: Configuration Option You can use the Legacy Full-Text tab to select the page types that can be converted to text. You can also specify multiple processors (up to 16 CPUs) for xplore full-text engine. You can use the OCR tab to select the OCR result format and specify multiple processors (up to 4 CPUs) for the OCR engine. You can use the Log File tab to set a log file path. You can use the Misc tab to configure stopping of jobs on error. You can use the Misc tab to configure the automatic removal of a job from a queue when the job is finished. For More Information, See Legacy Full-Text Tab, page 28 OCR Tab, page 31 Log File Tab, page 32 Configuring AppXtender Index Agent Error Behavior, page 33 Remove Job from Queue when Finished, page After you have finished configuring the AppXtender Index Agent, click OK. Note: Once you have configured the AppXtender Index Agent and created an OCR or full-text queue, make sure that the AppXtender Document Manager workstations in which users need to submit jobs to these queues are configured to allow OCR or full-text processing on the OCR and Full-Text tabs, respectively, of the AppXtender Document Manager Configuration dialog box. In addition, if the workstation that is submitting a full-text or OCR job has the local write path enabled on the Paths tab of the AppXtender Document Manager Configuration dialog box, or has documents stored in a local write path, the jobs result in error. If you update the configuration settings on the workstation, you must exit AppXtender Document Manager and re-login for the changes to take effect. The EMC ApplicationXtender Desktop Installation Guide provides more information about configuring AppXtender Document Manager Workstations. 22

23 Configuring AppXtender Index Agent Data Sources Tab The Data Sources tab allows you to locate and refresh the list of data sources and set one data source as default. The AppXtender Index Agent does not run unless a data source has been set as default. To configure a data source for the AppXtender Index Agent: 1. In the AppXtender Index Agent Configuration dialog box, click the Data Sources tab. 2. Click Locate. The Data Link Properties dialog box appears. On the Provider and Connection tabs, specify the database. (For more detailed instructions, see refer to the AppXtender Data Sources chapter of the EMC ApplicationXtender Core Components Administrator s Guide.) 3. Click OK. The list of data sources appears. 4. Double-click a data source to set it as default, or select the data source and click Set as Default. A white check mark within a black background appears next to it. Note: The data source that you set as default for the AppXtender Index Agent can be different than the data source that you set as default for AppXtender Document Manager. 5. Click the Setup tab. Setup Tab, page 24 provides more information on configuring the Setup tab. Note: If users submit documents to the AppXtender Index Agent from data sources other than the one set as default in the AppXtender Index Agent Configuration dialog box, see Documents Submitted from Other Data Sources, page 23. Documents Submitted from Other Data Sources When users submit documents to the AppXtender Index Agent from data sources other than the one set as default in the AppXtender Index Agent Configuration dialog box, the AppXtender Index Agent does not process the job until you perform the following procedure: 23

24 Configuring AppXtender Index Agent To allow the processing of documents from other data sources: 1. In the AppXtender Index Agent Configuration dialog box, set as default the data source that contains the document that has been submitted to the AppXtender Index Agent. For instructions, see Data Sources Tab, page If you have changed the data source that is set as default in the AppXtender Index Agent Configuration dialog box, you must re-enter the AppXtender login account information on the Setup tab. For instructions, see Setup Tab, page Click the Queues tab. Confirm that the correct queues have been configured for the selected data source. For instructions, see Queues Tab, page Click OK. Setup Tab The Setup tab allows you to specify the user account that the AppXtender Index Agent needs in order to access AppXtender. It also allows you to specify an AppXtender Index Agent impersonation account. You can also select the type of Index Agent role to handle OCR and Full-text jobs. Specifying an AppXtender Login Account, page 24 Specifying an AppXtender Index Agent Impersonation Account, page 25 Specifying an Index Agent Role, page 26 Specifying an AppXtender Login Account You must specify an AppXtender login account for the AppXtender Index Agent. To specify an AppXtender login account: 1. In the AppXtender Index Agent Configuration dialog box, click the Setup tab. 24

25 Configuring AppXtender Index Agent 2. Under AppXtender Login, specify the user account that the AppXtender Index Agent needs in order to access AppXtender. 3. In the User Name text box, enter a user name that is valid for the AppXtender Index Agent default data source. Note: Your login procedure may vary depending on the security provider in use for the current data source. The EMC ApplicationXtender Core Components Administrator s Guide provides more information about the login procedure. 4. In the Password text box, enter the password for the user name that you entered. 5. If you have not yet specified an AppXtender Index Agent impersonation account, see Specifying an AppXtender Index Agent Impersonation Account, page Click the Queues tab. Queues Tab, page 26 provides more information on configuring the Queues tab. Specifying an AppXtender Index Agent Impersonation Account You must create and specify an AppXtender Index Agent impersonation account. The impersonation account grants security privileges to AppXtender Index Agent and is essential for the AppXtender Index Agent to operate properly. AppXtender Index Agent uses the impersonation account to access remote and server resources. (For instructions on creating an AppXtender Index Agent impersonation account, see Creating the Impersonation Account, page 11.) To specify an AppXtender Index Agent impersonation account: 1. In the AppXtender Index Agent Configuration dialog box, click the Setup tab. 2. Make sure that the AppXtender Index Agent impersonation account has been created. For instructions, see Creating the Impersonation Account, page 11. Note: Because an actively logged on account is not required for AppXtender Index Agent to be running, some security context must be established. This is done with the AppXtender Index Agent impersonation account. AppXtender Index Agent sessions use the same security model as AppXtender to log in and access images and documents. It is recommended that the AppXtender Index Agent impersonation account be created as a domain account and then added locally. By adding the domain account to the local AppXtender Index Agent, you can ensure that only one account is used. 3. Type the domain name or server name in the Domain/Server text box. The AppXtender Index Agent searches in the specified domain or server for the AppXtender Index Agent impersonation account. 4. In the Account and Password text boxes, type the user name and password for the AppXtender Index Agent impersonation account. This password is case-sensitive. You can click Test to verify the impersonation account. The software first verifies whether the account is valid, by checking the user name and password. Then the software verifies whether the account has appropriate rights. If not, a message appears indicating which rights are missing. 25

26 Configuring AppXtender Index Agent 5. Click the Queues tab. Queues Tab, page 26 provides more information on configuring the Queues tab. Specifying an Index Agent Role The Index Agent Role allows you to specify the index agent to handle OCR or Full-Text jobs. Only the selected job type will be handled by the Index Agent. By default, the Index Agent is capable of handling both OCR and Indexing jobs. But to increase performance, you have the option to focus on one type of job. This boosts the overall job performance of multiple Index Agents setup. To specify an Index Agent Role: 1. In the AppXtender Index Agent Configuration dialog box, click the Setup tab. 2. From the drop-down list, select the Index Agent Role. Option Both Index and OCR Only OCR Only Index If you select this option, the Index Agent handles OCR and Index jobs. If you select this option, the Index Agent only handles OCR jobs. If you select this option, the Index Agent only handles Indexing jobs. 3. Click OK to exit the configuration. Queues Tab Queues are used by the AppXtender Index Agent to collect full-text jobs and OCR jobs. A full-text job is created in a full-text processing queue when a document is submitted to the AppXtender Index Agent for full-text indexing. An OCR job is created in an OCR processing queue when a document is submitted for OCR processing. Initially, there are no queues for the AppXtender Index Agent. Therefore, to add documents to the full-text database, you must create full-text queues. To submit OCR jobs to the AppXtender Index Agent, you must create OCR queues. The Queues tab allows you to add to the Available Queues list by creating new queues. To initiate processing of an available queue, you must move it to the Processing Queues list. When you add a document to the full-text database, you select an Available Queue for the document. The document is not processed, however, until that queue is added to the Processing Queues list. Once a queue has been added to the Processing Queues list, the documents in the queue are either processed and added to the full-text database or processed with OCR. 26

27 Configuring AppXtender Index Agent You can add new queues and move the queues from the Available Queues list to the Processing Queues list or remove queues from the Processing Queues list. Adding a New Queue Adding a new queue adds a new entry for the queue in the AE_QUEUE table in the AppXtender database for storing full-text database information for AppXtender documents. You can choose whether a queue will be a full-text queue (for documents that are already in text format or for images that have been processed using OCR) or an OCR queue (for documents that must be processed to extract text from an image). To add a new queue: 1. In the AppXtender Index Agent Configuration dialog box, click the Queues tab. 2. Click New. The Create New Queue dialog box appears. 3. In the Name text box, type a name. 4. From the Type drop-down list, select a type (OCR or FULL TEXT). OCR queues can be used by users submitting documents for OCR; full-text queues can be selected when a user submits a document for full-text indexing. 27

28 Configuring AppXtender Index Agent 5. In the Description text box, you can enter a description. This description appears in the AppXtender Queue Manager with the queue name. 6. Click Create. The new queue is listed in the Available Queues list. 7. For each additional queue you want to create, repeat Step 3 to Step Click Close. The Create New Queue dialog box closes. Adding Queues to be Processed In order for a queue to be used by AppXtender users, you must add it to the Processing Queues list. To add a queue to the Processing Queues list: 1. In the AppXtender Index Agent Configuration dialog box, click the Queues tab. 2. In the Available Queues list, select the name of the queue you want to add. Note: You can select multiple sequential queues by selecting the first queue in the sequence, pressing [SHIFT], and selecting the last queue in the sequence. You can select multiple non-sequential queues by pressing [CTRL] and selecting each queue, one at a time. 3. Click the right arrow button. The name of the queue is moved to the Processing Queues list. 4. Repeat the above procedure for each queue in the Available Queues list you want to add to the Processing Queues list. Removing Queues from the Processing Queues List When you remove a queue from the Processing Queues list, the definition of the queue is maintained for later use, but the queue is deactivated. Any jobs submitted to the queue for processing are not processed until the queue is returned to the Processing Queues list. To remove a queue from the Processing Queues list: 1. In the AppXtender Index Agent Configuration dialog box, click the Queues tab. 2. In the Processing Queues list, select the name of the queue you want to remove. 3. Click the left arrow button. The name of the queue is returned to the Available Queues list. 4. Repeat the above procedure for each queue you want to remove from the Processing Queues list. Legacy Full-Text Tab AppXtender supports Verity K2 and ProIndex legacy full-text engines that have been configured in earlier versions. The Legacy Full-Text tab is to access legacy full-text engines Verity K2 and ProIndex. Note: AppXtender Index Agent converts image, PDF, RTF, HTML, and foreign file types to text only if configured to do so on the Legacy Full-Text tab of the AppXtender Index Agent Configuration 28

29 Configuring AppXtender Index Agent dialog box. The configuration settings for non-image file types and non-pdf file types apply only to applications using the ProIndex full-text engine. These configuration settings do not apply to applications using K2 or xplore full-text engine because K2 and xplore do not need to convert RTF, HTML, or foreign files before processing them. To configure the Legacy Full-Text tab: 1. In the AppXtender Index Agent Configuration dialog box, click the Legacy Full-Text tab. 2. Enable or disable page types to be converted as appropriate. The page types are as follows: Bi-level images (black and white) PDF files (Portable Document Format) RTF files (Rich Text Format) HTML files (Hypertext Markup Language, with either an.html or.htm file extension) Foreign files (all types of files that AppXtender does not natively support) All of these file types need to be converted before the ProIndex full-text engine can process them. However, non image-only PDF files, RTF files, HTML files, and foreign files do not need to be converted for K2 full-text engine and xplore full-text engine. The EMC ApplicationXtender Document Manager User s Guide provides more information on these file types, as well as on which files are supported in AppXtender and which are considered foreign files. Note: Pages or documents added to AppXtender using the Insert Object command cannot be submitted to the AppXtender Index Agent for full-text indexing or OCR. If a user submits a document containing a mixture of pages that can be converted and pages that cannot be converted, only those page types enabled here will be converted and indexed. 3. If the computer on which the AppXtender Index Agent is installed has multiple processors, specify the number of processors to be used for full-text indexing. Using Multiple Processors, page 11 provides more information about multiple processors. Note: If K2 is the full-text engine, the maximum number of processors is 4. 29

30 Configuring AppXtender Index Agent 4. Click the xplore tab. xploretab The xplore tab is where you enter the path of the temporary file folder for xplore to search and process the job. To configure the xplore tab: 1. Open Index Agent, go to the AppXtender Index Agent Configuration, and click the xplore tab. Note: If the computer on which the AppXtender Index Agent is installed has multiple processors, specify the number of processors (up to 16 CPUs) to be used for full-text indexing with xplore engine in the Legacy Full-Text tab. Using Multiple Processors, page 11 provides more information about using multiple processors. 2. Specify a temporary folder path. Based on this path, xplore can search and process the job. The Index Agent will extract and store the content in this temporary file folder. Note: For xplore Instance account, if an xplore instance is launched from Windows Service, it is the Log On account of the Windows server. If an xplore instance is launched from xplore_home/jboss_version/server/startprimarydsearch.cmd, it is the account that runs the command. Index Agent impersonation account must have write access to this temp folder and xplore instance account must have read access. Specifying an AppXtender Index Agent Impersonation Account, page 25 provides more information about Index Agent impersonation account. 3. Enter the minutes in Check the index job status every (minutes). After submitting the document to xplore, Index Agent will check the job index status every 1 minute (by default), or any minutes you set in the field. 4. Enter the maximum number of concurrent jobs in Max number of concurrent jobs. 30

31 Configuring AppXtender Index Agent Note: Index Agent limits the concurrent jobs submitted to xplore. Before the indexing job is completed in xplore, Index Agent will stop submitting the document to xplore if the submitted jobs reach the maximum number set in Max number of concurrent jobs. However, when Index Agent receives a complete status, it continues to submit documents to xplore. 5. Click the OCR tab. OCR Tab The OCR tab allows you to configure the OCR engine to be used by AppXtender and to select the format of the page that results when a page is processed using OCR. To configure the OCR tab: 1. In the AppXtender Index Agent Configuration dialog box, click the OCR tab. 2. Select an OCR engine from the drop-down list. The FineReader OCR Engine is always available as an OCR engine because it is installed with AppXtender Document Manager. Note: The OCR engine selected here is the engine used when a user submits a document for OCR using the Submit OCR job option in the AppXtender Document Manager Result Set shortcut menu or the Document menu. If the user selects OCR Entire Document from the Document Display shortcut menu or OCR from the Document menu, the OCR engine configured on the AppXtender Document Manager workstation is used. The OCR tab also allows you to select the OCR result format (the OCR output file format). 3. From the OCR Processors (1-4), select the number of processors. It should be equal to or less than the CPU cores of the machine. For example, if the machine has dual-core CPU, the recommended thread number is Click OK and the configuration will be saved. If you enter a number larger than 4, an error message displays when you click OK. Note: In earlier versions of AppXtender, FineReader 9.x could not support multiple OCR engine objects. This only allowed Index Agent to have one OCR thread. In the current version of 31

32 Configuring AppXtender Index Agent AppXtender, Index Agent supports multiple OCR threads through FineReader 10.5 and up to four CPU cores. 5. Under OCR Result Format, you have the following choices: If you want the OCR result format to be plain text, click Plain text. If you want the OCR result format to be RTF, click Rich Text Format (RTF). The following table describes each available format: Option Plain text Rich Text Format (RTF) Description The plain text format is an ASCII text format, readable on MS-DOS/Windows operating systems. Rich Text Format is a format that can be read on all platforms. This format should be used if the document text will be exported for use on non-ms-dos/windows systems. Rich Text Format retains most formatting. Note: This OCR Result Format setting applies only to documents submitted to an AppXtender Index Agent OCR queue. If an image is submitted to an AppXtender Index Agent full-text queue and the OCR option is enabled, the AppXtender Index Agent always converts it to plain text before full-text indexing it, regardless of any OCR Result Format setting. 6. Click the Log File tab. Log File Tab The AppXtender Index Agent automatically logs index job events to a log file. The Log File tab allows you to configure the Log File Path. 32

33 Configuring AppXtender Index Agent To configure the Log File Path: 1. In the AppXtender Index Agent Configuration dialog box, click the Log File tab. 2. Type a path in the Log File Path text box or use the Browse button to browse for a file. 3. Click the Misc tab. Misc Tab The Misc Tab allows you to configure miscellaneous settings. Configuring AppXtender Index Agent Error Behavior When an error occurs, you can configure whether the AppXtender Index Agent stops processing the current job or continues processing the current job. To configure AppXtender Index Agent error behavior: 1. In the AppXtender Index Agent Configuration dialog box, click the Misc tab. 2. You have the following choices: If you want the AppXtender Index Agent to stop a job if an error is encountered and move to the next job in the queue, click the Abort current job when error occurs checkbox to enable the option. If you want the AppXtender Index Agent to continue processing the current job after encountering an error, click the Abort current job when error occurs checkbox to disable the option. 3. Click OK. The changes that you have made to AppXtender Index Agent configuration are applied and the dialog box closes. 33

34 Configuring AppXtender Index Agent Remove Job from Queue when Finished This option removes each job from its queue when it is successfully finished. When the job has been removed, you can no longer view it in the AppXtender Index Agent Monitor or Queue Manager. Note: If the AppXtender Index Agent configuration is opened and the Misc. tab is selected, the option Remove Job from Queue when Finished is disabled and grayed out. To set job removal on completion: 1. Start the AppXtender Index Agent. a. Open the Index Agent Configuration dialog box. b. Click on the Queues tab. c. Click on the Misc tab. 2. The option Remove job from queue when finished can be unchecked to disable this option, or checked to enable this option. 3. Click OK. The changes that you have made to AppXtender Index Agent configuration are applied and the dialog box closes. 34

35 Chapter 4 Managing the AppXtender Index Agent This chapter explains how to migrate index data to xplore, resubmit documents, and control Index Agent. Resubmitting Documents to the AppXtender Index Agent, page 35 Controlling the AppXtender Index Agent, page 40 Resubmitting Documents to the AppXtender Index Agent If you have changed the full-text engine for an application, keep in mind that full-text searching does not return any documents in this application until you submit them to the AppXtender Index Agent, even if they have already been full-text indexed by the previous engine. However, if you submit documents to the AppXtender Index Agent from AppXtender Document Manager result set, the number of documents that can be submitted at a time is limited. The EMC ApplicationXtender Desktop Installation Guide provides more information on this limitation. Consider using the AppXtender Full-Text Index Wizard to submit documents to the AppXtender Index Agent for full-text indexing, rather than submitting them from AppXtender Document Manager result set. This wizard allows you to submit more documents to the AppXtender Index Agent at a time than the AppXtender Document Manager result set allows. To use the AppXtender Full-Text Index Wizard: 1. From the Windows Start menu, select Programs > ApplicationXtender Desktop > Full Text Index Wizard. The wizard appears, starting with the Data Source Selection page. 35

36 Managing the AppXtender Index Agent 2. Select the data source in which you want to process documents. In the User Name and Password text boxes, enter your user name and password. (This user account must have the Administrator privilege.) Click Next. The Application Selection page appears. Note: If the application used Verity K2 or ProIndex in earlier versions of AppXtender, then you have the option of migrating the full-text index data to xplore. Managing Legacy AppXtender Full-Text Index Data, page 37 provides more information about migrating full-text index data to xplore or continue using Verity K2 or ProIndex engine. 3. Select the application in which you want to process documents. Click Next. The Queue Selection and Other Options page appears. This page lists the queues that are available for processing. 4. Under Queue Selection, select the queue in which you want to process documents. 5. If you want to add another queue, click Add. The Create New Full-text Queue dialog box appears. 36

37 Managing the AppXtender Index Agent 6. In Queue Name enter a name for the new full-text queue. You can also enter a description in the Description text box. Click OK. The Queue Selection and Other Options page reappears. 7. Click Next. The Query Documents page appears. 8. Enter criteria to match the documents that you want to process. (To select all documents in the application, leave all search fields blank.) Click Next. The Status page appears and the selected documents are submitted to the specified queue. When the documents is submitted, the Status page indicates the number of documents successfully submitted and the name of the queue to which they are submitted. 9. Click Finish. Managing Legacy AppXtender Full-Text Index Data If you have already integrated Verity K2 or ProIndex in earlier versions of AppXtender, you can migrate index data to use the new xplore full-text engine. The Full-Text Index Wizard keeps old engine information during index migration while performing a full-text search based on index 37

38 Managing the AppXtender Index Agent data from the previous engine. When the migration is completed, you can use the new engine to do full-text searches. The Full-text Migration Wizard gives you two options to manage legacy full-text index data. This section describes the following two options: Migrating to xplore Engine, page 38 Using Old (Verity K2 or ProIndex) Full-Text Engine, page 39 Migrating to xplore Engine 1. Shut down the Index Agent and the AppXtender Index Agent service before starting the migration of index data to xplore. 2. From the Windows Start menu, select Programs > ApplicationXtender Desktop > Full Text Index Wizard. Choose the application that you want to migrate the index data. 3. Choose Migrate to xplore engine. 4. Specify the following: xplore Server URL, xplore Domain Name, Collection Name. Creating xplore Domains and Collections, page 18 provides more information about xplore domains and collections. 5. Select Still use old engine for search during migration if you want to search documents with the original full-text engine. Click Next. Note: When the migration is complete, skip to step 10 and select Finish the migration process and switch to new engine for full-text search to switch to the new full-text engine. 6. Specify the Queue Selection, or you can Add a queue selection. 7. Click Next to submit all the documents for migration. 8. Click Finish to continue with the migration process. 9. Restart Index Agent to perform full-text jobs and migration process. 38

39 Managing the AppXtender Index Agent 10. Go back to the Full-text Index Wizard and click on the application that is being migrated. The Full Text Index Wizard displays the following two options: Finish the migration process and switch to new engine for full-text search: If you choose this and click Finish, the full-text index wizard removes the old engine information. The full-text search will be based on the new engine. Continue the migration process: If you choose this option and click Finish, the full-text index wizard will do nothing. 11. When the job is completed, click Finish. Using Old (Verity K2 or ProIndex) Full-Text Engine 1. From the Full-Text Index Wizard, choose the application you want to use the old engine for the index data. Choose Still use old engine for search during migration. This option allows you to keep and use existing Verity K2 or ProIndex engine information for full-text indexing and search. Note: Use old engine also lists the original full-text engine that was used for that particular application. 2. Click Next to proceed. 3. Specify the Queue Selection, or you can Add a queue selection. Click Next to submit all the documents to use the old engine. Note: If the selected application is using Verity K2 engine and contains documents that have already been processed by ProIndex, the Process only documents already indexed in ProIndex checkbox is available. Use this checkbox to specify whether you want to process only those documents. You have the following choices: If you want to process only the documents that have been indexed by ProIndex, select the checkbox. If you do not want to process the documents that have been indexed by ProIndex, clear the checkbox. Keep in mind that processing takes longer if this checkbox is enabled, because of the time it takes to determine which documents have already been processed. 39