Extended Search Administration

Transcription

1 IBM Extended Search Extended Search Administration Version 3 Release 7 SC

2

3 IBM Extended Search Extended Search Administration Version 3 Release 7 SC

4 Note! Before using this information and the product it supports, read the general information under Notices on page 227. Eighth Edition, May 2002 This edition applies to Version 3 Release 7 of the IBM Extended Search product and to all subsequent releases and modifications until otherwise indicated by new editions. Copyright International Business Machines Corporation 1997, All rights resered. US Goernment Users Restricted Rights Use, duplication, or disclosure restricted by GSA ADP Schedule Contract with IBM Corporation.

5 Contents Tasks Starting and stopping the serers Start the serer on UNIX Shut down the serer on UNIX Start the serer on Windows Shut down the serer on Windows Configuring the search domain Get started Configuration model Configuration roadmap Start the Administration interface Start the Administration applet Start the Administration application Naigate the interface Get help Configure serers Add another Extended Search serer Connect multiple Extended Search serers.. 18 Discoer data sources Run discoery as a standalone process Configure data sources Define natie fields Define mapped fields Set field usage controls Define data source parameters Define smart hitlist actions Export and import data sources Configure categories Create a category Categorize a data source Configure applications Define application-specific properties Export and import applications Configure links Define link parameters Define link actions Set up a remote EIP link Configure grammars Run configuration wizards Refresh Web source definitions Search my own Web sites Configure a Web crawler Refresh the search domain Use search templates to test changes Log and iew eents Enable eent logging View eent logs Enable custom broker logging Enable custom serlet logging Configure security options Enable impersonation Enable serlet-leel authentication Enable broker-leel authentication Enable agent-leel authentication Configure scalability options Separate brokers from the Web serer Configure remote links Partition agents and data sources Configure multiple brokers Configure performance options Configure copies of serer tasks Localize Notes data sources and agents Manage serer memory Manage data source connections Setting up a Web Client Configure the browser to launch Lotus Notes.. 60 Run the Web Client application Inoke the serlet Guidelines for new users Setting up a Notes Client Update the desclient.cfg file Configure an application in the CDB Create a client application database Define access controls Create an Application document Run the Setup Sources agent Schedule the Archie Docs agent Schedule the Scheduled Search agent Customize template documents Hide the design elements Verify the Notes Client setup NLS considerations for the Notes Client Saing and scheduling searches Sae and schedule queries Work with saed queries Access saed queries Saed query actions Fetch and store search results How documents are stored Fetch and store guidelines Monitoring serer actiity Run the Monitor in stand-alone mode View task details View a task summary Display data source statistics Log serer actiity Run the Monitor in the Administration interface 89 Refresh configuration data Performing backups and restores Running utilities Run DESAbort Run DESRefresh Concepts Configuration Actions Agents Applications Brokers Categories Data sources iii

6 Adding data sources Configuring data sources Date formats Discoery Predefined discoerers Discoering file system sources Discoering EIP sources Discoering LDAP sources Discoering Lotus Connectors sources Discoering Domino Domain Index sources 108 Discoering Domino.Doc sources Discoering Lotus Notes sources Discoering Microsoft Access sources Discoering Microsoft Exchange Serer sources Discoering Microsoft Index Serer sources 113 Discoering Microsoft Site Serer sources Discoering Microsoft SQL Serer sources 114 Discoering ODBC sources Eent logging Fields Resered field names Natie fields Mapped fields Grammars Predefined grammars Hitlist URL classes Links Predefined links National language support NLS configuration file content NLS configuration file access UTF-8 codeset Parameters Link parameters Data source parameters Application parameters Predefined parameters Web crawlers Web crawler configuration file Web crawler XML template Web sources Downloading Web source definitions Adding support for Web sites Security User authentication Authentication enforced by the Notes Client 146 Authentication enforced by the Web Client 147 Authentication enforced by the broker Access control Notes ACLs and impersonation Web Client access control Data source filtering Link-leel control Connecting to data sources Authentication enforced by an agent Message encryption Encryption of message authentication data 152 Encryption through SSL Notes database control Multiple client applications Database-leel access controls Document- and field-leel access controls 154 Scalability Assessing scalability Creating multiple client applications Separating brokers from the Web serer Separating agents from brokers Configuring remote links Assigning unique agent names Partitioning agents on a single machine Setting up multiple brokers Adantages of multiple brokers Partitioning the user population Broker network configurations Fully connected network Hub network Multi-hub network Performance Assessing performance Optimizing end-user performance Replicating databases Setting timeout alues Sorting search results Optimizing serer performance Understanding serer tasks Determining the optimum number of tasks 172 Localizing agents with data sources Reference Administration dialogs Application Properties Application-Specific Properties Assign Mapped Field to Data Source Category Properties Data Source Properties single data source 178 Data Source Properties multiple data sources 181 Discoer Data Sources File System Sources discoerer IBM Enterprise Information Portal discoerer 183 LDAP discoerer Lotus Connectors discoerers Lotus Domino 5.0 Domain Index discoerer 184 Lotus Domino.Doc discoerer Lotus Notes discoerer Microsoft Access discoerer Microsoft Exchange Serer discoerer Microsoft Index Serer discoerer Microsoft SQL Serer discoerer Microsoft Site Serer discoerer ODBC discoerer Sample Link discoerer Web Sources discoerer Grammar Properties Link Properties Mapped Field Properties Query Folder Properties Saed Query Properties Serer Properties Serer Position Properties System Properties Agent requirements i Extended Search Administration

7 Error messages Jaa serlet client error codes Notes client error codes LSX client error codes Serer side error codes Trace leel codes Log files Keyboard access Notices Copyright Trademarks Index Contents

8 i Extended Search Administration

9 Tasks This collection of topics represents the major actiities behind setting up, administering, extending, and maintaining an Extended Search system. Starting and Stopping the System Learn how to start the Extended Search serer components and how to gracefully shut down the system. Start the serer on UNIX on page 3 Shut down the serer on UNIX on page 4 Start the serer on Windows on page 5 Shut down the serer on Windows on page 6 Configuring the Search Domain Learn how to use the Administration interface to discoer data sources, configure options for how users can search and retriee information, and set adanced options to enhance, secure, and troubleshoot your Extended Search enironment. Understanding Configuration: Get started on page 8 Configuration model on page 8 Configuration roadmap on page 10 Start the Administration interface on page 10 Naigate the interface on page 14 Get help on page 16 Basic Configuration Tasks: Discoer data sources on page 19 Configure data sources on page 21 Configure categories on page 28 Configure applications on page 31 Refresh Web source definitions on page 40 Refresh the search domain on page 43 Use search templates to test changes on page 43 Log and iew eents on page 45 Adanced Configuration Tasks: Configure serers on page 16 Configure links on page 34 Configure grammars on page 38 Run configuration wizards on page 39 Search my own Web sites on page 41 Configure a Web crawler on page 42 Configure security options on page 48 Configure scalability options on page 52 Configure performance options on page 56 Setting Up a Web Client Learn about the Demo application proided with Extended Search and how to set up and run your own search applications through the Web Client. Configure the browser to launch Lotus Notes on page 60 Run the Web Client application on page 61 Inoke the serlet on page 61 Guidelines for new users on page 62 1

10 Setting Up a Notes Client Learn how to create a Lotus Notes database using a template proided with Extended Search and how to set up and run your own search applications through the Notes Client. Configure an application in the CDB on page 65 Create a client application database on page 65 Define access controls on page 66 Create an Application document on page 66 Run the Setup Sources agent on page 67 Schedule the Archie Docs agent on page 68 Schedule the Scheduled Search agent on page 69 Customize template documents on page 69 Hide the design elements on page 71 Verify the Notes Client setup on page 72 NLS considerations for the Notes Client on page 72 Saing and Scheduling Searches Learn how to use the Extended Search Saed Queries applet to store queries in the configuration database and run them at a scheduled time. This feature also allows you to sae search results for sharing with other users, or store the search results into a file system. Sae and schedule queries on page 74 Work with saed queries on page 76 Fetch and store search results on page 80 Monitoring Serer Actiity Learn how to use the Extended Search Monitor to keep track of processes running on remote serers. You can run the Monitor as a stand-alone program, or run it from within the Administration interface. This feature allows you to make real-time adjustments to the system configuration. Run the Monitor in stand-alone mode on page 84 View task details on page 85 View a task summary on page 86 Log serer actiity on page 88 Run the Monitor in the Administration interface on page 89 Refresh configuration data on page 91 Backing Up and Restoring Configuration Data Learn how to use the backup and restore programs proided by Extended Search to protect your configuration data. Performing backups and restores on page 92 Running Utilities Learn about utilities for shutting down or refreshing the system in the Windows enironment. Run DESAbort on page 94 Run DESRefresh on page 94 2 Extended Search Administration

11 Starting and stopping the serers If you installed Extended Search on IBM AIX or Sun Solaris, use the following procedures to start and stop the Extended Search serer: Start the serer on UNIX on page 3 Shut down the serer on UNIX on page 4 If you installed Extended Search on Microsoft Windows NT or Windows 2000, use the following procedures to start and stop the Extended Search serer: Start the serer on Windows on page 5 Shut down the serer on Windows on page 6 Start the serer on UNIX Use the following procedure to start an Extended Search system. The order of these steps is critical. Note that if you use Domino as your Web serer and Web application serer, Domino requires you to start both the Domino and Extended Search serers with the same user ID. You must also set certain enironment ariables in Domino before discoering or searching data sources. For details, see Extended Search Troubleshooting, which is aailable on the Resources page of the Extended Search web site ( 1. On the machine that hosts DB2 and the Extended Search configuration database: a. Ensure that the JDK or JRE that you installed is included in the.profile file. b. Ensure that the DB2 serer has been started and is running. 2. If necessary, follow the steps below to start the Extended Search RMI Serer. If you are using a Domino Web serer, you must perform these steps. If you are using WebSphere Application Serer and you did not deploy the Extended Search Enterprise JaaBeans (EJB) when installing Extended Search, you must perform these steps. If you are using WebSphere Application Serer and the Extended Search EJB, do not perform these steps. a. Log in to the machine that hosts the Extended Search configuration database. b. Change to the Extended Search /bin directory. The default path is /opt/extendedsearch3.7/bin. (On a Solaris platform that uses Domino as the Web Application Serer, the recommended path is /home/serer/extendedsearch3.7/bin.) c. Enter the following command:./desrmistart If you prefer, you can enter the following command to start the RMI serer in the background. This feature allows you to exit out of the telnet session without shutting down the serer../desrmistart -bg 3. Start the Web Serer and Web Application Serer on each machine where you installed the Extended Search Web Serer component. If you need assistance, consult the Domino or WebSphere product documentation. Note that it may take some time to start WebSphere Application Serer, especially if you enabled the option to load serlets at startup. Tasks 3

12 4. Start the Administration interface (for assistance, see the procedure on page 10). This action helps alidate your ability to communicate with the Web serer and proides eent information that can help resole potential startup problems. Using Authentication If you installed the Extended Search Web Serer component on a Web Serer where authentication has been enabled (or anonymous access has been disabled), you must update the following enironment ariables in the desstart script to identify the Extended Search administrator user ID and password before starting the Extended Search Serer: ESAdminUserid ESAdminPassword 5. On each machine that hosts an Extended Search Serer, log in by using a user ID that has write access to the Extended Search bin directory. If you are running Extended Search with a Domino Web serer, you should start both the Extended Search serer and Domino serer under the same user ID, and this user ID must be the Domino administrator user ID. 6. Make sure you are in the Extended Search /bin directory, and enter the following command to start Extended Search:./desStart If you prefer, you can enter the following command to start Extended Search in the background. This feature allows you to exit out of the telnet session without shutting down the Extended Search serer../desstart -bg Shut down the serer on UNIX How you shut down the Extended Search system depends on whether you are currently running the Monitor to track serer actiity. If you started the Extended Search Monitor for this serer, select Serer Shutdown. You will be prompted to specify the ES Administrator user ID and password to complete this task. Note that if you started Extended Search as a background process, you must use the Monitor to shut down the serer. If you are not currently running the Monitor: 1. Change to the Extended Search /bin directory. The default path is /opt/extendedsearch3.7/bin. (On a Solaris platform that uses Domino as the Web Application Serer, the recommended path is /home/serer/extendedsearch3.7/bin.). 2. Enter the following command to shut down the Extended Search serer:./desstop If you use the RMI serer and started it as a background process, enter the following command to shut it down:./desrmistop 4 Extended Search Administration

13 Start the serer on Windows Use the following procedure to start an Extended Search system. The order of these steps is critical. Keep in mind that proxy settings are different for each Windows user. You must ensure that the user ID that you use to start the Extended Search serer is able to access the Internet. If necessary, see Extended Search Installation for instructions on setting up the proxy serer for use with the Extended Search serer. 1. On the machine that hosts DB2 and the Extended Search configuration database: a. Ensure that the JDK or JRE that you installed is included in the system path. b. Open the Windows Control Panel and then double-click Serices. Ensure that the DB2 serice has been started. 2. If DB2 is running on a machine that is a member of a domain, select DB2 Security Serer and click Startup. Select Autostart as the Startup Type and click OK. 3. If necessary, follow the steps below to start the Extended Search RMI Serer. If you are using a Domino Web serer, you must perform these steps. If you are using WebSphere Application Serer and you did not deploy the Extended Search Enterprise JaaBeans (EJB) when installing Extended Search, you must perform these steps. If you are using WebSphere Application Serer and the Extended Search EJB, do not perform these steps. a. Select Extended Search RMI Serer and click Startup. b. Select Autostart as the Startup Type and click OK. Wheneer you restart the machine, the RMI Serer serice will check to see whether DB2 has been started and, if necessary, start it before starting itself. 4. Start the Web serer and Web application serer on each machine where you installed Extended Search Web serer support. (Typically these programs are autostarted.) If you need assistance, consult the Domino or WebSphere product documentation. Note that it may take some time to start a WebSphere serer, especially if you enabled the option to load serlets at startup. 5. Start the Administration interface (for assistance, see the procedure on page 10). This action helps alidate your ability to communicate with the Web serer and proides eent information that can help resole potential startup problems. Using Authentication If you installed the Extended Search Web Serer component on a Web Serer where authentication has been enabled (or anonymous access has been disabled), you must update the following entries in the Windows Registry to identify the Extended Search administrator user ID and password before you start the Extended Search Serer: HKEY_LOCAL_MACHINE SOFTWARE IBM Extended Search 3.7 AdminUserid AdminPassword Tasks 5

14 6. On each machine that hosts an Extended Search Serer, take one of the following actions to start the Extended Search Serer: Use the Windows Start menu to select Programs IBM Extended Search Extended Search Serer. Open the Windows Control Panel and then double-click Serices. Select Extended Search Serice, and then click Start. Note: If you are running Extended Search with a Domino Web serer, you cannot autostart the Extended Search serer serice unless they are on a separate machines and the Web serer is always running. If both serers are on the same machine, Extended Search may attempt to start before all the Web serer components hae started, and the startup attempt will fail. Shut down the serer on Windows It is recommended that you use the Extended Search Monitor to shut down an Extended Search serer. To do so: 1. Select Serer Shutdown in the Monitor. You can shut down the serer while running the Monitor as a stand-alone program or while you are monitoring serer actiity from within the Administration interface. You will be prompted to specify the ES Administrator user ID and password to complete this task. If you started Extended Search by using the Windows Start Programs menu, you can use the following procedure to shut it down. 1. Open the console window on the Extended Search Serer. 2. Enter one of the following commands: Quit Exit If you started Extended Search as a serice, you can use the following procedure to shut it down. Note, howeer, that if you use this approach, Windows is not aware of related tasks that may also need to be shut down. 1. On each machine where you installed Extended Search Serer software, open the Windows Control Panel and then double-click Serices. 2. On the Serices window, select Extended Search Serice and then click Stop. 3. Take the following steps only if you started the RMI Serer serice: a. On the machine where you installed the configuration database, open the Windows Control Panel and then double-click Serices. b. On the Serices window, select Extended Search RMI Serer and click Stop. CAUTION: You should neer stop Extended Search from the Windows Task Manager. If you stop Extended Search in this manner, the program stops, but the executables that are launched during startup do not. When this occurs, use the DESAbort utility to remoe the unwanted tasks. Note that you should use DESAbort only in critical situations. 6 Extended Search Administration

15 Configuring the search domain The Extended Search serer obtains information about the resources it is to manage from the Extended Search configuration database (CDB), a database managed by IBM DB2 Uniersal Database. To update the CDB, you run the Administration interface and specify the following types of information: The locations of the data sources you want to include in the Extended Search search domain Which data source fields users can search, iew as a search result, or retriee from the data source Which field names should be mapped to permit field-leel searching across dissimilar data sources Which data sources should be grouped into a common category Which categories should be accessible through a gien application Which broker should process requests for a gien data source Which agent, and how many agents, should serice transactions between the broker and data sources Which link should handle search and retrieal operations within the data source Which grammar should be used to translate the product s internal generalized query language (GQL) into the natie search language of the data source Extended Search can link to and search against a heterogeneous set of data sources, including the following data source types: File system directories and files, whether local or on a remote network drie IBM Enterprise Information Portal (EIP) sources Lightweight Directory Access Protocol (LDAP) sources Lotus Domino Connectors sources Lotus Domino.Doc sources Lotus Domino Domain Index sources Lotus Notes databases Microsoft Access sources Microsoft Exchange Serer sources Microsoft Index Serer sources Microsoft Site Serer sources Microsoft SQL Serer sources Open Database Connectiity (ODBC) sources Relational database management systems (RDBMS) Web sources (the product includes predefined connections to many popular Web search sites) IBM and Lotus business partners may offer support for other types of data sources. Furthermore, your enterprise can deelop custom connectors for additional proprietary data sources. Tasks 7

16 Get started Before starting to configure your Extended Search system, you need to understand the system components and how they interact. If you hae not already done so, be sure to read the product s Technical Oeriew in Extended Search General Information before proceeding. To help you prepare for the configuration tasks, you should also reiew the following topics: Configuration model Configuration roadmap on page 10 Adding data sources on page 100 Configuring data sources on page 100 Discoery on page 104 As discussed in Tasks on page 1, there are certain basic configuration actiities that eeryone should perform. Other more adanced actiities depend on the size and complexity of your system. Configuration model The Extended Search configuration data model is illustrated in Figure 1. It depicts the system as a set of inter-related boxes that are connected by directed arrows. Each box represents a configurable element in the CDB, as described below. The serers depicted in this diagram represent the physical composition of an Extended Search system. For interprocess communication to take place, the locations of the serers, including the broker and agent components they host, must be known to your network s Domain Name System (DNS) serer. The remaining components in this diagram represent the logical iew that you present to users of your Extended Search system. The data sources you plan to search, and the ways you plan to make data sources recognizable to and accessible by users, must be defined in the CDB. Extended Search Search Domain Logical View Physical View Links Data Sources Mapped and Natie Fields Brokers Grammars Categories Agents Applications Serers Figure 1. Configuration data model By using the Administration interface, you can configure or reconfigure the following information in the CDB. Serers When you configure a serer, you identify its physical location, the number of tasks that should be started, and information about the brokers and agents that run on that serer. You also run data source discoery from the serer and can run the Extended Search Monitor to track actiity on the 8 Extended Search Administration

17 serer. To enhance scalability and performance, you may want to install multiple Extended Search serers. If so, you can easily graph the relationships between serers by drawing connector lines between them. All network addressing actiity required to reflect these communication links is handled in the background for you. Data Sources After discoering a data source, you need to identify the link, grammar, and agent processes that should connect to the data source and perform search and retrieal operations. Although you can make changes, this data is generally proided for you by the discoerer. You also define which parameters, if any, the system should use when connecting to or searching the data source. Natie Fields and Mapped Fields When configuring a data source, you also define which fields should be accessible to end users. It is possible for two natie fields in separate data sources to contain the same information but be known by different names. To perform a single fielded search across these sources, each source must hae a common field that will enable it to be searched by a common query. You achiee this by configuring mapped fields. For example, if one data source has a field named Author and another data source has a field named Editor, you could create a mapped field named Owner and link both natie fields to it. You can then submit a query based on the field Owner and get appropriate results for the two data sources. Links If you hae a data source that is not supported by the set of predefined links, you can deelop a new link and define it in the CDB. See Extended Search Programming for information about deeloping custom links to support search and retrieal operations in new data sources. Grammars If you hae a data source that is not supported by the set of predefined grammars, you can deelop a new translator and define it in the CDB. See Extended Search Programming for information about deeloping custom functions to translate a GQL query into the natie search language of a data source. Categories Each data source you want to search must be associated with at least one category. When submitting a search request, users can select the categories applicable to the information they need to retriee. Categories also make it easier to manage potentially large sets of related data sources. Applications To ensure that users access only those data sources for which they hae a need, you must associate each category with an application and assign a unique application name (identifier). You can also assign application-specific controls, such as which fields can be searched or returned, the order in which fields should be presented in a hitlist, and the order in which documents should be retrieed from the data source. Tasks 9

18 Configuration roadmap Configuration of an Extended Search system is a multi-step process that assumes a successful installation of the product. Much of the configuration data gets generated for you through installation and data source discoery. There are additional steps you should take to meet the needs of your enterprise. Step 1. Identify and load the data sources to which Extended Search will link. The term data source refers to a logical entity that represents a collection of documents (or structured information) that shares common field definitions and is searchable. This collection of data sources constitutes your Extended Search search domain. Step 2. Decide how you want to configure fields in the data source. You should identify which fields contain information of interest to your users, and which fields users will want to search by name. You should also identify which fields to search by default when no other search criteria are aailable, and which fields constitute content. Options you specify here apply to the data source globally. Later, you can define additional options on an application-by-application basis. Step 3. Decide how you want to categorize groups of data sources. A category proides an easy way for you to group similar types of data sources so that a common set of rules applies to them all. To be searchable, each data source must belong to at least one category. The category, in turn, must be associated with at least one application. Step 4. Configure your client applications. The configuration of each client application includes tasks such as associating the categories of data sources to be searched by users of this application, selecting the fields to be searched, returned, or fetched, and specifying the order in which fields should be presented in a hitlist or retrieed from the data source. Step 5. Perform adanced configuration tasks as necessary. These tasks depend on the security, scalability, and performance requirements of your enterprise. If you install any serer components on remote machines, such as additional brokers or agents, you need to configure information about these remote components. You may also need to configure additional links or grammars to support the connection requirements and natie search languages of some sources. Step 6. Optionally enable eent logging. Choose the types of eents you want to log (from informational to critical). Then use the eent log iewer to reiew the eent messages and the correctie actions you should take. Step 7. Test your configuration changes. Extended Search proides sample search templates that enable you to immediately test your ability to search newly configured data sources. Start the Administration interface There are two ways to update the configuration database and administer your Extended Search search domain: You can run the Administration applet. This is the preferred method. You can run the Administration application. You might use this method if you enable Web authentication on the Web serer used by Extended Search. See Authentication enforced by the Web Client on page 147 for information about how Web authentication can impact the operation of the Administration applet. 10 Extended Search Administration

19 Regardless of how you start the Administration interface, the same Search Domain iew will be displayed, and you will follow the same procedures to naigate the interface and make changes to the CDB. Figure 2. Administration interface main iew Start the Administration applet To make changes to the CDB, you can use an Administration applet, ESAdmin. You can run this applet from any workstation that can access the Web serer and URL where the applet was installed. 1. If you did not preiously install the Jaa 2 Runtime Enironment (J2RE) plug-in on this machine, do so before starting the applet. You can download the plug-in from the following location: 2. If you are running Windows, select Start Programs IBM Extended Search Extended Search Administration. Otherwise, access the case-sensitie URL for the Administration applet. For example: Where: hostname is the fully qualified host name of the Web serer. :port is the Web serer port (required only if the port is not 80). serlet is the base directory on the Web serer where Extended Search serlet files are installed. On a Domino serer, this alue is serlet. Ona WebSphere serer, this alue is the Extended Search Web application name (the default alue is lotuskms). Tasks 11

20 For example: You may be prompted to log in. This occurs only if the security setup of your enironment requires user authentication. See Authentication enforced by the Web Client on page 147 for important information about how to use Web authentication with the Administration applet. The applet loads and starts. Wait until the applet has completely loaded and the main Search Domain iew has appeared (figure Figure 2 on page 11). Start the Administration application To make changes to the CDB, you can use a Jaa application, esadminapp. This method of starting the Administration interface enables you to bypass problems that can arise when you use Web authentication with the ESAdmin applet (for details, see Authentication enforced by the Web Client on page 147). Although the application prompts you for the ES Administrator user ID and password on startup, be aware that if you want to implement additional security, you must do so through custom programs that you write, by securing the serer within your firewall, or by physically securing the machines that host the application and configuration database. System Requirements: To run the application in a WebSphere Application Serer ersion 4.0 enironment, you can configure WebSphere to use either the Extended Search RMI serer or you can deploy the Extended Search EJB. To run the application in a WebSphere Application Serer ersion 3.x enironment, you must configure WebSphere to use the RMI serer; you cannot deploy the Extended Search EJB. If you use a Domino Web serer, you must use the RMI serer (the installation program set it up for you). See Extended Search Installation for information about configuring WebSphere Application Serer for use with Extended Search. See Starting and stopping the serers on page 3 for information about starting the RMI serer before starting the Extended Search serer. Update the Properties File: Before running the application, you must update the esadminapp.properties file to specify information about the serers and ports used by this Extended Search serer. Use the following guidelines to proide information about your system. web_serer Specify the fully qualified host name of the Web serer that you set up to use with this Extended Search serer. The host name you specify for this Web serer, which contains the Extended Search document tree, must match the Web serer host name that is specified in the configuration database. web_port Specify the port number used by this Web serer. rmi_host If you use the RMI serer, specify the fully qualified host name of the 12 Extended Search Administration

21 serer where you installed DB2 and the Extended Search configuration database. (This serer contains the RMI serer.) rmi_port If you use the RMI serer, specify its port number. This alue must match the RMI serer alue that is defined in the admin.properties file. doc_root Specify the logical path to the Extended Search document tree on the Web serer. For a WebSphere serer, the default alue is /lotuskms/. For a Domino serer, the alue is /serlet/. protocol If you use the RMI serer, specify rmi. If you use the Extended Search EJB, specify iiop. Start the Administration Application: 1. Ensure that Jaa has been installed on the machine where you run the application. 2. Ensure that the following files are co-located on your system. These files are installed into the \config subdirectory of your Extended Search installation directory. esadminapp.properties esadminapp.jar esnls.ini (the Extended Search national language configuration file) 3. You may need to set the system path to include Microsoft Internet Explorer (iexplore.exe). For example: set path=c:\programfiles\internet Explorer;%path% 4. Enter one of the following commands: If you are using a Domino serer, or if you configured WebSphere Application Serer to use the RMI serer, enter the following command: jaa -jar esadminapp.jar esadminapp.properties If you are using WebSphere Application Serer ersion 4.0.2, and you configured it to use the Extended Search EJB, use the WebSphere launchclient utility to launch the application, where x:\esjar_path is the fully qualified path to the location of the Extended Search Administration application files. (Enter this command on one line.) launchclient extendedsearchejb.ear -CCclasspath=x:\esjar_path\esadminapp.jar esadminapp.properties Note that if the Web application serer is remote from where you are launching the application, you may need to use additional launchclient parameters: CCBootstrapHost= Specify the host name of the serer you want to connect to initially. Use this format: your.serer.ofchoice.com CCBootstrapPort= Specify the serer port number. If omitted, the WebSphere default alue is used. 5. When prompted, enter the ES Administrator user ID and password. These alues must match the alues that were specified for the Extended Search DB2 Administrator during the installation of Extended Search. The application starts and the main Search Domain iew is displayed (figure Figure 2 on page 11). Tasks 13

22 Naigate the interface The CDB Naigator on the left side of the Administration interface lists the configurable elements in the Extended Search search domain. When the program first starts, the iew pane on the right side contains icons that represent each of the configurable elements. To select the element you want to work with, click its name in the CDB Naigator. Alternatiely, you can click the element s name with your second mouse button and select Explore. (The second button is typically on the right side of the mouse. Throughout this discussion, this directie is referred to as right-click.) After you select an element, the iew pane presents the indiidually configurable items currently defined for the selected element. There are a couple of ways to work with these items: Double-click the item name or the icon in the iew pane. This action opens the Properties window for that item, from which you can define or change configuration data. Right-click the item name or the icon to open a menu from which you can choose the actions aailable for that item, such as Copy or Delete. The Properties action opens the Properties window for that item, from which you can define or change configuration data. The following table describes the configurable elements and their icons. Icon Title Description Serers Data Sources by Link Data Sources by Agent Mapped Fields Categories Applications Lists all known serers. Use this option to identify the serer location and to configure brokers, agents, and the Web serer. Also use this option to discoer data sources you want to add to the search domain or to monitor serer actiity. Lists all known data sources organized by link type. Use this option to associate a link, grammar, and agent with a data source, and to define field usage controls, parameters, and hitlist actions for the data source. Lists all known data sources organized by agent name. Use this option to associate a link, grammar, and agent with a data source, and to define field usage controls, parameters, and hitlist actions for the data source. Lists all known mapped fields. Use this option to define a mapped field and make it aailable for use within indiidual data sources. Lists all data sources organized by category name. Use this option to define new categories and associate data sources with a category. Lists all applications organized by application name. Use this option to define new applications and associate categories of data sources with applications. You can also assign application-specific controls, such as which fields can be searched, returned, or fetched, and the order in which fields should be returned or fetched. 14 Extended Search Administration

23 Icon Title Description Search Templates: - Demo - Other Samples Links Grammars Documentation Lists search templates that you can use to run a test query after making configuration changes. The Demo templates are the most comprehensie examples. Other Samples illustrate basic search and retrieal options and adanced GQL query construction. Extended Search is preloaded with link definitions that enable many common data sources to be searched. You can use this option to define new links, link parameters, and hitlist actions, but you should not change the preloaded definitions. Extended Search is preloaded with grammar definitions that enable many common data sources to be searched. You can use this option to define new grammars, but you should not change the preloaded definitions. Lists the Extended Search product documents. Use this option to learn about the product and how to perform installation, administratie, and programming tasks. Eent Log Viewer Saed Queries Wizards Opens the log iewer, which enables you to iew the text of logged messages, print them, sae them to a file, or delete them from the eent log database. Lists saed queries and search results. As an administrator, you can iew and work with all saed queries and search results. Lists wizards that make it easy for you to configure mapped fields, data source parameters, or hitlists; export the configuration database; or export and import data sources and applications. The icons and buttons in and aboe the iew pane can help you iew the contents of the CDB and perform configuration tasks. Icon Title Description Icon View Displays configurable items as icons. List View Displays configurable items in a list. Detail View Tree View Lists the configurable items and proides details about each item. Shows the relationship between configurable items. Graph View Sort ascending Shows the connections between items in the current iew. For example, if you installed two or more serers, you would use this iew to draw the layout of the physical system. Sorts items in the current iew in ascending order. Tasks 15

24 Icon Title Description Sort descending Sorts items in the current iew in descending order. No sort Does not sort the items in the current iew. Refresh Refreshes the contents in the current iew of the CDB. Find Expand All button Collapse All button Apply Connections button Current program pull-down Searches the contents of the current iew to locate a term that you enter in the search box to the left of the icon. Expands the list of items in the current List iew. Collapses the list of items in the current List iew. Applicable only in the Serers iew, allows you to sae the arrangement of and connections between multiple serers. Displays the name of the currently running program. If you integrated Extended Search with another product, such as IBM Enterprise Information Portal, you can switch between the EIP and ES Administration interface programs. Get help There are a couple of ways that you can obtain help while using the Administration interface: In most of the iews, you can right-click in an open area of the iew pane and select Help. This action displays task-oriented information that can help you make selections and define options for the configuration items displayed in the current iew. After opening a configuration item to define properties for it, you can click the Help button in any Properties window. This action tells you about the information you must type or select to configure the selected item. The system displays help in a new browser window, which allows you to reiew the information while you complete the task at hand. To obtain additional help, click in the top frame of any help window, such as on the book icon. This action opens this document, Extended Search Administration. Configure serers The first time you install the Extended Search Serer component, information about its network location is written to the configuration database. The CDB also stores default information about the broker, agent, and Internet protocols that the serer uses to process search requests. Typically, you can use this initial configuration data to run Extended Search out-of-the-box, without making any changes to the CDB. Most of the actions you take to configure the serer occur as the result of completing another task. For example, to enhance security or performance, you may need to install another serer, alter the number of broker tasks that run on a particular serer, or associate a custom user exit to perform user authentication. The following table lists options that you can configure for an Extended Search serer along with pointers to where you can read about how to make changes. 16 Extended Search Administration

25 To learn about this task... Define additional Extended Search serers. Graphically connect two or more Extended Search serers to define how they interact with each another. Run data source discoerers to add data sources to your Extended Search search domain. Enable eent logging on this serer, and then use the eent log iewer to reiew and administer logged messages. Instruct the broker on this serer to use a custom function for logging. Instruct the broker on this serer to use a custom function for authentication. Instruct one or more agents on this serer to use a custom function for authentication or access control. Increase or decrease the number of copies of broker and agent tasks started on this serer. Configure the amount of shared memory managed by this serer. Separate the broker from the Web serer to balance resources. Configure agents and data sources on remote machines but allow them to interact with a central broker. Define multiple agents on one machine. Configure multiple brokers to balance the processing load across multiple machines. Run the Monitor to keep track of actiity on this serer. Read this procedure Add another Extended Search serer Connect multiple Extended Search serers on page 18 Discoer data sources on page 19 Log and iew eents on page 45 Enable custom broker logging on page 47 Enable broker-leel authentication on page 51 Enable agent-leel authentication on page 51 Configure copies of serer tasks on page 56 Manage serer memory on page 58 Separate brokers from the Web serer on page 52 Configure remote links on page 53 Partition agents and data sources on page 54 Configure multiple brokers on page 55 Run the Monitor in the Administration interface on page 89 Add another Extended Search serer To run more than one Extended Search serer or Web serer in your Extended Search domain, you need to install the serer software on another machine. You also need to configure options for the serer, and decide how you want to connect the new serer to one or more preiously configured serers. Note that you can configure options for the new serer before installing the Extended Search serer software on that machine. Howeer, you cannot refresh the domain and start using the new serer until the software has been installed. When you add a serer, you can use default configuration data or use data that you copy from an existing serer. When you copy a serer, you copy its entire definition. This approach makes defining a new serer easy. For example, the only options that you must specify for the new serer are a unique serer name and a unique broker name. 1. Follow procedures in Extended Search Installation to install serer software. (Select Extended Search Serer, Web Serer, or both options from the list of components to install, according to how you plan to use this new serer.) 2. Start the Administration interface, and use one of the following approaches to configure options for this new serer. Tasks 17

26 To define a new serer based on the default options, either right-click Serers in the CDB Naigator and select New, or right-click in an open area of the Serers iew and select New. To define a new serer based on the options defined for an existing serer, right-click an existing Extended Search serer in the iew pane and select Copy, and then right-click in an open area of the iew pane and select Paste. 3. When the Serer Properties window is displayed, specify configuration options for this serer (page 192 discusses these options). After you click OK to sae your settings, an icon for the new serer is displayed in the iew pane. 4. If you installed the Extended Search Serer component, you must connect the new serer to an existing serer. (See the procedure below for instructions.) Note that you do not need to connect serers on which only the Web Serer component is enabled. 5. If you need to change a serer configuration, right-click the serer s icon in the iew pane and select Properties. When the Serer Properties window is displayed, make your changes and then click OK. 6. If you need to delete a serer, right-click the serer s icon in the iew pane and select Delete. Be aware that deleting a serer also deletes all of the configuration data that was stored for that serer. 7. When you hae completed your configuration changes, refresh the search domain to make the new serer aailable to users. Connect multiple Extended Search serers If you hae more than one Extended Search serer in your Extended Search domain, you must specify the relationships between them. The Administration interface proides an easy way for you to do this. First, graphically position the serer icons in a manner that best represents the physical layout of your network. You can then draw connector lines to define the communication links between them. When you connect one serer to another, the sources become aailable to the Web serer that serices each serer. For example, when you configure a remote link, you need to connect the serer that hosts the remote agent with the serer that hosts the broker who serices that agent. Note that you should connect only those serers whose configurations specify that Extended Search is installed. If the Serer Properties for a gien serer specify that only the Web Serer is installed, you do not need to draw a connecting line. Use the following procedure to connect a new Extended Search serer with your existing network of serers. 1. Click the Graph View icon. 2. Click the new serer icon, and drag it to a location in the iew pane that represents how it fits in with your existing serer topology. Alternatiely, you can right-click the new serer icon, select Serer position, and specify the horizontal and ertical axes where the icon should be positioned. 3. Right-click the new serer and select Connect to. 4. Click the existing serer to which this serer should connect. The system draws a line between the two serers and, in the background, sets up the appropriate network addresses. Note: If MouseKeys is enabled for your operating system, you can perform steps 3 and 4 by using arrow keys on the keyboard to moe the mouse pointer and connect the two serers. 18 Extended Search Administration

27 5. If you need to delete a connection between two serers, click the connector line and then press the Delete key. 6. Click Apply Connections to sae your changes. The system recalculates the network addresses to reflect the correct locations of your data sources. Note that while you are working in Graph View mode, you can reposition the serers as needed. Any time that you connect two serers, or delete a line to disconnect them, be sure to click Apply Connections. This is necessary to commit your changes and recalculate the network addresses. Do not click the Refresh icon until you hae applied the connections or you will lose your changes. Discoer data sources Extended Search proides an automated data source discoery feature that makes it easy for you to add data sources to your search domain. Discoery simplifies the task of identifying data sources to Extended Search. It also improes the chance of a successful configuration by automatically detecting and correcting errors in data source definitions as it proceeds through the process. To discoer data sources, you must ensure that the software that supports the data source is installed on the serer being discoered. See Extended Search Installation for information about data source software requirements. You must also ensure that the user ID by which you log in to the system has the necessary permissions to access any remote network dries being discoered. If the user ID does not hae access permissions, you will see an access denied message instead of the list of data sources aailable to be discoered on that drie. The following procedure shows you how to use the Administration interface to discoer data sources. If you want to run discoery outside of this interface, see Run discoery as a standalone process on page Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all serers defined in the CDB. 3. Right-click the serer you want to work with and select Discoer Data Sources. You can also discoer data sources by opening the Data Sources by Link or Data Sources by Agent iew, right-clicking in an open area of the iew pane, and selecting Discoer Data Sources. If you hae only one Extended Search serer, the Discoer Data Sources window will be displayed. If you hae multiple serers, you will first be prompted to select the serer you want to discoer. 4. On the Discoer Data Sources window, select the discoerer that matches the type of data source you want to add to the search domain. 5. The system requests information about the selected data source type. For example, if you are adding a file system source, it asks you to select the system dries that you want to include in the discoery process. For other data source types, it may ask you for a database name or for a user ID and password that enables the discoery process to access the data source. Select or type the requested information and click Start Discoery. Note: If you need assistance when proiding this discoerer-specific information, see Discoer Data Sources on page 182. Tasks 19

28 6. The bottom half of the window lists all data sources of the selected type found on the serer that is being discoered. Select one or more data sources from the list and click Add to ES. 7. Repeat steps 4 on page 19 through 6 to add data sources of a different type. 8. When you hae finished discoering data sources on this serer, click Close. 9. After adding a data source, there are seeral steps you should take before making it aailable to users: a. Configure the field definitions. For example, you can add fields, delete fields, select fields that you want users to search or retriee, and so on. b. Categorize the data source. This step is required. You can create a new category or associate the data source with one or more existing categories. If you add a data source that is supported by one of the predefined link types, the discoerer automatically associates it with a category that has the same name as the link. c. Associate the category with an application. To be searchable by users, this step is required. You can create a new application or associate the category with one or more existing applications. Also, define application-specific controls to goern how users of the application should access and search the data source. 10. When you are ready to use the new data source, refresh the search domain to disseminate your changes to the user community. Run discoery as a standalone process If you prefer, you can run discoery outside the Administration interface. This feature may be useful if you want to share the workload among multiple users, or if you want to allow certain users to discoer data sources but not allow them to perform all the administratie tasks accessible through the Administration interface. To run discoery as a standalone process, start a Web browser and pass dbdiscoer.txt to the JKMSearchController serlet. Note that the Jaa 2 Runtime Enironment (J2RE) plug-in must be installed on any machine from which the discoery process is launched. For readability, the following example has been entered on two lines. You should type the entire string on a single line. Note, too, that the serlet name and its arguments are case-sensitie. &AppID=appname&desClientLocale=locale Where: hostname is the fully qualified host name of the Web serer. serlet is the base directory on the Web serer where Extended Search serlet files are installed. On a Domino serer, this alue is serlet. On a WebSphere serer, this alue is the Extended Search Web application name (the installation default is lotuskms). appname is the name (application ID) of the client application you want to run, such as the Demo application proided with Extended Search. locale is the language and country code that identifies the locale of the data sources that this application searches, such as enus for U.S. English. For information about specifying a alid language and country code, reiew National language support on page Extended Search Administration

29 For example: &AppID=Demo&desClientLocale=enUS When you run data source discoery in this manner, the dbdiscoer applet automatically stores data in the configuration database and writes it to an XML file. (You can later massage and import this file into Extended Search.) If you prefer, you can alter this default behaior. To do so: 1. Edit the dbdiscoermainpage.txt file. This file is located in the /templates/enus/jkm directory, where enus is the subdirectory for your locale (in this example, U.S. English). 2. To not store data in the configuration database, locate the following lines, and change the alue "Yes" to "No". For example: <PARAM NAME="WriteDB" VALUE="No"" WriteDB = "No" 3. To not write data to an XML file, locate the following lines and change the alue "/temp/discoer.xml" to blank (""). For example: <PARAM NAME="WriteXMLFile" VALUE=""> WriteXMLFile = "" Configure data sources Data source discoery automatically loads the CDB with information about the data source and the fields that are defined in the data source. You can update this information according to the needs of your enterprise. The procedure below shows you how to set options for accessing the data source. Refer to the following topics for information about configuring search and retrieal options for the data source and associating the data source with one or more categories: How do I define natie fields? How do I define mapped fields? How do I define field usage controls? How do I categorize the data source? How do I define parameters for accessing or using the data source? How do I specify actions for handling results returned from the data source? How do I export and import data source definitions? Use the following procedure to configure an indiidual data source. 1. Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB organized by link type or agent name. 3. Right-click the data source you want to configure and select Properties. 4. On the Data Source Properties window, reiew and optionally change information on the General tab. 5. In Data source name, type a short, unique, and descriptie name for the data source. This name is used throughout the Extended Search domain to identify the source as a distinct repository. It cannot match the name of any other data source or the name of any category. If you replicate a Notes database, be sure to assign a unique name to the replicated database. Tasks 21

30 6. Select the Disable data source check box only if you want to hide the data source from end users. For example, if you deeloped a custom link for this data source, you may want to temporarily disable the data source until you hae finished testing your changes. 7. The Link, Grammar, and Agent alues are all set as part of discoery and, typically, you do not need to change them. If you want to make a change, select the alue you want to use from the appropriate list. For example, your enterprise may hae deeloped a custom link or set up a separate agent to serice this data source 8. The Language used alue, which identifies the locale of the data source content, is specified at the time of discoery and, typically, you do not need to change it. 9. Either click OK to sae your changes and exit, or click Apply and select another tab to continue defining options for this data source. You can specify general options for more than one data source at a time. For example, if you configured a remote link, wrote a custom grammar, or installed a new agent, you can associate a group of data sources with the new link, grammar, or agent without haing to update each data source indiidually. Use the following procedure to configure multiple data sources. 1. Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB organized by link type or agent name. 3. Select two or more data sources you want to configure, right-click, and select Properties. 4. On the Data Source Properties window, select the Disable data source check box only if you want to disable all the data sources you selected. 5. Select the new Link, Grammar, and Agent alues as appropriate for this group of data sources. 6. Click OK to sae your changes and exit. Define natie fields When you define natie fields, you specify which fields should be accessible to end users through a client application. For example, you may want to remoe certain fields that you do not want users to search or retriee. In addition, you may want to add fields that were not defined during data source discoery. This is particularly true with Lotus Notes and ODBC-compliant databases which, typically, contain numerous fields that you opted not to spend time loading during the discoery process. When you add a field, make sure that the field name is a alid name that actually exists in the data source. Use the following procedure to define natie fields: 1. Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB organized by link type or agent name. 3. Right-click the data source you want to configure and select Properties. 4. On the Data Source Properties window, select the Natie Fields tab. 5. If you want to add a field to the list of fields that are accessible to client applications, click Add Row. A blank line is added beneath the currently selected row or, if none is selected, to the end of the list. 22 Extended Search Administration

31 a. In Natie field name, type the name of the field as it is defined in the data source. Note that you cannot use the following characters in field names: Forward Slash (/), Backslash (\), Comma (,), Semi-colon(;), Colon (:), Left Bracket([), and Right Bracket (]). b. In Type, click to display a list of aailable data types, and select the data type of this field. Note: If you add a field after the data source has been assigned to applications, the addition is reflected automatically in any application that includes the data source. Howeer, you must update the application-specific properties as appropriate to specify whether or not users are permitted to search the new field, iew it in a hitlist, or retriee it from the data source. 6. If you want to remoe a field from the data source, select the field from the Natie field name list and click Delete. Note that this action does not delete the field from the physical data source. It simply hides the field from the user s iew of the data source and preents users from accessing it. To preent users from performing certain operations on fields, as opposed to completely obscuring the field s presence in the data source, you can configure field usage controls (page 25 discusses this procedure). Note: If you delete a field after the data source has been assigned to applications, the deletion is reflected automatically in any application that includes the data source. 7. Either click OK to sae your changes and exit, or click Apply and select another tab to continue defining options for this data source. Define mapped fields Defining mapped fields is a two-fold process. First you must define the mapped field at a global leel where it can be known to all data sources and applications in your Extended Search domain. You must then identify the natie fields to which the mapped field should link. See Mapped fields on page 120 for complete information about how to take adantage of this feature when configuring fields. Step 1: Define a Mapped Field. 1. Start the Administration interface. 2. Click Mapped Fields in the CDB Naigator. The iew pane lists all mapped fields defined in the CDB. The default installation includes the following mapped fields: Default maps to all natie fields of type String, which allows users to enter simple queries without specifically choosing which fields to search. Doc$CC_Title maps to the source object s title. DocId maps to a alue that uniquely identifies a source object, such as a file name or URL. For data sources that use the ODBC link type, DocId maps to one or more natie columns in the database. 3. To create a new mapped field: a. Either right-click Mapped Fields in the CDB Naigator and select New, or right-click in an open area of the Mapped Fields iew pane and select New. b. Type a unique, descriptie name for the field. For example, you might create a mapped field named Author to encompass all natie fields that identify a document s author, such as Creator, Owner, Editor, and so on. c. Click OK. Tasks 23

32 4. To change a mapped field, right-click the mapped field s name or icon in the iew pane and select Properties. Type a new name and then click OK. 5. To delete a mapped field, right-click the mapped field s name or icon in the iew pane and select Delete. Note: If you delete a field after the data source has been assigned to applications, the deletion is reflected automatically in any application that includes the data source. Step 2: Assign Natie Fields to the Mapped Field. Note: After you define a mapped field, you can use an Extended Search wizard to map natie fields to it instead of using the steps below. For details, see Run configuration wizards on page Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB organized by link type or agent name. 3. Right-click the data source you want to configure and select Properties. 4. On the Data Source Properties window, select the Mapped Fields tab. 5. Click Assign. 6. On the Assign Mapped Field to Data Source window, select the field that you want to map within this data source and then click OK. 7. In the Mapped Fields with their Natie Fields area, first select the mapped field. Then, from the Natie field name list, select one or more natie fields to which the mapped field should link, and click the forward arrow button. For example, a mapped field named Author may need to link to three natie fields to adequately reflect the author s first name, middle name, and last name. 8. To specify how the data should be concatenated when the mapped field is returned in a hitlist, expand the contents of the mapped field, and select a field that you want to reposition. Click the up arrow button to moe it higher in the list, or click the down arrow button to moe it lower in the list. For example, you may want to position a field named First Name aboe a field named Last Name to ensure that the first name precedes the last name when search results are displayed. 9. To unmap a natie field, expand the mapped field to which it is linked. Then select the natie field that you want to disassociate, and click the backward arrow button. 10. To remoe the definition of a mapped field from this data source, select the mapped field and click Unassign. When asked to confirm that you want to remoe the mapped field, click Yes. 11. Either click OK to sae your changes and exit, or click Apply and select another tab to continue defining options for this data source. Note: If you configure a mapped field for a data source after the data source has been assigned to an application, the addition is reflected automatically in any application that includes that data source. Because mapped fields inherit their usage rules from the natie fields to which they link, you do not need to update application-specific properties for this change. 24 Extended Search Administration

33 Set field usage controls When you define field usage controls, you specify which fields should be accessible to end users for search and retrieal operations. Usage controls that you set at the data source leel apply to the data source on a global leel throughout the Extended Search domain. When you configure an application, you can modify these settings for the specific needs of that application. Restrictions you set at the data source leel can only be made more restrictie at the application leel. If you define a field as searchable, you can configure the field to be not searchable in a gien application. Howeer, if you define a field as not searchable at the data source leel, an application cannot later make that field searchable. Note: You cannot select usage options for mapped fields. A mapped field deries its accessibility from the natie fields it references. For example, if all the natie fields linked to a mapped field are searchable, the mapped field is searchable. If one or more of the natie fields is not searchable, then the mapped field is not searchable. Use the following procedure to define field usage controls: 1. Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB organized by link type or agent name. 3. Right-click the data source you want to configure and select Properties. 4. On the Data Source Properties window, select the Field Usage tab. 5. For each listed natie field, specify whether the field can be searched or returned: If the field can be used to construct a fielded query, click to put a checkmark in the Searchable column. Alternatiely, you can select one or more fields and click the Searchable button to toggle the checkmark for the selected fields. If the field can be returned whether for displaying in a hitlist, storing on disk, or retrieing from the data source click to put a checkmark in the Returnable column. Alternatiely, you can select one or more fields and click the Returnable button to toggle the checkmark for the selected fields. Note: Extended Search proides a wizard that makes it easy for you to configure the fields you want to include in a hitlist. For details, see Run configuration wizards on page If you change a field s usage definition after the data source has been categorized and assigned to applications, you must take the following steps to ensure that the change is reflected in any application-specific properties defined for the data source. a. If a field preiously could not be searched or returned, and you make the field searchable or returnable at the data source leel, you must decide, on an application-by-application basis, whether you want users to be able access this field when searching the data source, iewing results in a hitlist, or fetching documents from the data source. For each application that references the data source, you should configure application-specific properties as appropriate. b. If a field preiously could be searched, returned, or fetched by an application, and you make the field not searchable or not returnable at the Tasks 25

34 data source leel, you must reconfigure the application. When you iew the application-specific properties for the data source, the preiously aailable field appears grayed out and is unaailable. You must click OK or Apply for the change to take effect in the CDB. 7. Either click OK to sae your changes and exit, or click Apply and select another tab to continue defining options for this data source. Define data source parameters When you define data source parameters, you specify options that are passed to the agent at the time the data source is accessed. For example, you may want to instruct the agent to search with a data source-specific date format. You can define parameters at the data source leel and link leel. If you define the same parameter at both leels, the link parameter alue seres as the default for the data source parameter. You can oerride the default alue to inoke different rules for this data source. Note: Extended Search proides a wizard that makes it easy for you to configure data source parameters and associate them with specific data sources or all data sources of a gien link type. For details, see Run configuration wizards on page 39. You can also define application parameters. These parameters are distinguishable from data source parameters in that they are accessible to client applications through the Extended Search application programming interface (API). They enable client applications to coney information about the data source to end users. During data source discoery, many parameters are automatically configured for you. Predefined parameters on page 132 describes the parameters and proides information about which ones you can specify at the link or data source leel. Use the following procedure to configure parameters for a data source: 1. Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB organized by link type or agent name. 3. Right-click the data source you want to configure and select Properties. 4. On the Data Source Properties window, select the Parameters tab. 5. To add a new parameter, click Add Row. A blank line is added beneath the currently selected row or, if none is selected, to the end of the list. a. In Parameter name, type the parameter s name. b. In Value, type a alid alue for the parameter. c. In Usage, select Used by data source if the parameter is to be used exclusiely by the Extended Search serer. Select Used by application if the parameter is to be used by client applications. 6. To modify an existing data source parameter, select it from the list and type a new name, alue, or description as necessary. 7. To remoe a preiously defined parameter, select the parameter from the list and click Delete. If this parameter was also defined as a link parameter for data sources of this type, the alue defined at the link leel still applies to this data source. 8. Either click OK to sae your changes and exit, or click Apply and select another tab to continue defining options for this data source. 26 Extended Search Administration

35 Define smart hitlist actions If you define actions for a data source, you enable Extended Search to return a smart hitlist, one that enables users to do more than just browse and fetch documents returned in a hitlist. For example, you may want to enable users to sae the results, export results to an application, or results to other users. For information about how Extended Search supports smart hitlists, see Actions on page 96. You can define actions at the data source leel and link leel. If you define the same action at both leels, the link action alue seres as the default for the data source action. You can oerride the default alue to specify different actions for the results returned from this data source. Use the following procedure to configure hitlist actions for a data source: 1. Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB organized by link type or agent name. 3. Right-click the data source you want to configure and select Properties. 4. On the Data Source Properties window, select the Actions tab. 5. To add a new action, click Add Row. A blank line is added beneath the currently selected row or, if none is selected, to the end of the list. a. In Action name, type a name for this action. This name will either be displayed in the search results page or, depending on how your enterprise designed the results page, sere as the key to another label that is displayed in the search results. b. In Action URL, type the URL for the page that should be called when a user selects this action from the search results page. This page must contain logic to handle the requested action. 6. To modify an existing action, select it from the list and type a new name or URL, as necessary. 7. To remoe a preiously defined action, select the action from the list and click Delete. If this action was also defined as a link action for data sources of this type, the alue defined at the link leel still applies to this data source. 8. Either click OK to sae your changes and exit, or click Apply and select another tab to continue defining options for this data source. Export and import data sources You can export data source definitions and import them to another Extended Search domain. This feature, for example, can help you moe between test and production machines. To facilitate system testing, you can optionally import the exported data source data into the same domain. Exporting enables you to quickly deploy data sources without haing to go through data source discoery again. Note: Extended Search proides a wizard that makes it easy for you to export and import multiple data sources. For details, see Run configuration wizards on page 39. When you export a data source, the system creates an XML file that includes all the natie fields, mapped fields, field usage controls, parameters, actions, and categories configured for the data source (that is, it exports all the information defined in the Data Source Properties window). Tasks 27

36 Note that the export process does not include application-specific information that may hae been defined for the data source, such as application-specific field usage controls or the fetch order of fields returned in a hitlist. If you include this data source in applications that exist in the target Extended Search domain, you need to reconfigure your preferred application-specific properties. Make whateer changes you deem necessary for this data source in the new domain. Warning It is possible to modify the XML file that gets created before you import it. If you make changes, do so carefully. If you are unsure of what an XML entry is or what impact a change to its alue may hae, you are strongly cautioned to not change it; doing so may cause irreparable harm to the system. Use the following procedure to export data source data: 1. Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB organized by link type or agent name. 3. Select one or more data sources you want to export, right-click, and select Export. 4. In the Export Selected Items window, type the fully qualified path for a file on the Web serer where you want to export the data source data, and then click OK. Use the following procedure when you are ready to import data source data: 1. Ensure that a serer definition in the target system matches the serer from which you exported the data source. 2. Ensure that mapped fields and the link, grammar, and category definitions in the target system match the alues originally configured for the exported data source. 3. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. 4. Right-click in an open area of the Data Source iew and select Import. 5. In the Import Data window, type the fully qualified path for the file on the Web serer that contains the data source data you want to import, and then click OK. 6. If the name of the agent associated with this data source is not the default (agent), update the data source properties and select the correct agent name. Configure categories Categorization enables client applications to search data sources as a group and simplifies the user s iew of the system. When you add a data source to your Extended Search system, it may or may not be associated automatically with a particular category. When you use one of the predefined discoerers, the system automatically associates the target data source with a category that has the same name as the link type that goerns that data source. You can modify this association as necessary. If you deelop custom links and discoerers, you must explicitly associate each data source you add to the system with one or more categories. 28 Extended Search Administration

37 You can create hierarchies of categories, a feature that enables you to organize data sources into categories and subcategories much the way Windows Explorer organizes files into folders and subfolders. This arrangement allows you to fine-tune the way users search data sources. For example, a search that queries a subcategory searches only the data sources associated with that subcategory. A search that queries a top-leel category searches the data sources associated with it and the data sources associated with each of its subcategories. To configure a category, you need to first create the category and then assign data sources to it. Create a category Use the following procedure to create a category: 1. Start the Administration interface. 2. Either right-click Categories in the CDB Naigator and select New, or right-click in an open area of the Categories iew and select New. 3. On the Category Properties window, type a unique and descriptie name for this new category. Note that the category names and data source names must be unique. You cannot gie the category the same name you assign to a data source. 4. Select the locale for this category. This defines the language in which the category name will be displayed in the interface. 5. Click OK. There are a couple ways to create and work with subcategories. Follow the preceding steps to create a category but, when specifying the category name, type the name of an existing category you want to use as the top-leel category. Then type a backward slash (\) and the name of the new category. You can nest multiple leels of categories as necessary. For example: CategoryA\SubcategoryB CategoryA\SubcategoryB\SubcategoryC In the Categories iew, right-click the category that you want to make into a subcategory and select Copy. Then right-click the category you want to use as the top-leel category and select Paste. The category you copied, along with any data sources it contains, will be nested below the target category. Categorize a data source There are two ways that you can assign a data source to a category. If you are working in the Categories iew, you can select one or more data sources and assign them to a category. If you are working in the Data Sources iew, you can select an indiidual data source and, while defining other configuration options, assign the data source to a category. Note: When installed, Extended Search proides the following preconfigured categories. These groupings proide a conenient way for you to locate and select the data sources that you want to associate with a gien category. The [Not Categorized Data Sources] category contains only those data sources that hae not yet been categorized. The [System Defined Categories] category contains all the predefined categories, including: [All Data Sources], which contains all the data sources known to the Extended Search search domain, regardless of whether or not they hae Tasks 29

38 been categorized. After installation, this category lists the predefined Web Sources and ES Documentation, a data source that contains the Extended Search product documentation. When you discoer a new data source, the system automatically adds it to the [All Data Sources] category. Link-specific categories. For each predefined link type, Extended Search defines a category of the same name. When you discoer a data source of a gien link type, it is automatically added to the corresponding category. Categories View: Use the following procedure to categorize a data source from within the Categories iew: 1. Start the Administration interface. 2. Click Categories in the CDB Naigator. The iew pane displays all categories defined in the CDB and the data sources associated with each one. 3. In the iew pane, select one or more data sources that you want to categorize, right-click, and select Copy. 4. Right-click the target category and select Paste. The data sources you copied will be listed beneath the category s name. 5. Follow the procedures in Configure applications on page 31 to associate the category with one or more applications. 6. When you are ready to make your configuration changes aailable to Extended Search users, refresh the search domain. The following actions are also aailable for working with categories that are listed in the iew pane. Note that you cannot perform these actions if you select [System Defined Categories] or [Not Categorized Data Sources]. To remoe a data source from a category, right-click the data source name and select Remoe from this category. To delete an entire category, right-click the category name and select Delete. To rename a category, right-click the category name and select Properties. Type the new name and click OK. Data Sources View: Use the following procedure to categorize a data source from within the Data Sources iew: 1. Start the Administration interface. 2. Click Data Sources by Link or Data Sources by Agent in the CDB Naigator. The iew pane displays all data sources known to the CDB, and organizes them by link name or agent name. 3. Right-click the data source that you want to categorize and select Properties. 4. Select the Categories tab. 5. To see only the names of aailable categories, click Collapse All. To see the names of the data sources that belong to each category, click Expand All. 6. In the Aailable Categories list, select the category to which you want to assign this data source and click Assign. 7. To remoe the data source from a preiously assigned category, select the category from the Assigned Categories list and click Unassign. 8. Click OK. 9. Follow the procedures in Configure applications on page 31 to associate the category with one or more applications. 10. When you are ready to make your configuration changes aailable to Extended Search users, refresh the search domain. 30 Extended Search Administration

39 Configure applications When you add a data source to your Extended Search system, it is not associated automatically with any particular applications. To enable users to search the new data source, you must associate it with a category and then associate that category with at least one application. When you configure an application, you configure the particular fields a user can access through the application. You also specify the order in which fields are to be presented in the hitlist or fetched from the data source. For example, members of an accounting department may be interested in retrieing financial data from a data source that also contains personnel data about employees. By setting up separate Accounting and Human Resources applications, you can ensure that each user group accesses only those fields that are releant to their tasks. You can also present results in a manner meaningful to the intended use of the information. Follow the steps below to define an application. For each application, you must then define application-specific properties. If you are moing between systems, you may also want to reiew the procedures for exporting and importing applications. 1. Start the Administration interface. 2. Click Applications in the CDB Naigator. The iew pane displays all applications defined in the CDB. Listed beneath each application are the data sources assigned to the application, organized by category. Note: When installed, Extended Search proides two preconfigured applications: The Demo application is proided for example purposes. It allows you to search all predefined Web sources and the Extended Search product documentation immediately after installing the product. For information on how to use this sample application, see Setting up a Web Client on page 60. The [All Categories] application includes all the categories defined in the CDB. This list makes it easy for you to select the categories you want to associate with a gien application. The [All Categories] application is not aailable to client applications or end users. 3. To create a new application, either right-click Applications in the CDB Naigator and select New, or right-click in an open area of the Applications iew and select New. The Application Properties window is displayed. a. In Application name, type a unique, descriptie name for this new application, one that will enable you to readily recognize its intended purpose. Note that the application name can contain ASCII characters only, regardless of the language being used to run Extended Search. b. In Application description, type a description of the application. c. Select the Disable application check box only if you want to preent users from accessing this application. For example, you may need to preent access while you are working in test or debug mode. d. If you plan to use the Web Client, select the broker that should be used as the entry broker for this application from the Entry broker list. This is the first broker in a potential chain of brokers that receies queries and aggregates results for users of this application. (This item is not used by the Notes Client.) Tasks 31

40 Note: If a broker has been disabled or renamed, be sure to update this entry for all applications that pointed to that broker. Each application must identify the name of a alid, enabled broker. e. Click OK. 4. In the iew pane, right-click a category that contains the data sources you want to make aailable in this application and select Copy. Note: You must select a top- or root-leel category name. You cannot assign indiidual subcategories to an application. If a data source in a category being copied also belongs to a category that was preiously associated with the application, be aware that the system preseres the original application-specific properties. That is, it uses the application-specific properties defined for the data source in the category first associated with the application. 5. Right-click the target application and select Paste. The following actions are also aailable for working with applications that are listed in the iew pane. Note that you cannot perform these actions if you select the [All Categories] application. To remoe an entire category of data sources from an application, right-click the category name or icon and select Remoe from this application. Note that you cannot remoe indiidual data sources from an application. To delete an application, right-click the application name and select Delete. Define application-specific properties For each data source that you copied to the application, take the following steps to specify how fields in the data source should be accessed by users. Note: Extended Search proides a wizard that makes it easy for you to configure the fields you want to include in a hitlist. For details, see Run configuration wizards on page In the iew pane, right-click a data source that you want to configure for this application and select Properties. 2. On the Application-Specific Properties window, select the Field Usage tab. By setting the appropriate field usage controls, you can ensure that the intended users of this application access only those fields that contain data releant to their needs. If a field is marked Searchable, Returned in hitlist, orreturned in fetch, and you do not want users of this application to be able to search, iew, or retriee that field, click to remoe the checkmark. Alternatiely, you can select one or more fields, and then click Searchable, Hitlist Returned, orfetch Returned to toggle the checkmark for the selected fields. Note: If a field is not searchable or returnable in the base data source properties, it appears grayed out and is unaailable. You cannot make a field searchable returnable at the application leel unless you first make it searchable or returnable at the data source leel. If you are defining fields for an LDAP source, and plan to use the Notes Client application to search the source, you should configure the WebURL field as a returnable field. For greatest readability, you should also define this field as the second field returned in the Hitlist Field Order. 32 Extended Search Administration

41 3. Select the Hitlist Field Order tab. This tab lists the fields that can be returned in a hitlist. To specify the order in which the fields should be presented, select a field that you want to reorder. Next, click the up arrow button or down arrow button to position it. By setting the appropriate order, you can ensure that search results are presented to users in a format that enables them to easily identify the item they want to iew. 4. Select the Fetch Field Order tab. This tab lists the fields that can be returned to users and fetched from the data source. To specify the order in which the fields should be retrieed, select a field that you want to reorder. Next, click the up arrow button and down arrow button to position it. To fetch documents of a particular type, click Fetch file extension, and type or select the file extension for the type of document that users can retriee. 5. When you hae finished defining application-specific controls for the fields in this data source, click OK. 6. When you are ready to make your configuration changes aailable to Extended Search users, refresh the search domain. Export and import applications You can export application definitions and import them to another Extended Search domain. This feature, for example, can help you moe between test and production machines. To facilitate system testing, you can optionally import the exported application data into the same domain. Note: Extended Search proides a wizard that makes it easy for you to export and import multiple applications. For details, see Run configuration wizards on page 39. When you export an application, the system creates an XML file that includes all the data source, category, and application-specific data configured for the application (that is, it exports all the information defined in the Application Properties and Application-Specific Properties windows). Warning It is possible to modify the XML file that gets created before you import it. If you make changes, do so carefully. If you are unsure of what an XML entry is or what impact a change to its alue may hae, you are strongly cautioned to not change it; doing so may cause irreparable harm to the system. Use the following procedure to export application data: 1. Start the Administration interface. 2. Click Applications in the CDB Naigator. The iew pane lists all applications defined in the CDB. 3. Select one or more applications you want to export, right-click, and select Export. 4. In the Export Selected Items window, type the fully qualified path for a file on the Web serer where you want to export the application data, and then click OK. 5. After the export process successfully completes, a message indicates how many applications were exported. Click OK. Use the following procedure when you are ready to import application data: Tasks 33

42 1. Ensure that a serer definition in the target system matches the serer from which you exported the application. 2. Ensure that the application name does not already exist in the target system. 3. Ensure that any mapped fields defined for data sources in the exported application exist in the target system. 4. Click Applications in the CDB Naigator. 5. Right-click in an open area of the Applications iew and select Import. 6. In the Import Data window, type the fully qualified path for the file on the Web serer that contains the application data you want to import, and then click OK. Usage Guidelines The import process properly handles multi-leel categories and data sources that occur in multiple categories. It also automatically assigns applications to new broker names that were generated when serer definitions on the target system were created. Howeer, certain limitations exist: If any like-named data sources already exist in the target system, the import process appends an identifier, DB_ID, to the name of the imported data source. If any like-named categories already exist in the target system, the import process will not import the exported category data. Configure links When you add a data source to Extended Search, the data source discoery process automatically associates it with an appropriate link type. The link definition identifies a shared library that enables Extended Search to access data sources of that type. Extended Search comes with many predefined link types, which you should be able to use as proided. Howeer, you can create a new link or copy an existing link and modify it as necessary. For more information about how Extended Search uses links, see Links on page 123. For a description of the preconfigured link types, see Predefined links on page 123. Take the following steps to configure a new link: 1. Start the Administration interface. 2. Click Links in the CDB Naigator. The iew pane displays all links defined in the CDB. 3. Either right-click Links in the CDB Naigator and select New or, if you are working in the Icon View, right-click in an open area of the Links iew and select New. 4. On the Link Properties window, type a descriptie name for the link, one that indicates the type of data source that is targeted by this link. 5. Optionally type a description of the link. 6. In Hitlist URL exit, type the name of the Jaa class you want to use to control the format of URLs returned in the hitlist. For a description of the classes used by the predefined links, see Hitlist URL classes on page 122. For information 34 Extended Search Administration

43 about deeloping your own custom class, see the discussion of the CustomHitReturn Interface in Extended Search Programming. 7. Click Add Row. A blank line is added beneath the currently selected row or, if none is selected, to the end of the list. 8. In Discoerer library name, type the name of the shared library that contains logic to discoer data sources of this type. This discoerer will be called when adding data sources of this link type. You can use an existing discoerer or deelop your own. For information about Deeloping Discoerers, see Extended Search Programming. 9. In Link library name, type the name of the shared library that contains logic to perform search and retrieal operations in data sources that are associated with this link type. You can use an existing link or deelop your own. For information about Deeloping Links, see Extended Search Programming. Note: Links typically work with an agent to fetch documents that are returned in a hitlist from the target data source. Howeer, a hitlist returned from a Web source contains URLs that allow the browser to fetch the document instead of the link and agent. 10. In Platform, select the operating system where this discoerer and link will run. 11. If you want to associate another discoerer or shared library with this link (for example, to use on a different platform), repeat steps 7 through If you want to delete a shared library that was preiously associated with this link, select its name in the list and click Delete. 13. If you want to define or modify link parameters for this link, see Define link parameters on page If you want to define or modify hitlist actions for this link, see Define link actions on page Click OK to sae your changes and exit. Take the following steps to modify a link or disseminate new and changed links to your user community: 1. To change an existing link, right-click the link s name or icon in the iew pane and select Properties. You cannot change the name of a preiously defined link. You can change the description as needed. You can change the names of the shared libraries that are associated with the link. You can also add or change link parameters and hitlist actions. After making your changes, click OK. 2. To copy a link, click the Icon View icon, right-click the link s icon in the iew pane and select Copy. Then right-click in an open area of the iew pane and select Paste. You must specify a new name for this link. Make other changes as necessary and then click OK. Note that when you copy a link, you copy its complete definition, including any link parameters and hitlist actions that may be defined for it. 3. To delete a link, right-click the link s name or icon in the iew pane and select Delete. 4. To associate the link with one or more data sources: a. Click Data Sources By Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB. b. Select one or more data sources that you want to associate with this link, right-click, and select Properties. Tasks 35

44 c. At Link, select the name of your new link. d. Either click OK or refer to Data Source Properties single data source on page 178 for information about specifying other information about the data source. 5. When you are ready to make your configuration changes aailable to Extended Search users, refresh the search domain. Note that if you deeloped a custom shared library for a link, you must shut down and restart Extended Search to actiate it. Define link parameters Link parameters proide a way for you to specify options for all data sources associated with a gien link type. If you define the same parameter at the data source leel, the link parameter s alue seres as the default alue for the data source parameter. The alue of the data source parameter oerrides the link-leel alue for that particular data source. During data source discoery, many parameters are automatically configured for you. Predefined parameters on page 132 describes the parameters and proides information about which ones you can specify at the link leel. Note: Extended Search proides a wizard that makes it easy for you to configure data source parameters and associate them with specific data sources or all data sources of a gien link type. For details, see Run configuration wizards on page 39. Use the following procedure to configure a link parameter: 1. Start the Administration interface. 2. Click Links in the CDB Naigator. The iew pane displays all links defined in the CDB. 3. Right-click the link type for which you want to configure parameters and select Properties. 4. On the Link Properties window, select the Parameters tab. 5. Take the following steps to define a new link parameter: a. Click Add Row. A blank line is added beneath the currently selected parameter or, if none is selected, to the end of the list. b. In Link parameter name, type the name of the parameter you want to set for this link. c. In Value, type the parameter s alue. When the link gets initialized, the parameter will be set to this alue, and will apply to all operations within data sources of this link type. d. In Description, briefly describe the purpose of this parameter. 6. To modify an existing link parameter, select it from the list and type a new name, alue, or description as necessary. 7. To remoe a preiously defined parameter, select the parameter from the list and click Delete. If this parameter was also defined as a data source parameter, the alue defined at the data source leel still applies. 8. Either click OK to sae your changes and exit, or click Apply and select another tab to continue defining options for this link. Define link actions If you define actions for a link, you enable Extended Search to return a smart hitlist, one that enables users to do more than just browse and fetch documents returned in a hitlist. For example, you may want to enable users to sae the 36 Extended Search Administration

45 results, export results to an application, or results to other users. For information about how Extended Search supports smart hitlists, see Actions on page 96. You can define actions at the data source leel and link leel. An action defined at the link leel applies to all data sources of that link type. If you define the same action at both leels, the link action alue seres as the default for the data source action. The alue of the data source action can oerride the link-leel alue for that particular data source. Use the following procedure to configure hitlist actions for a link: 1. Start the Administration interface. 2. Click Links in the CDB Naigator. The iew pane displays all links defined in the CDB. 3. Right-click the link type you want to configure and select Properties. 4. On the Link Properties window, select the Actions tab. 5. To add a new action, click Add Row. A blank line is added beneath the currently selected row or, if none is selected, to the end of the list. a. In Action name, type a name for this action. This name will either be displayed in the search results page or, depending on how your enterprise designed the results page, sere as the key to another label that is displayed in the search results. b. In Action URL, type the URL for the page that should be called when a user selects this action from the search results page. This page must contain logic to handle the requested action. 6. To modify an existing action, select it from the list and type a new name or URL, as necessary. 7. To remoe a preiously defined action, select the action from the list and click Delete. If this action was also defined as a data source action, the alue defined at the data source leel still applies. 8. Either click OK to sae your changes and exit, or click Apply and select another tab to continue defining options for this link. Set up a remote EIP link If you hae IBM Enterprise Information Portal, it is not installed on the Extended Search serer, and you want to include EIP sources in your Extended Search domain, you need to configure a remote link to support them. 1. Using procedures in Extended Search Installation, install the required software. The following steps assume that Serer A is the Web Serer that supports Extended Search and Serer B is the EIP serer. a. Install the Extended Search Web Serer and Reflector components on Serer A. b. Install and configure EIP with the data sources you plan to support on Serer B. Consult your EIP documentation for assistance. c. Install the Extended Search Serer component on Serer B. Make sure that you specify the same Web Serer host name and port number that you specified when you installed the Extended Search Web Serer component on Serer A. d. If you expect users to access Extended Search from a Notes client, install the Extended Search Notes Client component on a Domino serer, and then use the EIP Administration tool to map the Lotus Notes ID (long name) to an EIP user name and password. Tasks 37

46 Note that you do not need to perform this step if you plan to use the EIP C++ and Jaa samples for Extended Search. 2. Configure a remote link for EIP: a. Start the Administration interface. b. Follow the procedure in Add another Extended Search serer on page 17 to add Serer B to the configuration database. Use the following guidelines when specifying Serer Properties for Serer B: On the Broker tab, make sure that the Disable Broker check box is checked. On the Agents tab, assign a unique name to the agent, such as EIP-Agent. c. Follow the procedure in Connect multiple Extended Search serers on page 18 to connect Serer B to Serer A. d. Start Extended Search on Serer B. e. While still in the Serers iew, right-click Serer B and select Discoer Data Sources. If you need assistance when proiding discoery information, see IBM Enterprise Information Portal discoerer on page 183. f. Follow the procedures in Configure categories on page 28 and Configure applications on page 31 to associate the data sources with a category and to assign the category to an application. g. Refresh the search domain to update Extended Search with your configuration changes. h. Stop and restart Extended Search on Serer B. 3. Take the following steps to erify your changes: a. Start the Monitor for Serer A. Make sure that all the brokers and agents are actie. You should not see the newly added EIP agent on this serer. b. Start the Monitor for Serer B. You should see only the newly added EIP agent on this serer. Configure grammars When you add a data source to Extended Search, the discoery process automatically associates it with an appropriate grammar. The grammar definition identifies a shared library that is responsible for translating a GQL statement into the search grammar natie to the data source s link type. Extended Search comes with many predefined grammars, which you should be able to use as proided. Howeer, you can create a new grammar or copy an existing grammar and modify it as necessary. For more information about how Extended Search uses grammars, see Grammars on page 121. For a description of the preconfigured grammars, see Predefined grammars on page 121. Take the following steps to configure a new grammar: 1. Start the Administration interface. 2. Click Grammars in the CDB Naigator. The iew pane displays all grammars defined in the CDB. 3. Either right-click Grammars in the CDB Naigator and select New, orifyou are working in the Icon View, right-click in an open area of the Grammars iew and select New. 38 Extended Search Administration

47 4. On the Grammar Properties window, type a descriptie name for the grammar, one that indicates the type of data source or link that is targeted by this grammar. 5. Optionally type a description of the grammar. 6. Click Add Row. A blank line is added beneath the currently selected row or, if none is selected, to the end of the list. 7. In Library name, type the name of the shared library that should be called when a user submits a query against data sources that are associated with this grammar. You can use an existing grammar or deelop your own. For more information about deeloping an Extended Search Translator, see Extended Search Programming. 8. In Platform, select the operating system where this grammar will run. 9. If you want to associate another shared library with this grammar (for example, to use on a different platform), repeat steps 6 through If you want to delete a shared library that was preiously associated with this grammar, select its name in the list and click Delete. 11. Click OK. Take the following steps to modify a grammar or disseminate new and changed grammars to your user community: 1. To change an existing grammar, right-click the grammar s name or icon in the iew pane and select Properties. You cannot change the name of a preiously defined grammar. You can change the description as needed. You can also type the name of a different shared library that you want to use to handle the translation of GQL into the natie search grammar. After making your changes, click OK. 2. To copy a grammar, right-click the grammar s name or icon in the iew pane and select Copy. Then right-click in an open area of the iew pane and select Paste. You must specify a new name for this grammar. Make other changes as necessary and then click OK. 3. To delete a grammar, right-click the grammar s name or icon in the iew pane and select Delete. 4. To associate a new grammar with one or more data sources: a. Click Data Sources By Link or Data Sources by Agent in the CDB Naigator. The iew pane lists all data sources defined in the CDB. b. Select one or more data sources that you want to associate with this grammar, right-click, and select Properties. c. At Grammar, select the name of your new grammar. d. Click OK. 5. When you are ready to make your configuration changes aailable to Extended Search users, refresh the search domain. Note that if you deeloped a custom shared library and associated it with a grammar, you must you must shut down and restart Extended Search to actiate it. Run configuration wizards The Administration interface proides seeral wizards that can make it easier for you to perform certain configuration tasks. Before using a wizard, you should be familiar with the basic task that is being facilitated and understand any requirements that apply when the task is performed manually. Tasks 39

48 Wizard Configure mapped fields Description Use this wizard to select a mapped field to which you want to map one or more natie fields. Selection criteria enables you to easily map fields that belong to similarly named data sources or data sources of a specific link type, fields that hae similar names or are of a specific data type, or any combination of these criteria. Reiew the following topics before using this wizard: Mapped fields on page 120 Define mapped fields on page 23 Configure data source parameters Use this wizard to define a data source parameter and apply it to a specific data source, data sources with similar names, or data sources of a specific link type. Reiew the following topics before using this wizard: Parameters on page 130 Define data source parameters on page 26 Configure hitlist fields Use this wizard to design a hitlist and specify whether it applies to all data sources in a search application, or to data sources within the application that hae similar names or that are of a specific link type. Reiew the following topics before using this wizard: Set field usage controls on page 25 Define application-specific properties on page 32 Export and import data Use this wizard to export data sources, applications, or the configuration database, and to import data sources and applications that you preiously exported. Reiew the following topics before using this wizard: Export and import data sources on page 27 Export and import applications on page 33 Performing backups and restores on page 92 Take the following steps to run a configuration wizard: 1. Start the Administration interface. 2. Click Wizards in the CDB Naigator. The iew pane displays a list of aailable wizards. 3. Right-click the wizard you want to run and select Display. 4. After the wizard starts, follow the online instructions to complete the task. Refresh Web source definitions As discussed in Web sources on page 142, the Extended Search CDB comes preconfigured with data source definitions for many popular World Wide Web sites. Each Web data source is preconfigured with a standard set of field and parameter definitions. A single file contains the interface descriptions for all of the supported Web sites. If you install an Extended Search serer on a Windows platform, you can configure the serer to read this file at startup and at a scheduled frequency. Use the following procedure to specify how often Extended Search should refresh these predefined Web source definitions. 1. Start the Administration interface. 40 Extended Search Administration

49 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer you want to work with and select Properties. 4. On the Serer Properties window, select the Extended Search tab. Page 195 describes these options. 5. In the Internet settings area, reiew the proxy settings for the Web serer. Typically you do not need to change these settings. 6. Ensure that the Disable download check box is clear. 7. In Download frequency, specify how often, in hours, you want the serer to refresh the definitions of the Web data sources. 8. Click OK. Search my own Web sites As discussed in Web sources on page 142, Extended Search is preconfigured with data source definitions for many popular World Wide Web sites. If there are other Web sources that you want to make aailable for searching, you can easily add them to your Extended Search search domain. The procedure below describes one approach for adding support for your own Web sites. For guidelines on using other approaches, see Adding support for Web sites on page 143. The first step is to contact Intelligent Algorithms Enterprises, Ltd., to obtain an infogist toolkit that enables you to build your own Web source definition file (.sbb file). For more information about the toolkit, and access to it, select the Business Partner Offerings option on the Extended Search Web Site ( Note that the Intelligent Algorithms code is not fully enabled for national language support. Through the Extended Search Web Sources link and a.sbb file, you can search for languages that use the ISO code page (such as English, French, German, Portuguese, Swedish, and so on). You cannot search double byte character set (DBCS) languages such as Korean, bi-directional (bi-di) languages such as Hebrew, or other non-iso languages. After creating a.sbb file that defines your Web sources, use the following procedure to configure support for your new Web sources in Extended Search: 1. Ensure that your custom.sbb file is stored in the base Extended Search installation directory. 2. Start the Administration interface. 3. Discoer the new Web sources: a. Click Serers in the CDB Naigator. The iew pane displays all serers defined in the CDB. b. Right-click the serer where the new Web sources are aailable and select Discoer Data Sources. c. In Type of source to discoer, select Web Sources. d. In ESWebConfig link parameter alue, type the name of your custom.sbb file. e. In Language used, select the locale of the Web sources. f. Click Start Discoery. Tasks 41

50 g. After the Web sources that match those defined in your.sbb file are displayed, select the ones that you want to add to your search domain, and then click Add to ES. h. Click Close to close the Discoer Data Sources window. 4. Configure the Web Sources link: a. Click Links in the CDB Naigator. The iew pane displays all links defined in the CDB. b. Right-click the Web Sources link in the iew pane and select Properties. c. Click the Parameters tag, and then select the ESWebConfig Value field. d. Append a question mark (?) and the name of your.sbb file to the existing ESWebConfig alue. For example: deswebdef.sbb?custom.sbb. e. Click OK. 5. Group the new Web data sources into a category, and associate the category with at least one application. 6. To update the system with your configuration changes, refresh the search domain. Configure a Web crawler As discussed in Web crawlers on page 137, you can configure an Extended Search serer to automatically traerse Web sites and download data to a local repository. You can then use an indexing program to index the data, include it in your Extended Search domain, and make it aailable to users through a client application. 42 Extended Search Administration To use Web crawling, you must enable the Extended Search serer to use it, and then specify options to control when the crawlers should run and which Web sites they should run against. Each crawler that you enable uses a unique configuration file that identifies the specific Web site to be crawled. Web crawler configuration file on page 138 describes escrawler.xml, a sample configuration file included with the product. Use the following procedure to configure a Web crawler. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer on which you want to enable Web crawling, and then select Properties. 4. Select the Web Serer tab, and then select Scheduled Crawling to expand the options. 5. Ensure that the Enable crawling check box is selected. 6. In Check for scheduled crawling, specify how often (in minutes) the crawler process on the Web serer should check for the existence of newly scheduled requests. If the crawler process detects a crawler request that is ready to run, it sends the request to the Extended Search serer for processing. 7. Select the Extended Search tab, then select Crawling to expand the options. 8. Click Add Row. A blank line is added beneath the currently selected crawler or, if none is selected, to the end of the list. 9. Select the Enable check box to enable the newly added Web crawler process on this serer. 10. In Crawler, select the type of Web crawler you want to use. In this release, you must select the escrawler crawler.

51 11. In Configuration File, type the fully qualified path for the XML configuration file used by this Web crawler. 12. In Start Day, type the day of the week that this Web crawler should begin crawling the Web. The default start day is Sunday. 13. In Start Time, type the time of day, using a 24 hour clock, that this Web crawler should begin crawling the Web. The default start time is midnight (24:00). 14. In Frequency, type a alue that indicates how often Web crawling should be repeated. (This alue works in conjunction with Interal.) The default frequency is once (1), such as once eery week. 15. In Interal, select a alue that indicates how often Web crawling should be repeated. (This alue works in conjunction with Frequency.) You can configure the Web crawler to run on a monthly, weekly, daily, or hourly basis. The default interal is weekly. 16. Click OK. 17. To update the system with your changes, refresh the search domain. You can also take the following actions when configuring a Web crawler: If you do not want any Web crawling to be performed, clear the Enable crawling check box. If you want to temporarily suspend a particular crawler, clear the Enable check box by its name to disable it. If you want to remoe a preiously defined crawler from this serer, select it from the list and click Delete. Refresh the search domain Any time that you update the configuration database, you need to refresh the search domain to disseminate the changes to your user community. Note: If you change configuration alues on the Extended Search tab of the Serer Properties window, you must also shut down and restart the Extended Search serer for the changes to take effect. These tasks include modifying the serer heap size, Broker settings, Agent settings, and Internet settings. If you change eent logging options for discoerers and Web crawlers, you must also shut down and restart the Extended Search serer for these changes to take effect. Take the following steps to enable users to access new or changed configuration data. 1. Refresh the CDB. Right-click Search Domain in the CDB Naigator and select Refresh Domain. This action refreshes the Extended Search search domain with your configuration changes. 2. Web Client users will not experience the changes until they start a new browser session. Instruct your users to close and then re-open their Web browsers. 3. If you use the Notes Client, refresh the Notes Client application. Open the Notes Client Administration View and click Setup Sources. See Run the Setup Sources agent on page 67 for assistance with this step. Use search templates to test changes Extended Search proides seeral search templates that enable you to erify changes you make to the configuration database. Tasks 43

52 Note: The templates demonstrate many of the Extended Search features that you can implement when deeloping your own Web Client applications. These templates hae been partially tested, and are proided to illustrate functionality. They may not look and function the same in all enironments. Use the following procedure to test your configuration changes: 1. Refresh the search domain. 2. Click Search Templates in the CDB Naigator, and then select one of the following categories: The Demo template is the most comprehensie example. You should use this template to erify the installation of Extended Search and to test configuration changes. The Other Samples illustrate different ways that you can alter the look and feel of Web Client applications, and demonstrate the use of adanced GQL queries. 3. When the iew pane lists the aailable search templates, right-click the template you want to use and select Display. The search templates hae been implemented in different ways to illustrate different application deelopment approaches. The search templates aailable to you depend on the Web serer used with this Extended Search system: JKM templates. These search templates illustrate how to incorporate Extended Search functionality in HTML pages by embedding Extended Search proprietary JKM tags and beans. JSP 1.0 templates. These search templates illustrate how to incorporate Extended Search functionality in JSP pages by embedding Jaa code and Extended Search beans. The templates conform to the Sun JaaSerer Pages 1.0 standard. JSP 1.1 templates. These search templates illustrate how to incorporate Extended Search functionality in JSP pages by embedding an Extended Search tag library. The templates, which conform to the Sun JaaSerer Pages 1.1 standard, hae names that begin with tag. If you use a Domino serer, you must select a JKM template. If you use a WebSphere serer, you can select either a JKM template or a JSP template. Display Name Template Name Description Demo AllOptions.txt (JKM) AllOptions.jsp (JSP 1.0) tagalloptions.jsp (JSP 1.1) This template, which uses a tabbed page format and is the most comprehensie example, includes most of the options supported by Extended Search. It allows you to easily construct a complex GQL query, schedule the query, sae the query and search results, and specify constraints. It includes a scrolling headline ticker for retrieing hitlist documents. The hitlist includes options for iewing detailed or summary information, and for selecting actions to process hitlist items. Other Samples 44 Extended Search Administration

53 Display Name Template Name Description Framed Sample Adanced GQL Sample Framed.txt (JKM) Framed.jsp (JSP 1.0) tagframed.jsp (JSP 1.1) gqlad.txt (JKM) gqlad.jsp (JSP 1.0) This template is diided into three frames. The top frame contains the search terms box and a list of searchable data sources. A frame on the left displays the search results. After you select a document to retriee, it is displayed in a frame on the right. This template, which requires Microsoft Internet Explorer ersion 4.0 or later, demonstrates the power of searching with GQL. It uses a tabbed format and includes features that enable you to build a complex GQL query by selecting fields, operators, and boolean constructs. 4. When the template is displayed, specify search terms, select the categories or data sources you want to search, and optionally set search and retrieal options. When ready, select the action to submit your search request. Note: If you are testing the initial installation of Extended Search, you should submit a query against the Web Sources or ES Documentation category. Web sources for many Web search sites are preconfigured in Extended Search, enabling you to submit queries without configuring your own data sources. If you experience problems when you try to connect to the Web, consult the FAQ on the Extended Search Web Site. 5. After the search results are returned, click arious entries to erify your ability to iew documents that are listed in the hitlist. Usage Notes When you launch one of these search templates, the Demo application name is hard-coded in the link from the Administration interface. To test your Web Client application, change the application name in the URL displayed in the browser. For example, change &AppID=Demo to &AppID=Finance. If you open a search template from within the same browser session, you need to refresh the browser to effect the change. Press the Shift key while you click the browser s Refresh button. By default, the Extended Search templates allow up to 100 matching hits to be returned from each data source. Be aware that certain Web search engines impose their own limitations on the number of results that can be returned. This number may be less than the default Extended Search limitation. Log and iew eents Extended Search proides features that enable you to track serer actiity and resole problems that may arise. Eent management is a two-step process: You must first enable eent logging on indiidual Extended Search serers and Web serers. You can then use the eent log iewer to reiew the messages and determine the appropriate correctie action. You can optionally enable the broker to log message traffic. Tasks 45

54 46 Extended Search Administration If you use the Web Client application, you can optionally configure a serlet user exit to perform custom logging. Enable eent logging Use the following procedure to log error messages according to the seerity of the eent that occurs. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer on which you want to enable eent logging and select Properties. Note that you can enable eent logging only at the serer leel. You cannot enable logging on a product component basis. 4. On the General tab of the Serer Properties window, click Eent logging to expand the entries beneath it. 5. For each type of eent (critical, non-critical, warning, or informational), select the Write to log check box if you want errors of that type to be written to the eent log. Clear the check box if you do not want to log error messages for a particular eent type. 6. If your Extended Search support representatie instructs you to trace a component s processing, click Tracing to expand the entries beneath it. a. Select the Enable tracing check box. b. In Trace leels, type the trace leel (or range of leels) appropriate to the process you need to diagnose (see page 223 for a description of alid trace codes). Separate trace codes by a comma (,). Join a range of trace codes with a dash ( ). For example: , 20 40, c. In Trace directory, type the fully qualified path for a directory where the trace log file should be created. This directory should already exist on the Web serer. 7. Click OK. 8. To hae your eent logging choices take effect, refresh the search domain. Note that the Extended Search discoerer and Web crawler responders do not get refreshed on a Refresh Domain request. To hae your eent logging choices take effect for discoery and Web crawling, you must stop and restart the Extended Search serer. View eent logs If you enabled eent logging for particular types of eents, error messages will be logged to the Extended Search configuration database. You can then use the eent log iewer to iew and administer the messages. 1. Start the Administration interface. 2. Click Eent Log Viewer in the CDB Naigator. The iew pane displays the contents of the eent log organized by the names of the Extended Search serers that logged eents. For each eent, the iewer displays the name of the serer that logged the eent, the eent type or seerity leel of the eent, the time and date the eent was logged, and the error message that describes the eent. 3. By default, eents are listed in descending order. You can toggle the sort order on a column-by-column basis. To sort entries in ascending order, click the column you want to reorder. Click the column again to return to descending order.

55 4. To get details about a particular eent, either double-click it, or right-click it and select Properties. The Eent Properties window shows the name of the file that generated the eent, the method that defines the eent, and the line number in the code where the eent originates. 5. If you want to print the eent log, click Print Preiew. A printable ersion of the eent log is displayed, enabling you to use your browser s print function to print it. This feature is proided as a template, msglistprintpreiew.txt, which you can customize for your reporting purposes. 6. If you want to sae the contents of the eent log to a file on the serer, click Sae As. When prompted, type the fully qualified path for a file in which to sae the eent log messages. This feature is proided as a template, msglistsae.txt, which you can customize for your reporting purposes. 7. If you want to clear the contents of the eent log, click Clear All. To consere disk space and to keep the size of the eent log to a manageable length, you should clear the eent log on a regular basis. Enable custom broker logging You can instruct the broker to keep a log of its actiities and to record message traffic between client applications and the target data sources. To implement broker-leel logging, you need to deelop a custom user exit to handle it. For instructions, see the discussion of User Exits in Extended Search Programming. You must then use the following procedure to configure the broker to call this exit. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer that runs the broker you want to configure and select Properties. 4. On the Serer Properties window, select the Extended Search tab and then click Broker to expand the entries beneath it. 5. In Custom logging library, type the name of the custom shared library that you want the broker to use. If the library is not stored in the Extended Search installation directory, you must type its fully qualified pathname. 6. In Encrypt logged passwords, select the check box if you want to pass passwords to the library in an encrypted format. If you clear the check box, passwords will not be passed. 7. Click OK. 8. To begin using the exit with future requests that this broker receies, refresh the search domain. Note that you must shut down and restart Extended Search to actiate your custom exit. Tip The Extended Search installation comes with sample shared libraries for customized logging: sbloggingdll.dll (Windows), libsbsamplelog.a (AIX) and libsbsamplelog.so (Solaris). To test broker logging, you can enter one of these file names in step 5. For more information about these samples, see the discussion of User Exits in Extended Search Programming. Tasks 47

56 Enable custom serlet logging You can instruct the Extended Search serlets to keep a log of their actiities and to record message traffic between Web Client applications and the Web serer. To implement serlet-leel logging, you need to deelop a custom Jaa class to handle it. For instructions, see the discussion of User Exits in Extended Search Programming. You must then use the following procedure to configure the serer to call this exit. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer that uses the Web serer you want to configure and select Properties. 4. On the Serer Properties window, select the Web Serer tab and then click Logging Exit to expand the entries beneath it. 5. Select the Enable logging user exit check box to enable custom serlet logging for this Extended Search serer. 6. In Custom log handler, type the name of your custom Jaa class. This class must exist in the Web application serer s serlet directory. 7. In Log file name, type the name of a file to which logged output should be written. 8. In Logging directory, type the fully qualified path for a directory where the log file should be created. 9. Click OK. 10. To begin using the exit with future Web Client requests, refresh the search domain. Note that you must shut down and restart Extended Search to actiate your custom exit. Tip The Extended Search installation comes with a sample Jaa class, CustomLogger.jaa, that you can use as a guideline for writing an enterprise-specific logging class. For more information about this sample code, see the discussion of User Exits in Extended Search Programming. Configure security options To enhance the security of your Extended Search system, you can alidate users against a data source s access control list, and enable authentication at both the broker and agent leels. Security-related configuration tasks include the following: Using impersonation to erify user access permissions Enabling serlets to perform Web Client authentication Enabling the broker to authenticate users Enabling the agent to authenticate users Before performing any of these tasks, be sure to reiew the discussion of Security (beginning on page 145) to learn about the arious ways that security has been implemented in Extended Search. 48 Extended Search Administration

57 Enable impersonation When a user submits a query to search a Notes data source, Extended Search does not automatically check the permissions granted to that user per the access control list (ACL) defined for that data source. You can enhance security by setting a parameter, Impersonate, that instructs Extended Search to compare the requesting user s name against the Notes ACL. Before you configure the Impersonate parameter, be sure to read about how Extended Search interacts with Lotus Notes to implement Notes access controls. This discussion begins on page 149. If you set Impersonate as a link parameter, the system performs ACL erification against all Notes and Notes 5.0 data sources. If you set Impersonate as a data source parameter, the system performs ACL erification only against the data source in which you configured the parameter. Use the following procedure to set Impersonate as a link parameter: 1. Start the Administration interface. 2. Click Links in the CDB Naigator. The iew pane displays all links defined in the CDB. 3. Right-click a link for Notes or Notes 5.0 and select Properties. 4. On the Link Properties window, select the Parameters tab and then click Add Row. A blank line is added beneath the currently selected parameter or, if none is selected, to the end of the list. 5. In Link parameter name, type Impersonate. 6. In Value, type Yes. 7. In Description, briefly describe the purpose of this parameter. 8. Click OK. Use the following procedure to set Impersonate as a data source parameter: 1. Start the Administration interface. 2. Click Data Sources By Link or Data Sources By Agent in the CDB Naigator. The iew pane displays all links defined in the CDB. 3. Right-click a Notes or Notes 5.0 data source and select Properties. 4. On the Data Source Properties window, select the Parameters tab and then click Add Row. A blank line is added beneath the currently selected parameter or, if none is selected, to the end of the list. 5. In Parameter name, type Impersonate. 6. In Value, type Yes. 7. In Usage, select Used by data source. 8. Click OK. Tasks 49

58 ODBC and Web Source Impersonation You can also set Impersonate as a parameter for ODBC sources and Web Sources to indicate whether or not the data source requires a user ID and password for access. If enabled, Extended Search will attempt to establish a connection to the target data source by using the user ID and password passed in on the client request. See Predefined parameters on page 132 for information about how impersonation works in conjunction with the DatabaseRemainOpen parameter when setting a persistent connection to an ODBC data source. Enable serlet-leel authentication To control which Web Client users are able to access Extended Search data sources, you can enable the serlets to perform user authentication. To set up this leel of support, you must enable user authentication on the Web serer and encode enterprise-specific rules in a custom Jaa class. For instructions, see the discussion of User Exits in Extended Search Programming. You must then use the following procedure to configure the serer to call this exit. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer that uses the Web serer you want to configure and select Properties. 4. On the Serer Properties window, select the Web Serer tab and then click Authentication to expand the entries beneath it. 5. In Authentication type, select Custom. Extended Search will first perform basic client authentication, in which the serlet serer uses its own security methods to authenticate the user, and then call your custom authentication class. Note: Be sure to read about how enabling authentication on non-domino Web serers impacts the operation of the Administration interface (see page 147). 6. In Authentication realm, type a realm name only if your Web serer s configuration defines a realm that collects users into groups for authentication purposes. Some Web serers use a realm alue along with the user ID and password when performing user authentication. If true for this Web serer, type the realm name. Note that this alue is ignored by Domino serers, which use a public address book (PAB) and the user s ID and password to perform authentication. 7. In Custom authentication library, type the name of your custom Jaa class. This class must exist in the Web application serer s serlet directory. 8. Click OK. 9. To begin using the exit with future Web Client requests, refresh the search domain. Note that you must shut down and restart Extended Search to actiate your custom exit. 50 Extended Search Administration

59 Tip The Extended Search installation comes with a sample Jaa class, CustomAuthenticator.jaa, that you can use as a guideline for writing an enterprise-specific authentication class. For more information about this sample code, see the discussion of User Exits in Extended Search Programming. Enable broker-leel authentication Upon receipt of a search request, the broker can be instructed to determine whether or not the user is authorized to access all the target data sources, access a data source with the parameters specified in the request, issue a search request against a particular data source, or retriee content from a data source. To implement this leel of access control, you need to deelop a custom user exit to handle it. For instructions, see the discussion of User Exits in Extended Search Programming. You must then use the following procedure to configure the broker to call this exit. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer that runs the broker you want to configure and select Properties. 4. On the Serer Properties window, select the Extended Search tab and then click Broker to expand the entries beneath it. 5. In Custom authentication library, type the name of the shared library that you want the broker to use. If the library is not stored in the Extended Search installation directory, you must type its fully qualified pathname. 6. Click OK. 7. To begin using the exit with future requests that this broker receies, refresh the search domain. Note that you must shut down and restart Extended Search to actiate your custom exit. Tip The Extended Search installation comes with sample shared libraries for customized authentication: sbvalidatedll.dll (Windows), libsbsamplevalid.a (AIX), and libsbsamplevalid.so (Solaris). To test broker authentication, you can enter one of these file names in step 5. For more information about these samples, see the discussion of User Exits in Extended Search Programming. Enable agent-leel authentication Prior to sending a query to a search engine, the agent can be instructed to determine whether or not the requesting user has permission to access the target data source. After search results are returned, the agent can assess the responses one by one. It can determine whether the user is allowed to iew particular references in a hitlist or retriee referenced documents. Tasks 51

60 To implement this leel of access control, you need to deelop a custom user exit to handle it. For instructions, see the discussion of User Exits in Extended Search Programming. You must then use the following procedure to configure the agent to call this exit. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer that runs the agent that you want to configure and select Properties. 4. On the Serer Properties window, select the Extended Search tab and then click Agents to expand the entries beneath it. 5. Select the agent that you want to configure from the list of agents. 6. In Agent exit library, type the name of the custom shared library that you want the agent to use. If the library is not stored in the Extended Search installation directory, you must type its fully qualified pathname. 7. Click OK. 8. To begin using the exit with future tasks that this agent handles, refresh the search domain. Note that you must shut down and restart Extended Search to actiate your custom exit. Configure scalability options To enhance the scalability of your Extended Search system, you can perform such tasks as configuring multiple brokers and separating brokers from the Web serer. You can also configure remote links and partition multiple agents on a single machine. Scalability-related configuration tasks include the following: Separating brokers from the Web serer Configuring remote links Partitioning agents and data sources on a single machine Configuring multiple brokers Before performing any of these tasks, be sure to reiew the discussion of Scalability (beginning on page 156) to learn about the arious ways that you can extend your Extended Search system. Separate brokers from the Web serer During installation, you hae the option of installing Extended Search serer software separate from the Web serer software. To accommodate future growth in the number of users and data sources that constitute your Extended Search domain, you may want to install these two options on separate machines. On one of these machines, you would enable the broker. On the other machine, you would enable the Web serer. This approach allows the resources on each machine to be dedicated to a gien task. The broker handles all search requests and aggregates and sorts the resulting hitlist. The Web serer handles all Web and configuration database processing required by Extended Search. If you already installed Extended Search on the same machine with the Web serer, it is still possible to separate the two processes. The following procedure assumes 52 Extended Search Administration

61 that you want to retain the original machine (machine A) to use as your Extended Search serer and that you will set up the Web serer on the new machine (machine B). 1. Follow procedures in Extended Search Installation to install only the Web Serer component on machine B. 2. Start the Administration interface, and click Serers in the CDB Naigator. 3. In the iew pane, right-click the icon for machine A and select Copy. Right-click in an open area of the Serers iew and select Paste. 4. When the General tab of the Serer Properties window is displayed: a. Type the serer name, host name, and IP address to identify machine B and click Apply. b. Select the Extended Search tab, clear the Extended Search installed check box, type a unique Broker name, disable the broker, and click Apply. c. Select the Web Serer tab and ensure that the Web Serer installed check box is selected. Confirm the remaining configuration settings, and click OK. 5. Right-click the icon for machine A and select Properties. 6. On the Serer Properties window, select the Web Serer tab, clear the Web Serer installed check box, and click OK. If you had been using the Notes Client application on machine A, use the following procedure to update the Reflector: 1. Follow procedures in Extended Search Installation to install the Reflector component on machine B. 2. On the Web serer, there should be a subdirectory for holding CGI programs. Usually, this subdirectory is named cgi-bin. The configuration file for the Reflector, reflector.cfg, is installed into this directory. Use a text editor such as Notepad to edit this file and make the following changes. Note that any line that starts with a pound sign (#) is a comment line. 3. Scan the file to locate the line that begins with EnterpriseSerer=. Make sure that you locate this exact line, and not one for other serer definitions, such as EnterpriseSerer2=. 4. Ensure that the alue of the EnterpriseSerer parameter matches the fully qualified host name of the serer on which the broker is running (machine A). 5. Locate the line that begins with PortNumber= (this is usually the next line in the file). Typically, the PortNumber alue is set to Ensure that the alue of this parameter matches the port number of the serer on which the broker is running (machine A). 6. Sae and close the reflector.cfg file. Configure remote links Agents are serer tasks and thus need an Extended Search Serer to start them up, handle their communications, and proide other client-serer architectural needs. Accordingly, there needs to be configuration data for the Extended Search Serer and its agent tasks on the remote serer. The remote serer, howeer, does not hae to host the broker. You may want to set up remote links to allow agents to be co-resident with their respectie data sources, yet allow a central broker to distribute incoming search requests to the appropriate agents, and then aggregate and sort the resulting hitlists into a single hitlist. Take the following steps to set up such a configuration. Tasks 53

62 1. Follow procedures in Extended Search Installation to install a new Extended Search serer. (Select only the Extended Search Serer option from the list of components to install.) 2. Start the Administration interface. 3. Follow the procedure in Add another Extended Search serer on page 17 to configure the new serer in the CDB. Note that on the Serer Properties window, you must select the Extended Search tab, click Broker settings to expand it, and then select the Disable Broker check box. This action preents a broker process from running on this serer. When you configure the host name, be sure to specify it in the same format as the Extended Search serer that hosts the broker you plan to use with remote agent. For example, both serers must specify the short host name, or both must specify the fully qualified host name. 4. In the iew pane, click the Graph View icon. 5. Right-click the new serer and select Connect to. To connect the remote link to a broker, click an existing serer where a broker that should be part of this broker chain is enabled. A line shows that the two are connected. 6. Click Apply Connections. 7. Run data source discoery on the new serer to add data sources. Configure data source fields, group the data sources into categories, and associate the categories with applications as appropriate. Note: You must update the configuration data for all applications that identified this now disabled broker as the entry broker. An application must point to a alid, enabled broker. 8. To disseminate your changes to the user community, refresh the search domain. When a search request that targets the data sources on the new serer is receied, the broker automatically forwards it to the agent on that machine. The agent processes the request and returns search results to the broker. The broker sorts the results, aggregates them with hitlists receied from other agents (if applicable), and returns a single hitlist to the requesting user. Partition agents and data sources A single agent can access many different types of data sources. Howeer, to support this flexibility, the agent must load shared libraries to support each type of data source that it can serice. To reduce the number of libraries that get loaded, you may want to set up multiple agents and dedicate them to particular data sources. Before you begin this task, be sure to read about how partitioning agents on a single machine can enhance your system s scalability and security. This discussion begins on page 160. To distribute resources, you need to define multiple agents, determine the optimum number of serer tasks that should be started for each agent, and associate each agent with the data sources it is to serice. Multiple agents can exist on broker-resident serers or on serers that host remote links. The following example walks you through the process of partitioning agents on the same serer. Note that this task is different than configuring multiple copies of tasks for a single agent. 54 Extended Search Administration

63 Consider a serer named Paris that hosts a set of Notes databases and a number of DB2 databases. You decide to dedicate an agent to handle requests for each type of data source. To better utilize the system s resources, you also decide to grant the agent for the Notes sources more copies of serer tasks. The following settings in the CDB establish an agent named NotesAgent with 60 copies, and an agent named DB2Agent with 10 copies. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. 3. In the iew pane, right-click the Paris serer and select Properties. 4. On the Serer Properties window, select the Agents tab. 5. Click Add Row. A blank line is added beneath the currently selected agent or, if none is selected, to the end of the list. 6. In Agent name type NotesAgent and, in Copies, type Click Add Row again. This time, type DB2Agent as the agent s name and 10 for the copies of agent tasks that should be started. 8. Click OK. 9. Click Data Sources by Link or Data Sources by agent in the CDB Naigator. 10. In the iew pane, select all the data sources that should be sericed by the NotesAgent agent, right-click, and select Properties. 11. On the Data Source Properties window, select NotesAgent from the list of agents and click OK. 12. In the iew pane, select all the data sources that should be sericed by the DB2Agent agent, right-click, and select Properties. 13. On the Data Source Properties window, select DB2Agent from the list of agents and click OK. 14. To disseminate your changes to the user community, refresh the search domain. Configure multiple brokers One of the most extendable features of Extended Search is the ability to install and configure multiple brokers to achiee load balancing. In this scenario, data sources and processing resources are spread across multiple machines. A request coming in to one broker is distributed to the other brokers in the chain. Each broker then communicates the request to the responsible agents, and handles the aggregation and sorting of returned search results. Be sure to reiew Setting up multiple brokers on page 161 for broker scalability guidelines. Note: Wheneer you add a broker or rename an existing broker, you must update the name of the Entry broker in any applications that will use that broker to serice search requests. If you omit this step, applications will continue to use the preiously configured broker as their entry broker. Consequently, your processing load will not be distributed in the manner you hoped to achiee. Use the following procedure to install and configure another broker. 1. Follow procedures in Extended Search Installation to install a new Extended Search serer. (Select only the Extended Search Serer option from the list of components to install.) 2. Start the Administration interface. 3. Follow the procedure in Add another Extended Search serer on page 17 to configure the new serer in the CDB. 4. In the iew pane, click the Graph View icon. Tasks 55

64 5. Right-click the new serer and select Connect to. Then click an existing serer where a broker that should be part of this broker chain is enabled. A line is drawn to indicate that the two serers are connected. 6. Click Apply Connections. 7. You can now use this new broker to be the entry broker for some of your applications. Take the following steps to implement this support: a. Click Applications in the CDB Naigator. b. In the iew pane, right-click an application that you want this broker to serice and select Properties. c. In Entry broker, select the name of this new broker and then click OK. 8. You can also take the following steps to fully configure the serer where this new broker resides: a. Run data source discoery on the new serer to add data sources to the Extended Search search domain. b. For each new data source, configure fields, including field-leel access controls. c. Group the data sources into categories. d. Associate the categories with applications. 9. To disseminate your changes to the user community, refresh the search domain. Configure performance options To enhance the performance of your Extended Search system, you can configure multiple copies of serer tasks and set up data sources on the same machine as the agent that serices them. You can also specify options to manage memory on the Extended Search serer and control how sessions are opened when an agent connects to a data source. Performance-related configuration tasks include the following: Configuring copies of serer tasks Localizing data sources with their responsible agents Managing memory usage on the serer Managing how Extended Search connects to data sources Before performing any of these tasks, be sure to reiew the discussion of Performance (beginning on page 167) to learn about the arious ways that you can improe performance in an Extended Search enironment. Configure copies of serer tasks Each broker and agent can hae one or more copies started by the Extended Search serer. Increasing the number of copies increases the number of requests for serice that the system can process at the same time. As the Extended Search Serer initializes, it starts all copies of each type of serer task, and then listens for client requests. The installation process sets the number of copies for each task to a default alue. For a higher powered machine, it is almost certain that you will need to adjust these initial numbers. Take care when increasing the number of serer tasks because each copy is a separate component that consumes its own set of resources. Setting an optimum number of copies offers a way to optimize search performance in accordance with the frequency of access. 56 Extended Search Administration

65 Use the following procedure to specify how many copies of serer tasks the Extended Search serer should start. Before making any changes, be sure to reiew the guidelines (beginning on page 172) on how to determine the optimum number of serer tasks. 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer you want to work with and select Properties. 4. On the Serer Properties window, select the Extended Search tab. 5. Click to expand the entries under Broker settings. 6. In Number of copies, type the number of broker copies you want the serer to start and click Apply. 7. Click to expand the entries under Agent settings. 8. In Number of copies, type the number of agent copies you want the serer to start and click OK. 9. To begin using begin using the new number of serer tasks, refresh the search domain. Localize Notes data sources and agents If you added remote Notes data sources through data source discoery, you may want to consider replicating the Notes database onto a local machine. When using Extended Search to search Notes data sources, you can achiee optimum performance by setting up the agent on the same machine with its target database. For information about why you may want to replicate databases and co-locate them with agents, see Localizing agents with data sources on page 173. An agent is told how to access a Notes source by the LNSerer data source parameter. For a remote data source, the discoery process automatically sets LNSerer to the name of the remote host. The agent accesses the Notes database remotely through this specified serer. After replicating a database to make it local to the agent s machine, you should set LNSerer to blank. This causes the agent to assume local access and attempt to read the database directly. Use the following procedure to localize an agent with its data source: 1. Follow the procedures in your Notes documentation to replicate the database onto the serer where the agent is installed. 2. Start the Administration interface. 3. Click Data Sources By Link or Data Sources By Agent in the CDB Naigator. The iew pane displays all data sources defined in the CDB. 4. Right-click the data source you replicated and select Properties. 5. On the Data Source Properties window, select the Parameters tab. 6. Examine the entry for the LNSerer parameter. 7. If the LNSerer Value field contains text, delete the text. 8. In Usage, select Used by data source. 9. Click OK. 10. Repeat steps 4 through 9 for each Notes data source you replicated or localized on the agent s machine. Tasks 57

66 Manage serer memory The Extended Search Serer manages a segment of shared memory for interprocess communication between the serer tasks that run under its control. The size of this shared memory segment is obtained from the alue set for the heap usage field in the serer s configuration properties. The heap size alue is defined in bytes, and it is set initially to 20 megabytes. If search results are returned when aailable, as opposed to being sorted, the broker reseres half of the allocated heap size for caching hitlist pages. The remainder of the allocated space is used for internal messages between serer tasks. To determine whether an appropriate heap size has been allocated, run the Monitor and obsere the Heap Use alue. If usage is generally at 80% or higher, you should increase the heap size. Likewise, if you obsere error messages concerning memory allocation or the inability to create messages in the Extended Search eent log iewer, it generally indicates an insufficient amount of heap size. Take the following steps to increase the heap size configured for an Extended Search Serer: 1. Start the Administration interface. 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer you want to reconfigure and select Properties. 4. On the Serer Properties window, select the Extended Search tab. 5. In Heap size, type a new alue for the amount of space you want to allocate. For example, you might start by increasing the alue from 20 MB ( ) to a alue between 70 to 100 MB. 6. Click OK. 7. For this change to take effect, you must shut down and restart the Extended Search serer. If the Monitor indicates that heap usage is still around 80%, or if memory warnings still appear in the Eent Viewer, repeat this procedure to increase the heap size. If heap usage appears to be low, and you occasionally receie a Message Create error, it may be because a user has fetched a large document (for example, 10 MB or more), or that the broker has generated an unusually large hitlist. Manage data source connections In a production enironment where many users perform searches with Extended Search, you could hae many agents and a large number of sources that run simultaneously. Some of these sources might include ODBC and Notes database tables or LDAP directories. By default, agents open a connection to each ODBC table, Notes table, or LDAP directory at the time the Extended Search Serer is started. Thus, if an enterprise is running 50 instances of an agent, one table could prompt Extended Search to open 50 connections to the database an undesirable scenario for most enterprises. To limit the number of open connections, you can configure the DatabaseRemainOpen parameter to serice specific ODBC tables (such as tables for DB2 or Oracle), Notes databases, and LDAP directories. 58 Extended Search Administration

67 If you set the DatabaseRemainOpen parameter alue to No, the agent opens a connection at query initiation and closes the connection at query completion. This option may hae performance implications because session handles get opened and closed on each query. If you set the DatabaseRemainOpen parameter alue to Yes, the agent maintains a persistent connection between queries. By using a combination of uniquely named agents and setting the DatabaseRemainOpen parameter to an appropriate alue, you can better control system resources. You can actiate this parameter at either the link leel or data source leel. If you set it at the link leel, it applies to all data sources associated with the link type. If you set it at the data source leel, it oerrides the link-leel setting and applies only to the data source for which it is configured. Use the following procedure to set DatabaseRemainOpen as a link parameter: 1. Start the Administration interface. 2. Click Links in the CDB Naigator. The iew pane displays all links defined in the CDB. 3. Right-click the link that you want to configure. 4. On the Link Properties window, select the Parameters tab and then click Add Row. A blank line is added beneath the currently selected parameter or, if none is selected, to the end of the list. 5. In Link parameter name, type DatabaseRemainOpen. 6. In Value, type Yes or No, as appropriate for this type of data source. 7. In Description, briefly describe the purpose of this parameter. 8. Click OK. Use the following procedure to set DatabaseRemainOpen as a data source parameter: 1. Start the Administration interface. 2. Click Data Sources By Link or Data Sources By Agent in the CDB Naigator. The iew pane displays all links defined in the CDB. 3. Right-click the data source you want to configure. 4. On the Data Source Properties window, select the Parameters tab and then click Add Row. A blank line is added beneath the currently selected parameter or, if none is selected, to the end of the list. 5. In Parameter name, type DatabaseRemainOpen. 6. In Value, type Yes or No, as appropriate for this data source. 7. In Usage, select Used by data source. 8. Click OK. Tasks 59

68 Setting up a Web Client The installation process automates many of the steps required to set up a Web Client application. For example: If you use Domino as your Web application serer, the installation program automatically configures the Domino Serlet Manager. If you use IBM WebSphere Application Serer, the administrator must follow a procedure after installing Extended Search to configure the system correctly. Extended Search comes with seeral serlets. If you use Domino as your Web application serer, the installation program updates the serlets.properties file to include the names of these serlets. When you start the application serer, the Extended Search serlets will be started. Typically, you do not need to make any changes to the serlets.properties file. If you use IBM WebSphere Application Serer, WebSphere adds the Extended Search serlet definitions to a DB2 table. To use a Web Client application, take the following steps following the installation of Extended Search: Optionally configure your Web browser to launch Lotus Notes. Run the Web Client application. Initially, you should run a search from a remote Web browser by using the Demo application and Demo template proided with Extended Search. Configure the browser to launch Lotus Notes If you use Microsoft Internet Explorer to access an Extended Search client application, you can configure it to recognize the Lotus Notes protocol. If the hitlist includes results from Notes data sources, the browser will be able to launch Lotus Notes, enabling you to access the referenced document. To set up Internet Explorer to handle links to Notes documents, you must make the following changes in the Windows Registry. 1. Open the Registry Editor. Select Start Run, type regedit, and click OK. 2. Right-click [HKEY_CLASSES_ROOT] and select New Key. 3. Rename the new key to notes. 4. Right-click the (Default) entry for the notes key and select Modify. 5. At Value Data, type URL:Notes Protocol and click OK. 6. Right-click the notes key and select New String Value. 7. Rename the new alue URL Protocol. 8. You can leae the URL Protocol alue data blank, or right-click the alue, select Modify, and specify a protocol-specific string. Contact your Web serer administrator if you are unsure of this information. 9. Right-click the notes key and select New Key. 10. Rename the new key to DefaultIcon. 11. Right-click the (Default) entry for the Default Icon key and select Modify. 12. At Value Data, type the file name to use as an icon for the Note URL protocol and click OK. For example, if Notes exists on your C: drie, you could specify c:\notes\notes.exe. 13. Right-click the notes key and select New Key. 14. Rename the new key to shell. 60 Extended Search Administration

69 15. Right-click the notes key and select New Key. 16. Rename the new key to open. 17. Right-click the notes key and select New Key. 18. Rename the new key to command. 19. Right-click the (Default) entry for the command key and select Modify. 20. At Value Data, type the command that should be used to call the Notes client application and click OK. For example, if Notes exists on your C: drie, you could specify c:\notes\notes.exe % Confirm your changes. The Registry should hae an entry similar to the following structure: [HKEY_CLASSES_ROOT] [notes] (Default) = "URL:Notes Protocol" URL Protocol = "" [command] (Default) = "c:\notes\notes.exe %1" [DefaultIcon] (Default) = "c:\notes\notes.exe" [open] (Default) = (alue not set) [shell] (Default) = (alue not set) 22. Select Registry Exit to exit the Registry Editor. Run the Web Client application Running the Web Client application from a browser inoles the use of two Extended Search serlets. To start the application, an HTTP request must explicitly inoke the JKMSearchController serlet and specify information about the client application and the sources to be searched. After results hae been returned, users can inoke the JKMFetchMgr serlet, which retriees the requested search object, simply by clicking a URL in the hitlist. To make it easy for users to submit a query, and to ensure that the query targets the appropriate data sources in the appropriate language, you will probably want to create a front-end Web page that proides links to the arious applications configured in your Extended Search search domain. By encoding URLs in this page, you can hide the complexity of the HTTP request from your users. Users can make simple selections on this page, and the serlet will be inoked with the appropriate parameters. Note: The installation process installs seeral template files that demonstrate Extended Search search and retrieal operations. You can change the look and feel of the Web Client application, or create your own Web-based search application, by customizing these templates or creating your own. For information about how to do so, see the discussions about deeloping Web Client applications in Extended Search Programming. Extended Search comes preconfigured with an application that has the ID Demo. This application is configured to search all of the Web sources predefined in Extended Search and the Extended Search product documentation. If you are running Extended Search on a Windows platform, you can use the Demo application to run the system out-of-the-box and search both of these categories. If you are running Extended Search on an UNIX platform, you can use the Demo application to search the product documentation. Inoke the serlet To inoke the JKMSearchController serlet, your custom search page needs to call a URL similar to the following for each application and language your enterprise Tasks 61

70 supports. If you do not proide a front-end Web page, users must type this URL into the location toolbar of a Web browser. Note: For readability, the following examples hae been entered on two lines. You should type the entire string on a single line. Note, too, that the serlet name and its arguments are case-sensitie. &AppID=appname&desClientLocale=locale Where: hostname is the fully qualified host name of the Web serer. serlet is the base directory on the Web serer where Extended Search serlet files are installed. On a Domino serer, this alue is serlet. On a WebSphere serer, this alue is the Extended Search Web application name (typically lotuskms). template must be destemplatefile or desjspfile, depending on whether you are inoking a search page that embeds Extended Search JKM tags, or a search page that embeds Extended Search JSP beans or JSP tags, respectiely. filename is the name of the file you want to use for searching the Extended Search domain, such as the AllOptions.txt, AllOptions.jsp, or tagalloptions.jsp sample pages that are proided with Extended Search. appname is the name (application ID) of the client application you want to run, such as the Demo application proided with Extended Search. locale is the language and country code that identifies the locale of the data sources that this application searches, such as enus for U.S. English. For information about specifying a alid language and country code, reiew National language support on page 125. Example: &AppID=Demo&desClientLocale=enUS &AppID=Finance&desClientLocale=frFR Guidelines for new users If you are testing your ability to submit a search request from a remote browser following the initial installation of the product, you should submit a query against the preconfigured Web Sources or ES Documentation categories. (You can search Web Sources only if you installed Extended Search on a Windows machine.) To test another type of data source, you must first run the Administration interface to configure the data source, assign it to a category, and associate the category with at least one application. After search results hae been returned, click on arious entries to ensure that you are able to iew or fetch documents that are listed in the hitlist. By default, the Extended Search search templates allow up to 100 hits to be returned. Be aware that certain Web search engines impose their own limitations on the number of results that can be returned, which may be less than this default limitation. If you want to switch applications, change the URL displayed in the browser to identify the application ID of the application you want to run, and press Enter. If you use a Netscape browser with the JKM-based search templates, performance can be slow. For example, if you click the Sources tab in the JKM-based Demo search template, it may take awhile for the aailable sources to be displayed. 62 Extended Search Administration

71 Selecting the Schedule or Options tabs can also be slow. To resole this problem, use an Internet Explorer browser. Note that this performance problem is not an issue when you use JSP-based search templates with a Netscape browser. If you experience problems when connecting to the Web, consult the FAQ page on the Extended Search Web Site for assistance. Tasks 63

72 Setting up a Notes Client The database for the Notes Client application gets created from a Notes template file named esclient.ntf. You can use the template file to create a ariety of client application databases, such as client applications for financial searches or employee personnel information. The template, which contains search forms and iews from which a user can run searches, gets installed in the Domino Serer s \data directory when the Notes Client component of Extended Search gets installed on a Windows machine. If you want to customize this sample application, or deelop a Notes template for a custom application, see the discussion about deeloping Notes Client applications in Extended Search Programming. Because the client configuration process depends on information that is supplied at run-time by the Extended Search serer, you must configure and start the serer before configuring a Notes Client application. The first time that a user accesses a Notes Client application database, the client application prompts the user to install files required by Extended Search. To create and configure a Notes Client application, you must perform the following procedures: 1. Optionally update the desclient.cfg file. 2. Configure an application in the Extended Search configuration database. 3. Create a client application database in Notes. 4. Define access controls for the new database. 5. Create an Application document for the new database. 6. Run the Setup Sources agent. 7. Optionally schedule the Archie Docs agent. 8. Optionally schedule the Scheduled Search agent. 9. Optionally customize the template documents. 10. Optionally hide the application design elements. 11. Verify the configuration of the Notes Client application. If you plan to support locales other than U.S. English, you should also reiew NLS considerations for the Notes Client on page 72. Update the desclient.cfg file A Notes Client configuration file, desclient.cfg, gets installed in the directory where you choose to install the Notes Client component. If you use a Domino Web serer, you typically do not need to update this file. If you plan to use the Notes Client with a WebSphere Web application serer, or if your system administrator did not accept certain default alues when installing the Extended Search serer, you must modify alues in this file before using the Notes Client application. Use a text editor to modify the following configuration alues as needed. Contact your local Extended Search administrator for assistance with specifying these alues. 64 Extended Search Administration

73 DESCGIPATH Specifies the location of the Extended Search Reflector component on the Web serer. This path is a relatie path to the Web serer home directory. The default alue is /cgi-bin/desreflector. If the cgi-bin directory on your Web serer is in a different location, change this alue to specify the correct path. DESREQURI Specifies the location of the ESAdmin.jar file on the Web serer. This path is a relatie path to the Web serer home directory. The default alue is /serlet/esadmin. If your system uses the WebSphere application serer for Extended Search instead of a Domino serer, change this alue to /lotuskms/esadmin. DESPORT Specifies the port number where the Extended Search serer listens for requests from clients to search and fetch documents. The default alue is If the Extended Search administrator did not accept the default port for the Extended Search serer, change this alue to specify the correct port number. Configure an application in the CDB For each Notes Client application database, a corresponding application should exist in the Extended Search configuration database. The configuration data specifies the application s unique identifier, which is used to control the sources aailable to users. It also specifies the name of the broker that should handle search requests and process results. The configuration database also stores application-specific information for each data source associated with a gien application. For example, it identifies which fields the users of this application are allowed to search, iew in a hitlist, or fetch from the data source. You can also specify the order in which fields should be presented in a hitlist or fetched from the data source. To create an application in the configuration database, take the following steps: 1. If necessary, discoer the data sources you want to search with this Notes Client application. 2. Configure the data source fields for search and retrieal. 3. Categorize the data sources. 4. Define the application. First, associate the data source categories with the application. Then configure application-specific options for the fields that can be searched or retrieed in each data source. Create a client application database Use the following procedure to create an Notes Client application database from the template proided with Extended Search. Note: Before you instantiate a database from a Notes Client database template, you must ensure that the user ID being used to create the database has been granted ESAdmin access priileges. To do so, follow the procedure in Define access controls on page 66, except for step 1. In place of that instruction, you should select the Notes Client template icon (esclient.ntf). 1. Start the Notes Administration client. 2. Select File Database New. Tasks 65

74 3. Type or select the name of the serer where the application database should be created. 4. Gie the new database a descriptie and unique title and file name. 5. Click Template Serer, type or select the name of the serer where the esclient.ntf file is installed, and click OK. 6. Select IBM Extended Search Client R3.7 (esclient.ntf), and click OK. Define access controls Follow the steps outlined below to set up the access control list (ACL) in the new Notes Client application database. 1. From the Notes workspace, select the icon for the newly created application database. If the database icon is not on the Notes workspace, select File Database Open. Select the database name from the serer s list and click Open. (If an About This Database screen appears, press ESC.) Close the database, and then select its icon from the Notes workspace. 2. Select File Database Access Control. 3. On the Access Control List window, click Add to bring up the Add User window. 4. Add the name of a user who will be administering Extended Search and click OK. 5. In the Access drop-down list, select Manager Access to assign the user administrator priileges. 6. In the Roles box, click to place a checkmark next to [ESAdmin] to add the user to the ESAdmin role. 7. Click OK. 8. Repeat these steps to assign other users Extended Search administrator priileges, as desired. To grant priileges to users who are not administrators, repeat these steps but select Editor Access instead of Manager Access from the Access drop-down list. Make sure that the [ESAdmin] entry in the Roles box does not hae a checkmark. Create an Application document After you define the application in the Extended Search configuration database, you need to make the configuration data known to the Notes Client. To do this, you must first create an Application document for the client application database. You must then run the Setup Sources agent. Use the following procedure to create an Application document: 1. Ensure that the Extended Search Serer is running. 2. Access Notes as a user who has ESAdmin priileges. 3. From the Notes workspace, double-click the icon for the new application database. 4. Open the Administration View. If necessary, select Actions Toggle Naigator to switch to this iew. 5. Click the New Application Form button. 6. On the Application Configuration form: 66 Extended Search Administration

75 Type the fully qualified host name or IP address of the Web Serer you configured for Extended Search. For example, mars.aka.ibm.com or Type the Web Serer s port number (typically 80). Type the name of the application. The application name (also known as its ID) must exactly match the name you gae to the application when you configured it in the Extended Search configuration database. 7. Sae and close the Application document. Tip: Deleting Search Results By default, only the search author who created a query and users with ESAdmin priileges can delete search result documents. If you prefer to allow anyone to delete search results, select Options in the Application document, and then select anyone for the Query documents can be deleted by option. Run the Setup Sources agent After you create an Application document, you must run the Setup Sources agent. The Setup Sources agent contacts the Extended Search configuration database and retriees information necessary to set up the following two documents for each source: Target displays source information to the end user. Source Configuration stores client-leel information about each data source. Each Source Configuration document has ten positions for field names. Each position is numbered 1 through 10. Next to each number is a list of aailable fields for that source. When you run the Setup Sources agent, Extended Search automatically fills information in these fields based on data defined in the configuration database. Use the following procedure to run the Setup Sources agent: 1. Ensure that the Extended Search Serer is running. 2. Access Notes as a user who has ESAdmin priileges. 3. From the Notes workspace, double-click the icon for the new application database. 4. Open the Administration View. If necessary, select Actions Toggle Naigator to switch to this iew. 5. Click the Setup Sources button. When the Setup Sources agent is finished, it presents a list of Target and Source Configuration documents that represent the data sources aailable to the application, as defined in the Extended Search configuration database. 6. You may want to oerride the initial settings by updating the Source Configuration documents. For example, you may want to specify a different order for the fields returned in a hitlist or preent a field from being returned. To do so: a. Open the Source Configuration document for the data source you want to modify and edit it. b. Sae your changes and close the Source Configuration document. 7. Exit the application database. Tasks 67

76 Any time that you make changes to the Extended Search configuration database, you should run the Setup Sources agent to refresh the Notes Client with new or reised configuration data. When you click the Setup Sources button: The Setup Sources agent erifies that the fields defined in each Source Configuration document exist in the configuration database. If they do, no changes are made. If a field has been remoed from a particular data source definition in the configuration database, the Setup Sources agent remoes it from the Source Configuration document for that data source. If you want the alues in a Source Configuration document to match the alues specified in the configuration database, you must first delete the Source Configuration document, and then run the Setup Sources agent again. A new Source Configuration document for this data source will be created. When you configure a Source Configuration document for a File System data source, be aware that the discoery process automatically loads the data source with internal fields (Doc$Filename and Doc$Content). These fields enable the entire document to be retrieed. If the Doc$Content field is enabled to be Returned in Fetch, it is automatically listed in the Choose which fields make up the document content area of the Source Configuration document. This feature enables users to retriee files from a file system source when using Extended Search product as shipped. Schedule the Archie Docs agent The Archie Docs agent is set initially to manual operation and hidden from regular (non-esadmin) users. Although this step is optional, you should set the Archie Docs agent to run on a schedule appropriate for the needs of your enterprise. The Archie Docs agent can automatically archie selected queries at a scheduled time. You configure the Archie Docs agent by setting Query Archie options in the Application document that you created for the client application database. These options specify criteria that the agent will use to select query documents for archiing. Use the following procedure to set up the Archie Docs agent: 1. Access Notes as a user who has ESAdmin priileges. 2. From the Notes workspace, double-click the icon for the new application database. 3. Open the Administration View. If necessary, select Actions Toggle Naigator to toggle to this iew. 4. Double-click the Application document to open it. 5. Expand the Options section and set your preferred Query Archie options. 6. Select File Sae to sae and close the Application document. 7. From the Notes workspace, select the icon for the new application database. 8. Select View Agents to switch to the Agents iew. 9. Double-click the (Archie Docs) agent to open it. 10. Select a schedule preference from When should this agent run? (such as on a daily or weekly schedule). 68 Extended Search Administration

77 11. Click Schedule and specify the day and time that the agent should run, and then click OK. 12. Press ESC to exit the Archie Docs Agent iew, and click Yes to sae the changes. 13. You must set up and enable the Domino serer for this agent to run. See the Domino Administration Guide for information about scheduling agents. Any queries that meet the Query Archie criteria you specified in the Application document will be archied at the scheduled date at time. Schedule the Scheduled Search agent The Scheduled Search agent is set initially to manual operation and hidden from regular (non-esadmin) users. Although this step is optional, you may want to set up the Scheduled Search agent to run on a schedule appropriate for the needs of your enterprise. Used in conjunction with a flag in the search form, the Scheduled Search agent can automatically run selected queries at a scheduled time. If you hae a query that targets numerous data sources, for example, you may want to use this feature to run the query after hours. Use the following procedure to set up the Scheduled Search agent: 1. Access Notes as a user who has ESAdmin priileges. 2. From the Notes workspace, select the icon for the new application database. 3. Select View Agents to switch to the Agents iew. 4. Double-click the (Scheduled Search) agent to open it. 5. Select a schedule preference from When should this agent run? (such as on a daily or weekly schedule). 6. Click Schedule and specify the day and time that the agent should run, and then click OK. 7. Press ESC to exit the Scheduled Search Agent iew, and click Yes to sae the changes. 8. You must set up and enable the Domino serer for this agent to run. See the Domino Administration Guide for information about scheduling agents. Customize template documents Reiew the following tips if you want to tailor the esclient.ntf template documents to meet the needs or preferences of your organization. To complete these tasks, you must first log in as a Notes administrator and then access the database design forms: 1. Access Notes as a user who has ESAdmin priileges. 2. From the Notes workspace, double-click the icon for the new application database. 3. Open the Administration View. If necessary, select Actions Toggle Naigator to switch to this iew. 4. In the ES Client R3.7 naigator on the left, select Design Forms. Replace or remoe the IBM-Lotus Extended Search image The default PriateSearch form includes an image that seres as an Extended Search product identifier. You can either replace the default image with one of your own, or you can remoe the image totally. If you want to replace the image, you should choose a thin and small image in Tasks 69

78 order to maintain or improe search performance. If you remoe the image, search performance will improe slightly because Notes will not hae to display the image for eery search result document. Use the following procedure to remoe or replace the default image: 1. Double-click the PriateSearch form, and then select the image. 2. Press the Delete key to remoe it. 3. If you want to use a different image, select File Create, and then select the picture you want to insert. 4. Press Ctrl-S to sae your changes. Increase the number of result fields The default SourceConfiguration form presents 10 fields for search results and 5 fields for fetched result content. You can increase this limit, howeer, if you want to hae more result fields displayed. For example, you can display 15 search result fields for each match originated from a certain data source instead of 10. In the SourceConfiguration form, fields that are configured to hold the search results are named DisplayField_#, where # is the actual search field sequence (such as DisplayField_1, DisplayField_2, and so on through DisplayField_10). Fields that are configured to hold fetched content are named ContentField_# (ContentField_1 through ContentField_5). Use the following procedures if you want to configure additional result fields. Note that you must maintain the numerical sequence in the field names in order for Notes and the Extended Search LotusScript to display the new fields correctly. For example, the first new search result field you add must be named DisplayField_11, the second must be named DisplayField_12, and so on. The first new fetch result field you add must be named ContentField_6, the second must be named ContentField_7, and so on. Take the following steps to add a new search result field: 1. Click at the beginning of the last search result field (DisplayField_10). 2. Press the Shift and End keys simultaneously to mark the entire line, and then press Ctrl-C to copy the line to the clipboard. 3. Press the Enter key to insert a new line in the form, and then press Ctrl-V to paste the line you just copied. 4. Change the static text at the start of the line from 10 to Right-click the new search field (DisplayField_10_1) and select Field Properties. 6. In the Name field of the Properties dialog, change the field name from DisplayField_10_1 to DisplayField_11. Do not change any other field properties. 7. Repeat the preceding steps for each new result field that you want to add. Make sure that you properly increment the numbers that uniquely identify each new field name. 8. Press Ctrl-S to sae your changes. Take the following steps to add a new fetch result field: 1. Click at the beginning of the last fetch result field (ContentField_5). 2. Press the Shift and End keys simultaneously to mark the entire line, and then press Ctrl-C to copy the line to the clipboard. 70 Extended Search Administration

79 3. Press the Enter key to insert a new line in the form, and then press Ctrl-V to paste the line you just copied. 4. Change the static text at the start of the line from 5 to Right-click the new fetch field (ContentField_5_1) and select Field Properties. 6. In the Name field of the Properties dialog, change the field name from ContentField_5_1 to ContentField_6. Do not change any other field properties. 7. Right-click the new file extension field (FileExt_5_1) and select Field Properties. 8. In the Name field of the Properties dialog, change the field name from FileExt_5_1 to FileExt_6. Do not change any other field properties. 9. Repeat the preceding steps for each new result field that you want to add. Make sure that you properly increment the numbers that uniquely identify each new field name. 10. Press Ctrl-S to sae your changes. After changing the SourceConfiguration form, you must perform the following procedure for the Extended Search LotusScript to pick up the new field definitions. You must perform these steps before running the Setup Sources agent to load or reload configuration data. 1. Click Script Libraries in the Naigator on the left. 2. Double-click ESLoadSources to display the script. 3. Click the Eent drop-down list box, and select Declarations. 4. Change the maximum configured search display fields to match the number of fields you added to the SourceConfiguration form. For example, if you added 5 new search result fields, change Const MAX_DISPLAY_FIELDS=10 to Const MAX_DISPLAY_FIELDS= If you added fetch fields to the SourceConfiguration form, change Const MAX_CONTENT_FIELDS=5 to reflect correct number of fields. 6. Press Ctrl-S to sae the changes in the ESLoadSources script library. Hide the design elements Although this is an optional step, you should replace the design after completing the setup of the client application database to hide the design elements. To do so: 1. Access Notes as a user who has ESAdmin priileges. 2. From the Notes workspace, select the icon for the new application database. 3. Select File Database Replace Design. 4. On the Replace Database Design window, click Template Serer, and then type or select the name of the serer where the database template resides. 5. Select the IBM Extended Search Client R3.7 template (the esclient.ntf file). 6. Make sure that the Show adanced templates check box is clear, and that the Inherit future design changes and Hide formulas and LotusScript check boxes are selected. 7. Click Replace. Tasks 71

80 Verify the Notes Client setup Use the following procedure to erify that the Extended Search Notes Client application has been configured correctly. If you need assistance with specifying a query, select Help Using this database after opening the application database. 1. From the Notes workspace, double-click the icon for the new application database. 2. Click the New Search button. 3. Expand Sources Options, and select the check box for an indiidual data source, such as the predefined Alta Vista - Web source. 4. Type the following text into the Search for box: lotus Note: You can use any source listed under Sources for erification testing. To use a different source, select the source name from the Sources list. To obtain a greater number of releant hits, type a search key word that appears in the data source name. For example, type alta ista in the Search for box when searching the Alta Vista - Web source. 5. Click Go! to start the search. If Extended Search returns an error instead of a hitlist, refer to the FAQ page on the Extended Search Web Site for troubleshooting assistance. NLS considerations for the Notes Client Be sure to read National language support on page 125 before reading this section so that you are familiar with the NLS configuration file (esnls.ini). When the Extended Search Notes Client application is installed on a client machine, seeral enironment ariables are added to the client machine. The following enironment ariables hae national language support implications for the Notes Client application. ES_NLS_INI This enironment ariable contains the location of the esnls.ini file. A copy of the esnls.ini file is installed in the directory where the Notes Client application is installed. LOCPATH This enironment ariable contains the location of the X/Open Portability Guide Issue 4 (XPG4) locale definition and conersion files that are used by the Extended Search Notes Client application. These files are installed along with the other Notes Client application files. LANG This enironment ariable is set to the XPG4 locale definition file that should be used by the Notes Client application code when communicating with the backend Extended Search serers. It is set by querying the Windows enironment to see which regional setting is being used. 72 Extended Search Administration

81 Saing and scheduling searches The Saed Queries applet enables users to work with saed searches and search results. Settings in the Administration interface enable an administrator to make this feature aailable, and to set options that enable users to schedule saed queries and hae them run at specific or periodic times. These features, which are aailable only for searches that are submitted through a Web Client application, enable you to process search requests at a conenient time. For example, if you hae a search that requires connections to many data sources, and that returns numerous hits, you can schedule the query to run after hours when other users and resources are not contending for processor time. Saed queries enable multiple users to collaborate and are useful for specifying syntactically complex queries that are difficult to duplicate. For example, one user can submit a complex search that is of interest to all the members of a work group, and all users will be able to iew the results and retriee releant documents. A key feature is the ability to write search results to a file system data source. Extended Search can operate as an intelligent crawler, automatically retrieing content that is based on query criteria. Once a document or hitlist has been stored, you can use other products to render, index, or analyze the content. For example, you may want to embed returned URLs in a spreadsheet, or index the search results and feed them into a data mining or knowledge management application. Fetch and store search results on page 80 describes this feature in detail. The security features of Extended Search enable you to control the accessibility of saed queries and their resulting hitlists. If the Web serer does not challenge a user to specify a user ID and password when setting up a search request, Extended Search saes the query and its resulting hitlists in the public domain. All users will be able to iew the query and work with the search results. If you configure Extended Search to use either basic or custom authentication, the Web serer will prompt users to specify a user ID and password when they inoke a search form. If a user saes the query, Extended Search saes the query data and hitlist under the specified user ID, and only that user (and the administrator) will be able to iew the search request and search results. If you want to allow multiple users to share queries and hitlists, you should create a group ID and assign indiidual users to it. Users can then specify this group ID when accessing the search form and the Saed Queries applet. Note: Be aware that Extended Search stores only the user ID of the user who set up the search; the system does not store passwords. Tasks 73

82 Using Impersonation Because the system does not store passwords, you will not be able to re-run saed queries that target ODBC sources or Web sources for which you hae enabled impersonation. When impersonation is enabled, the user must proide both a user ID and password to be able to access and search the data source. The saed query proides only the user ID. Note that because the Notes serer requires only the user ID when erifying a user against the Notes ACL, you can use impersonation with saed queries that target Notes data sources. To take adantage of the saed query features of Extended Search, you need to understand the following: How to sae and schedule queries How to work with saed queries and search results How to retriee search results and store them into a file system Sae and schedule queries How you sae and schedule a query depends on whether you are an administrator or end user. The sample Demo search template includes options that enable an administrator to schedule and sae queries when testing the configuration of the Extended Search search domain. The search form accessed by an end user must include options for scheduling and saing the query. For complete information about creating search forms for your custom Web Client applications, see Extended Search Programming. Use the following procedure to set up Extended Search to check for scheduled queries, set up and schedule a search request, and sae the query and results. 1. Set up the query scheduler. 1. As an administrator, start the Administration interface and then select Serers in the CDB Naigator. When the iew pane lists the known serers, right-click the serer you want to configure and select Properties. 2. Select the Web Serer tab, then select Scheduled Query to expand the options. 3. Select the Enable scheduled query check box to allow users to sae queries and to hae the Extended Search scheduler serlet start checking for scheduled queries. Note that you can clear this check box at any time if you need to preent scheduled queries from being run. For example, you may need to disable the scheduler serlet to troubleshoot problems, such as a rogue query that you suspect is consuming too many resources. 4. In Check for scheduled queries, specify how often, in minutes, the serer should check for queries that are due to be run. 5. In Concurrent fetch and store processes, type the number of fetch and store processes that can run concurrently. At a minimum, this number should equal the number of brokers that are configured for this Extended Search serer. You should also ensure that the number of 74 Extended Search Administration

83 aailable agents is at least one greater than the number of brokers. See Fetch and store search results on page 80 for configuration guidelines. 6. In Timeout fetch and store, specify how long, in seconds, the broker should wait for a response from an agent before cancelling the request to retriee and store a document. The default alue is 60 seconds. 7. Click OK. 8. Refresh the search domain to hae your changes take effect. 2. Access the search form. If you are an administrator, start the Administration interface, select Search Templates in the CDB naigator, and open a Demo search template. If you are an end user, follow instructions from your local administrator to access the URL for a search form that includes options to sae and schedule searches in Extended Search. 3. Set up the search request. Described below are options that are typically aailable for scheduling and saing queries. The search form that you access may or may not include these options. 1. Assign a name to the query. Type a descriptie name, one that will enable you and other users to recognize its objectie. 2. Type a brief description of the query. 3. Assign the query to a query folder. If you preiously created query folders, you can select one from the list of existing folders and add this search to that group of search requests. Alternatiely, you can create a new folder for this query by typing a name for it. Note that if you type the name of an existing folder, the query will be saed in that folder. At this point, your search form may allow you to sae the query without running it or scheduling it. If you want to sae the query for later refinement or processing, click the option to sae the query only. 4. Schedule the saed query: a. Select an interal for how often the query should be run (1 time per day, 1 time per week, 2 times per month, and so on). b. Select the day of the week that the query should be run the first time (this sets the start date for when the scheduled frequency should be repeated). c. Type the time of day that the scheduled query should be run. Specify the time by using the 24 hour clock (09:30, 12:45, 23:00, and so on). d. Optionally, but recommended, select the option to retain all the hitlists returned by this query. If you do not select this option, the latest search results will oerwrite the existing hitlist each time the query is run. e. Optionally select the option to return a detailed hitlist. The detailed iew shows all the returnable fields that were defined for the hitlist item s data source in the configuration database. The summary iew identifies the hitlist item by name, data source, and category. f. Optionally write the search results to an existing file system data source. (It is recommended that only users with administratie responsibilities use this option.) Extended Search lists the file system data sources aailable to this query; that is, those that were Tasks 75

84 configured for this application in the configuration database. See Fetch and store search results on page 80 for more information about this feature. If you want to store the documents returned in the hitlist, select a target file system data source. If you want to store the hitlist itself, select a target file system data source. g. Click the option to schedule the query. The next time the Extended Search scheduler serlet checks for queries that are due to be run, this newly saed query will be added to the queue of scheduled requests. The query will be processed the next time the serlet runs after its scheduled start time occurs. See Work with saed queries for information about how to work with saed queries and search results. Work with saed queries Extended Search presents saed queries and search results in a familiar tree structure, much the way Windows Explorer organizes files into folders and subfolders. At the top leel is the query folder, which you can create when iewing saed queries or, depending on options in the search form, at the time that you set up the search request. If you expand a query folder, you see a list of the queries associated with it (each query must be associated with a folder at the time that it is saed). The names of the queries match the names they were assigned when the query was saed. Icons help you identify the saed state of a query: Indicates that this saed query is also scheduled. Indicates that this saed query was preiously scheduled, but is now disabled (currently, it is not scheduled to be run again). Indicates that this saed query was neer scheduled. To schedule it, you must modify the query, specify schedule information, and select the option to schedule as well as sae the query. If you expand a query, you may see a saed hitlist (if the query has been run), or you may see multiple hitlists (if the query specified that preious results were to be retained). The names of the hitlists match the dates and times that the queries were run. How you access saed queries depends on whether you are an administrator or end user. The actions you can take also ary according to your role and whether the saed query is public, owned by you, or owned by another user. Administrators can iew and work with all saed queries and hitlists. End users can iew and work with only those saed queries and hitlists that were created under their user ID (that is, the user ID they specified when accessing the applet, if any), or that were created as public queries (that is, queries that do not require a user ID for authentication). 76 Extended Search Administration

85 Access saed queries If you are an administrator, there are two ways for you to access saed queries. First, start the Administration interface. Then use one of the following methods: Select Saed Queries in the CDB Naigator. The iew pane lists all the queries that were saed by you or by end users, and organizes them by query folder name. Select Search Templates in the CDB Naigator, and then right-click one of the Demo templates and select Display. Select the Schedule tab, and click the link for iewing saed searches. The Saed Queries iew is displayed, which lists all the queries saed by you or end users, organized by query folder name. If you are an end user, there are also two ways for you to access saed queries. Note that the Jaa 2 Runtime Enironment (J2RE) plug-in must be installed on any machine on which you want to work with saed queries. After the Saed Queries applet starts, you see a list of all public queries and the queries to which you hae access, organized by query folder name. Depending on options that are included in the search form, you may be able to access saed queries by clicking a button or link that automatically inokes the Saed Queries applet. The Schedule tab in the Demo search template demonstrates this action. Start a Web browser and pass saedqueries.txt to the JKMSearchController serlet. For readability, the following example has been entered on two lines. You should type the entire string on a single line. Note, too, that the serlet name and its arguments are case-sensitie. &AppID=appname&desClientLocale=locale Where: hostname is the fully qualified host name of the Web serer. serlet is the base directory on the Web serer where Extended Search serlet files are installed. On a Domino serer, this alue is serlet. Ona WebSphere serer, this alue is the Extended Search Web application name (the installation default is lotuskms). appname is the name (application ID) of the client application you want to run, such as the Demo application proided with Extended Search. locale is the language and country code that identifies the locale of the data sources that this application searches, such as enus for U.S. English. For information about specifying an ISO-compliant language and country code, reiew National language support on page 125. For example: &AppID=Demo&desClientLocale=enUS When the list of aailable saed queries is displayed, you can take any number of actions to work with them. Tasks 77

86 Saed query actions After accessing saed queries, you can take any of the following actions to iew or work with the saed queries and hitlists: Create or modify a query folder: 1. To create a query folder, right-click in an open area of the saed queries iew pane and select New Folder. 2. On the Query Folder Properties window, type a name for this folder, type a brief description of its purpose, and click OK. This window also displays the user ID of the user who created the query (the user ID is blank if the query was created by an administrator). This field is proided for informational purposes only; you cannot modify it. 3. To change the name or description of an existing query folder or query, right-click the folder you want to change and select Properties. Make your changes, and click OK. Delete a query folder, query, or hitlist: 1. Select one or more query folders, queries, or hitlists you want to delete, right-click and select Delete. When you delete a query folder, all queries and hitlists associated with it are also deleted. When you delete a saed query, all hitlists stored for it are also deleted. To aoid cluttering the database, you should periodically reiew saed queries and their results, and delete those that no longer need to be retained. Temporarily preent a query from being run: 1. Select one or more queries you want to suspend, right-click and select Properties. 2. On the Saed Query Properties window, select the Disable query check box and click OK. The next time the scheduler serlet checks for scheduled queries, this request will be remoed from the set of scheduled queries. When you are ready to process the search again (for example, after debugging or modifying the query), clear the Disable query check box and click OK. Modify information about a query: 1. To change the name or description of a query, right-click the query you want to change and select Properties. 2. On the Saed Query Properties window, type a new name and description and click OK. This window also displays information about when the query was created. If the query is scheduled, it also shows when it was last run and when it is scheduled to be run again. These fields are proided for informational purposes only; you cannot modify them. Modify the content of a query: 1. To change a query s search terms or search options, or to change options for saing and scheduling a query, right-click the query and select Modify Query. 2. When the search form for the query is displayed, make your changes and select the option to sae and optionally schedule the query. If the query is scheduled, the next time the scheduler serlet runs the query, the changed query will be processed. 78 Extended Search Administration

87 Note: If you attempt to change the name of the query folder or the name of the query from within the search form, the following results occur: If you are an administrator working with saed queries in the Administration interface, the requested changes will be saed in the configuration database. If you are an end user working with the Saed Queries applet, either a new folder or query will be created, or an existing folder or query of that same name will be oerwritten. (To rename a query folder or query, update the Query Folder Properties or Saed Query Properties instead of changing this information in the search form.) Retriee search results: 1. To display search results, right-click the hitlist you want to reiew and select Display Query Results. 2. When the hitlist is displayed, click the URL or DocID of a document you want to retriee. Depending on how the hitlist page was designed, you may be able to modify the query at this point and resubmit the search request. Depending on features that are included in your search page, you may be able to take the following actions when working with search results. Note that these actions are not aailable in the Saed Queries applet. Howeer, the Demo search template proided with Extended Search demonstrates these features. Retriee documents by selecting them from a scrolling ticker display. The esticker applet proided with Extended Search can display information, such as a hitlist, while you are working with a query. As the hitlist scrolls across your screen, select a hitlist item to access the item s URL. Toggle between the detailed and summary iews of the hitlist. The detailed iew shows all the returnable fields that were defined for the hitlist item s data source in the configuration database. The summary iew proides the hitlist item name and the names of its parent data source and category. Select an action for processing hitlist items. When you select a smart action, the system inokes the URL that was defined for that action in the configuration database. For example, the action may allow you to the hitlist item to other users. The sample smart actions proided in the Demo template enable you to link to the Extended Search home Web site (ES Home) and the Extended Search troubleshooting page (FAQ). As discussed in Define smart hitlist actions on page 27, it us up to your local Web designer and administrator to define Web pages that support the intended actions. Tasks 79

88 Usage Guidelines A maximum of 254 characters can be stored in the configuration database per search option. This limitation is typically not a concern unless your application deeloper chose to use the DESSources ariable in the JKM-based search template and you selected numerous indiidual data sources to search. If you receie an error message indicating that you exceeded the 254 character limitation, either select fewer data sources to search or select categories instead of data source names. Your Web browser may limit your ability to re-display a saed query. Een though Extended Search is able to store your query in the configuration database, certain browsers impose limitations on the amount of data they can retriee (such as 2048K characters). If the total amount of data in the saed query exceeds the browser limitations, the search page will appear blank. Fetch and store search results In addition to saing queries and search results in the Extended Search configuration database, you can sae search result data (both hitlists and documents) to a file system data source. Typically, only users with administratie responsibilities choose this option. With this feature, Extended Search is able to operate as an intelligent, query-drien crawler. It can automatically retriee content of different formats on a scheduled basis, and store the retrieed data in a file system data source. Once stored, a document can be rendered in its natie form or be analyzed and indexed by another product, such as Lotus Discoery Serer. Examples of how this feature can benefit an enterprise include: The ability to directly search the results wheneer you submit a query against the file system data source. The ability to direct result data to an applet. The esticker applet proided with Extended Search demonstrates how to stream the contents of a hitlist across a screen. Users are able to access a referenced document by clicking on it as it is displayed in the scrolling ticker. The ability to add file extensions to the stored data, thus enabling a document to be rendered by its natie iewer or by a Web browser. For example, adding a.doc extension enables Microsoft Word to open the document. Adding the extension.html enables a browser to display the document regardless of whether the natie application is aailable. This feature makes it possible to iew a Lotus Notes document, for example, een if Notes is not installed or aailable. The ability to import the stored data into another application. This feature proides flexibility with regard to how you intend to use the data: You can crawl and index the data, thus making it meaningful for data mining and knowledge management purposes. Users can rapidly access releant information by using keywords that were associated during the indexing process. You can make the data aailable to users without first indexing it. When storing a document, Extended Search can create an XML wrapper that identifies the original document and the location of the document as stored in 80 Extended Search Administration

89 the file system. The document identifier, which typically takes the form of a URL, enables users to access the original source of the document, not just the saed ersion in the file system data source. You should reiew the following topics before storing search results to a file system: How documents are stored describes how Extended Search stores the data it fetches. You need to understand how to access a stored document to be able to index it or use with other applications. Fetch and store guidelines on page 82 discusses performance considerations that you need to be aware of when configuring fetch and store features in the configuration database. How documents are stored When Extended Search fetches documents as the result of a scheduled search, you control how you want to store the data: You can store the natie content of the document into the file system as is. You can hae the natie content eneloped in XML. You can store both the natie content and its XML wrapper. Your choice depends on the presence and content of an XML template named desschedule.xml, which is located in the Extended Search installation directory. If desschedule.xml does not exist in the installation directory, then the natie content of the document will be stored into the file system. This is usually an HTML document, but it could be a document of any type, such as a document produced from typical office software products. When stored, the natie documents are named consecutiely with a unique alphanumeric suffix that increments with each document and field returned. The file name consists of the characters hit followed by a unique number for each document returned, a unique alphabetic character for each field returned, and the file extension.des. For example: hit0a.des, hit0b.des, hit0c.des (and so on) hit1a.des, hit1b.des (and so on) hit2a.des, hit2b.des, hit2c.des, hit2d.des (and so on) If desschedule.xml exists in the installation directory, Extended Search will use it to format and store the fetched document into the file system. For each fetched document, Extended Search creates an XML file whose content is determined by the desschedule.xml template. When stored, the XML files are named consecutiely with a unique alphanumeric suffix that increments with each document and field returned. The file name consists of the characters hit followed by a unique number for each document returned, a unique alphabetic character for each field returned, and the file extension.xml. If you store both the natie content and a companion XML file, the filenames correlate. For example: hit0a.xml, hit0b.xml, hit0c.xml (and so on) hit1a.xml, hit1b.xml (and so on) hit2a.xml, hit2b.xml, hit2c.xml, hit2d.xml (and so on) Note: If you store hitlists instead of or in addition to documents, the stored files are named hitlist0, hitlist1, hitlist2, and so on. The Extended Search product ships with the following desschedule.xml template file: Tasks 81

90 <?xml ersion="1.0"?> <!DOCTYPE rdf:rdf SYSTEM " <rdf:rdf xmlns:rdf=" xmlns:dc =" <rdf:description about="$es:storepath:es$"> <dc:identifier> $ES:DOCID:ES$ </dc:identifier> <dc:body> $ES:DOCCONTENT:ES$ </dc:body> </rdf:description> </rdf:rdf> You can modify or replace this file with your own XML as long as your file includes at least one of the following Extended Search replacement ariables. $ES:STOREPATH:ES$ This ariable, if present, tells Extended Search to store the natie content of the document into the file system (with a.des extension) and to replace this ariable with the file system path of where it was stored. You might choose this option if the content of the document is binary in nature (such as a natie spreadsheet file) and thus cannot be wrapped in XML. $ES:DOCID:ES$ This ariable, if present, will be replaced with the identifier of the document. Normally, the identifier is a URL, but it can be anything depending on the source repository of the document. $ES:DOCCONTENT:ES$ This ariable, if present, will be replaced with the actual content of the document. Note that the content of the document should be all text and not contain any nondisplayable characters. You can combine these ariables as needed. For example, if the documents you are retrieing contain a mixture of text and binary content, you might use the $ES:STOREPATH:ES$ and $ES:DOCID:ES$ ariables only. This would result in.des and.xml files being produced (the default in Extended Search). Alternatiely, you could choose to omit the $ES:STOREPATH:ES$ ariable and generate only XML, complete with the document identifier and content contained in the one.xml file. Note that the maintenance of this data is beyond the control of Extended Search. You must ensure that there is sufficient free space aailable in the file system to store the saed data. You should also periodically reiew the data and delete information that is no longer needed. Fetch and store guidelines The configuration of fetch and store features in the configuration database is critical to your system s performance and to your ability to process fetch and store requests concurrently. The default installation of Extended Search defines three brokers and fie agents. This configuration is acceptable for small production and test enironments, but for smooth performance in a typical or large production enironment, additional brokers and agents will be needed. When configuring the number of fetch and store processes that can run concurrently, you should specify a number that is at least as great as the number of aailable brokers. This ensures that fetch and store requests can be processed in parallel. 82 Extended Search Administration

91 If the agents and brokers doing the fetching and storing are all located on the same machine, the number of agents should be double the number of brokers. To aoid deadlock, you must ensure that the number of agents is always at least one greater than the number of brokers. If none of the fetching is being done by agents on the machine that is doing the storing, the number of agents should equal the number of brokers for the maximum number of concurrent stores. For optimum performance, you should install multiple Extended Search serers and configure multiple brokers and multiple agents on each machine. Ideally, you would also hae a separate machine with extensie disk space aailable. The file systems on this machine should be configured as data sources within your Extended Search search domain, enabling them to be aailable for saing search result data. To achiee the greatest throughput, you should schedule saed queries to run after hours or during non-peak hours of the work day. Doing so can aoid resource contention with users who submit queries for immediate processing. This recommendation is especially true if your scheduled query targets numerous data sources or is expected to return ery high numbers of search results. Be aware that if you use a Netscape browser, ersion 4.73 or older, to iew or modify saed queries that were created with JKM-based search templates, the query data will not be restored properly by the browser and will appear to be lost. To resole this problem, either upgrade to ersion 6.0 of Netscape or use a Microsoft Internet Explorer browser. (This performance problem is not an issue with JSP-based search templates.) Tasks 83

92 Monitoring serer actiity The Extended Search Monitor proides a graphical interface to Extended Search serers. It enables you to obsere serer actiity and, when necessary, make adjustments to improe processing. Because it runs independently of the Extended Search serer, you can start it and stop it without affecting serer processing. You can run the Monitor in two modes: As a stand-alone C++ program The stand-alone ersion of the Monitor runs on Windows systems only. Howeer, you can start the Monitor on a Windows machine and then use it to monitor an Extended Search serer that you installed on a UNIX machine. You should run the Monitor on a machine other than the machine where the Extended Search serer is running. This approach enables you to check the status of the serer without haing to be at the host console. Furthermore, because the Monitor uses a graphical interface, it consumes machine resources that might otherwise be dedicated to Extended Search serer actiity. While running the Administration interface This feature enables you monitor actiity, make real-time changes to the system (such as increasing or decreasing the number of data sources accessed by the serer), and refresh the Extended Search domain with those changes from a single point of control. You can refresh Extended Search serers while running the Monitor. Refreshing a serer s configuration causes each serer task to reload its configuration data from the CDB. This feature enables an Extended Search serer to remain online. You do not hae to shut down and restart the serer each time you make a change to its configuration. The Monitor is a primary tool for helping you assess system performance and make adjustments. Although the following topics show you how to use and naigate the Monitor s graphical interface, you should also reiew the discussion of Performance on page 167. It may help you understand how to interpret reported statistics and make the appropriate changes to optimize serer performance. Run the Monitor in stand-alone mode Perform the following steps to start the Monitor on a Windows machine where you installed the Extended Search Serer component: 1. Select Start Programs IBM Extended Search Extended Search Monitor. 2. On the Monitor an Extended Search Serer window, type or select the host name of the Extended Search serer you want to monitor, type its port number (typically 6001), and then click OK. If the host name is correct, and the Monitor is able to connect to the serer, a detailed iew of the serer s run-time statistics is displayed. If the Monitor is unable to connect, you see a message that asks whether you want to cancel or retry the request. If you click No, an empty Monitor is displayed. Select Monitor Open, and re-select or retype the serer s host name. For example, you may hae typed a wrong alue the first time, or the serer may not be online or aailable. Note that selecting Monitor Open causes statistics to be displayed in the current Monitor window. If you want to display statistics for this serer or a different serer in a new Monitor window, select Monitor New. 84 Extended Search Administration

93 After the Monitor has been started, you may want to select View Always on top. This feature ensures that the Monitor window is always isible, enabling you to instantly monitor the results of any serer actiity that is currently occurring. To moe the window to the background, toggle this selection. Note: To shut down the Monitor, either close the Monitor window or select Monitor Exit. If you select Serer Shutdown, you shut down the Extended Search Serer, not just the Monitor. Be aware that to shut down the Extended Search Serer, you must specify the ES Administrator user ID and password. View task details The Serer Tasks (Detail) iew of the Monitor proides detailed information about each copy of each serer task that was started on the serer. This mode is the default iew in the stand-alone ersion of the Monitor, and it is the only iew aailable within the Administration interface. For an explanation of serer tasks, and for guidelines on configuring copies of tasks, see Understanding serer tasks on page 171 and Determining the optimum number of tasks on page 172. To access the Detail iew at any time, either select Serer Tasks (Detail) in the Monitor area on the left side of the interface or select View Serer task details to switch to this iew. Figure 3. Extended Search Monitor details iew Serer area The Serer area displays the host name and port number of the serer being monitored. It also shows the date and time when the serer was last started. Message counter A counter in the lower left corner of the window displays the total number of messages that hae been receied by the serer since it was last started. Tasks 85

94 Memory heap usage A scale along the bottom of the window reflects the percentage of memory resources that hae been used, based on the configured heap size for this serer. Each Extended Search serer allocates a segment of shared memory for interprocess communication between the serer tasks that run under its control. To manage serer memory, you may need to adjust the serer s heap size (see page 58). Serer Tasks area The Serer Tasks area proides information about each copy of a serer task. Each line represents a unique copy. Scroll through the list to examine information about each task: The icons in the Status column change color and text as the processing state of a gien task changes: The gray Idle icon indicates that the task is actie but no actiity is occurring. The green Busy icon indicates that the task is processing a message. The yellow Wait icon indicates that the task is waiting for another task to process a request before it can continue. The blue Conf icon indicates that the task is getting information from the internal configurator. You see this icon only when the serer is started or refreshed. The red Down icon indicates that the task is not actie. This generally indicates an abnormal condition. The Title column displays the name of the task. The PID column displays the process identifier. A PID is a code that uniquely identifies each copy of each task. The PID also enables you to distinguish among the serer tasks when you use a system analysis tool such as the Windows NT Task Manager. The Count column specifies the number of messages that hae been completely processed by a gien task. The Time column displays the aerage length of time that it takes to process a message through a gien serer task. View a task summary The Serer Tasks (Summary) iew of the Monitor is aailable only if you are running the Monitor as a stand-alone program. It is not accessible from within the Administration interface. To access the Summary iew at any time, either select Serer Tasks (Summary) in the Monitor area on the left side of the interface, or select View Serer task summary to switch to this iew. 86 Extended Search Administration

95 Figure 4. Extended Search Monitor summary iew The Serer, message counter, and memory heap usage areas of the Summary iew mirror the same areas in the Detail iew. The Serer Tasks area, howeer, differs in that it lists each type of serer task. It does not list the indiidual process IDs for each copy of a serer task. Howeer, a status icon is displayed for each copy of a serer task. The color and content of the status icons are identical to those that are displayed in the Detail iew. Display data source statistics The Data Source Statistics iew of the Monitor proides information about the target data sources to which this serer is connected. You can scroll through the list and reiew the number of requests that hae been directed to each data source since the serer was last started. You can also assess the aerage elapsed time that it took for the agent associated with a gien data source to process the requests. Note that this aerage elapsed time reflects the time that elapsed during an agent s processing of the request, not the oerall response time that users experience. To access the Data Source Statistics iew at any time, either select Data Source Statistics in the Monitor area on the left side of the interface, or select View Data source statistics to switch to this iew. Tasks 87

96 Figure 5. Extended Search Monitor data source statistics iew Log serer actiity You can keep track of a serer s actiity by writing statistics to a log file. This feature enables you to run the Monitor in an unattended mode and reiew a log of the serer s actiity at a later time. To log serer actiity, you must specify how often the log file needs to be refreshed, actiate logging, and specify a location for the log file. Logging is aailable only if you are running the Monitor as a stand-alone program, not if you are monitoring serer actiity from within the Administration interface. Use the following procedure to record serer actiity in a log file: 1. Select Monitor Refresh eery, and then select a alue to specify how often messages should be written to the log file. Typically, you should not select.05 or 1.0. Logging will occur only if the refresh rate is greater than one second. Also, you typically would not select Continuously to constantly refresh the log file. This selection can cause heay resource consumption. Select Set Interal to specify a refresh rate in milliseconds: a. On the Set monitor refresh interal window, specify how often messages should be written to the log file. The default alue is 200 milliseconds. b. Click Set. 2. Select Monitor Log actiity. The Log serer actiity to a file window is displayed. 88 Extended Search Administration

97 Figure 6. Extended Search Monitor log serer actiity 3. To enable logging, select the Actiate logging to file check box. 4. Type the fully qualified path and file name for a file where log messages should be written. 5. If you want to append messages to an existing file, select Add to file. Ifyou want to replace an existing log file and start a new one, select Replace file. 6. If you want to log messages about serer task actiity, select the Include Message Statistics check box. 7. If you want to log messages about data source search actiity, select the Include Database Statistics check box. 8. Click Set. Run the Monitor in the Administration interface The following procedure shows you how to use and naigate the Monitor while you are using the Administration interface. Howeer, before using this method, you should reiew the following topics to fully understand how the Monitor works in Extended Search: Run the Monitor in stand-alone mode on page 84. Note that if you encounter problems with Jaa resource contention while running the Administration program and the Monitor, you should run the Monitor in stand-alone mode. Performance on page 167. This discussion may help you interpret reported statistics and make appropriate changes to optimize serer performance. It also explains serer tasks, and proides guidelines for determining how many copies of tasks are best for your enironment. Use the following procedure to monitor serer actiity from within the Administration interface: 1. Start the Administration interface. Tasks 89

98 2. Click Serers in the CDB Naigator. The iew pane displays all Extended Search serers defined in the CDB. 3. Right-click the serer you want to monitor and select Monitor. The Monitor starts and proides detailed information about each copy of each serer task that was started on the serer. Figure 7. Extended Search Monitor Administration interface iew 4. Use the following guidelines to interpret the run-time statistics displayed by the Monitor: Serer area Displays the host name and port number of the serer being monitored. It also shows the date and time when the serer was last started, and the total number of messages that were receied by the serer since it was last started. The Heap Use field reflects the percentage of memory resources that hae been used, as determined by the configured heap size for this serer. Search Statistics area Proides information about the target data sources to which this serer is connected. You can scroll through the list of data sources and reiew the number of search and retrieal requests that hae been directed to each data source (Count column). You can also assess the aerage elapsed time that it took for the agent associated with a gien data source to process the requests (Response Time (ag) column). Note that this aerage elapsed time reflects the time that elapsed while an agent processed the request, not the oerall response time that the end user experienced. Serer Tasks area The Serer Tasks area proides information about each copy of a serer 90 Extended Search Administration