Presentation References

Transcription

1 Presentation References Knowledge Articles How can I estimate how long a full FTS Reindex will take? - KA FTS Configuration Options for Recovery Environments - KA FTS - Reindexing a single form - KA The ft_pending table continues to grow even though FTS indexes appears to be working - KA How to Troubleshoot FTIndexer problems in 9.x - KA Video High Availability Setup - BLOG The Pulse: High Availability for FTS in a Server Group - Webinar Q&A Q: Why does support recommend to not use a shared drive/nfs for collection directory? A: Like an RDBMS, the FTS database engine should be as close to the database files as possible. To introduce a network connection between the DB engine and its files can cause many problems. The Lucene engine performs a huge number of file accesses and they are expected to be very fast. When the drive is remote, the speed is a fraction of what is expected. In fact, in testing we do see problems when the files are remote and recommend that they always be local. Q: Your references mention how to index a specific form, but is there any other way to index without having to perform a full FTS reindex? A: All indexing occurs because of entries on the ft_pending table. Theoretically you could add something yourself. This is not expected behavior so we don t document it. But you should be able to easily figure out what values to insert. Another way is this; modify the thing that you want index. If you want a field to be indexed (for every record), remove and re-add the FTS setting for that field. If you want one record to be reindexed, modify that record. Q: The activities are different from what we saw in ? A: I believe you mean Operation Types - yes, they are different than 8.x-. Refer this: Q: What are the implications of setting Private-RPC-Socket: in ar.cfg on FTS indexing server? A: It allocates the configured number of threads for indexing. For 4 threads, total threads allocated are 5. I.e. always 1 more than configured.

2 Q: Is there a Max memory setting for the FTS JVM Max memory setting? A: Yes you can set up in FTS process in armonitor.conf by changing the value of the Xmx setting. In case of indexer server (or standalone setup) - FTS code runs within the server process itself, so memory settings of server are used. In case of secondary server making a call to FTS secondary plugin, FTS secondary plugin server JVM parameters are in use. The max setting for Xmx is based on the physical memory of the system. Q: Can the collection directory path be a shared path? Or both AR servers defined as indexer(s) must maintain their local copy of index files? A: The collection directory should not be shared. Yes, each indexer will maintain its own collection. Q: FT Index is only on Primary Server with Rank 1 A: FTS secondary plugin in this Primary Server must be enabled in armonitor.conf and in all other servers ar.cfg - it should point to this secondary plugin of primary server. Q: It does, and are there index fragmentation implications or performance implications of using more than one thread on ? A: No. In versions prior to 9.x, there was some file lock contention that occurred and we often recommended not increase the threads. In 9.x, we use a newer version of Lucene that does not have the same issues. There should not be a problem using more than one thread. In fact, we recommend 2 and 2 and most of our testing was done that way. Q: in that configuration vendor form where you specify server-group indexer or search-indexer, there's a field where you can specify memory setting (example was 3 gigs) is there a maximum value that FTS uses or that can be specified there? A: This is a java setting. The limit of this setting is a practical limit based on the available resources on the system. Q: If I am running 8.1 SP2 in a server group with FTS Searcher/Indexer partitioned across servers, if I upgrade in place to 9.X, will it make all the necessary changes in the configuration to run appropriately? A: Yes. If you were successfully running searcher/indexer in 81SP2 - upgrade to 9.x will keep those config and will run fine once your ft_pending is empty after upgrade. Q: Also, do I have to re-run a full FTS Re-Index after upgrade? A: Not required if you are upgrading to AR 9.0 or later. Q: Is there any benefit to increase the number of selector threads? A: For the numcorethreads, It will allocate more thread for indexing but the load on server in general should be considered before increasing the the number of threads. The selector threads should remain the default.

3 Q: Can secondary AR servers point to a load balancer name (which has multiple indexers AR servers behind it) instead of a dedicated indexer AR server? A: It may work (we haven't tested it this way), but failover could result in some issues, since that entry will be looked up in ranking form. Q: suggests that more than one thread on port can result in a fragmented index and slow down performance. Is having 4 threads configured on a problem A: If you are using AR 9.x - this is not an issue.- Because in 9.x, there are indexes per form. Q: When performing a reindex on our primary server user searches are not working on the secondary server. In arftsindx.log on both servers are messages that searching is not possible because indexing is underway??? A: Yes that is possible but if there are inconsistent results it might be a case that the record which you are searching might not have been indexed yet. Q: I do see only 0, 1 and 7 operation type when query is done on ft_pending A: 0 and 1 are the common ones; Create and Modify, respectively. 7 is a change was made to the field property. Maybe a field was modified or added to a form with an FTS or MFS Index. Q: Could this potentially be an issue on ? We're trying to figure out why a customer consistently has 100,000 records in ft_pending. A: Records which are getting inserted in ft_pending is based on the updates on records which users do. If constant updates that will be pushed for reindex. Q: So, does the upgrade to 9.0 update the collection directly automatically as part of the upgrade process? Or is the collection directory and files associated with 8.1 re-usable for 9.X? (just wanting to evaluate the time it will take for me to upgrade) A: Collections are upgraded as part of upgrade installer - an FTS migration utility runs during installation that fixes those collections. Q: What's the results for when someone tries to search on a 0 length (clob) field with the wild cards? A: Not sure if I understand question correctly, but let me try to answer - If that field is FTS / MFS enabled - it will be searched via Lucene indexes, there is no query in the database. Also see KA for info on the impact of wildcards on Performance.

4 Q: For 9.1 Centralized Configuration, there is a new entry in ar.cfg for "Configuration-Name". Inside a server group, should each server-group member set Configuration-Name to a specific server-group member (e.g. server1)? A: Each server has to have separate name (typically server name itself) because each of their configurations will be different. Q: After upgrading to 9.1 the schema_reindex (OperationType 6) entries are taking very long (Almost 8-10 hours) to complete. What might be the possible reason? A: That should be OperationType 5 - I would suggest see my suggestions on this thread: Q: For customers who want to stop using FTS...Disabling Indexer and Searcher via checkboxes on Server Information FTS tab...the result seems to be that records are still written to ft_pending table. Records pile up in this table in large numbers. A: Correct, until the time there is FTS license available - it will keep on writing to ft_pending in anticipation that FTS will be enabled in future. Q: It seems like disabling only stops the indexing of records in table ft_pending. But does not seem to stop records being created in table ft_pending. What is a good way to prevent this from happening? Other than disabling FTS on all form fields one by A: Remove FTS license from the server. Q: Hi - what is the recommended FTS threshold setting for ARS 9.1? A: It is the number records to fetch from Lucene on search. Generally the lesser the better from a performance perspective. Having said that it depends on how many relevant records you want to show. Q: Operation Type 6 entries get created for Help Desk and SRM:Request whenever we run data wizard A: Data Wizard uses the special Application Command found in KA This uses Operation 6 to reindex a form based on a date range, if provided. Q: Queries against ft_pending to monitor outstanding indexes should be accessible from an API call A: It's not available directly in API, but you may always use GLSQL API to query the database using admin credentials. Q: Can we query to get a list of all forms that have at least one indexed field? A: There is no ready API available. You can use the SQL statements found in KA as a guide to querying yourself from the DB.

5 Q: How can I determine how long a reindex will take and how to monitor the progress of reindex? A: Check value of the updatetime field in ft_pending. After every 5000 entries are indexed, server writes number of rows in this field. So it would be in multiples of This will keep on growing till it reaches count(*) from T table/view for your form. Q: Is it possible to have one or more FTS servers indexing/searching the same index? A: Only one indexer server can operate on one index. Though multiple searcher can search from same index. Q: What about through the RKM you can re-index a single form A: Yes, RKM allows to reindex single data sources (a form) but for other forms that are not integrated with RKM, the special run process can index those forms.-application-fts-reindex-form "form" [ "scantime" [ fieldid 1 ]... [ fieldid N ] ] See KA Q: also why do i see -2 in the updatetime field in ft_pending? A: -2 = failure in reindexing and it will be picked up by the recovery thread that runs every 1 hour. Q: Do you have a SQL query to get a list of all forms that have at least one indexed field? Looking to do a full re-index one form at a time with important forms done first. A: FTS indexing info is stored as part of object properties (field.objprop) - but its proprietary format and can change in future. If you want to find out - other way is to - disable indexer, and then perform reindex - see what gets populated in ft_pending. Those are your schema Ids with at least 1 FTS field. Take a look at KA The SQL Query there will help you. Q: If we archive data and run an FTS search, will it search the archive data/form A: If FTS is enabled on archive form - yes, it will search. Q: Does the Max Entries Returned by GetList affect the results returned in an FTS Search? A: Yes. If you are using ITSM - it's anyway a few records at a time. If yours is a custom app, this setting will play a role. Q: Do we have to reindex FTS if we enable a field to be FTS and MFS only? A: No, server will take care of this.

6 Q: If the local DB for ARS is case sensitive, and FTS is applied to additional fields to allow for case insensitive searches on those fields, at what point do you think switching database products makes sense. This is in terms of index size and performance. A: Switching Databases is a major task. I assume you mean switching from Oracle to SQL Server. These databases are very different in terms of manageability and control. Before switching, make sure you understand all the differences and the impact. Many customers have switched to Oracle Case Insensitivity and are happy. Many customers use SQL Server and are happy. So it is impossible for us to tell you at which point you should switch. Most FTIndexed fields are not indexed for the sole purpose if Case. They are used by Global Searches. Even after converting the database, you may have the same FT indexes in place. Q: We currently have 3 ARS servers in our environment - 1 Admin and 2 AR user facing. FTS indexing is set on each server. Is this advisable? A: There is no problem with this, but indexing will consume some CPU on each server and 3 servers is probably too much redundancy. Typically customers use only 2 servers as indexers for backup. Q: When indexing will happen on a form? When any field which has FTS enabled is modified or any field which is not indexed is modified? A: When FTS property is changed (on to off or off to on) - it will be reindexed via ft_pending entry. Q: in server group environment, collection folder from primary server is copied on to Secondary server, will that work? A: if your primary server was stopped and ft_pending was empty when you copied - yes, it will work if secondary server has fts license and FTS search is enabled.