Migration Reference. For migration from Spool File Archive to Common Server

Transcription

1 IBM Content Manager OnDemand for iseries Migration Reference For migration from Spool File Archive to Common Server Updated 6/7/2013 Visit our web site to make sure that you have the latest version of this document. Go to directly to the document at or go to and use as search criteria.

2 INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any performance data contained herein was determined in a controlled environment. Therefore, results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs are provided AS IS, without warranty of any kind. IBM shall not be liable for any damages arising out of your use of the sample programs. 6/7/2013 Page 2

3 Table of Contents Migration Reference Introduction to this Guide...4 Part 1: Migration The Basics...7 Migration overview...8 Why you might use a different approach when migrating...11 What can be migrated...11 Migration prerequisites...12 Reports that can NOT be migrated...13 Reports that require attention BEFORE they are migrated...15 Reports that require attention either BEFORE or AFTER they are migrated...23 Reports that require attention AFTER they are migrated...28 Backups as part of the migration process...35 Create an OnDemand System Administrator user to run the migration program...36 Create a TEST instance for definition migration testing...36 Do not rename Application Groups after migration...37 Migration status information...37 Description of the phases of migration...38 Phase 0: Coexistence...39 Phase 1: Definition Migration...40 Users and Groups...40 Migration Policies...46 Report Definitions...52 Phase 2: Definition Conversion...53 Phase 3: Index Migration...55 Phase 4: Index Conversion...59 Phase 5: Cleanup...61 Part 2: Migration The Details...64 More information on PTF SI More reasons why you might use a different approach when migrating...66 Restart migration if a new version of a report definition is created in SFA...68 Additional Reference Information...69 Tracking the status of migration...71 More details of Phase 0: Coexistence...73 More details of Phase 1: Definition Migration...76 How DOC reports are migrated...79 How PAGE reports are migrated...81 How NODX reports are migrated...85 How segmentation conditions are migrated...88 How posting date is migrated...90 How sequence number is migrated...92 How translate print control is migrated...93 How SCS reports with an AFP overlay are migrated...95 How key security is migrated...97 How application groups are created...98 Part 3: Appendices Appendix A: Reading reports from Report Definition migration Appendix B: Differences between the Spool File Archive indexer and the OS/400 indexer Appendix C: Difference between Spool File Archive and Common Server dates used for expiration Appendix D: Tips for customizing the XML files created by migration Appendix E: Migrating only certain users to Common Server Appendix F: Automating the re-spooling of Spool File Archive reports Appendix G: Changing the Spool File Archive server port number Appendix H: Archiving migration reports /7/2013 Page 3

4 Introduction to this Guide Purpose This document replaces the information provided in Appendix A. Migration from Spool File Archive to Common Server in the IBM Content Manager OnDemand for iseries: Common Server Planning and Installation Guide Version 5 Release 4. This document includes additional information that was not available when the version 5.4 information was published. Summary of Changes Fifth Edition, May 8, 2013 Added note regarding getting current on Content Manager OnDemand PTFs prior to starting work on migration. Added an SQL statement that can be used to determine if you have any User Defined AnyStore data archived. Fourth Edition, August 8, 2012 Added information regarding migration of reports that are integrated between Content Manager for iseries and OnDemand Common Server that are *OTHER data type with an associated AFP overlay. Updated information regarding reports with report type of ANYS where object class is set to a numeric value other than 1-7 or 255. Added information about the differences between Spool File Archive and Common Server dates used for expiration of data. Added a warning not to use migrated definitions when choosing to re-spool and re-archive data instead of migrating. Also updated Phase 3 and Phase 4 descriptions to indicate that PRTRPTRDAR no longer works (use PRTRPTOND instead) after completion of Phase 3, not Phase 4. Third Edition, November 7, 2011 Added details regarding integration between Content Manager for iseries and OnDemand Common Server, and information regarding the removal of a second set of PAGERANGE indexer parameters under certain circumstances for PAGE reports. Second Edition, August 29, 2011 Added comments regarding the requirement for 'permit blank password' during migration processing, additional line required in the ARS.CFG file to enable coexistence, clarification of which optical storage groups are selected for migration, and simplification of prerequisite for Xerces Parser as described in the Batch Admin Guide. Some of these changes coincide with the OnDemand server version upgrade PTFs, and are noted in that section of the document. Also slightly modified cover page and header of document to clearly reflect Migration Reference publication name. First Edition, March 25, 2011 Replaces Appendix A of the Common Server Planning and Installation Guide version /7/2013 Page 4

5 Terminology This document uses the term OS/400 to include the Operating System/400 (OS/400) and i5/os operating systems. Use of the term iseries includes AS/400, iseries, and System i hardware that are running version 5.4 of the operating system. Audience This document is for IBM support personnel, and for customer technical staff that have a very detailed knowledge of OnDemand Spool File Archive (SFA) and Common Server. This document assumes that you, the reader, are already familiar with all aspects of Spool File Archive and Common Server, and that you only need to learn the details of migration. Minimum software requirements This document applies to Content Manager OnDemand for iseries version 5.4 with PTF SI40481 (or its superceding PTF) applied. Running migration on version 6.1 or higher is not supported. The Common Server option of 5722-RD1 must be at server level or higher. See the Information APAR for version 5.4, II14053, for a list of the required PTFs. It is also imperative that you load and apply all OnDemand PTFs for the product options you have installed. This must be done before you begin the migration process. Information APAR II14053 provides a list of all non-superseded PTFs to apply. The OnDemand Administrator client must be at level or higher. The OnDemand PTFs listed below, or their superseding PTFs, are required to enable Common Server integration with Content Manager for iseries. In addition to these PTFs, your OnDemand server must be running version (To determine the server version, log on to either the OnDemand Administrator or end-user client. Once logged on, the server version is shown on the message line in the lower right portion of the panel.) If your system does not already have the PTFs applied, the PTFs will be sent along with the integration PTFs as prerequisites. Version SI44867 and SI44906 Integration of the OnDemand Common Server feature is only supported with Content Manager for iseries version 5.3. Integration of Common Server with Content Manager for iseries version 5.1, VisualInfo/400, and Workfolder Application Facility (WAF) is not supported. Changes to Migration at PTF SI43848 (server version ) Upgrading to the version of the OnDemand server by applying PTF SI43848 and its corequisite PTFs introduces two changes to the migration process. The changes include the following: If you have configured OnDemand to retrieve both Spool File Archive and Common Server data from a single OnDemand Client logon (called "coexistence"), you must now add two lines to the ARS.CFG file for each instance that uses coexistence. The ARS.CFG file is located in IFS in the /QIBM/UserData/OnDemand/<instancename> directory on your IBM i server. The ARS.CFG file for any instance that already uses coexistence will contain a line that begins with ARS_MIGR_SERVERS= followed by 6/7/2013 Page 5

6 some additional parameters. You must insert a line below it that contains the following: ARS_USE_ODTBL_PREFIX=0 If the OnDemand server job is currently running for the instance, you must end the instance server and then restart it before retrieving documents or the retrieval will fail. Requirements for the installation of the Xerces parser as part of the arsxml configuration have been simplified. See the updates for server version in the Batch Admin Guide (also referenced below for PTF SI40481) at Changes to Migration at PTF SI40481 The migration of users, report groups, report definitions, and migration policies (storage sets) from Spool File Archive now uses Extensible Markup Language (XML) to migrate the objects to Common Server. The XML created by migration is stored in the SFAMIGR directory under the Common Server instance. For example, for instance QUSROND the directory is /QIBM/UserData/OnDemand/QUSROND/SFAMIGR. The arsxml command processes the XML files created by migration. You must have arsxml configured on your system before starting migration. Read and follow all instructions in the Batch Admin Guide at or you will be unable to run the migration tool. The XML files created by migration are not deleted after migration finishes. The XML files provide an audit trail of migration. The XML files also provide the possibility of customizing the XML. See Appendix C for some examples. Data area QRLRMIGJOB is now created during the installation of this PTF. See Optional data area to suppress job logs on page 69 for more details. The performance of user and report definition migration is substantially improved. See More information on PTF SI40481 on page 65 for more details. 6/7/2013 Page 6

7 Part 1: Migration The Basics 6/7/2013 Page 7

8 Migration overview Introduction Migration Reference At Version 5 Release 1, IBM Content Manager OnDemand for iseries (OnDemand) introduced a new server implementation known as the OnDemand Common Server. The Common Server provides enhanced indexing, searching, viewing, security, PDF Indexing, and web enablement capabilities for OnDemand users and administrators. Indexing enhancements up to 32 index keys up to 254 positions for index key length default key values string, decimal, integer, and date data types Searching tools full text search default search operators required search fields search result volume limits prevent performance degradation from inadequately defined searches Viewing unique logical views for each user, or group of users reports can be viewed from multiple folders creation of graphical annotations Security enhancements permit or deny viewing, printing, or faxing documents allow or prohibit the creation of logical views permit or deny adding or deleting annotations Optional capabilities The PDF Indexer can capture and index PDF output from Integrated File System (IFS ) directories; adding to existing support for line data, SNA character string (SCS), SCS-extended, and Advanced Function Presentation data stream (AFPDS) spooled files. The OnDemand Web Enablement Kit (ODWEK) provides browser access to your OnDemand archive. 6/7/2013 Page 8

9 With the AFP2WEB Transform you can view your AFP data in a PDF or HTML format when using an OnDemand Web Enablement Kit (ODWEK) interface. IBM Web Interface (WEBi) is designed as an out-of-the-box, easy-to-use, highly interactive Web Client that employs open standards and supports Web 2.0 and AJAX technologies. Common Server works with Kofax Ascent Capture to archive scanned documents and other PC files. Current OnDemand customers who have implemented Spool File Archive (with or without the AnyStore feature or the Server feature) can migrate to Common Server using these instructions. Note that throughout these instructions, references to the migration of Spool File Archive data also imply AnyStore data as well, if AnyStore is installed. For those customers who have had experience upgrading previous releases of R/DARS or OnDemand, each migration was incremental. Often, new fields were introduced to existing database files, new files were created, and the program function was enhanced. Since Common Server is a complete replacement for Spool File Archive, migrating from Spool File Archive to the new Common Server is quite a different experience. The files are all new, the terminology has changed, and the program architecture is client/server for most functions. A number of OS/400 commands are provided to perform operational tasks, but the majority of administrative and end-user functions are performed using the workstation client software. Visit the OnDemand for iseries support page at: Management/Content_Manager_OnDemand_for_i and search for comparison to find a document that provides a comparison between Spool File Archive and Common Server functions, commands, and APIs. There are two main areas of focus during the migration from Spool File Archive to Common Server: Migrating the definitions: Definition migration includes report definitions, report group definitions, public and private logical view definitions, key security, OnDemand users and user groups, migration policies, and optical storage groups. Named queries are not migrated. Users can re-create their named queries after the definition has been migrated. See Report Definitions that have named queries on page 22 for more information. Migrating the index information: After index migration to Common Server, the data already stored in Spool File Archive is accessible from Common Server. Index migration includes index records, annotations, and AFP resources. Migration is done through a series of program calls which first migrate the definitions, and then migrate the index information for your current Spool File Archive data. The actual archived data is not migrated. Common Server has been designed to access data stored in both Spool File Archive format and Common Server format. If you want to migrate your archived data from the control of the Report Management Cycle (RMC) to the control of Archive Storage Manager (ASM), use the Media Migration Facility (MMF ). More information about MMF can be found on the Content Manager OnDemand for i support web site, using MMF as search criteria. 6/7/2013 Page 9

10 Notes: 1. The migration steps described in this document do not apply to OnDemand Object Archive or Record Archive. 2. You must read these instructions in their entirety before beginning your migration activities. Migration steps are very interdependent, and a thorough understanding of the entire process is mandatory for a successful migration. 3. In addition to reading this document thoroughly, you must also read the Content Manager OnDemand for iseries: Common Server Planning and Installation Guide. In addition to installing the prerequisite Common Server software and related workstation software, you need to have done some configuration work described in that publication. Terminology is used in this document that is introduced in that publication as well. 4. Visit our support web site to make sure that you have the latest version of this document. Search for migration at: Management/Content_Manager_OnDemand_for_i 5. There are some characteristics of Spool File Archive report definitions that do not have a corresponding function in Common Server. See the sections of this document titled Reports that can NOT be migrated on page 13 and Reports that require attention BEFORE they are migrated on page 15 for a list of areas that need to be reviewed before migration. 6. When using the OnDemand end-user client to access data stored in both Spool File Archive and Common Server, the user is prompted for a single user ID at logon. The system logs the user onto Common Server and to Spool File Archive using the same user ID and password. This is known as Coexistence and is explained in more detail in Phase 0: Coexistence on page Changes you make to a Spool File Archive report definition after it is migrated to Common Server are not reflected in Common Server. If you require these changes to be reflected in Common Server, you must make the required changes in Common Server to match the changes made in Spool File Archive. 8. The migration tool is available only with OnDemand 5.3 and 5.4. Note that OnDemand 5.3 is out of support. You must migrate from Spool File Archive to Common Server before upgrading your operating system and OnDemand to 6.1 or higher. 6/7/2013 Page 10

11 Why you might use a different approach when migrating If you want to take advantage of features available in Common Server that are not available in Spool File Archive, you might consider reprinting and re-archiving the data using new definitions (Application Groups, Applications, and Folders) created in Common Server. See Appendix F: Automating the re-spooling of Spool File Archive reports on page 131 for more information. You might also consider using a mixed approach, where you use the IBM migration tool for some reports, and where you re-spool and re-archive some reports using newly created definitions. See More reasons why you might use a different approach when migrating on page 66 for more details. What can be migrated You can migrate almost all report definitions from Spool File Archive to Common Server. However, there are a few exceptions. For those report definitions that you cannot migrate, you need to create their definitions directly in Common Server by using the OnDemand Administrator client. The following two sections list categories of reports that either you can not migrate, or that you might need to change to prepare them for migration. Because migration can be done in pieces, specifying individual report names, report group names, or generic names, you are able to select or exclude particular reports if desired or necessary. Note that the report definition analysis program in Phase 1 provides a report to identify any of your report definitions that fall into either of these categories. In addition, the actual report definition migration program lists any attempts to migrate report definitions in any of these categories. The following tables show the relationship between SFA and Common Server data types, and between SFA IFS object types and Common Server data types. Table 1: Data Type Migration SFA OTHER SCS AFPDS LINE AFPDSLINE BUFFER FILE Common Server SCS SCS-Extended AFP LINE IFS See Table 2 SPLF Not supported, convert to AFP Not supported AnyStore data type Not supported AnyStore data type Not supported AnyStore data type 6/7/2013 Page 11

12 If the SFA data type is IFS, you might need to know the SFA object class to determine how the report definition will be migrated to Common Server. See Reports with report type of ANYS where object class is set to a numeric value other than 1-7 or 255 on page 36 for more information. Table 2: Data Type IFS Migration SFA Object Class BMP (object class 1) GIF (object class 2) PCX (object class 3) PDF (object class 4) PS (object class 5) TIFF (object class 6) JFIF (object class 7) 8 through 255 (uses file extension to launch application external to the OnDemand Client) Common Server Data Type BMP GIF PCX PDF User Defined (uses file extension to launch application external to the OnDemand Client) TIFF JFIF (JPEG) User Defined (uses file extension to launch application external to the OnDemand Client) Migration prerequisites Before you can begin migrating your data from Spool File Archive to Common Server, you must already have Common Server installed on the same iseries server on which Spool File Archive is installed. (Common Server is product option 10 of the 5722-RD1 licensed program.) You must install the OnDemand Administrator client and end-user client on a Windows workstation so you can validate the migrated definitions and index data. The OnDemand Administrator client must be at level or higher. Before migrating users from Spool File Archive to Common Server you must add the user profile that will be used to run the migration program to the Common Server instance. If migration is run with a user profile not already authorized to the Common Server instance, migration fails with error 64 Export Failure. See Create an OnDemand System Administrator on page 36 for more information. The user profile used to run migration from Spool File Archive to Common Server should have the same language settings as your Common Server instance. For example, if you are migrating to an instance created with language code FRA and locale FR_FR.LOCALE, you should verify that the user profile used to run program QRLRMIG has those same settings. You can use the DSPUSRPRF command to compare your user profile to the instance user profile. If you do not run migration with the same language settings as your Common Server instance, unexpected results can occur, including but not limited to the Application Load Information not being correctly populated. 6/7/2013 Page 12

13 If you are using the Kofax Ascent Capture Release Script PRPQ to archive images using Spool File Archive and AnyStore, you need to acquire the Common Server Release Script to replace the Spool File Archive Release Script you are using. The Common Server Release Script is available directly from Kofax on their web site. If you are using the AnyStore or Advanced Spool File Archive APIs in your own customwritten application programs, you need to replace them with the corresponding Common Server APIs. Documentation on the Common Server APIs is available in the Content Manager OnDemand for iseries Common Server Administration Guide. Documentation on the Common Server APIs and how they relate to the Spool File Archive APIs is available at: Management/Content_Manager_OnDemand_for_i Use comparison as your search. A migration local server instance was used, prior to PTF SI40481, to contain work files from migration. The migration local server instance consisted of the OND_MIG_INST directory and LOAD, TABLE, and VIEW subdirectories as shown below. After PTF SI40481 is applied to your system, you can delete these directories, if they exist, and all the files in them at your convenience. Directory.... : /OND_MIG_INST Opt Object link Type LOAD DIR TABLE DIR VIEW DIR After PTF SI40481 is applied to your system, the file /QIBM/UserData/OnDemand/CONFIG/ARS.INI no longer must contain the following entries to define the migration local server instance. If they exist, these entries can be deleted from the ars.ini at your convenience. [@SRV@_ONDMIGINST] HOST=LOCAL PROTOCOL=1 DIRECTORY=/OND_MIG_INST Reports that can NOT be migrated Content Manager (CM) for iseries integration (without Common Server to CM integration PTFs applied) Without the Common Server to CM integration PTFs applied as described in Minimum software requirements on page 5, there is no integration between Common Server and Content Manager. Integration of new archives ceases after report definition conversion (Phase 2) is completed. The remainder of this section applies to you only if you have not applied the integration PTFs. If you wish to continue integration between OnDemand and Content Manager, disregard this section, apply the required PTFs, and then follow the instructions in Content Manager (CM) for iseries integration (with Common Server to CM integration PTFs applied) on page 32 in the Reports that require attention AFTER they are migrated section. 6/7/2013 Page 13

14 If you no longer require integration between OnDemand and CM, you can migrate the report definition. A warning message is issued on the report definition migration report to indicate that you have just migrated a CM-integrated report. Until report definition conversion (Phase 2) occurs, the Spool File Archive to CM integration continues, even though you have begun the migration process to Common Server. During the Cleanup Phase (Phase 5) for the report definition, all CM integration records that were previously written to the CM control files are deleted. Until you run Cleanup, data that was integrated into CM from Spool File Archive continues to be retrievable from the CM client. If you require continued integration between OnDemand and Content Manager and choose not to use the integration support described in Content Manager (CM) for iseries integration (with Common Server to CM integration PTFs applied) on page 32, you should consider other options that offer a consolidated hit list across both OnDemand Common Server and Content Manager. These options include: a. IBM Content Integrator (ICI) b. IBM eclient Unbundled (UBND) report definitions Reports with report type of UBND are not migrated. You must now separate the bundled spooled file into individual spooled files before processing in Common Server. There is no Common Server equivalent to Spool File Archive UBND definitions. The UBND definitions are not migrated. However, the individual report definitions WITHIN the bundle can be migrated. Certain AnyStore (ANYS) report definitions Most AnyStore report definitions are migrated. However, AnyStore reports that fall into the following categories are not migrated: Reports with report type of ANYS where report data type is *SPLF, *FILE or *BUFFER. You might be able to store these AnyStore reports using the Common Server generic indexer. See the Content Manager OnDemand for i: Common Server Advanced Indexing Reference, available on our support Web site, for more information about the generic indexer. *AFPDSLINE report definitions You can not migrate reports with report data type of *AFPDSLINE to Common Server. You should consider modifying the way these reports are created so that the printer device type of their spooled file attributes is *AFPDS instead of *AFPDSLINE, if possible. 6/7/2013 Page 14

15 Reports that require attention BEFORE they are migrated Reports with an index exit Before considering the need to write new exit programs for Common Server, review the Common Server capabilities available that might eliminate the requirement for an index exit: Application options such as the removal of leading, trailing, or imbedded characters before index values are stored. Indexing options such as concatenating two fields into one index value. Folder options such as the ability to use a common date format when searching, regardless of the date format in the spooled file. The powerful string manipulation capabilities available using the qshell stream editor called sed can often eliminate the need for exit programs. Sed scripts and other options are specified as postprocessor parameters in the OnDemand application definition. See the OnDemand Administrator client online help or the Content Manager OnDemand for iseries Support Web site at: Management/Content_Manager_OnDemand_for_i using postprocessor as your search criteria for more information about various options available for postprocessor parameters. Reports with an input exit Replacing an input exit requires you to write a preprocessor program. If you do not write a Common Server preprocessor program, there is a high probability that the report will not store correctly after migration of the definition. Note that you must run the preprocessor program outside of OnDemand. There is no capability within Common Server to specify a preprocessor program. Reports with a viewer exit A viewer exit does not exist in Common Server. Data is presented in the standard OnDemand Client viewer. If you are using a viewer exit in Spool File Archive, you should review your need for that exit and see if your requirements can be met by the OnDemand Client viewer, the OnDemand Web Enablement Kit, or IBM WEBi. If you determine that you do need a different viewer, consider writing a program that uses the ODWEK Java APIs. *OTHER reports with an AFP overlay Background SFA allows you to store an SCS spooled file (report data type *OTHER) that specifies an AFP overlay in the spooled file attributes. 6/7/2013 Page 15

16 If you view the report by using a 5250 session, the data is displayed without the overlay. If you view the report using the OnDemand client, and you want the data displayed with the overlay, you must specify a printer file name that contains the overlay and you must change the value of Channel Code 7 in file QARLRACT. The value of this channel code directs the OnDemand client to convert the data to AFP at view time. After conversion to AFP the data is displayed in the client with the overlay. If you never use the OnDemand client, you probably have not changed the setting of Channel Code 7. Valid values for Channel Code 7 are: Value 0 or blank Before migration Meaning No special processing. The OnDemand client never shows the overlay and host print uses the current overlay as defined by the printer file in the report definition. 1 The OnDemand client will display the report with the stored overlay. 2 Host print uses the stored overlay. 3 Both the OnDemand client and host print use the stored overlay. If you have *OTHER reports that were stored with AFP overlays, you should verify the setting of Channel Code 7 before migration and change the setting if it is blank or 0. During and after migration If Channel Code 7 is NOT blank or 0, the report migrates as AFP. Any reports already stored in SFA are converted to AFP at view time. Any new reports stored after migration are converted to AFP during the archive process. If Channel Code 7 is blank or 0, the report migrates as SCS. After migration any reports previously stored in SFA are viewed as SCS. Any new reports stored after migration are stored as SCS. Determining which reports are *OTHER with an AFP overlay The following Structured Query Language (SQL) statement will list all SFA report definitions with data type *OTHER that also have stored AFP resources. SELECT CDTYPE, VERSION, CHANCODE7 FROM QUSRRDARS/QARLRRSC, QUSRRDARS/QARLRACT WHERE INTEGER(CHANCODE1/100)*100=AGID AND PRTDEVTYP = '*OTHER' The output will look similar to the following: Report Version Channel Type Code 7 INVOICETST 01 0 HCFAOVL 01 3 In this example there are two data type *OTHER reports that also have stored AFP resources. One report, HCFAOVL, already has Channel Code 7 set to 3; no further action is required before migrating this report. The other report, INVOICETST, has a 6/7/2013 Page 16

17 zero in Channel Code 7. Before migrating report INVOICETST you should change Channel Code 7 to a value of 1 or 3. See table above for valid values. Changing Channel Code 7 iseries Navigator You can change Channel Code 7 using either iseries Navigator or using SQL. In System i Navigator, open the properties of the report definition. On the Retrieval tab, specify the printer file name that contains the overlay. Then click on the desired options for using the overlay. The following figure shows changing the Channel Code 7 value to 3 for report HCFAOVL. The printer file HCFAOVL in library ONDSAMPLES contains the name of the overlay to be displayed with the data. Changing Channel Code 7 SQL The following SQL statement changes the Channel Code 7 value to 3 for report INVOICETST: UPDATE QUSRRDARS/QARLRACT SET CHANCODE7 = '3' WHERE CDTYPE = 'INVOICETST' 6/7/2013 Page 17

18 Reports with text-based report overlays Spool File Archive text-based overlays (created using option 3 - Work with Report Overlays on the OnDemand Report Administration Menu) are not supported in Common Server. The report definition migrates, but no text (screen) overlays are displayed when viewing. These non-graphical, character-based overlays should not be confused with AFP overlays, which are supported. SCS Extended reports with overstrike lines Spool File Archive SCS report definitions used to archive data containing overstrike lines might require the creation of a data area to continue archiving after migration without changes to the Common Server application. In Spool File Archive, if a report definition is defined as SCS, the resulting applications after migration have a data type of SCS-Extended. When Spool File Archive processes SCS data, it processes the overstrike lines. When Common Server processes SCS-Extended data, it does not process the overstrike lines. While not processing the overstrike lines is preferred, to avoid having to alter your applications after migration, you can define a data area to force Common Server to process the overstrike lines. Use the following command to create the data area: CRTDTAARA DTAARA(QUSRRDARS/QRLMSCSSFA) TYPE(*CHAR) LEN(10) It is important to note that this data area is global in scope, which means that ALL Common Server SCS-Extended applications with data containing overstrike lines are processed the same way. Once you have created this data area and archived SCS- Extended data, if you delete the data area, SCS-Extended data might fail to archive. It is also important to note that, if you have defined other Common Server SCS-Extended applications natively (rather than through migration), creation of this data area might require you to make changes to those applications. Reports with key security that have *ALL for Low and High key values When migrating report definitions, some user permissions might not migrate as expected. If you have key security enabled for a report in Spool File Archive with users having key security records of *ALL for both the low and high value and these same users are also members of a group profile with a more restrictive key security record (for example A* for the low value and M* for the high value), the results might not be as expected. This is due to the fact that *ALL for the low and high value in the key security record is not recognized by the OnDemand Administrator client in Common Server. This situation results in the user's authority to the report not being migrated, such that the user is only given permission to the application group and folder based on the permission of the group profile the user belongs to. In this example, the user might be able to see everything for a specific report in Spool File Archive. However, after being migrated to Common Server, the user is only allowed to see documents in a range of A through M for the field they are searching on. Before migrating report definitions with key security enabled and users with a key security record containing low and high value *ALL, you should consider changing these values to achieve the expected results for the application group permissions after migration is completed. The SQL statement shown below can be used to globally change the key security records having a low and high value of *ALL for any report in 6/7/2013 Page 18

19 Spool File Archive with key security enabled to a low value of '.' and a high value of ' '. This provides the same results as low and high value *ALL and also allows the user permissions to be correctly migrated. The SQL statement below needs to be executed for each one of the key security files in library QUSRRDARS, which are QARLRSEC1, QARLRSEC2, QARLRSEC3, QARLRSEC4 and QARLRSEC5. Also, when running the SQL statement for the key security files 2 through 5, you not only need to change the name of the file in the SQL statement, but also change the field names. For example, KEY1_SEC_LOW_VALUE and KEY1_SEC_HI_VALUE need to be changed to KEY2_SEC_LOW_VALUE and KEY2_SEC_HI_VALUE when running the SQL statement for file QARLRSEC2. UPDATE QUSRRDARS/QARLRSEC1 SET KEY1_SEC_LOW_VALUE = '.', KEY1_SEC_HI_VALUE = ' ' WHERE HEX(KEY1_SEC_LOW_VALUE) = ' ' AND HEX(KEY1_SEC_HI_VALUE) = 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF' Reports with duplicate key names Reports with more than one key with the same key name can be a problem during migration. For example, a page report might have Part Number defined as the key name for key 1 and key 2. If you have reports with duplicate key names, you must edit the report definition and rename one of the keys before migration to Common Server. You can use the following SQL statement to determine if you have any reports that have duplicate key names. Note that if you skip key names (For example, if you use keys 1, 2, and 5), this SQL statement will not find the duplicate key names. SELECT DISTINCT A.CDTYPE, A."VERSION", A.KY1NM, A.KY2NM, A.KY3NM, A.FL1NM, A.FL2NM FROM QUSRRDARS/QARLRACT AS A, QUSRRDARS/QARLRACT AS B WHERE A.CDTYPE = B.CDTYPE AND A."VERSION" = B."VERSION" AND A.ARPTTYP!= 'UBND' AND ((A.KY3NM = ' ' AND A.FL1NM = ' ' AND A.FL2NM = ' ' AND A.KY1NM = B.KY2NM) OR (A.KY2NM!= ' ' AND A.KY3NM!= ' ' AND A.FL1NM = ' ' AND A.FL2NM = ' ' AND (A.KY1NM = B.KY2NM OR A.KY1NM = B.KY3NM OR A.KY2NM = B.KY3NM)) OR (A.KY2NM!= ' ' AND A.KY3NM!= ' ' AND A.FL1NM!= ' ' AND A.FL2NM = ' ' AND (A.KY1NM = B.KY2NM OR A.KY1NM = B.KY3NM OR A.KY1NM = B.FL1NM OR A.KY2NM = B.KY3NM OR A.KY2NM = B.FL1NM OR A.KY3NM = B.FL1NM)) OR (A.KY2NM!= ' ' AND A.KY3NM!= ' ' AND A.FL1NM!= ' ' AND A.FL2NM!= ' ' AND (A.KY1NM = B.KY2NM OR A.KY1NM = B.KY3NM OR A.KY1NM = B.FL1NM OR A.KY1NM = B.FL2NM OR A.KY2NM = B.KY3NM OR A.KY2NM = B.FL1NM OR A.KY2NM = B.FL2NM OR A.KY3NM = B.FL1NM OR A.KY3NM = B.FL2NM OR A.FL1NM = B.FL2NM))) The output will be similar to the following (bold type highlights the duplicate key names): Report Type Version Key1 Name Key2 Name Key3 Name Field1 Name Field2 Name CHECK$ 01 Account # Account # CHECK$$ 01 Account Num Account Num Cust Name CHECK$$$ 01 Account Num Customer Num Account Num CHECK$$$$ 01 Account Number SSN / Tax ID SSN / Tax ID CHECK# 01 Account # Account # SSN / Tax ID Balance 6/7/2013 Page 19

20 CHECK## 01 Account # Account Name Account # Balance CHECK### 01 Account # Account Name SSN / Tax ID Account # CHECK#### 01 Account # Account Name Account Name SSN / Tax ID CHECK##### 01 Account # Account Name Customer # Account Name CHECK####1 01 Account # SSN/ Tax ID Account Name Account Name CHECK@ 01 Account # Account # SSN / Tax ID Balance Code CHECK@@ 01 Account # Customer Account # Balance Code CHECK@@@ 01 Account # Customer SSN / Tax ID Account # Code CHECK@@@@ 01 Account# Customer SSN / Tax ID Balance Account# CHECK@@@@1 01 Account # Customer Customer Balance Account# CHECK@@@@2 01 Account # Customer Balance Customer Account# CHECK@@@@3 01 Account # Customer Balance SSN / Tax ID Customer CHECK@@@@4 01 Account # Customer Balance Balance Code CHECK@@@@5 01 Account # Customer Balance SSN / Tax ID Balance CHECK@@@@6 01 Account # Customer Balance Code Code If you do not change the duplicate key names, creation of the folder during migration will fail. The error messages will be similar to the following: 08/09/10 22:47:31: Adding folder-field, RD4575-Acct # 08/09/10 22:47:31: Add of folder-field, RD4575-Acct # was successful. 08/09/10 22:47:31: A field object named 'Acct #' already exists. Reports with blank key names If you have reports with blank key names, and you have archived data into the keys with the blank names, you must edit the report definitions and enter a key name before migration to Common Server. Report Definitions with different report types in different versions If you have a Spool File Archive report definition with different report types (DOC, PAGE, or NODX) in different versions, you need to re-spool these stored reports, create a new report definition (in Common Server) and re-archive the reports. If you have a Spool File Archive report definition with different report types in different versions, your users are probably having trouble finding the data they need in SFA. Why perpetuate that mistake to Common Server? Create a new definition in Common Server, re-spool all your SFA data, and archive it correctly. You can use the following SQL statement to determine if you have any reports that have different types in different versions. SELECT DISTINCT ACT1.CDTYPE, ACT1."VERSION", ACT1.ARPTTYP FROM QUSRRDARS/QARLRACT AS ACT1, QUSRRDARS/QARLRACT AS ACT2 WHERE ACT1.CDTYPE = ACT2.CDTYPE AND ACT1.ARPTTYP!= ACT2.ARPTTYP The output will be similar to the following: Report Version Report Type Type CHECKSAB 01 DOC CHECKSAB 02 NODX CHECKSAB 03 PAGE 6/7/2013 Page 20

21 Report Definitions with no reports stored If you have any report definitions that do not have any reports stored, you can delete them before migrating your report definitions. This simplifies your migration. You can use the following SQL statement to determine if you have any report definitions that do not have any reports stored. SELECT CDTYPE, "VERSION", RPTCOUNT FROM QUSRRDARS/QARLRACT WHERE RPTCOUNT = 0 The output will be similar to the following: Report Version Report Count Type CHECKSTMTS 01 0 TSTINV 01 0 HCFA 01 0 Migrating Report Definitions with a key field named Date If you have a Spool File Archive report definition for which you have defined one of the five key fields with a name of Date (in upper, lower, or mixed case), then migration of the report definition finishes successfully but you might experience problems after migration, including a data loading failure where system log message 88 might contain one of the following: Row 1: The string " " has a length of 10 and the field has a maximum length of 8. Row 1: The string "Date reformat failed" has a length of 20 and the field has a maximum length of 8. Row 1: The string "12/20/05" could not be converted to a date from the format of %Y-%m-%d. This occurs because the Spool File Archive report date is migrated in addition to the other five key fields, and it becomes a Common Server key field the same as the five user-defined Spool File Archive keys. When it is migrated, the Common Server field that is created is named Date and therefore creates a conflict if you also have one of your user-defined keys named Date. You can use the following SQL statement to determine if you have any report definitions that have a key name of Date. SELECT CDTYPE, "VERSION" FROM QUSRRDARS/QARLRACT WHERE KY1NM = 'Date' OR KY2NM = 'Date' OR KY3NM = 'Date' OR FL1NM = 'Date' OR FL2NM = 'Date' The output will be similar to the following: Report Version Type CHECKS 01 In order to resolve this issue, simply update your Spool File Archive report definition before migrating, changing the user-defined key from Date to Report Date or any other meaningful name. This change eliminates the conflict after migration. 6/7/2013 Page 21

22 Report Definitions that have named queries Named queries are not migrated from Spool File Archive to Common Server. If you have created any named queries, they must be re-created in Common Server after migration. The following SQL statement can help you determine if you have any named queries in Spool File Archive. SELECT DISTINCT QARLRNQ.FID, QARLRACT.CDTYPE, QARLRNQ.UID, QARLRNQ.NQNAME FROM QUSRRDARS/QARLRNQ, QUSRRDARS/QARLRACT WHERE QARLRNQ.FID = ((QARLRACT.CHANCODE1/100)*100) The output will be similar to the following: Folder ID Report User ID NQ Type Name 25,400 CHECKSB 0 Test 25,400 CHECKSB ,400 CHECKSB 3, ,500 CHECKSDB 0 PETER 11,700 CHKS_G1 0 PETER 11,700 CHKS_G1 0 PSGROUP 18,400 XMLLATE1 0 Smith 18,400 XMLLATE1 70,000 Jones The user id is 0 for public named queries. For private named queries, the user id is the UID of the OS/400 user profile. 6/7/2013 Page 22

23 Reports that require attention either BEFORE or AFTER they are migrated Reports with report type of ANYS where object class is set to a numeric value other than 1-7 or 255 In Spool File Archive, when a report definition is defined as Report type ANYS, the user is prompted to enter an Object class for the ANYS report definition. Object classes 1 through 7 are predefined to Spool File Archive, such as Object class 1 for BMP data or Object class 6 for TIFF data. During migration, if the AnyStore report definition has an Object class specified of 1 through 7, the definition is migrated with Data type set to User defined and File extension set based on the Object class specified in the Spool File Archive ANYS report definition, such as BMP or TIFF using the examples above. If an Object class greater than 1 through 7 is specified for the Spool File Archive report definition, such as 12, the user can provide the file extension to OnDemand so that the OnDemand client knows what application to launch for viewing the specific data. This is done by adding a record to the QARLRAPP file in library QUSRRDARS for the specific report definition and indicating the file extension, such as EXTENSION=RTF. The addition of the record in the QARLRAPP file is not a requirement, however, so you might have a situation where the Object class for your AnyStore report definition is something other than the predefined Object classes of 1 through 7, but there is no QARLRAPP file to indicate the actual file extension to be used for the data. During migration these two scenarios are handled differently: When the Spool File Archive ANYS report definition has an Object class specified that is not one of the predefined classes 1 through 7, and a record exists in the QARLRAPP file in library QUSRRDARS for the report definition specifying the specific file extension for the data, OnDemand migration from Spool File Archive to Common Server will handle the definition as follows: The ANYS report definition is migrated to Common Server successfully. The resulting application definition is created with the Data type specified as User Defined and the File extension set to match what was specified in the Spool File Archive QARLRAPP file for the ANYS report definition. So for example, migrating a report definition from Spool File Archive of report type ANYS with Object class 12 and the extension set to RTF in the QARLRAPP file would result in an application definition in Common Server that is Data type User Defined and File extension RTF. When the Spool File Archive ANYS report definition has an Object class specified that is not one of the predefined classes 1 through 7, and a record does not exist in the QARLRAPP file for the report definition that specifies the specific file extension for the data, OnDemand migration from Spool File Archive to Common Server will handle the definition as follows: Migration of the ANYS report definition will fail to complete successfully. In the definition migration job log, the following messages will be listed: OND2798 A parsing error occurred in file, Line 28, Column 24: OND2798 cvc-minlength-valid: Value ' ' with length = '0' is not facet-valid with respect to minlength '1' for type 'fileextstring'. 6/7/2013 Page 23

24 The messages indicate an error while parsing the xml data that is generated for the report definition during migration. This occurs when there is no value for the file extension in the xml data and is caused because OnDemand was unable to determine the file extension due to no record for this report definition in the QARLRAPP file indicating the file extension. There are a number of different ways to address this issue, either before or after migrating the ANYS report definition. Changing before migration If you know you have ANYS report definitions that fit this scenario, you might choose to address this issue before migrating the report definition, so that you never see the errors described above. To use this approach, prior to migrating the report definition, add a record for the ANYS report definition to the QARLRAPP file in library QUSRRDARS specifying the file extension for this report definition. The instructions on how to do this can be found in Chapter 4 in the IBM Content Manager OnDemand for iseries Administration Guide for Version 5 Release 3 or Version 5 Release 4 in a section titled Setting The PC File Extension to Launch a PC Application. Once the record has been added to the QARLRAPP file, the report definition can be successfully migrated to Common Server. Changing after migration If you have already attempted to migrate ANYS report definitions that fit this scenario, and migration failed, you can still address this issue after migration. You can choose either of two methods to address this. The first method is to add a record for the ANYS report definition to the QARLRAPP file in library QUSRRDARS specifying the file extension for this report definition. The instructions on how to do this can be found in Chapter 4 in the IBM Content Manager OnDemand for iseries Administration Guide for Version 5 Release 3 or Version 5 Release 4 in a section titled Setting The PC File Extension to Launch a PC Application. Once the record has been added to the QARLRAPP file, the xml that was generated for this report definition when definition migration was first run should be removed from IFS on your IBM i system. Use the WRKLNK command with the following object specified: /QIBM/UserData/OnDemand/instance/SFAMIGR/rptdef.XML (where instance is the name of your OnDemand Common Server instance and rptdef is the name of the report definition you are migrating). Then select option 4=Remove to delete the xml file from the SFAMIGR directory. The xml file must be deleted so that when you run migration again, the new xml can be created successfully. After the xml file is deleted, then rerun report definition migration for this ANYS report. The second method for addressing this issue is to manually enter the file extension in the xml that was generated for this report definition when definition migration was first run. The xml can be found down the following path using the WRKLNK command: /QIBM/UserData/OnDemand/instance/SFAMIGR/rptdef.XML (where instance is the name of your OnDemand Common Server instance and rptdef is the name of the report definition you are migrating). Select option 2 to edit this file and either scroll down or do a find to locate <uddata fileext=" " />. Position your cursor between the double quotes and enter the file extension. Using the above example, the line would look like this: <uddata fileext="rtf" />. There are two locations in the file where the extension needs to be added. You will need to scroll down or do a find to locate the second <uddata fileext=" " /> and enter the extension again. Once the extension has been entered in both places, press F3 twice to save the changes and exit the file. 6/7/2013 Page 24

25 The xml for this report definition is now ready to import into the Common Server instance. In the original job log for the failed migration attempt for this report definition, a message will be listed that contains the ARSXML ADD command needed to import the definition into Common Server. You can simply copy this statement, then enter QSH on the command line and paste the complete ARSXML ADD command into QSH and press enter to execute the command. Once the command has completed, verify that the Common Server application group, application and folder now exist in the instance for this report definition using the OnDemand Administrator client. Also verify that the Data type in the application definition is User defined and the File extension is the file extension that you manually entered into the xml for this report definition. You can use the following SQL statement to determine if you have any report definitions that are defined as Report Type ANYS with Object Class defined as something other than 1 through 7 or 255. SELECT CDTYPE, ARPTTYP, CHANCODE2 FROM QUSRRDARS/QARLRACT WHERE ARPTTYP = 'ANYS' AND CHANCODE2 <> 255 AND CHANCODE2 NOT BETWEEN 1 AND 7 The output will be similar to the following: Report Report Object Type Type Class ANYSPOOL ANYS 0 TESTANYS ANYS 0 TESTREPT ANYS 12 ANYS128 ANYS 128 Reports that use Overprinted Lines Do you still have reports that use overprinting? (Overprinting was used on line printers to produce bold print by striking the same character twice in the same location.) Because the overprinted line is present twice in the spooled file, it also is displayed twice when viewed by the OnDemand Client. Spool File Archive supports two methods of merging overprinted lines. The first method is named server merge, and uses the iseries server to merge the overprinted lines before sending them to the OnDemand Client. The second method is named client merge, and uses the OnDemand client to merge the overprinted lines after they have been received by the client. Common Server supports only one method of merging overprinted lines, that is the client merge method. Report definitions migrated from Spool File Archive to Common Server have to be changed from server merge to client merge. This change can be made before or after migration. To determine if you have any reports using the server merge method, use this SQL statement. SELECT CDTYPE, VERSION, CHANCODE5 FROM QUSRRDARS/QARLRACT WHERE CHANCODE5!= 0 The output will be similar to the following, any non-zero value in CHANCODE5 means that server merge is used for that report: 6/7/2013 Page 25

26 Report Version Client Type Overprint Merge CHECKSTMTS 01 1 CHECKSTMTS 02 1 FUELMAN 01 1 CHECKS Changing before migration Now that you know the report definitions and versions that need to be changed, you can use the iseries Navigator OnDemand Archive plugin to create a logical view specifying to merge overprinted lines. The following steps describe how to do this: Start iseries Navigator Open the OnDemand Archive plug-in, click on Report Definitions Right click on the Report Definition, and select Properties Click on the Client - Public Views tab, then click on the Add button Name the logical view, click on the Default view box, then select the correct Overstrike mode for your report. The choices are: Separate Lines: A line with a Suppress Space carriage control character is displayed on a separate line. For example, if the word TOTAL is underlined using the Suppress Space carriage control character on the next line in the document, then the word TOTAL is displayed on one line (without the underline characters) and the underline characters are displayed on the following line. This is the default setting, and is how the OnDemand Client has worked previously. Merge Lines: If two characters are in the same column position, an alphabetic character takes precedence over any other characters. For example, if the word TOTAL is underlined using the Suppress Space carriage control character on the next line in the document, then the word TOTAL is displayed without the underline characters. This has the same effect as setting channel code 5 in the QARLRACT file. Overstrike Lines: Enables true overstrike mode. If two characters are in the same column position, then the second character is displayed on top of the first. For example, if the word TOTAL is underlined using the Suppress Space carriage control character on the next line in the document, then the word TOTAL is displayed underlined. For information about the limitations of the overstrike lines option, see the OnDemand Windows Client help text. Click on the OK button to save the logical view Click on the OK button again to save the report definition To determine if your report definition uses the server merge method, click on the Client > General tab. If the Merge Overprinted lines box is checked, then the server merge 6/7/2013 Page 26

27 method is being used. After creating a logical view you can remove the check from this box. Changing after migration After the report definition is migrated, you still need to create a logical view to merge the overprinted lines. After migration you must use the OnDemand Administrator Client to create the logical view. You need to create a logical view for each application created (both the -xx and the non-dash applications.) Do not make any changes to the -xx versions other than creating the logical view. The following steps describe how to create a logical view: Start the OnDemand Administrator Client Logon to the instance Click on Applications Right click on the Application, and select Update Click on the Logical Views tab, see figure below 1. Name the logical view 2. Click on the Default view box 3. Select the correct Overstrike mode for your report 4. Click on the Add View button 5. Click on the OK button to save the application You can set other values while creating the logical view, if necessary. 6/7/2013 Page 27

28 Reports that require attention AFTER they are migrated Reports with a different number of keys across multiple versions Spool File Archive report definitions that have more than one version defined (for example, 02, 03, and so on, rather than just having the 01 version), that have a different number of keys defined in some of the versions, need to be modified. They need to be modified after the definition migration step is run and before you load any new spooled files, or before you migrate any indexes into the migrated applications in Common Server. The reason for this is that the migrated Common Server application group definition is a composite of all the keys for all the versions in the Spool File Archive report definition. If the current spooled file of the report has fewer keys than a previous version (due to the removal of a field from the report, for example), then an attempt to store the report will end in error. Conversely, if the current spooled file of the report has more keys than a previous version (due to the addition of a field to the report, for example), then an attempt to migrate indexes for the previous versions will end in error. The error is a result of the need to load data into all the fields defined in the application group, and only having values for some of the fields. Fewer Keys in the Current Version In the case where the current version has fewer keys, you need to update the migrated application definition to add a default value before loading reports in Common Server. For example, if version '01' of the report definition CHECKSTMTS in Spool File Archive has 5 keys defined and version '02' of the same CHECKSTMTS report definition has 4 keys defined, the resulting applications for this report definition in Common Server after migration are: CHECKSTMTS-01 with 5 fields (keys) represents version 01 in SFA, used for migrating indexes CHECKSTMTS-02 with 4 fields (keys) represents version 02 in SFA, used for migrating indexes CHECKSTMTS with 4 fields (keys) used for loading new reports into Common Server The CHECKSTMTS application has only 4 fields (keys) because the final (non-dash) version is the same as the highest Spool File Archive version. The CHECKSTMTS application group, because it is a composite view of all the versions of the report definition from Spool File Archive, has 5 fields defined. Loading data into the migrated CHECKSTMTS application group/application in Common Server requires a change to the CHECKSTMTS application definition. The change is required because the current data being loaded to the CHECKSTMTS version of the application CHECKSTMTS contains just 4 keys, but the application group definition CHECKSTMTS contains 5 fields. If the CHECKSTMTS application definition is not changed, the load fails with a message that 5 fields were expected, but only 4 fields were received. 6/7/2013 Page 28

29 To resolve this problem, use the OnDemand Administrator client to update the Common Server application definition. Update the application definition with the name that does not include the version number appended to the end of the definition name. In the example above, you would modify the CHECKSTMTS application definition. When updating the application, go to the Load Information tab and look for the field that no longer is displayed on the report. Define a default value for that field, so that OnDemand does not fail when it doesn t find the field on the printed page. If the field is a string field, then simply entering a blank satisfies the requirement for a default value. If the field is a decimal or integer field, use a 0 as the default value. You should also note that in the case where the current version has fewer keys, you need to update the highest dash version of the migrated application definition to add a default value before migrating indexes to Common Server. In the example above, you need to make the same update to application CHECKSTMTS-02 that you made to application CHECKSTMTS. More Keys in the Current Version In the case where the current version has more keys, you need to update some of the migrated application definitions to add a default value before migrating indexes to Common Server. For example, if version '01' of the report definition CHECKSTMTS in Spool File Archive has 4 keys defined and version '02' of the same CHECKSTMTS report definition has 5 keys defined, the resulting applications for this report definition in Common Server after migration are: CHECKSTMTS-01 with 4 fields (keys) represents version 01 in SFA, used for migrating indexes CHECKSTMTS-02 with 5 fields (keys) represents version 02 in SFA, used for migrating indexes CHECKSTMTS with 5 fields (keys) used for loading new reports into Common Server The CHECKSTMTS-01 application has only 4 fields (keys) because it represents the first (and oldest) Spool File Archive version. The CHECKSTMTS application group, because it is a composite view of all the versions of the report definition from Spool File Archive, has 5 fields defined. Migrating indexes to the migrated CHECKSTMTS application group and CHECKSTMTS- 01 application in Common Server requires a change to the CHECKSTMTS-01 application definition. The change is required because the migrated indexes being loaded to the application CHECKSTMTS-01 contains just 4 keys, but the application group definition CHECKSTMTS contains 5 fields. If the CHECKSTMTS-01 application definition is not changed, the index migration will fail with a message that 5 fields were expected, but only 4 fields were received. To resolve this problem, use the OnDemand Administrator client to update the Common Server application definition. Update the application definitions with fewer keys than the current version. In the example above, you would modify the CHECKSTMTS-01 application definition. When updating the application, go to the Load Information tab and look for the field that is not displayed on the old reports. Define a default value for that field, so that OnDemand does not fail when it doesn t find the field on the printed page. If the field is a string field, then simply entering a blank satisfies the requirement for a default value. If the field is a decimal or integer field, use a 0 as the default value. 6/7/2013 Page 29

30 PAGE reports Spool File Archive reports with report type of PAGE migrate successfully but must be modified before use in Common Server. The Common Server application definitions that result from PAGE report migration contain a mask that identifies which data on the print page should be used as index data. For example, a mask of ####.## causes the indexer to select the field only if the data in the field (from left to right) contains four numeric characters, followed by a decimal point, followed by two numeric characters. Because the migration program has no way of knowing the nature or format of the data that is to be used for indexing, it creates a very generic mask that allows ANY data to be considered valid index data. You need to update the resulting application definition for each migrated Spool File Archive PAGE report to specify a mask that more appropriately describes the data, or else the header lines of the report will probably be picked up as index values, making the index values meaningless and making retrieval of the data unpredictable. See How PAGE reports are migrated on page 81 for more detail on PAGE reports. PAGE reports with keys 4 and 5 defined Most Spool File Archive PAGE reports have only keys 1, 2, and 3 defined, which are the default keys provided by OnDemand for this report type. However, OnDemand does permit you to define keys 4 and 5 as well. If you have defined key 4 or key 5 for a PAGE report, there might be an error in the field mapping for these keys in the Common Server folder definition that is created when you run report definition migration (run type *MGRDFN). Key 4 might get mapped to a field named F_05 when it should have been mapped to F_06. Similarly, key 5 might get mapped to a field named F_06, when it should have been mapped to F_07. This error might not be encountered until you try to retrieve a migrated report, at which time you might receive an unhandled exception error on the OnDemand Client in a program named arsgui32. To eliminate this error, you can simply correct the field mapping as described below. OnDemand application group fields are mapped to OnDemand folder fields in the folder definition, under the Field Mapping tab. If you need to change the field mapping, you simply remove the incorrect mapping and add back the correct mapping. To do this, log on to the OnDemand Administrator Client. Select Update for the folder that contains the field mapping you need to change. Go to the Field Mapping tab. In the lower display box, locate the key with the incorrect field mapping defined. For example, locate the key 4 field of your PAGE report, mapped incorrectly to F_05. Click that key name under the Folder Field heading, and then click the Remove button in the top right corner of the panel. This removes the erroneous field mapping. Then find that same field name in the pulldown list under Name at the top left of the panel. Find the CORRECT field it should be mapped to in the upper display box (such as F_06) and click once to select it. Then click the Add button in the top right corner of the panel. This adds the new (corrected) field mapping to the list of all defined mapped fields in the lower display box. Scroll if you need to, to confirm that the new (corrected) one is displayed in the list of mapped fields. If it looks correct, click OK to save your changes and exit. 6/7/2013 Page 30

31 Reports with AFP overlays and Translate print control The combination of *OTHER data with AFP overlays and Translate Print Control is not supported in Common Server. Spool File Archive report definitions of data type *OTHER with an AFP overlay named in the associated printer file AND Translate Print Control set to Yes migrate from Spool File Archive to Common Server, but any attempt to load data in the migrated definition using the Add Report to OnDemand (ADDRPTOND) command fails. You need to modify the migrated report definition or create a new report definition in Common Server. SFA report groups with individual reports that have *PUBLIC authority set to *EXCLUDE For any Spool File Archive report group that has the authority for individual reports in the group set to *EXCLUDE for *PUBLIC, you should verify that the Common Server application groups created by migration for the individual reports within the group have the permissions set correctly for all the users that were authorized to the Spool File Archive report group. You need to set the permissions in the application groups for each user or user group that was only authorized to the Spool File Archive report group. The reason for this is that, in Spool File Archive, anyone who is authorized to the report group is automatically authorized to all the reports in the group. This means that most individual reports within a report group do not have explicit authorities defined for these users because the report group authority provides the individual report authority. The migration tool uses the authority for the report group to set the permissions for the folder that is created for the report group. That folder then acts to combine all the reports that are in the report group by combining all the application groups that are created for the reports in the group. There is no application group that is created for the report group. The authorities for the individual reports in the group are used by the migration tool to set the permissions for the application groups that are created for the individual reports. This means that any authority that was a result of the report group is not set in the permissions for the application group. For individual reports where *PUBLIC is set to *EXCLUDE, users are not able see the data in the application group unless you explicitly give them permission. You may find that the easiest approach is to change your application groups to have *PUBLIC permissions of Access and Logical Views. You can then control access to your data by modifying the folder permissions. Reports with data type of AFPDS that use imbedded indexes A report definition that has both a data type of *AFPDS and Use AFP Imbedded Indexes set to Yes requires manual changes after migration. (imbedded indexes are also known as Tagged Logical Elements or TLEs.) Due to differences in how Spool File Archive handles AFPDS reports, as compared to Common Server, keys 1 through 5 are shifted one position to the right by migration. If the keys are located on the printed page itself, this shift works as intended and the keys are correctly extracted by the Common Server OS/400 Indexer. However, any keys located on the imbedded indexes lines of data (such as those shown in the figure below) have the left most position truncated by the Common Server OS/400 Indexer. You can clearly see this problem by viewing sample data using the OnDemand Administrator client graphical indexing tool as shown in the figure below. Looking at 6/7/2013 Page 31

32 the defined indexes in Display mode, you can see the truncation of the first position of the key values for ACCOUNT NUMBER, CUSTOMER NAME, and DOCUMENT DATE that are extracted from these three imbedded index lines. To fix this problem, you must first determine which indexes are extracted from the imbedded index lines (TLEs) and which indexes are found within the actual printed page. For the indexes that are extracted from the imbedded index lines, use the OnDemand Administrator client to update the application definition s indexer parameters. Go to the Indexer Information tab of the application definition, click Keyboard for the Parameters Source, and then click the Modify button to change the second numeric value on the FIELDx entries as shown below: FIELD1=-1,88,9,(TRIGGER=2,BASE=0) << change 88 to 87 FIELD2=1,86,11,(TRIGGER=3,BASE=0) << change 86 to 85 FIELD3=0,18,12,(TRIGGER=4,BASE=0) << change 18 to 17 FIELD4=0,87,15,(TRIGGER=5,BASE=0) << change 87 to 86 No change is required for any indexes that are extracted directly from the printed page. The best way to verify the modified field definitions is to use the OnDemand Administrator Client graphical indexer. Click the Toggle between Display and Add Parameters icon. Toggle to Display mode and visually check the fields that are outlined to verify correct column selection. Any fields that are incorrect can be changed as described above, and then verified again by using the graphical indexer. Content Manager (CM) for iseries integration (with Common Server to CM integration PTFs applied) Without the Common Server to Content Manager integration PTFs applied as described in Minimum software requirements on page 5, there is no integration between Common Server and Content Manager for iseries. Integration of new archives ceases after report definition conversion (Phase 2) is completed. The remainder of this section applies only if you have applied the integration PTFs. If you do not wish to continue integration between OnDemand and Content Manager, disregard this section and refer to the information in Content Manager (CM) for iseries integration (without Common Server to CM integration PTFs applied) on page 13. After the integration PTFs have been applied, enable integration by creating a Content Manager integration postprocessor exit program, creating a symbolic link for it, and specifying it in the OnDemand application definitions that were created during report definition migration (Phase 1). You must add the postprocessor program to the migrated application definition that has no suffix (no -01, for example) AND to all application definitions that do have a suffix appended (such as -01, -02, etc. which correspond to the Spool File Archive report definition versions that existed before migration.) 6/7/2013 Page 32

33 The postprocessor program must be added to each of the Common Server application definitions before you run index migration (Phase 3). If you run index migration before you add the postprocessor program to your Common Server application definitions, no Spool File Archive index information will be integrated with Content Manager and you will not have an opportunity to reprocess the data without deleting the migrated indexes and starting over. Detailed instructions for creating the postprocessor exit and other CM integration setup and usage requirements can be found on the OnDemand for i support website at: ment/content_manager_ondemand_for_i using "Content Manager integration" for your search criteria. For reports that are *OTHER data type with an associated AFP overlay, a printer file (with the overlay named within it) must be specified in the Spool File Archive report definition in order for the overlay to be displayed when the data is retrieved by the Content Manager client. To confirm that a printer file is specified in the Spool File Archive report definition, click the Retrieval tab within the report definition using iseries Navigator or see the Environment section of the report definition using a 5250 session. The printer file information must remain in the Spool File Archive report definition even after the report has been migrated to Common Server in order for the Content Manager client to retrieve and display the data with the overlay. For reports that are *OTHER data type without an associated AFP overlay, a printer file can be specified in the Common Server application definition if desired to improve formatting or readability when it is retrieved by the Content Manager client. To confirm or add a printer file to the Common Server application definition, see the Print Parameters on the Miscellaneous Options tab within the application definition using the OnDemand Administrator client. An entry such as PRTF=MYLIB/MYPRTF in the Print Parameters, for example, causes OnDemand to use the MYPRTF printer file in MYLIB library to present the data for viewing in the Content Manager client. After you create your postprocessor exit program and specify it in the migrated application definitions, it is very important that you test the migrated report definition and verify that your postprocessor program works correctly to integrate the OnDemand data with Content Manager. If you do not test adequately and find an error weeks or months after you have begun storing the data into OnDemand, you must remove the integration links from Content Manager and store the OnDemand data again to cause integration to occur. During testing, it is recommended that you rename or delete the QRLRMIGJOB data area if one exists in library QUSRRDARS. If this data area exists, it suppresses migration job logs if there are no significant error messages generated during migration (as described elsewhere in this document). However, during the testing of your postprocessor program, it is good to be able to review all available messages. Once you have tested thoroughly and confirmed that the integration is working as required, you can rename or create the data area if desired and then continue with the migration process by running Definition Conversion (Phase 2). When you run the Index Migration (Phase 3) step, OnDemand will run the postprocessor exit and create new integration records in Content Manager for the data now migrated to Common Server. At this point there will be records for BOTH Spool File Archive and Common Server reports in the integration file. End users will see duplicate entries on the hit list when they perform a search by using the Content Manager Client. 6/7/2013 Page 33

34 Then, in the Cleanup (Phase 5) step of migration, the Remove from CM for OnDemand (RMVVIRDAR) command will run for all Spool File Archive reports integrated with Content Manager (report definitions with index exit QRLWEXITV or QRLWEXITVM only), removing the Spool File Archive integration records from Content Manager. After Cleanup completes successfully, there will no longer be duplicate entries on the Content Manager hit list. If the steps above have been followed for a migrated report, moving data to *ASM (the Common Server Archive Storage Manager) using the Media Migration Facility (MMF) is fully supported. Note that after Index Migration (Phase 3) is run and before Cleanup (Phase 5) is run, two sets of indexes will be integrated with Content Manager. The SFA indexes, which can no longer successfully retrieve the documents using the CM client; and the Common Server indexes, which can successfully retrieve the documents using the CM client. If you attempt to retrieve a document using the SFA indexes, you will receive message FRN4214W: An error occurred in loading the file. After receiving that error, you must close the CM client and restart it before you can retrieve any other documents. 6/7/2013 Page 34

35 Backups as part of the migration process It is very important that you take backups of your data at various points during the migration process. Ideally, you would take backups as described below before and after each step so that if you have an unrecoverable error, you can recover. However, realistically you might not be able to do that due to time constraints. You need to determine the appropriate balance between the amount of risk and the amount of time you are willing to accept. You also need to consider that restoring from a backup causes you to write over (and therefore lose) any new data that was loaded since the backup was taken. The more current the backup, the less data that is overwritten. At a very minimum, it is recommended that you perform backups at the following times: Before you begin the migration process Before you begin Phase 3: Index migration Before you run Phase 5: Cleanup Backup steps The following items should be backed up before beginning migration: QUSRRDARS library (which is the library that contains the Spool File Archive index data and control information that points to your archived Spool File Archive data, as well as the Common Server Archive Storage Management (ASM) information for all instances). SAVLIB SAVLIB(QUSRRDARS) DEV(your device name) Spool File Archive IFS path (which contains the Spool File Archive data that still resides on disk, along with AFP resources, and so forth, if they exist). SAV DEV(your device name) OBJ('/QIBM/UserData/RDARS/SpoolFile/*') QUSROND and/or other instance libraries (which are the Common Server instance libraries that contain index data and control information that points to your archived Common Server data). You need only backup instance libraries for instances that you are migrating to during migration processing. If you use the default instance named QUSROND: SAVLIB SAVLIB(QUSROND) DEV(device_name) If you have additional Common Server instances on your system that are being migrated to during migration processing: SAVLIB SAVLIB(instance_library) DEV(device_name) where your instance library is the name of the instance. Common Server instance IFS path (which contains the Common Server data that still resides on disk, along with configuration files, and so forth, for each of your instances that are being migrated to during migration processing). SAV DEV(device_name) OBJ('/QIBM/UserData/OnDemand/instance_name/*') where instance name is the name of any Common Server instances you have on your system that are being migrated to during migration processing. 6/7/2013 Page 35

36 Create an OnDemand System Administrator user to run the migration program Before you begin your migration project, you must manually create one OnDemand Common Server user using the OnDemand Administrator Client. This OnDemand Common Server user must be defined as a user type of System Administrator. This user should be created for the person who will be signed on to the iseries server to run all of the migration program calls. This OnDemand System Administrator user name might correspond to an existing OS/400 user profile, or you might choose to create a new OS/400 user profile and use that new user profile name for the new OnDemand System Administrator user that you create to run migration In either case, the reason that this OnDemand System Administrator needs to be created before you even migrate your other Common Server users is rather simple: You must be recognized as a valid Common Server System Administrator in order to create additional Common Server users. So remember to create this first Common Server System Administrator user manually before running the user migration step for your other users, and then continue to run the remainder of the migration steps with this same System Administrator user. Remember, too, that this user must have *ALLOBJ special authority to run the migration program calls, as stated in the requirements for running the migration steps. Also, this OS/400 user should have the same language and locale settings as the instance you are migrating to. If you are using key security in Spool File Archive, both a new OS/400 user profile and OnDemand user (one that is not specified in any of the key security files) should be created to be used to run the migration program calls. Do not use an existing OS/400 user profile. This will ensure that all Spool File Archive key security data is "transcribed" correctly into the corresponding Common Server application group's Query Restriction field. See How key security is migrated on page 97 more information on key security. After you complete Phase 5, Cleanup, this special Common Server user can be deleted from OnDemand by using the OnDemand Administrator client. At that time, the OS/400 user profile can also be deleted if it is no longer needed. Create a TEST instance for definition migration testing You should create a test instance (such as ONDTEST) for use in Phase 1: Definition Migration. This allows for parallel testing and verification Before running your production Definition Migration. To do verification for Phase 1, you have to store data using the migrated definitions. The test instance can then be deleted after you have finished migrating to your production instance(s). If you actually load data directly into an instance during this parallel testing without using (and then deleting) a test instance, you might end up with duplicate data after Index Migration (Phase 3) unless you manually delete all the loaded data with the Remove Report (RMVRPTOND) command. In most cases, your production instance is QUSROND, which is the default Common Server instance. QUSROND is the default value for the instance parameter on the 6/7/2013 Page 36

37 OnDemand OS/400 commands, so using this instance name as your production instance saves you from having to specify an instance name when you issue a command. Note that at version 7.1 you can create a data area to set your default instance. Do not rename Application Groups after migration After migrating from Spool File Archive to Common Server, you cannot rename the migrated Application Groups. Doing so breaks the connection between those new Application Groups and your archived data, which remains in the location used by Spool File Archive. If you rename a migrated Application Group and then attempt to retrieve data using the OnDemand client, you receive errors such as: Server failed while retrieving document." The system log will contain message 20, for example 'SM Error: ASM FAILED TO RETRIEVE OBJECT' and message 24, for example 'Object > < in Application Group >LCA< not found in node > LS RDARSTEST<.' Migration status information Migration commands may end with status code 3 Migration commands may end with status code 3, even when the migration completes successfully. For example, when running *MGRUSR or *MGRDFN, if arsxml ends with a non-zero return code, the migration command will end with status code 3. You need to check the reports and job logs created by migration to determine if there is actually a problem that needs correction. Migration reports can list different counts A number of different steps throughout the migration process generate reports. Do not be concerned if the reported counts are not correct or do not match from one step to the next. For example, the Folders generated count on the Migrate Definitions report can indicate that a non-zero number of folders were generated when, in fact, an error message in the body of the report actually indicates an error that prevented any folders from being generated. Or, the Number of AFP Resources count on the Analyze Indexes report might not match the number reported by the Migrate Indexes report. Error message can contain blank migration status If you encounter an error during migration processing, your migration report might contain a blank instead of an actual status value in the following error message. The message is indicating that there was a status value for one or more of the reports that prohibited processing from proceeding. Review the reports from the previous migration steps to determine when a previous error occurred. For example, your migration report might contain: Error - The migration status of does not allow index processing to proceed. At least one version is not at the correct status. 6/7/2013 Page 37

38 Description of the phases of migration These sections describe each phase of migration in detail. The following table provides an overview of each phase. Phase Phase Name Description 0 Coexistence The ability to access both Common Server and Spool File Archive data with a single logon to the OnDemand Client. 1 Definition Migration The migration of users, groups, migration policies, report group definitions, and report definitions. 2 Definition Conversion The report definitions that were migrated now use ADDRPTOND, STRMONOND, or ARSLOAD to store reports. 3 Index Migration The index records are moved to Common Server. 4 Index Conversion The index records have been migrated to Common Server, then manually verified in Common Server, and now are no longer accessible in Spool File Archive. 5 Cleanup The Spool File Archive index, annotation, and AFP resource data are removed. 6/7/2013 Page 38

39 Phase 0: Coexistence After Phase 0 is complete When Phase 0 is complete, users are able to access both Common Server and Spool File Archive data with a single logon to the OnDemand Windows Client. Overview What is coexistence? Coexistence is the ability to access both Common Server and Spool File Archive data with a single logon to the OnDemand Client. When you logon to the Common Server instance that is configured for coexistence, the list of folders contains a combination of user-authorized Spool File Archive folders that contain archived data as well as all of the user-authorized Common Server instance folders. Why would I want to use coexistence? To access your existing Spool File Archive data while at the same time accessing data migrated from SFA to Common Server and also accessing new data stored in Common Server. Where would I load my production reports, into Spool File Archive or Common Server? You would continue to use the Start Monitor for OnDemand (STRMONRDAR) or Start Coded Data Store (STRCDSRDAR) commands to store reports into Spool File Archive. New reports defined in Common Server and reports migrated from Spool File Archive to Common Server would now be archived in Common Server using ADDRPTOND or STRMONOND. How do I configure coexistence? Coexistence is configured by adding two lines to the ARS.CFG file for the Common Server instance. For example, if you want to enable coexistence for the QUSROND instance, you would edit the ARS.CFG file located at the path /QIBM/UserData/OnDemand/QUSROND/. At the end of the file, add two lines. The first line will contain the keyword ARS_MIGR_SERVERS. The parameters on this keyword are specified as follows: ARS_MIGR_SERVERS=RDARS my400.network.company.com If Spool File Archive uses the default TCP/IP port of 1445 you can also specify the coexistence information as follows: ARS_MIGR_SERVERS=my400.network.company.com The second line should be specified exactly as follows: ARS_USE_ODTBL_PREFIX=0 More information on coexistence See More details of Phase 0: Coexistence on page 73 for additional information on the parameters for the ARS_MIGR_SERVERS keyword. 6/7/2013 Page 39

40 Phase 1: Definition Migration The definition migration phase includes the migration of users, groups, migration policies, report group definitions, and report definitions. Organization This phase is organized with a section for users and groups, a section for migration policies, and a section for report group and report definitions. After Phase 1 is complete When Phase 1 is complete the following occurs: The Start Coded Data Store (STRCDSRDAR) command and the QrlsOpenAnyStoreObject API give warning messages that the definition has been migrated. The Work with Admin for OnDemand (WRKADMRDAR), Work with Security for OnDemand (WRKSECRDAR), and iseries Navigator functions give warning messages that the definition has been migrated. When Phase 1 is complete, it is imperative that you test the migrated definitions by actually storing data in Common Server using the newly-migrated definitions. Note that you only need to test storing data to the non-dash version of the migrated definitions. Storing data to the -xx versions of the migrated definitions is not supported. The -xx versions are only used for migrating index data in Phase 3, and for viewing reports using that index data. Pay special attention to the index (key) values that are extracted for the stored data. Converting the indexer parameters (i.e. where the keys and segmentation criteria are ) is the most critical part of definition migration. Users and Groups Overview This section will help you understand how users and groups authorized to Spool File Archive migrate to users and groups in Common Server. Basics of Users and Groups migration 1. All users with access to Spool File Archive migrate to Common Server. 2. All groups with access to Spool File Archive migrate to Common Server. 3. A user successfully migrated from the Spool File Archive environment to the Common Server environment has the authority to access the same reports in Common Server as in Spool File Archive. 4. A group successfully migrated from the Spool File Archive environment to the Common Server environment has the authority to access the same reports in Common Server as in Spool File Archive. 6/7/2013 Page 40

41 5. A group successfully migrated from the Spool File Archive environment to the Common Server environment has the same user members in Common Server as in Spool File Archive. Source of information The migration of Users and User Groups gathers that information from user profiles and authorization lists associated with Spool File Archive. Notes 1. You must add the user profile that you use to run migration to the Common Server instance before migrating users from Spool File Archive to Common Server. If you run migration with a user profile not already authorized to the Common Server instance, migration fails with error 64 Export Failure. If you are using key security in Spool File Archive, we recommend that you create a separate user to run migration, and delete that user when migration is complete. 2. Because passwords cannot be determined from the OS/400 user profiles, the internal OnDemand passwords for migrated users are set to blanks when the users are migrated. This requires that the Minimum Password Length setting must be set to Permit Blank Password within the System Parameters of the Common Server instance. This should be verified by using the OnDemand Administrator client, although it is the default when an instance is created and most systems will already have this parameter set correctly. Permitting blank passwords should not be a problem because most customers use the Common Server default which links OS/400 passwords to OnDemand passwords rather than using a separate, internal OnDemand password. Using the default OnDemand configuration, any OnDemandspecific passwords are ignored. If you have disabled the default linkage between OS/400 passwords and OnDemand internal passwords, all your users must change their password the first time they logon to Common Server. 3. If the QRDARS400 authorization list has *PUBLIC set to anything other than *EXCLUDE, and at least one report authorization list has *PUBLIC set to anything other than *EXCLUDE, then all non-ibm user profiles (user profiles that do not begin with the letter Q) are migrated as OnDemand users. This could create a large number of OnDemand users that you do not need. If this is the case on your system, you might want to take the time to analyze the OnDemand authorization lists and make changes to limit the number of OnDemand users created by the migration process. 4. All users with access to Spool File Archive are migrated to Common Server as a user type of User. You should review your users and manually change those needing additional capabilities to User Administrator, Application Group/Folder/Cabinet Administrator or System Administrator as appropriate. 5. Before running report definition migration, your Spool File Archive users and user groups must exist in the instance to which you are migrating. You can accomplish this by using the *MGRUSR migration option or simply by adding them to the Common Server instance manually with the OnDemand Administrator client or by using a program. See Appendix D for several example programs. The Common Server user and group names must be identical to the Spool File Archive names. If the users and groups do not exist in the Common Server instance before migrating report definitions, the permissions are not correctly set when the report definitions are migrated. 6/7/2013 Page 41

42 6. The maximum number of users that you can migrate is 20,000. If you have more than 20,000 users to migrate, contact software support before you begin. Command Run the users and user groups migration program. There are two required parameters, the run type (*MGRUSR) and the name of the instance to which the users and user groups are migrated. All users and groups are migrated when this program is called. The migration program produces a report that lists all OnDemand users and user groups that were created for Common Server from the Spool File Archive authorization lists. The migration of users and groups is performed by running the command: CALL PGM(QRDARS/QRLRMIG) PARM('*MGRUSR' 'QUSROND') where QUSROND is the name of the instance to which the users are migrated. A warning message, OND2798, is returned. IMPORTANT: YOU MUST REVIEW ALL THE MIGRATION STATUS REPORT AND JOB LOG CREATED DURING THIS STEP TO BE SURE THE ENTIRE PROCESS COMPLETED SUCCESSFULLY A completion message, RDR7099, is returned. The status code is 0 (zero) if the program finishes without errors. Migration of users and groups completed with status x. You might want to migrate users and groups to your test instance, then delete those users and groups that do not need access to Common Server. The remaining users and groups could be exported to your production instance using the OnDemand Administrator client. Another option is to modify the XML created by the migration tool and then use arsxml to add the users and groups in that XML file to your production instance. See Appendix D: Tips for customizing the XML files created by migration on page 114 for more information. How users are migrated Scheme for creating Common Server users from existing Spool File Archive authorization lists An OS/400 user profile becomes an OnDemand Common Server user if one of the following is true. ("Authorized" means the authority is not *EXCLUDE, and "user" is any profile not beginning with 'Q', but includes QRDARS400 and QRDARSADM.) all users, if *PUBLIC is authorized on the QRDARS400 authorization list and at least 1 report authorization list user has *ALLOBJ authority user is authorized on QRDARS400 authorization list user is authorized on any report authorization list and authorized as part of the QRDARS400 group profile user is a member of a group which has become an OnDemand group 6/7/2013 Page 42

43 How groups are migrated Migration Reference Scheme for creating Common Server user groups from existing Spool File Archive authorization lists An OS/400 group profile becomes an OnDemand Common Server user group if one of the following is true. ("Authorized" means the authority is not *EXCLUDE, and "group" is any group profile not beginning with 'Q', but includes QRDARS400 and QRDARSADM. ) The OS/400 group profile is QRDARS400 or QRDARSADM An OS/400 group profile that is authorized on QRDARS400 authorization list An OS/400 group profile that is authorized on any report authorization list 6/7/2013 Page 43

44 How to read the Migrate OnDemand Users and Groups report The *MGRUSR option creates a spooled file, QPRLRCVUG, that contains three sections, one section each for user migration and group migration, and a summary. User section The user section of the report lists each user profile that is eligible for migration. The reference column specifies why the user profile is eligible. For example if the user profile has *ALLOBJ special authority, or belongs to a group profile that is eligible, then it is eligible for migration. The message column indicates if the profile was migrated, and error codes if the profile was not migrated RD1 V5R4M OnDemand for eserver i5 Migrate OnDemand Users and Groups Instance : QUSROND User profiles: Profile Reference Message DBRYANT QRDARS Already exists in instance DAVISMJ *ALLOBJ 62 - Exported successfully TSTAMP *GROUPMBR 62 - Exported successfully Group section The group section of the report lists each group profile that is eligible for migration. The reference column specifies why the group profile is eligible for migration. The message column indicates if the profile was migrated, and error codes if the profile was not migrated RD1 V5R4M OnDemand for eserver i5 Migrate OnDemand Users and Groups Group profiles: Profile Reference Message QRDARS400 QRDARS Exported successfully QRDARSADM QRDARS Exported successfully 6/7/2013 Page 44

45 Summary section The summary section lists the number of group profiles and user profiles analyzed and migrated. OnDemand for eserver i RD1 V5R4M Migrate OnDemand Users and Groups Users and Groups migration statistics Group profiles: Processed : 4 Not required by OnDemand : 2 Exported : 2 Exist in specified instance : 0 Export failed : 0 User profiles: Processed : 94 Not required by OnDemand : 71 Exported : 22 Exist in specified instance : 1 Export failed : 0 6/7/2013 Page 45

46 Migration Policies Migration Reference Migrating Migration Policies from Spool File Archive to an existing Common Server instance During this migration step (identified by run type *MGRPCY), your Common Server target instance is populated with Migration Policies of the same name as the migration policies you have defined in Spool File Archive. Also during this step, an important piece of the new Common Server migration policy gets created behind the scenes that is critical to the successful retrieval of migrated data once the migration process is complete. It is important to understand that if you are migrating to a Common Server instance where you have already created one or more migration policies using the iseries Navigator OnDemand Archive plug-in to natively create Common Server migration policies, you need to verify that the names of those Common Server migration policies do not match the names of any Spool File Archive policies that will be migrated to Common Server. If a migration policy already exists in the Common Server instance to which you migrate your Spool File Archive migration policies, the *MGRPCY process skips that migration policy and goes on to the next policy, resulting in the important behind the scenes piece not being created. The *MGRPCY process completes successfully, because it assumes you had migrated those policies previously and were rerunning *MGRPCY to add some additional migration policies at a later date. It cannot tell that those Common Server migration policies are missing the critical behind the scenes piece. If you have created migration policies in your Common Server instance using the iseries Navigator OnDemand plug-in, and those policy names match the names of migration policies in Spool File Archive, then you need to perform three steps before running the *MGRPCY process. 1. Before running the policy migration step (*MGRPCY), run the following SQL statement to determine if the same policy name exists in Spool File Archive and in your Common Server instance. Change QUSROND to the name of your instance. SELECT QARLRCOL.COLLECTION_NAME AS SFA_POLICY_NAME, QARLCPT.POLICY_NAME AS CS_POLICY_NAME FROM QUSRRDARS/QARLRCOL, QUSRRDARS/QARLCPT WHERE QARLRCOL.COLLECTION_NAME = QARLCPT.POLICY_NAME AND QARLCPT.POLICY_INSTANCE = 'QUSROND' The output will look similar to the following: SFA_POLICY_NAME CS_POLICY_NAME DISK DISK 2. Create a new Spool File Archive migration policy that is a copy of the Spool File Archive migration policy with the matching name in your Common Server instance. Except for the migration policy name, the old and new Spool File Archive migration policies should be identical. 3. Change all Spool File Archive report definitions that use the old name (which matches the name already in Common Server) so that they now use the new migration policy name. You can use SQL or Query on the QARLRACT physical file in library QUSRRDARS to create a list of report definition names that require changing. If you have a relatively small number of Spool File Archive report definitions, your query could simply sort the QARLRACT file by migration policy name (field name 6/7/2013 Page 46

47 ACOLLN) so that you can easily find the report definitions you need to change. If you have a relatively large number of report definitions, you might prefer to include in your query only the records that contain the specific migration policies that need to be changed. 4. It is critical that you also make this name change in the SRT file for any stored reports that used the old policy name! Basics of Migration Policy migration The basic characteristics of the migration of SFA migration policies to Common Server are: Each migration policy in Spool File Archive creates a migration policy in Common Server. Each type of media that is used in a Spool File Archive migration policy creates a migration policy level in Common Server. After migration to Common Server, all migration policies have level 0001 as a disk pool storage group for ASP01. If a tape or optical level exists in Spool File Archive, it is level 0002 after migration to Common Server. Each optical storage group in Spool File Archive creates an optical storage group in Common Server. Migrated policies can also be used to control storage management for new reports that are archived into Common Server. All application groups are created to not cache data. All application groups are created to send data to the storage manager at load time. Source of information Information for migration policies is gathered from the migration policies that exist in Spool File Archive. (The RDARSRPT menu refers to migration policies as report policies.) Command The migration of migration policies is performed by running the command: CALL PGM(QRDARS/QRLRMIG) PARM('*MGRPCY' 'QUSROND') where QUSROND is the name of the instance to which the migration policies are migrated. All migration policies are migrated when the program is run. There is no method to select a subset of policies for migration. A warning message, OND2798, is returned. IMPORTANT: YOU MUST REVIEW ALL THE MIGRATION STATUS REPORT AND JOB LOG CREATED DURING THIS STEP TO BE SURE THE ENTIRE PROCESS COMPLETED SUCCESSFULLY A completion message, RDR7099, is returned. The status code is 0 (zero) if the program finishes without errors. Processing of migration policies completed with status x. 6/7/2013 Page 47

48 How migration policies are migrated Scheme for creating Common Server migration policies from existing Spool File Archive migration policies A Common Server migration policy is created for each existing Spool File Archive migration policy. Scheme for creating Common Server migration policy levels from existing Spool File Archive migration policies A Common Server migration policy level is created for each type of media used in the Spool File Archive migration policy, including disk. Note that migration policy levels which refer to external media (tape or optical) are disabled after migration. You must manually enable each migration policy level that refers to external media after adding media (tapes or optical volumes) to OnDemand and before running the Archive Storage Manager (ASM) in Common Server. Scheme for creating Common Server optical storage groups from existing Spool File Archive optical storage groups A Common Server Optical Storage Group is created for each existing Spool File Archive Optical Storage Group that is specified in at least one of the migrated Spool File Archive migration policies. How to read the Migrate Report Migration Policies report The *MGRPCY option creates a spooled file, QPRLRCVPCY, that contain two sections. One section for migration policy migration, and one section containing a summary. Migration Policy section The migration policy section lists each migration policy existing in Spool File Archive and the action taken on each during migration. Migration of a policy that uses optical media generates an action message that an optical storage group was created for that policy. A reminder message is generated that you should add optical volumes to the storage group RD1 V5R4M OnDemand for eserver i5 Migrate Report Migration Policies Instance : QUSROND Policy Action Taken Message ACCTRPT Migrated Storage group RDARSOPT created 04 - Reminder: add optical volumes D90OPTICAL Migrated Storage group RDARSOPT created 04 - Reminder: add optical volumes D90TAPE Migrated RDARSTEST Migrated 6/7/2013 Page 48

49 Summary section The summary section lists statistics on the number of policies processed and the action that was taken. OnDemand for eserver i RD1 V5R4M Migrate Report Migration Policies Policy migration statistics Processed : 4 Migrated : 4 Failed : 0 Disk-only : 0 Already migrated : 0 An example of Migration Policy Migration Existing Spool File Archive Migration Policy example 1 The following graphics show the existing migration policy in Spool File Archive. The sample Spool File Archive migration policy, D90OPTICAL, shown below, migrates to a Common Server migration policy as shown in the second figure. 6/7/2013 Page 49

50 Migration Policy after migration to Common Server The following graphic shows the Common Server migration policy created by migration. The Days allowed on disk value of 90 migrates to 90 days in level 0001, the ASP01 disk pool. The Days allowed on optical value of 900 is migrated to 900 days in level 0002, the RDARSOPT optical storage group. Note that optical levels are always migrated as 'Disabled'. After adding optical volumes to Common Server, you should enable the optical level before running Archive Storage Manager (ASM). Existing Spool File Archive Migration Policy example 2 The Spool File Archive migration policy PERM, shown below, keeps archived data on disk forever. It migrates to a Common Server migration policy shown in the second figure. 6/7/2013 Page 50

51 Migration Policy after migration to Common Server The following graphic shows the Common Server migration policy created by migration. The Days allowed on disk value of migrates to days in level 0001, the ASP01 Disk Pool. In the Common Server migration policy a duration of days specifies No Maximum. 6/7/2013 Page 51

52 Report Definitions Migration Reference The basics of report definition migration are covered here. More detail can be found in More details of Phase 1: Definition Migration on page 76. Commands Before actually migrating report definitions, you should run the analysis function. The analysis function allows you to find any report definitions that can not be migrated, or that require changes before migration. The analysis function is run using the command: CALL PGM(QRDARS/QRLRMIG) PARM('*ANZDFN' 'QUSROND' '*ALL') The three parameters required to run the analysis program are: 1. Run type (*ANZDFN) 2. Instance name 3. Report name (specific name, generic, or *ALL). All versions of a report are analyzed at one time. Both report definitions and report group definitions can be processed in one call to the program. If a report group name is specified, migration analyzes all report definitions in the group. Note that a maximum of 200,000 report or report group definitions can be analyzed in a single run of this program. After you run the analysis function and address any issues that are found, you are ready to run the actual migration function. Note that if you use report groups in Spool File Archive, you should migrate the report group definition instead of the individual report definitions within the group. Migrating the report group definition migrates all the individual reports within the group, and also ensures that the report group folder is created for searching across the entire group in Common Server. The migration function is run using the command: CALL PGM(QRDARS/QRLRMIG) PARM('*MGRDFN' 'QUSROND' '*ALL') The three parameters required to run the migration program are the same as used for the analysis program. 6/7/2013 Page 52

53 Phase 2: Definition Conversion Overview When this phase is complete, the data and indexes for migrated report definitions continue to exist in Spool File Archive. The report definitions that were migrated now use ADDRPTOND, STRMONOND, or ARSLOAD to store reports. After you run this step, you cannot start over without assistance from software support, because you risk the creation of duplicate data in your Common Server archive. For this reason, you should be absolutely certain that you are satisfied with the results of your parallel testing described in Phase 1. If you run this step and then discover problems, contact software support immediately. This conversion phase protects you from loading the same spooled file data in both Spool File Archive and Common Server. If you were able to load the same report into both environments, you would end up with duplicate data after Phase 3, Index Migration. After Phase 2 is complete At the conclusion of Phase 2 you must convert to storing the data using: ADDRPTOND instead of STRCDSRDAR for storing individual reports STRMONOND *OUTQ instead of STRMONRDAR for output queue monitoring arsload API instead of calling the QrlsOpenAnyStoreSegment, QrlsStoreAnyStoreSegment, and QrlsCloseAnyStoreSegment APIs for AnyStore STRMONOND *DIR or arsload API instead of STRMONANYS or calling the QRLSSTART program for Kofax integration At the conclusion of Phase 2, the following changes take place: Command STRCDSRDAR and QrlsOpenAnyStoreObject send an escape message that data archiving for this report definition is no longer permitted. STRMONRDAR fails to store the report, and the spooled file is placed in HLD (Held) status on the Error output queue specified on the monitor command. WRKADMRDAR and WRKSECRDAR prohibit changes to the report definition, since it has already been migrated. Operations Navigator prohibits update or adding of a new version of the migrated report definitions. Run the definition conversion program. Three parameters are required: 1. Run type (*CVTDFN) 2. Instance name 6/7/2013 Page 53

54 3. Report name (specific name, generic name, or *ALL). All versions of a report are converted at one time. Both report definitions and report group definitions are processed in this one program run. a. If a report definition is specified, it must have been migrated successfully during Phase 1. b. If a report group definition is specified, the reports which are members of the group are also converted, if they were migrated in Phase 1. c. If all reports in the group were not migrated successfully during Phase 1, the report group definition is not converted. d. If a generic name is specified, the program processes only report definitions (not report group definitions) that satisfy the generic specification. The reason for this is that, if report group definitions are processed when using a generic specification, there is a high probability that report definitions within the group will be converted even though they do not match the original generic specification. In all cases, the program monitors for duplicates and insures that none are processed. The conversion of report definitions is performed by running a command similar to the following: CALL PGM(QRDARS/QRLRMIG) PARM('*CVTDFN' 'QUSROND' 'PAY*') Changes to capture production reports using Common Server After converting your report definitions, you are ready to capture production reports in Common Server. If you have Spool File Archive commands set up in a job scheduler or in a production CL program, they need to be modified to use the new Common Server commands. The following table shows the mapping of commands from Spool File Archive to Common Server. Spool File Archive Command STRMONRDAR STRCDSRDAR CALL QRLSSTART or STRMONANYS Common Server Command STRMONOND TYPE(*OUTQ) ADDRPTOND STRMONOND *DIR Notes You can also use asrload, however, this is not as user friendly as STRMONOND *DIR. 6/7/2013 Page 54

55 Phase 3: Index Migration The process of migrating the Spool File Archive index data to Common Server is outlined below. Overview Only the index records are moved during migration. The report objects themselves are not moved to Common Server IFS directories. They remain in their current Spool File Archive location and are accessed from that location. When the migration is complete, the Media Migration Facility (MMF) can be used to move objects from Spool File Archive directories to Common Server directories where they are then managed by Common Server s Archive Storage Manager (ASM). More information about MMF can be found at in the document OD SFA Media Migration Facility at: Index migration also moves AFP resources into Common Server. The AFP resources are migrated to the ASMREQUEST directory. Changes have been made in Common Server's Archive Storage Management (ASM) such that when an object for a migrated report is needed, ASM knows that the object is located in Spool File Archive and runs the correct process to retrieve that object. No Spool File Archive functions, other than storage management, are used after index migration is complete. If a date range has been specified when running the migration program (in order to perform a gradual migration over time), end-users should continue to use Spool File Archive for production retrievals until all indexes for all dates are migrated. If endusers use Common Server while the individual date ranges of index records are being migrated, they are only working with a subset of the entire archive until all indexes are migrated. After Phase 3 is complete When this phase is complete, the index records, annotations, and AFP resources for the migrated Spool File Archive reports can be retrieved by Common Server using the migrated index records. Index data, resources, and annotations are migrated to Common Server. Note that a date range parameter is provided for Phase 3 to allow for gradual migration of index data. Any Spool File Archive index records migrated to tape or optical are returned to disk before migration to Common Server. If the *RCLIDX option is used to recall SFA index records existing on tape or optical, those index records are recalled to disk and remain on disk. After the *RCLIDX completes successfully, a separate *MGRIDX step must then be run to migrate the SFA indexes to Common Server. If the *MGRIDX option is used to recall SFA index records existing on tape or optical, those index records are recalled to disk, migrated to Common Server, and then deleted from disk all within the *MGRIDX step. 6/7/2013 Page 55

56 Each stored report occurrence in Spool File Archive is flagged when its indexes are migrated to Common Server. PRTRPTRDAR no longer works. Use PRTRPTOND instead. DLTRPTRDAR is not allowed. The command gives an escape message that the report cannot be deleted from Spool File Archive because it has been migrated to Common Server. Other useful information Work files are placed into the home directory of the user running the migration program. For example, /home/dbryant/mgridx_hcfaovl_ ind. If the user does not have a home directory, the work files are placed in the / directory. The file name is MGRIDX, an underscore, the report name, an underscore, the report date, and the sequence number. An extension of.ind is used for indexes,.res is used for AFP resources, and.ann is used for annotations. Before running *ANZIDX, you can quickly run an SQL statement to determine if you have any index records on tape or optical. SELECT DISTINCT CDTYPE, "VERSION", COLLN, IDXSTAT FROM QUSRRDARS/QARLRSRT WHERE IDXSTAT!= 'D' The output will be similar to the following, where O indicate optical and T indicates tape: Report Version Management Index Type Policy Status Name INVNEW1 01 OPT2 O INVNEW1 02 OPT2 O INVNEW2 01 OPT2 O INVNEW3 01 OPT2 O INVNEW4 01 OPT2 O MMF_DOC_01 01 MMFTAPEI T MMF_LWP_01 01 MMFTAPEI T MMF_PDF_01 01 MMFTAPEI T MMF_PPT_01 01 MMFTAPEI T MMF_PRZ_01 01 MMFTAPEI T MMF_XLS_01 01 MMFTAPEI T Command 1. Run the index analysis program. Index analysis can be run at any time. You might want to run it early in your migration planning to determine if you need to recall (pre-fetch) indexes. A report is produced that lists the number of index records that require migration (as well as the number of index records that must be recalled from optical or tape, if you have chosen to archive indexes in Spool File Archive). Five parameters are required: a) Run type *ANZIDX b) Instance name or *NONE 6/7/2013 Page 56

57 The instance name is not used when running Analyze Index. c) Report name Specific name, generic name, or *ALL Index analysis should not be run massively but in small, controlled pieces that coincide with your plan for the actual index migration. d) Date range - starting date in YYYYMMDD format or *AVAIL e) Date range - ending date in YYYYMMDD format or *CURRENT For example: CALL PGM(QRDARS/QRLRMIG) PARM('*ANZIDX' 'QUSROND' 'PAYCHECKS' ' ' ' ') 2. Run the pre-fetch program for any Spool File Archive index records that have been archived to optical or tape. THIS STEP IS AN OPTIONAL STEP that most customers will not run, since archiving index records was not recommended. Also, it is not mandatory that you run the pre-fetch program before migrating indexes. If you prefer, you can allow the index migration program to recall the indexes as part of its processing. If you prefer to pre-fetch your indexes, the pre-fetch program can be run at any time. Since index pre-fetch can run for a very long time, you might want to begin running it even during Phase 0. Five parameters are required: a) Run type *RCLIDX b) Instance name or *NONE The instance name is not used when running Recall Index. c) Report name (specific name) - No generic names or report group names allowed. d) Date range - starting date in YYYYMMDD format or *AVAIL e) Date range - ending date in YYYYMMDD format or *CURRENT For example: CALL PGM(QRDARS/QRLRMIG) PARM('*RCLIDX' '*NONE' 'PAYCHECKS' ' ' ' ') Because the recall indexes function uses the existing Spool File Archive index retrieve program, messages are issued for each index recalled. Message RDR0046 is issued and is similar to the following: The recall of ' ' to directory 'CHECKSDB ' is complete. The report is now available for viewing or printing. 3. Run the index migration program. (This step migrates annotations and AFP resources in addition to indexes.) Five parameters are required: a) Run type *MGRIDX b) Instance name 6/7/2013 Page 57

58 c) Report name (specific name) - Index migration should not be run massively but in small, controlled pieces since the process can be very time-consuming and very disk space-consuming if large amounts of index records are involved. d) Date range - starting date in YYYYMMDD format or *AVAIL e) Date range - ending date in YYYYMMDD format or *CURRENT Verification For example: CALL PGM(QRDARS/QRLRMIG) PARM('*MGRIDX' 'QUSROND' 'PAYCHECKS' ' ' ' ') To verify the migrated index data, a variety of retrievals should be performed in parallel between Spool File Archive and Common Server. For example, a Spool File Archive retrieval of *ALL in the first key could be compared to a Common Server retrieval using LIKE as the operator and % (percent) as the value. Compare the counts of items on the list, and ALSO compare the actual values for the keys. If mismatches are found, contact your OnDemand support provider. 6/7/2013 Page 58

59 Phase 4: Index Conversion Overview This phase marks the index records in Spool File Archive as having been converted to Common Server. Conversions in this context means that the indexes have been migrated to Common Server, then manually verified in Common Server, and now are no longer required to be accessible in Spool File Archive. When this phase is complete, migration is complete for the specified report. Spool File Archive storage management is still required, including the running of the Report Management Cycle (RMC). No other Spool File Archive functions are required. Before running index conversion, you should have manually verified the migrated index data, annotations, and AFP resources. After Phase 4 is complete At the completion of this phase the following restrictions on Spool File Archive functions are in effect. FNDKEYRDAR, FNDRPTRDAR, QrlsRetrieveAnyStoreSegment, QrlsRetrieveAnyStoreList, QrlrRetrieveReportSegment, and QrlrRetreiveReportKeyList all give escape messages indicating that retrieval is not allowed. RCLRPTRDAR works as long as the archived data is still in the expected location. Command To run the index conversion program, five parameters are required: 1. Run type *CVTIDX 2. Instance name 3. Report name - Index conversion should be run specifically for reports ready for conversion; not massively with the risk of converting before you are really ready. 4. Date range - starting date in YYYYMMDD format or *AVAIL. IMPORTANT NOTE: A date range can be specified, but is ignored. Index records for the ENTIRE date range for ALL versions of a report definition must already be migrated successfully for the *CVTIDX function to complete successfully. If all index records for all dates have been successfully migrated, then running the *CVTIDX function converts them all, even if a date range is specified. 5. Date range - ending date in YYYYMMDD format or *CURRENT. IMPORTANT NOTE: A date range can be specified, but is ignored. Index records for the ENTIRE date range for ALL versions of a report definition must already be migrated successfully for the *CVTIDX function to complete successfully. If all index records for all dates have been successfully migrated, then running the *CVTIDX function converts them all, even if a date range is specified. 6/7/2013 Page 59

60 For example: CALL PGM(QRDARS/QRLRMIG) PARM('*CVTIDX' 'ONDMIGR3' 'CHECKS' '*AVAIL' '*CURRENT') Index conversion messages If the index conversion program finishes without errors, the job log contains the message: *CVTIDX SUCCESSFUL for REPORT - xxxxxxx. where xxxxxxx is the report name. If the index conversion does not complete successfully, the job log contains an error message. For example: Error - The migration status of 3 does not allow index processing to proceed. At least one version is not at the correct status. 6/7/2013 Page 60

61 Phase 5: Cleanup Migration Reference Overview of cleanup following successful migration Cleanup of migrated Spool File Archive definition records and index records does not automatically occur after migration, even if the migration APPEARS to have run successfully. Cleanup is a separate step that can be deferred until all parallel testing and verification is complete to the satisfaction of all interested parties. It is important that YOU (and not OnDemand software) own the responsibility to determine when the time is appropriate to remove the old Spool File Archive data that has been migrated to Common Server. Additionally, if there is some kind of error or oversight in the migration process that is discovered late in the process, you have not lost the input data that allows you (along with software support) to fix and rerun. After Phase 5 is complete When this phase is complete, you have removed the Spool File Archive index, annotation, and AFP resource data for the specified report(s) that has been migrated to Common Server, along with any OnDemand control data if applicable. Command The steps to run cleanup are outlined below. The cleanup program is initiated using a program call (rather than a command). As an added control feature, none of the parameters have defaults; all parameters must be specified, or the program call fails. 1. Backup QUSRRDARS library (which is the Spool File Archive library with all the index data that will be cleaned up) SAVLIB QUSRRDARS 2. Run cleanup program. Five parameters are required: a. Run type *RMVIDX b. Instance name or *NONE The instance name is not used when running Remove Index. c. Report name (specific name) - Cleanup should not be run massively but in small, controlled pieces since the process can be time-consuming if large amounts of index records are involved. d. Date range - starting date in YYYYMMDD format or *AVAIL. IMPORTANT NOTE: A date range can be specified, but is ignored. Index records for the ENTIRE date range of ALL versions of a report definition must already be migrated and converted successfully for the *RMVIDX function to complete successfully. If all index records for all dates/versions have been successfully converted, then running the *RMVIDX function removes them all, even if a date range is specified. e. Date range - ending date in YYYYMMDD format or *CURRENT. IMPORTANT NOTE: A date range can be specified, but is ignored. Index records for the ENTIRE date range of ALL versions of a report definition must already be migrated and converted successfully for the *RMVIDX function to complete successfully. If all index records for all dates/versions have been successfully converted, then running the *RMVIDX function removes them all, even if a date range is specified. 6/7/2013 Page 61

62 For example: Final steps Migration Reference CALL PGM(QRLRMIG) PARM('*RMVIDX' 'QUSROND' 'PAYCHECKS' '*AVAIL' '*CURRENT') The cleanup phase does not actually reclaim the disk space from the data that the cleanup process deletes. The operating system requires that you run the Reorganize Physical File (RGZPFM) command in order to actually reclaim the disk space. However, access paths will shrink immediately upon record deletion. The following is a list of files in library QUSRRDARS that you should reorganize after ALL reports have been migrated successfully: QARLRRSC QARLRANN QARLR000PF QARLRxxxPF - where xxx is the 1, 2, or 3-character report group abbreviation (used only if you are using report groups in Spool File Archive) How to read the Remove Indexes report The *RMVIDX option creates a spooled file, QPRLRMIDX, that contains one section with a summary of the removal activity. Summary section The summary section lists statistics on the number of indexes processed and action taken RD1 V5R3M OnDemand for iseries Migrate Indexes Report name : MIGRCHKS Instance : ONDMIGR3 Date range Starting date : Ending date : Run type : *RMVIDX Index processing statistics: Total report occurrences : 1 Report occurrences already processed.... : 0 Report occurrences processed : 1 Index records processed : 48 Index records recalled : 0 Space needed for recalled indexes (MB).... : 0 Annotation records processed : 0 AFP resources processed : 0 Index processing successful. 6/7/2013 Page 62

63 Error messages If the index clean up does not complete successfully, the QPRLRMIDX report contains an error message. Some examples: Error - Index processing failed. Check the joblog for specific errors. Error - The migration status of x does not allow index processing to proceed. At least one version is not at the correct status. Stopping the SFA server jobs from starting after you migrate to Common Server After you complete the final phase (Phase 5 - Cleanup) of migration from OnDemand Spool File Archive to Common Server, you no longer need the Spool File Archive server jobs to start. (Typically, the server jobs are started by issuing the Start TCP/IP Server (STRTCPSVR) command with *ONDMD specified for the SERVER parameter.) To stop the Spool File Archive server jobs from starting when you issue the STRTCPSVR command for SERVER(*ONDMD), simply save the OnDemand licensed program and then delete option 5 of the OnDemand licensed program, which is 5722RD1. (Be careful to only delete option 5, which appears as "Content Manager OnDemand Server" when you display installed licensed programs on your iseries system. This is not to be confused with "Content Manager OnDemand Common Server," which is product option 10, which should not be deleted.) There are several ways to do the save and several ways to delete option 5. Below we have provided an example for one way to do the save and delete. You might prefer to use a different method. One of the easiest ways to save the licensed program is to select option 13 from the Work with Licensed Programs menu. (Enter GO LICPGM from the command line.) Scroll through the list until you find the 5722RD1 licensed program, and then enter a 1 to Save in front of each of the product options (including *BASE). This saves the entire licensed program and all PTFs that have been applied, so that if you delete the incorrect option by mistake, you can restore it again easily. You might choose to save to a save file (*SAVF), which simplifies the process because it does not require any special media (such as tape) to be mounted for the save (or restore, if necessary). After the save of the licensed program is complete, one way to perform the delete of option 5 is to issue the following Delete Licensed Program command as shown: DLTLICPGM LICPGM(5722RD1) OPTION(5) Again, be careful to specify OPTION(5) to delete only product option 5. Turning off coexistence after you migrate to Common Server Once you have completed migration from Spool File Archive to Common Server, if you had configured your Common Server instance(s) to use coexistence, you can now remove or comment out the following lines from the ars.cfg file for your instance. The ars.cfg file can be found in /QIBM/UserData/OnDemand/instance where instance is the name of your instance. The two lines to remove or comment out are: ARS_MIGR_SERVERS=XXXXXXXXXX ARS_USE_ODTBL_PREFIX=0 6/7/2013 Page 63

64 Part 2: Migration The Details 6/7/2013 Page 64

65 More information on PTF SI40481 Users Users are now migrated using the arsxml and the file /QIBM/UserData/OnDemand/instance_name/SFAMIGR/USERS.XML Performance improvements Comparing the performance on a 5.4 test system: Without PTF SI40481: Migrating 1204 users and 8 groups required 353 minutes Migrating 2500 users and 30 groups required 593 minutes. With PTF SI40481: Migrating 1204 users and 8 groups required 3 minutes. Migrating 2500 users and 30 groups required 7 minutes. Your performance will vary depending on your system environment. Migration Policies Migration policies are now migrated, in part, using the file STORAGESET.XML. Not all information used by migration policy migration is contained in the XML file. Migration policies are now created with the option selected to close aggregates after 3 days. The Archived Storage Management process will close an aggregate after it has been open for 3 days, or when the aggregate reaches the specified maximum size, whichever occurs first. Report Definitions Report definitions are now migrated using a file named report_name.xml. For example, migrating the CHECKSTMTS report definition creates the file CHECKSTMTS.XML. If you use the special value *ALL to migrate report definitions, the XML file is named Z_ALL_Z.XML. If you use a generic report name, the XML file is named the generic value followed by the underscore character. For example, the generic name CHK* creates an XML file named CHK_.XML. If you migrate a report definition containing a number sign (#); the # is replaced with a dash (-). For example, the report name DB#CHECKS creates an XML file named DB-CHECKS.XML. If you migrate a report group, all members of the report group are migrated. Common Server definitions for all members of the group, and the group itself, are included in an XML file with the same name as the report group. 6/7/2013 Page 65

66 The SFA posting date field is now migrated using the Common Server indexer parameter DEFAULT='_*FINDONCE*_'. This means that, in Common Server, the date is now located only in the first document of the input file. The date located in that document is now carried forward and used in all documents within the particular input file. See Posting Date on page 111 for more details. Performance improvements Comparing the performance on a 5.4 test system: Without PTF SI40481: Migrating 188 report definitions and 8 report groups required 106 minutes. With PTF SI40481: Migrating 188 report definitions and 8 report groups required 12 minutes. Both migrations resulted in the creation of 188 application groups, 520 applications, and 196 folders in Common Server. Your performance will vary depending on your system environment. Indexes Migration of indexes is not affected by this PTF. More reasons why you might use a different approach when migrating Features not available in Spool File Archive and not available during the migration 1 to Common Server include: Use of field data types other than string 2. Re-spool if you want to use these field data types for existing archives: Integer: Good for numeric data such as customer numbers and account numbers. Decimal: Good for currency amounts. Date and Time: Good for dates, but only if the dates are between January 1, 1970 and September 17, Use of string data type field lengths longer than 25 characters 3 : Re-spool if you want the existing archives to use the longer fields. 1 Footnoted items can be customized during migration by editing the XML produced by the migration tool. See Appendix D: Tips for customizing the XML files created by migration on page 114 for more details. 2 The field data type can be changed from string to integer or decimal by modifying the XML created by the IBM migration tool. See Appendix D: Tips for customizing the XML files created by migration on page 114 for more details. 3 The string key lengths can be increased to as much as 254 bytes by modifying the XML created by the IBM migration tool. In addition, the string type can be changed from fixed length to variable length. See Appendix D: Tips for customizing the XML files created by migration on page 114 for more details. 6/7/2013 Page 66

67 If not, migrate the existing definitions, then create new definitions and bring existing and new definitions together in a new folder. The maximum field length is 254 bytes; you can create both fixed length and variable length string fields. Definition of more than 5 keys plus posting date: Re-spool if you want the existing archives to have more keys If you DO NOT want existing archives to have more keys, migrate the existing definitions. Then create new definitions with more keys and bring the existing and the new definitions together in a new folder. The ability to change the field type from index to filter 4 : All Spool File Archive keys that are searchable are migrated as indexes A filter field is a field not used to identify a document or a field that is always used with an index field to refine the results of a query A filter field does not create a logical view (access path) over the index data Using Large Object Support: Re-spool if you want the existing archives to use large object support. If you DO NOT want the existing archives to use large object support, migrate the existing definitions, then change the migrated definitions to use large object support. Only data archived after you change the application is stored with large object support. Data migrated from SFA will not use large object support. 4 The field type can be changed from index to filter by modifying the XML created by the IBM migration tool. See Appendix D: Tips for customizing the XML files created by migration on page 114 for more details. 6/7/2013 Page 67

68 Restart migration if a new version of a report definition is created in SFA When running migration for a Spool File Archive report definition, all existing versions of the report definition are analyzed to determine the greatest number of keys used across all versions, the longest key length for a given key field that exists across all versions, and so forth. If, after running the Migrate Definition (*MGRDFN) step for a particular report definition, you have the need to create a new version of that report definition in Spool File Archive and then attempt to migrate this new version to Common Server, all things previously considered when *MGRDFN was originally run could be at risk. If the new version of the report definition now contains an additional key or the length of a key was increased to a value greater than a key length already existing in a previous version, the Common Server application group (that was created when the original versions of the report definition were migrated) is now incorrect. The correct way to handle the situation where you need to create a new version of the Spool File Archive report definition and you have already run *MGRDFN for a report definition (but not yet completed testing and therefore not yet run the Convert Definition (*CVTDFN) step), would be to delete the Common Server application group and applications that were created during the original *MGRDFN step, and then run migration again for this report definition. If you have a reason that prohibits you from being able to delete the previously migrated definitions, you should call your software support provider to discuss what other options might be available for your specific situation. 6/7/2013 Page 68

69 Additional Reference Information Test only report definitions that you are currently using for archiving If you are no longer storing data for a particular Spool File Archive report definition, you only need to migrate the definition and indexes; no testing of the actual storing of data to the migrated definition is required since future archival will not be done. (You do need to consider if you have annual reports that might not have been stored yet for the current year; they will also have to be tested.) In other words, if you have old report definitions that are no longer used to archive data, then you can migrate all your definitions and index data verify that you can retrieve all of your data test the storing of data only for report definitions that are currently being used skip testing the storing of data for report definitions that are not being used To determine which report definitions were used to archive data during 2010 for example, you can run this SQL statement: SELECT CDTYPE, COUNT(*) FROM QUSRRDARS/QARLRSRT WHERE ONAME LIKE '2010%' GROUP BY CDTYPE The output will look similar to the following: Report COUNT ( * ) Type CHECKS 2 CHECKS01 12 CHKSTMTS1R 1,000 Now you know the report definitions that have been used during You can focus your migration archive testing efforts on these reports. Optional data area to suppress job logs When running the QRLRMIG migration program, numerous job logs might be produced, even when processing is successful. You can choose to minimize the job logs produced by creating a data area, named QRLRMIGJOB, in library QUSRRDARS. Create the data area with Type = *CHAR, with Length = 1, and with no initial value. The existence of this data area causes the migration program to generate job logs only if errors occur. This data area is now created by default when PTF SI40481 is applied. CDROM Mastering for data migrated from Spool File Archive to Common Server CDROM Mastering is not supported for data that is migrated to Common Server from Spool File Archive. If you attempt to use CDROM Mastering with data migrated from SFA, the server job log will contain MCH3601 Pointer not set for location referenced. 6/7/2013 Page 69

70 Do not specify Postprocessor Parameters in the -xx versions of applications (in most cases) Postprocessor Parameters should only be specified in the version of the application used to load new data, which is the non-dash version (CHECKSTMTS for example, and not CHECKSTMTS-01.) Specifying Postprocessor Parameters in the -xx versions of applications might cause index migration to fail. The only exception to this rule applies if you are integrating OnDemand Common Server with Content Manager for iseries. If you are integrating with Content Manager for iseries, you will specify a Postprocessor Parameter exit program in both the non-dash and -xx versions of any integrated applications as described previously in this document. Using ARSDOC GET with migrated data and the -X parameter The -X parameter of the arsdoc get API can be used in an application program to retrieve Common Server documents for a specified load ID. Note that this -X parameter is not supported for data that has been migrated from Spool File Archive. If you run the arsdoc get command for migrated data, and use the -X parameter, the QRLCSFARO1 job log contains MCH0601 Pointer not set for location referenced. Using ARSDOC GET with migrated data and the -n parameter If you use the arsdoc get API to retrieve data migrated from Spool File Archive to Common Server, you must use the -n parameter. The -n parameter specifies 'Do not use bulk retrieve'. If you do not use the -n parameter the data retrieval will hang without any error messages in qshell. The server job log will contain MCH3601 Pointer not set for location referenced. The following figure shows the messages from a typical retrieval of migrated data using a Named Query created using the OnDemand Client. The folder name and application group are both named CHKIDXMGR1. The Named Query is named KATIE, and the output file is prefixed by KATIE.OUT. $ arsdoc get -h QUSROND -q KATIE -v -o KATIE.OUT -f CHKIDXMGR1 -G CHKIDXMGR1 -g -N -c -n 08/20/09 23:18:55: Starting arsdoc. Version: /20/09 23:18:55: QRDARS/ARSDOC get -h QUSROND -q KATIE -v -o KATIE.OUT -f CHKIDXMGR1 -G CHKIDXMGR1 -g -N -c -n 08/20/09 23:18:55: Searching for folder 'CHKIDXMGR1'... 08/20/09 23:18:56: Search successful 08/20/09 23:18:56: Searching for documents in 'CHKIDXMGR1'... 08/20/09 23:18:56: Search successful 08/20/09 23:18:56: 1 document(s) have been queried. Retrieving 1 document(s). 08/20/09 23:18:56: (1): Retrieving document for userid ''... 08/20/09 23:18:57: Document successfully retrieved and stored in file 'KATIE.OUT.0.CHKIDXMGR1.CHKIDXMGR1-01.out' 08/20/09 23:18:57: arsdoc completed. $ 6/7/2013 Page 70

71 Tracking the status of migration Report definition The status of the migration of your report definitions can be tracked at any time by creating a Report Definition Status Report. The report can be generated in a number of different sort sequences, depending on your requirements. The Report Definition Status Report is generated by running the command: CALL PGM(QRDARS/QRLRMGRSTS) PARM('*DFNSTS' '*RPTVER') The first parameter (*DFNSTS) indicates that you are running the Report Definition Status Report. The second parameter (*RPTVER in our example) indicates the sort sequence to use, and can be any one of the following values: *RPTVER (Report Name, Version) (Default, if parameter is not provided) *RPTGRP (Report Group, Report Name, Version) *STATUS (Migration Status, Report Name, Version) *STSDTE (Migration Status, Migration Date (in YY-MM-DD.HH:MM format)) *INSTANCE (Instance, Report Name, Version) All sort sequences are in ascending order. If you use the *RPTVER sort sequence as in the example above, the status report will look similar to the following: 12:28:51 ONDEMAND REPORT DEFINITION MIGRATION STATUS 3/23/2011 SEQUENCE: REPORT NAME, REPORT VERSION PAGE: 1 REPORT NAME VERSION MIGRATION DATE AND TIME MIGRATION INSTANCE MIGRATION STATUS INVOICE :20 QUSROND DEFINITION MIGRATED JIFF :20 IMAGES DEFINITION MIGRATED JOBLOG :20 QUSROND DEFINITION MIGRATED JOBLOG :20 QUSROND DEFINITION MIGRATED JOBLOG :20 QUSROND DEFINITION MIGRATED JOBLOG :20 QUSROND DEFINITION MIGRATED JPEG :20 IMAGES DEFINITION MIGRATED JPEGH :20 IMAGES NOT MIGRATED JPEG :20 IMAGES DEFINITION MIGRATED LATECHARGE :20 QUSROND DEFINITION MIGRATED PAYROLLREG :20 QUSROND DEFINITION MIGRATED Note that if you migrate the same report definition to more than one instance, the most recent date and time and most recent instance name will be listed on the report. An escape message, OND2798, is returned if a failure occurs. Report occurrence The status of the index migration of particular report occurrences (for specific dates) can be tracked at any time by creating a Report Occurrence Status Report. The report can be generated in either of two sort sequences, depending on your requirements. 6/7/2013 Page 71

72 The Report Occurrence Status Report is generated by running the command: CALL PGM(QRDARS/QRLRMGRSTS) PARM('*RPTSTS' '*RPTVER') The first parameter (*RPTSTS) indicates that you are running the Report Occurrence Status Report. The second parameter (*RPTVER in our example) indicates the sort sequence to use, and can be either one of the following values: *RPTVER (Report Name, Version, Object Name) (Default, if parameter is not provided) *STATUS (Migration Status, Report Name, Version) All sort sequences are in ascending order. The report will look similar to the following: 1:08:33 ONDEMAND REPORT OCCURRENCE MIGRATION STATUS 3/24/2011 SEQUENCE: REPORT NAME, REPORT VERSION PAGE: 1 REPORT NAME VERSION OBJECT NAME REPORT LOCATION INDEX LOCATION MIGRATION STATUS INVOICE DISK DISK REPORT MIGRATED INVOICE DISK DISK REPORT MIGRATED INVOICE DISK DISK REPORT MIGRATED INVOICE DISK DISK REPORT MIGRATED JPEGH DISK DISK NOT MIGRATED JPEGH DISK DISK NOT MIGRATED An escape message, OND2798, is returned if a failure occurs. 6/7/2013 Page 72

73 More details of Phase 0: Coexistence Parameters The parameters of the ARS_MIGR_SERVERS keyword are explained in the following table: Value shown in example RDARS Meaning Name that appears in parentheses after the folder name in the folder hit list. This value appears in parentheses beside all Spool File Archive folders to distinguish them from Common Server folders. This parameter is optional and, if left blank, defaults to the value specified for the host name parameter. In the graphic below, the character string RDARS was used for this parameter. my400. network. company. com Host name or TCP/IP address of the iseries server. This is the iseries server where both Spool File Archive and Common Server reside. In the graphic below, no value was specified for the first parameter of the ARS_MIGR_SERVERS keyword, so the host name is shown in parentheses after all Spool File Archive folders. 2 This value must always be Port number of the Spool File Archive environment. If left blank, this parameter defaults to 1445 which is the default that Spool File Archive uses for communications with the iseries server. (You might notice that the server definition on your OnDemand Client actually says Port 0. This is normal. Specifying Port 0 indicates to the client that you want to use the default port.) 0 Protocol indicator. This value must always be 0, which specifies TCP/IP. 0 Unified logon indicator. This value must always be 0, since the iseries server does not support unified logon. 6/7/2013 Page 73

74 ARS_MIGR_SERVERS Only one ARS_MIGR_SERVERS statement is supported per Common Server instance. If more than one ARS_MIGR_SERVERS statement is present in a single ARS.CFG file, only the last statement is used. The Spool File Archive server referred to in the ARS_MIGR_SERVERS statement must be on the local iseries server. If the Common Server instance is started, but the Spool File Archive server is not running, attempting to logon results in the error message: Note that the server name given in the message is the first parameter as described in the table above. The OnDemand Windows Client Security Changing the logon password using the OnDemand Windows Client while coexistence is configured might be confusing for end users. Since there are really two connections established to the OnDemand server (one for Spool File Archive and one for Common Server), the password change request is made first through the Spool File Archive session and then again through the Common Server session. The password is successfully changed through the Spool File Archive session. The Spool File Archive server job log contains message "CPC2205 User profile xxxxxxxxxx changed." However, no message to this effect is presented to the user. Then the the client makes a second attempt to change the password through the Common Server session. This second attempt fails and generates a message that the current password is incorrect (because it has already been changed). The Common Server job log contains "CPF22E2 Password not correct for user profile xxxxxxxxxx" and the client displays a similar message, also indicating that the password specified is not the same as the password defined in the user profile. Although the message to the end-user implies that the password attempt has failed, the password really has been successfully changed. The end-user should begin using the new password. 6/7/2013 Page 74

75 Considerations for using OS/400 MiXed case password support When logging on to the client, if OS/400 allows MiXed case passwords, and the Common Server instance supports UPPER CASE only, and your password is UPPER CASE only, then you must enter your password in UPPER CASE or you receive the error message Login succeeded, however additional server logons failed., which indicates that the Spool File Archivce server logon failed. This behavior is caused by Common Server changing the password to upper case before passing it to OS/400 for validation, while the Spool File Archive server does not. Note that the server name given in the message is the first parameter as described in the table above. Key security Spool File Archive reports with key security defined are not accessible when coexistence is implemented. It is recommended that report definitions that use key security be some of the first definitions that you migrate if you plan to use the coexistence function. Annotations Note Marks are always saved and displayed in yellow on Spool File Archive coexistence documents. If you specify a different color using Options > Document Page Note Marks > Set Color for Added Notes, the note marks start out in the selected color, but do not stay that color after closing the document. The note marks are always yellow after closing and reopening the document. Server print Server printing of Spool File Archive coexistence data from the OnDemand client is not supported. Using the server print function of the client might result in message A document or document segment could not be retrieved. being displayed for each Spool File Archive document selected in the document list. The QRLGSVR job log might contain the message Procedure ArcDB_GroupQuery(uid=0) has been called but is unsupported by this OnDemand Server. Print All Selected The "Print All Selected" button on the OnDemand Client's "Search Criteria and Document List" panel is not supported if any of the documents you are attempting to print are Spool File Archive documents that have not been migrated to Common Server. Instead, the end-users should simply print these documents individually. (The "Print All Selected" button works correctly when printing migrated Spool File Archive documents or Common Server documents in this environment.) The OnDemand Web Enablement Kit The OnDemand Web Enablement Kit (ODWEK) does not support Spool File Archive coexistence. Some functions work and others do not, therefore IBM can not support Spool File Archive coexistence with ODWEK. 6/7/2013 Page 75

76 More details of Phase 1: Definition Migration Contents of this section The types of Spool File Archive report definitions described are: Document reports: These are reports that can be segmented into unique, separately-retrievable items, such as checking account statements or invoices. The abbreviation for this report type is DOC. Page reports: These are reports that can be logically indexed by a range of values based on the sort order of the data within the report. The abbreviation for this report type is PAGE. No index reports: These are reports that do not contain any meaningful indexes within the data. The abbreviation for this report type is NODX. Unbundled reports: Note that unbundled reports (report type UBND) are not migrated, but the report definitions that belong to the bundle can be migrated. The common Spool File Archive functions discussed are: Segmentation: The process of splitting the spooled file into separate items, such as individual invoices, that can later be retrieved individually. Posting date: The posting date is the date assigned to the report at the time it is stored. This date can either be the date the report was stored, or a date found within the report data. Translate print control: Translate print control uses the First Character Forms Control (FCFC) information in the input data to create a secondary view of the report page with each line of data placed as it would be on the printed page. This secondary view is then used to index the report. SCS with an AFP overlay: A spooled file with a data type of *SCS that also has an AFP overlay specified. The data type in Spool File Archive is specified as *OTHER. Terminology Some common terms used in this document are: Pivot value: In Spool File Archive, a string of one or more characters beginning at a known column on a report. It is used to specify the location of an index (key), segmentation value, or report date when such data does not appear on the same line of each page. Also known as a reference string. Trigger: In Common Server, data values that the OS/400 Indexer searches for in a print data stream to identify the beginning of a new group of pages. The first trigger is then the anchor point that the OS/400 Indexer uses to locate index values. Index: An index is used for search and retrieval of documents. In Spool File Archive, indexes are also known as keys. 6/7/2013 Page 76

77 Overview of Report Definition migration When migrating reports that belong to a report group, you should migrate the report group instead of the individual reports in the group. Migrating the report group also migrates all the individual reports in the group, and ensures that the report group folder is created in Common Server. Use of Spool File Archive emulation functions created during migration, as described in the following sections, requires support by the OnDemand Administrator client used to create and modify the indexing control statements (also known as indexer parameters) contained within the OnDemand application definitions. These Spool File Archive emulation functions include some distinct departures from the previous standards for indexer parameters: 1. Non-consecutive numbering of FIELD and INDEX statements. For example, a set of indexer parameters created by the migration process might have fields 1 through 4, then field 6, then fields 29 and Allowing the definition and use of two grouprange INDEX statements in a single application. 3. Use of the INDEXSTYLE statement with a value of PAGE or NODX. The OnDemand Administrator client tolerates such statements when they are added manually to the indexer parameters using the Keyboard modify function. However, entry of these statements using the Graphical Indexer is not supported. Note that these modifications can only be done to the migrated application definitions WITHOUT the -xx suffix. For example, CHECKSTMTS application definition can be modified, but CHECKSTMTS-01 cannot be modified, except under very specific circumstances described in this document. Control characters The Spool File Archive emulation functions are initiated and controlled by special values in constant field statements. These special values begin with the characters _* and end with the characters *_. The underscore plus asterisk pattern is unlikely to occur in normal data strings. The asterisk character (*), which is represented as X 5C in hex, and the underscore character (_), which is represented as X 6D in hex, are not changed by code page shifts in multiple language environments. An example would be the field statement FIELD2=X 6D5CE2C5C7C4C1E3C55C6D, where the hexadecimal characters shown are equal to _*SEGDATE*_ in EBCDIC. However, using this pattern requires that data strings which begin with the characters _* are not allowed as data values for constant fields. Trigger 1 Migration of a report definition to the Common Server environment, in most cases, adds a standard definition of TRIGGER1 to the indexer parameters: TRIGGER1=*,1,X F1,(TYPE=GROUP) This definition can then be used as the base trigger (reference point) for data fields which had been located in Spool File Archive by referring to an absolute line number. 6/7/2013 Page 77

78 Reference strings Migration of a report definition to the Common Server environment creates TRIGGER definitions for any reference strings that exist in the original report definition. Such a reference string might exist for the posting date specification (1 reference string), for either or both of the segmentation criteria (1 or 2 reference strings), and for any of the key values (up to 5 reference strings). This means that the theoretical maximum number of triggers required for the new application is nine, since migration also generates the TRIGGER1 statement described previously. However, in the Common Server environment, the maximum number of triggers is eight. Actually reaching the theoretical requirement for nine triggers seems very unlikely. To manage this situation, the default definition of TRIGGER1 is eliminated if all of the keys, both segmentation values, and the posting date all use reference strings. In this case, the TRIGGER1 base for an absolute line number reference in Spool File Archive is not needed. This is a condition which is checked during migration. If the existence of all eight reference strings in the report definition is detected, the first segmentation reference string is used as the alternative TRIGGER1. Indexes All indexes are migrated with BREAK=NO. That means that a change in the index value does not cause a new document to be created. A new document is created only when the segmentation criteria are met. Graphical indexer The graphical indexer that is part of the OnDemand Administrator client does not support the specification of Spool File Archive emulation functions. These functions must be entered manually into the indexer parameters of the application definition. 6/7/2013 Page 78

79 How DOC reports are migrated Overview Spool File Archive report definitions with report type of DOC can have up to 5 keys. All key values in a DOC report are extracted from the report data. Each key defined in the Spool File Archive report definition is migrated to one FIELD and one INDEX statement in the Common Server application definition s indexer parameters. Each reference string, used to locate a key, is migrated to a TRIGGER statement in the application indexer parameters. Example Key 1 migrates to FIELD1 and INDEX1. The key 1 reference string, if used, migrates to TRIGGER2. Key 2 migrates to FIELD2 and INDEX2. The key 2 reference string, if used, migrates to TRIGGER3. Key 3 migrates to FIELD3 and INDEX3. The key 3 reference string, if used, migrates to TRIGGER4. Key 4 migrates to FIELD4 and INDEX4. The key 4 reference string, if used, migrates to TRIGGER5. Key 5 migrates to FIELD5 and INDEX5. The key 5 reference string, if used, migrates to TRIGGER6. This example shows how the key specifications from a Spool File Archive report definition are migrated to the application s indexer parameters in Common Server. For an example report definition we use CHECKSTMTS, which is a sample report definition shipped with OnDemand. Spool File Archive report definition The CHECKSTMTS report definition has 4 keys, each of which is located using a reference string. The definition of each key is shown below. Key # Key Name Key Length Starting Column Reference String Reference String Length Starting Column 1 Account Number 9 88 ACCOUNT SSN / Tax ID CONTENTS Cust Name ACCOUNT Ending Balance ENDING BAL Common Server application s indexer parameters Key Offset from Reference String After migration to Common Server, the CHECKSTMTS application s indexer parameters are as shown below. Some standard and optional indexing statements are omitted from this example to focus better on the migrated keys. 6/7/2013 Page 79

80 TRIGGER1=*,1,X'F1',(TYPE=GROUP) /* 1 */ TRIGGER2=*,72,X'C1C3C3D6E4D5E3',(TYPE=FLOAT) /* ACCOUNT */ TRIGGER3=*,2,X'C3D6D5E3C5D5E3E2',(TYPE=FLOAT) /* CONTENTS */ TRIGGER4=*,3,X'C1C3C3D6E4D5E3',(TYPE=FLOAT) /* ACCOUNT */ TRIGGER5=*,18,X'C5D5C4C9D5C740C2C1D3',(TYPE=FLOAT) /* ENDING BAL */ FIELD1=-1,88,9,(TRIGGER=2,BASE=0) FIELD2=1,86,11,(TRIGGER=3,BASE=0) FIELD3=0,18,12,(TRIGGER=4,BASE=0) FIELD4=0,87,15,(TRIGGER=5,BASE=0) INDEX1=X'C A495A3D5A ',FIELD1,(TYPE=GROUP,BREAK=NO) /* AccountNumber */ INDEX2=X'E2E2D5E381A7C9C4',FIELD2,(TYPE=GROUP,BREAK=NO) /* SSNTaxID */ INDEX3=X'C A495A3D ',FIELD3,(TYPE=GROUP,BREAK=NO) /* AccountName */ INDEX4=X'C C ',FIELD4,(TYPE=GROUP,BREAK=NO) /* EndingBalance */ DOC reports that span segments What about the use of key 5 on DOC reports that span multiple segments? In Spool File Archive, a DOC report greater than the 100-page maximum automatically displays a key 5 value that identifies each sequential 100-page segment for each of the 100-page segments that have been archived. For example, a 150-page DOC report has two entries to select from, the first one being a 100-page segment with a key 5 value of #0001 and the second one being a 50-page segment with a key 5 value of #0002. To quote from a FAQ titled: DOC report that exceeds 100-page segments -- display segment # If there are more than 100 pages per logical segment in a DOC report (or more pages than the segment size specified in the report definition), the report will be divided into 100-page physical segments and will all show up with multiple entries on the hit list. If you create a key 5 for this report, the key 5 value will be overlaid with the segment number (#0001, #0002, etc.). If you don't need a Key 5 but would like to see the segment number displayed on the hit list, just create a key 5 with the heading "Segment #", length 5, and minimum search characters 0. This feature is NOT migrated to Common Server. You must change to large object support or change the segment size to greater than 100 pages in the Common Server application definition. 6/7/2013 Page 80

81 How PAGE reports are migrated Overview Spool File Archive reports with report type of PAGE have three standard keys. Key 1 is the starting value of the user-defined key. Key 2 is the ending value of that same key field within the report segment. Together, Key 1 and Key 2 define a range of values within which a particular key value might be contained. Key 3 is the starting page number for the segment. Key 4 and key 5 are optional. The range values, keys 1 and 2, are migrated as a standard grouprange INDEX1. The page number, key 3, is migrated as a second grouprange INDEX2. The INDEXSTYLE=PAGE statement designates INDEX2 as a grouprange index based on generated page numbers. Migration of a PAGE report from Spool File Archive creates FIELD6 and INDEX6 statements for the report posting date, and FIELD7 and INDEX7 statements for the report sequence number. Migration of a PAGE report creates an application with trigger, field, and index statements similar to the following example. Each of these indexer statements is described in greater detail below. Some standard and optional indexing statements are omitted from this example to focus better on the emulated functions. INDEXSTYLE=PAGE TRIGGER1=*,1,X F1,(TYPE=GROUP) FIELD1=*,*,7,(OFFSET=(6:12),MASK= =======,ORDER=BYROW) FIELD6=X 6D5CD9E4D5C4C1E3C55C6D /* _*RUNDATE*_ */ FIELD7=X 6D5CE2C5D8D5E4D45C6D /* _*SEQNUM*_ */ INDEX1=PrimaryRange,FIELD1,(TYPE=GROUPRANGE,BREAK=NO) /* PrimaryRange */ INDEX2=PageNumber,FIELD1,(TYPE=GROUPRANGE,BREAK=NO) /* PageNumber */ INDEX6=PostingDate,FIELD6,(TYPE=GROUP,BREAK=NO) /* PostingDate */ INDEX7=SequenceNumber,FIELD7,(TYPE=GROUP,BREAK=NO) /* SequenceNumber */ GROUPMAXPAGES=100 Page number as an index value Migration of a PAGE report from the Spool File Archive environment creates field and index statements based on Key 3 = page number in the new application in the Common Server environment. Logical design Current Spool File Archive function Reports with report type of PAGE have a key 3 which is the page number of the first page in each segment. This page number value is generated by OnDemand simply by counting the pages as the report is archived. Because page number is always numeric, key 3 is migrated to an integer field. 6/7/2013 Page 81

82 Numeric key values Spool File Archive stores numeric page report key values with leading zeros. In our example, the PAGE report is sequenced on account number, the first account number in the segment is 11 and the last account number in the segment is The key 1 value is stored as and the key 2 value as Migration detects the leading zeros and uses this information to create an integer field to contain the indexes. Migration scans the index values for key 1 and key 2 to verify that they are both numeric. If even one key value is alphabetic, then key 1 and key 2 are migrated to a string field. ACTIVITY REGISTER ACCT.NO. NAME ACTIVITY DATE BALANCE 11 ABBOTT, BENJAMIN 18OCT08 1, GUTIERREZ, SEBASTIAN 17OCT08 2, BRYANT, DARRELL 18OCT08 1, Decimal key values Spool File Archive also stores page report key values having embedded decimal points with leading zeros. In our example, the PAGE report is sequenced on dollar amount. The first amount in the segment is 3.00 and the last amount in the segment is The key 1 value is stored as and the key 2 value as Migration detects the leading zeros and uses this information to create a decimal field to contain the indexes. Migration scans the index values for key 1 and key 2 to verify that they are both numeric and to count the number of decimal positions. If even one key value is alphabetic, then key 1 and key 2 are migrated to a string field. DOLLAR SEQUENCE REPORT T/R ACCOUNT ITEM SEQ. AMOUNT NUMBER NUMBER NUMBER P P P B N N Indexer parameters Output from the migration tool; input to the Common Server OS/400 Indexer Migration of a PAGE report creates field and index statements based on Spool File Archive Key 3 = page number. 6/7/2013 Page 82

83 INDEXSTYLE=PAGE FIELD1=*,*,7,(OFFSET=(6:12),MASK= =======,ORDER=BYROW) INDEX1=PrimaryRange,FIELD1,(TYPE=GROUPRANGE,BREAK=NO) /* PrimaryRange */ INDEX2=PageNumber,FIELD1,(TYPE=GROUPRANGE,BREAK=NO) /* PageNumber */ The FIELD1 and INDEX1 statements shown above actually illustrate a definition of the primary range index for the PAGE report. They are shown in this example only because the INDEX2 statement must also refer to FIELD1, a reference made only for syntactical consistency. Function changes The FIELD and INDEX statements shown in the example above conform to the standard syntax rules for indexing statements. But some specialized processing is required for some of these combinations. When a group break occurs, the values for a grouprange INDEX are transferred to the appropriate elements in the found indexes arrays. Usually, these values are the first and last values extracted from the report text for the transaction field specified in the grouprange INDEX statement. However, for a grouprange INDEX2 in conjunction with INDEXSTYLE=PAGE, the transaction field referenced in the INDEX statement is not used. Instead, the first and last page number values within the group are used for the low and high values for that grouprange index. Mandatory action after migration IMPORTANT: Spool File Archive reports with report type of PAGE migrate successfully but must be modified before use in Common Server. The Common Server application definitions that result from PAGE report migration contain a mask that identifies which data on the print page should be used as index data. For example, a mask of ####.## would cause the indexer to select a key value only if the data in the field (from left to right) contains four numeric characters, followed by a decimal point, followed by two numeric characters. For additional information on mask characters, see the OnDemand Administrator help text. Because the migration program has no way of knowing the nature or format of the data that is to be used for indexing, it creates a very generic mask that allows ANY data to be considered valid index data. You need to update the resulting application definition for each migrated Spool File Archive PAGE report to specify a mask that more appropriately describes the data, or else the header lines of the report might be picked up as index values, making the index values meaningless and making retrieval of the data unpredictable. You might also consider using the STARTTRANSACTIONFIELDSONLINE indexer parameter to further define the first line on which your sorted data begins. The OnDemand for i Advanced Indexer Reference includes more information about this parameter. You might also want to pay attention to the Load Information tab in the application definition. The specifications to strip leading, embedded, and trailing characters are defined on this tab, and can be a significant benefit, especially if you had written index exit programs in Spool File Archive to accomplish the same function. 6/7/2013 Page 83

84 PAGE reports with no Key 2 explicitly defined Before the graphical report definition capabilities were available with the OnDemand Archive plugin of iseries Navigator, you were only able to create Spool File Archive report definitions using a 5250 (green screen) interface. The 5250 interface allowed you to create a PAGE type report definition and leave Key 2 blank. OnDemand would index the reports using the same set of values for Key 2 that were defined for Key 1. Later, when the OnDemand Archive plugin of iseries Navigator became available, if Key 2 was left blank when creating a PAGE report definition, the plugin would automatically populate the Key 2 parameters with the same values used to define Key 1. Because of this, you might have PAGE report definitions in Spool File Archive that were created using the older 5250 interface that have no Key 2 explicitly defined. If Key 2 is blank when migrating PAGE report definitions to Common Server, this will cause the new Common Server application definition resulting from migration to have a second set of offset numbers for Field 1 in its indexer parameters. For example, after migration, the indexer parameters for Field 1 might look like the following: FIELD1=*,*,5,(OFFSET=(2:6, 4:2),MASK='=====',ORDER=BYROW) The second set off offset numbers for Field 1 (the "4:2" in this example) might cause the loading of new data into the migrated Common Server application to fail with a message similar to the following: Message.... : Message 'MCH0603' in program object 'QRLMIDX' in library 'QRDARS' (C D F G). Cause..... : Message 'MCH0603' was detected in COBOL statement 1850 of COBOL program 'QRLMIDX' in program object 'QRLMIDX' in library 'QRDARS'. In order to successfully archive data into the newly migrated application, the second set of offset numbers must be removed from the indexer parameters of the application definition without a numeric suffix. For example, Field 1 needs to be changed using the OnDemand Administrator client to remove the second set of offsets numbers in the Common Server application definition to look similar to the following (using our example from above). Note that your indexer parameters will be different, depending on the layout of your report. Simply remove the second set of offset numbers so that there is only one range included inside the parentheses as shown below. Leave the first set of offset numbers unchanged. FIELD1=*,*,5,(OFFSET=(2:6),MASK='=====',ORDER=BYROW) No changes are required for Spool File Archive PAGE report definitions created using the OnDemand Archive plugin of iseries Navigator or created using the 5250 interface with Key 2 explicitly defined. 6/7/2013 Page 84

85 How NODX reports are migrated Overview Spool File Archive reports with report type of NODX have three keys. None of these keys have values which are extracted from the report data itself; their values are generated as the report is archived. Key 1 is the segment number within the report. It is migrated as INDEX1. Key 2 is the segment date (this is the same date as the posting date). It is migrated as INDEX2. Key 3 is the starting page number for the segment. It is migrated as a grouprange INDEX3. The INDEXSTYLE=NODX statement designates INDEX3 as a grouprange index based on generated page numbers. The special value _*PAGE*_ in the MASK designates FIELD3 as a dummy transaction field, preventing normal transaction field processing for this field. Migration of any report from Spool File Archive creates FIELD7 and INDEX7 statements for the report sequence number. Migration of a NODX report creates an application with trigger, field and index statements similar to the following example. Each of these indexer statements is described in greater detail below. Some standard and optional indexing statements are omitted from this example to better focus on the emulated functions. INDEXSTYLE=NODX TRIGGER1=*,1,X F1,(TYPE=GROUP) FIELD1=X 6D5CE2C5C7D5E4D45C6D /* _*SEGNUM*_ */ FIELD2=X 6D5CE2C5C7C4C1E3C55C6D /* _*SEGDATE*_ */ FIELD3=*,*,8,(OFFSET=(1:8),MASK= _*PAGE*_,ORDER=BYROW) FIELD7=X 6D5CE2C5D8D5E4D45C6D /* _*SEQNUM*_ */ INDEX1=X'E A34095A ',FIELD1,(TYPE=GROUP,BREAK=NO) /* Segment number */ INDEX2=X'E A A385',FIELD2,(TYPE=GROUP,BREAK=NO) /* Segment date */ INDEX3=X'D A ',FIELD3,(TYPE=GROUPRANGE,BREAK=NO) /* Page number */ INDEX7=X'E28598A A ',FIELD7,(TYPE=GROUP,BREAK=NO) /* Sequence number */ Segment number as an index value Migration of a NODX report from the Spool File Archive environment creates field and index statements based on Spool File Archive Key 1 = segment number in the new application. 6/7/2013 Page 85

86 Current Spool File Archive function Migration Reference Reports with report type of NODX have keys which are generated by OnDemand. These keys are segment number, date, and page number. The first of these keys, segment number, is simply a sequential numbering of the segments of the report. Indexer parameters Migration of a NODX report creates field and index statements based on Key 1 = segment number. For example: FIELD1=X 6D5CE2C5C7D5E4D45C6D /* _*SEGNUM*_ */ INDEX1=X'E A34095A ',FIELD1,(TYPE=GROUP,BREAK=NO) /* Segment number */ In these examples, a descriptive index name is used for clarity. The migration actually inserts the corresponding data base field name, expressed in hexadecimal because the name is in EBCDIC. Function changes FIELD1 in the example is a constant field. The constant value _*SEGNUM*_ is not transferred as ordinary constant data would be. When the indexer finds this special value for a field, it places the current group count value into the corresponding array element as the value of FIELD1. A length value of 11 bytes is used. Segment date as an index value Migration of a NODX report from the Spool File Archive environment creates field and index statements based on Spool File Archive Key 2 = segment date for the new application. Current Spool File Archive function Reports with report type of NODX ( no index ) have keys which are generated by OnDemand. These keys are segment number, segment date, and page number. The second of these keys, segment date, is the current job date; i.e., the date when the report is archived. Indexer parameters Migration of a NODX report creates field and index statements based on Key 2 = segment date. For example: FIELD2=X 6D5CE2C5C7C4C1E3C55C6D /* _*SEGDATE*_ */ INDEX2=X'E A A385',FIELD2,(TYPE=GROUP,BREAK=NO) /* Segment date */ Function changes FIELD2 in the example is a constant field. The constant value _*SEGDATE*_ is not transferred as ordinary constant data would be. When the indexer finds this special value for a field, it does nothing. This forces all subsequent processing steps to treat 6/7/2013 Page 86

87 the field, and thus INDEX2, as if the field value could not be found. When the indexer output is created with the index name but without an index value, the report load process uses the default value t, which is the current job date. Page number as an index value Migration of a NODX report from the Spool File Archive environment creates field and index statements based on Spool File Archive Key 3 = page number in the new application in the Common Server environment. Current Spool File Archive function Reports with report type = NODX have a key 3 which is the page number of the first page in each segment. This page number value is generated by OnDemand simply by counting the pages as the report is archived. Indexer parameters Migration of a NODX report creates field and index statements based on Key 3 = page number. For example: INDEXSTYLE=NODX FIELD3=*,*,8,(OFFSET=(1:8),MASK= _*PAGE*_,ORDER=BYROW) INDEX3=X'D A ',FIELD3,(TYPE=GROUPRANGE,BREAK=NO) /* Page number */ Function changes The FIELD and INDEX statements shown in the examples above conform to the standard syntax rules for indexing statements. But some specialized processing is required for some of these combinations. FIELD3 (in the NODX example above) is defined as a transaction field associated with the grouprange INDEX3, but it is present only to meet the syntax requirements for INDEX3. When the indexer finds the special MASK value _*PAGE*_ for a transaction field it does nothing. When a group break occurs, the values for a grouprange INDEX are transferred to the appropriate elements in the arrays of indexes found. Usually, these values are the first and last values extracted from the report text for the transaction field specified in the grouprange INDEX statement. For a grouprange INDEX3 in conjunction with INDEXSTYLE=NODX, the transaction field referenced in the INDEX statement is not used. Instead, the first and last page number values within the group are used for the low and high values for that grouprange index. 6/7/2013 Page 87

88 How segmentation conditions are migrated Overview Migration of a report from Spool File Archive which includes entries for segmentation condition 1 creates indexer parameter statements for FIELD29 and FIELD30 as described in indexer parameters below. If the migrated report from Spool File Archive also includes entries for segmentation condition 2, the migration process creates indexer parameter statements for FIELD31 and FIELD32. Current Spool File Archive function Report segmentation specifications in reports defined in Spool File Archive control how the report is divided into manageable pieces or groups. The most basic approach to this is specifying the maximum number of pages in each segment. This segment size control has a direct equivalent in the OS/400 indexer, the GROUPMAXPAGES indexing parameter, so no emulation of this function is required. The usual method in Spool File Archive of segmenting a report based on either the matching of a report data value or detecting a change in such a value has no direct equivalent in the Common Server environment. The Common Server OS/400 indexer performs group (segment) control based on changes in one or more index values. Such an index value might or might not be the same data element used for segmentation by Spool File Archive. Emulation of these Spool File Archive segmentation functions has been added to the Common Server OS/400 indexer. Group control based simply on matching a value in the data Using two such segmentation criteria in an and or or relationship Indexer parameters Migration of a report from Spool File Archive which includes entries for segmentation condition 1 creates indexer parameter statements for FIELD29 and FIELD30. FIELD29 contains the change or match specification. There are two possibilities: FIELD29=X 6D5CC3C8C1D5C7C55C6D /* _*CHANGE*_ */ FIELD29=X 6D5CD4C1E3C3C85C6DA7A7A7...A7 /* _*MATCH*_xxx...x */ where xxx...x is the data value to be matched. FIELD30 is a trigger-based field specifying the location of the data to be examined. The trigger referenced is either TRIGGER1, with the field displacement equal to the record number based on record 1, or it is a trigger based on the reference string for the segmentation condition 1 from the original Spool File Archive report definition. If the migrated report from Spool File Archive also includes entries for segmentation condition 2, the migration process creates indexer parameter statements for FIELD31 6/7/2013 Page 88

89 and FIELD32. FIELD31 specifies the logical relationship and the change or match specification. There are four possible combinations: FIELD31=X 6D5CC1D5C4C3C8C1D5C7C55C6D /* _*ANDCHANGE*_ */ FIELD31=X 6D5CD6D9C3C8C1D5C7C55C6D /* _*ORCHANGE*_ */ FIELD31=X 6D5CC1D5C4D4C1E3C3C85C6DA7A7A7...A7 /* _*ANDMATCH*_xxx...x */ FIELD31=X 6D5CD6D9D4C1E3C3C85C6DA7A7A7...A7 /* _*ORMATCH*_xxx...x */ where xxx...x is the data value to be matched. FIELD32 is a trigger-based field specifying the location of the data to be examined. The trigger referenced is either TRIGGER1, with the field displacement equal to the record number based on record 1, or a trigger based on the pivot data for the segmentation condition 2 from the original Spool File Archive report definition. Function changes FIELD29 and FIELD31, as in the examples above, are constant fields. 1. During processing of field statements, the constant value in the field statement is not transferred as ordinary constant data would be. 2. When the indexer finds the special emulation control value starting with the characters _* for either of these fields it does nothing related to that particular field during fields processing. 3. FIELD30 and FIELD32 are processed normally, finding and extracting the data value(s) from the location(s) specified and storing the value(s), and so forth, in the field value array and related arrays in the normal manner. (These fields are not normally used in any standard INDEX statement.) During group break processing (after processing triggers, fields, and indexes): 1. The indexer program first searches for a group break based on a change in value for any index with TYPE=GROUP and BREAK=YES. 2. If such a group break is not detected, then the contents of FIELD29 and FIELD31, if defined, are used to determine if a group break should occur. 3. If the specified match or change in values has occurred, group break processing is initiated and handled exactly as if a change in index values had occurred. This segmentation emulation processing requires retaining the prior values when change testing is specified, and extraction, retention, and comparison of the match value(s) from the constant string(s) in FIELD29 and FIELD31. The order of tests for a group break is: 1. Checking for any change in index values for indexes with TYPE=GROUP and BREAK=YES 2. Checking the conditions specified for segmentation emulation 3. Comparing the number of pages in the current group with the GROUPMAXPAGES value, if specified 6/7/2013 Page 89

90 How posting date is migrated Migration of any report definition from Spool File Archive creates one or more field statements which allow access to a posting date, as described below. Current Spool File Archive function All reports archived by Spool File Archive have a report date or posting date which is either a date extracted from the print page of the spooled file data or the actual date on which the report was archived. The posting date is effectively an additional key field though it is not designated as such in Spool File Archive. Indexer parameters Migration of any report definition from Spool File Archive creates one or more field statements which allow access to a posting date. Example 1: If the posting date is not extracted from the spooled file data, then FIELD6 is a constant field and INDEX6 contains the current date. FIELD6=X'6D5CD9E4D5C4C1E3C55C6D' /* _*RUNDATE*_ */ INDEX6=X'C481A385',FIELD6,(TYPE=GROUP,BREAK=NO) /* Date */ Example 2: If the posting date is extracted from the spooled file data, both FIELD6 and FIELD28 are created in the indexer parameters. FIELD6 is a trigger-based field specifying the location of the date. The trigger referenced can be either TRIGGER1, with the field displacement equal to the record number based on record 1, or it can be a trigger based on the reference string for the posting date from the original Spool File Archive report definition. Effective with PTF SI40481, the SFA posting date field is now migrated using the Common Server indexer parameter DEFAULT='_*FINDONCE*_'. This means that, in Common Server, the date is now located only in the first document. The date located in that document is now carried forward and used in all other documents found within that particular input file. FIELD28 is a constant field specifying the Spool File Archive date format parameter. INDEX6 contains the date found in the report. TRIGGER4=*,67,X'C4C1E4C5',(TYPE=FLOAT) /* DATE */ FIELD6=0,73,8,(TRIGGER=4,BASE=0,DEFAULT='_*FINDONCE*_') FIELD28=X'6D5CC4C1E3C5C6D4E35C6DF140' /* _*DATEFMT*_1 */ INDEX6=X'D796A2A C481A385',FIELD6,(TYPE=GROUP,BREAK=NO) /* PostingDate */ Function changes 1. FIELD6, as in Example 1 above, is a constant field. The constant value _*RUNDATE*_ is not transferred as ordinary constant data would be. When the indexer finds this special value for a field it does nothing. This forces all subsequent 6/7/2013 Page 90

91 processing steps to treat the field, and thus INDEX6 as well, as if the field value can not be found. When the indexer output is created with the index name but without an index value, the report load process uses the job date. 2. FIELD6, as in Example 2 above, is a trigger-based field. This trigger, field, and index are processed normally by the Common Server OS/400 indexer, with a single exception. If FIELD28 exists in the indexer parameters, the extracted date value is converted from the date format specified in FIELD28 to YYYY-MM-DD before storing the date as the value of FIELD6. This date conversion uses the existing date format conversion program from Spool File Archive. Important Note on Posting Date Migration After the migration of a report definition from Spool File Archive (SFA) to Common Server, the format of the posting date in the Application > Load Information is always %Y-%m-%d. This date format should not be changed. Migrated applications use a SFA date routine to build the posting date. The output of that SFA date routine is always in the format of %Y-%m-%d. The Application > Load Information > Date Format does not have to match (and probably will not match) the format of the date in the spooled file. 6/7/2013 Page 91

92 How sequence number is migrated Migration of any report from the Spool File Archive environment creates FIELD7 and INDEX7 statements for the report sequence number in the indexer parameters for the new application. Current Spool File Archive function All reports archived by Spool File Archive have a 3-digit report sequence number generated by OnDemand. This allows a user to know, for reports archived with the same posting date, the order in which these reports were archived. Indexer parameters Migration of any report from Spool File Archive creates FIELD7 and INDEX7 statements for the report sequence number. For example: FIELD7=X 6D5CE2C5D8D5E4D45C6D /* _*SEQNUM*_ */ INDEX7=X'E28598A A ',FIELD7,(TYPE=GROUP,BREAK=NO) /* Sequence number */ Function changes FIELD7 in the example is a constant field. The constant value _*SEQNUM*_ is not transferred as ordinary constant data would be. When the indexer finds this special value for a field, it places a unique report sequence number value (a timestamp) as the value of FIELD7. The sequence number field uses a timestamp to indicate when a particular copy of the report was actually archived. All documents within a particular input file receive the same sequence number. 6/7/2013 Page 92

93 How translate print control is migrated Details For the Common Server OS/400 indexer there is a new indexer parameter. TRANSLATEPRINTCONTROL=NO YES If TRANSLATEPRINTCONTROL is not specified, a value of No is assumed. General notes The need for translate print control arose from the way that data was archived in the Spool File Archive server. To be indexed, all input spooled files must be first converted to line data. SCS data is converted to line data by using an operating system function that converts all line spacing to First Character Form Control (FCFC) characters. The resulting line data is sometimes very difficult to index because what you need to index can vary greatly in placement on the page from document to document. Appropriate trigger values (also known as reference strings or pivot values in Spool File Archive) were sometimes impossible to find, making correct indexing impossible. Data that seemed to be variable to the indexer always printed on the same line on the printed page, so why could it not be indexed? Translate print control was created to enable the indexing of reports in these circumstances. When translate print control is set to Yes, the Spool File Archive indexer takes the input line data for a page and, rather than process it in the order that it came in, it reads the page buffer and creates a secondary view of the page. This secondary view of the page has each line of data placed as it would be on the printed page. See Example translated page buffer on page 94 for an example of translate print control in action. The TRANSLATEPRINTCONTROL function is primarily for reports that were migrated from Spool File Archive to Common Server. There is nothing to stop you from using this approach on new applications. However, there is no support in the graphical indexer for Translate Print Control. This means that if you want to use the TRANSLATEPRINTCONTROL indexer parameter you must enter it manually using the OnDemand Administrator. Important Note on Translate Print Control After migration from SFA to Common Server, reports that specify translate print control index and store correctly, but do not display correctly in the graphical indexer. No changes are necessary, but you should be aware of this limitation if you are updating migrated applications that use translate print control. The graphical indexer used to display sample spooled file data for Common Server applications does not support the translate print control indexer parameter. If specified in the application indexer parameters, translate print control is ignored by the graphical indexer. If you need to create a new version for the report or modify the current version; you must either use the keyboard to make your changes or use the graphical indexer to create indexer parameters that do not use Translate Print Control. 6/7/2013 Page 93

94 You can use the following SQL statement to determine which SFA report definitions use translate print control: SELECT CDTYPE, TRANSPRTCC FROM QUSRRDARS/QARLRACT WHERE TRANSPRTCC = 'Y' The output will look similar to that shown in the figure below. Report Type EDFACT HCFAOVL Translate Print Control Y Y After migration, you can use the following SQL statement to determine which applications use translate print control (replace QUSROND with the name of your instance): SELECT SUBSTR(NAME,1,13) AS APPLICATION, INDEXER FROM WHERE INDEXER LIKE '%TRANSLATEPRINTCONTROL=YES%' QUSROND/ARSAPP The output will look similar to that shown in the figure below. APPLICATION EDFACT-01 EDFACT HCFAOVL-01 HCFAOVL Example translated page buffer INDEXER CCTYPE=A CONVERT=NO... TRANSLATEPRINTCONTROL=YES CCTYPE=A CONVERT=NO... TRANSLATEPRINTCONTROL=YES CCTYPE=A CONVERT=NO... TRANSLATEPRINTCONTROL=YES CCTYPE=A CONVERT=NO... TRANSLATEPRINTCONTROL=YES Below is a short example of how the raw and translated page buffers appear. The first column of the data field is the FCFC character. Raw Page Buffer Line Data Bill Smith First St. 5 0 Somewhere, NC Line Data John Smith Second Ave. 5 Appt 5C 6 0 Somewhere, NC Translated Page Buffer Line Data Bill Smith First St Somewhere, NC Line Data John Smith Second Ave. 6 Appt 5C 7 0 Somewhere, NC /7/2013 Page 94

95 How SCS reports with an AFP overlay are migrated The following information applies to Spool File Archive report definitions that have a data type of *OTHER (input spooled file data type of *SCS) and also include an AFP overlay specified in their associated printer file. This information does not apply to Spool File Archive reports that have a data type of *AFPDS, because the overlays for *AFPDS files are handled the same as any other AFP resource within the AFPDS data stream. Note that the options described below assume that you have correctly set the "Use archived AFP overlays" setting for these report definitions before migration. See the section entitled Reports that require attention before they are migrated for more information. Options for handling SCS spooled files that have AFP overlays Change device type The preferred method of handling SCS spooled files that have AFP overlays is to change the device type (DEVTYPE) parameter of the printer file used to create the spooled file to *AFPDS. This causes the operating system to put the data into spool as *AFPDS. It is the most efficient way for OnDemand to capture (load) this type of spooled data. However, making this change requires the generated spooled file be printed on an AFPDS printer. If you really are printing it with an AFP overlay this should not be a problem, but if you are printing it on a line printer with preprinted forms this does not work. OnDemand conversion If it is not possible to change the printer device type to *AFPDS, OnDemand can do the conversion to AFP, thus allowing the spooled file to be viewed and printed with fidelity. This method is more resource intensive than by using the operating system to do the conversion using the printer device type parameter of the printer file (as described above). To enable this conversion, specify the data type in the Common Server application definition as AFP rather than SCS. When OnDemand encounters a *SCS spooled file that also has an AFP overlay and the application specifies a data type of AFP, OnDemand converts the SCS data to AFPDS and stores the newly created spooled file. Caution Do not specify data type AFPDS for any other type of non-afp spooled file. Background Spool File Archive history The term SCS with overlays describes iseries spooled files that contain *SCS data and use an AFP overlay when printed. When SCS with overlay documents are printed, the overlay is applied by the system, the data is converted to AFP, and the document must be printed on an AFP-capable printer. Printing these documents is never a problem; however, viewing the spooled file, including the overlay, from within OnDemand presents a problem. The only way the overlay can be viewed is if the data type is AFP. 6/7/2013 Page 95

96 A method was developed in Spool File Archive that converts the data from its stored format to AFP at retrieval time. Spool File Archive signals the OnDemand client that the data is AFP; and then the document, including the overlay, can be viewed successfully. This process depends on information that is stored in Spool File Archive files to correctly handle the conversion. Migrated reports For migrated reports, a slightly modified version of the process works. However, for new Common Server reports, the needed information is not there nor is it available at retrieval time. All the needed information is there at load time because the attributes of the spooled file contain all of the required information. If the OnDemand application definition has a data type of AFPDS, the attributes of the spooled file are *SCS, and an AFP overlay is specified in the spooled file attributes, then Common Server converts the input data to AFP. This conversion is done using a modified version of the process used at retrieval time in Spool File Archive. The conversion process If the OnDemand application definition s data type is AFPDS, Common Server checks the attributes of the spooled file to see if it is really AFP or if it is SCS with an AFP overlay. If it is SCS with an AFP overlay, then the original spooled file is converted to AFPDS by creating a secondary spooled file with the correct data type. This secondary spooled file is what is actually stored. The secondary spooled file is deleted after it is processed. 6/7/2013 Page 96

97 How key security is migrated During the migration of Spool File Archive report definitions, Spool File Archive key security files are "transcribed" into an SQL statement to be placed into the corresponding Common Server application group s Query Restriction field. The user running the migration is considered the administrator of the Spool File Archive report definition and is given all authority / permissions to the migrated reports. Therefore, no key security information for that user is migrated and no query restriction is created for that user in the Common Server application group. To ensure that all Spool File Archive key security data is "transcribed" correctly into the corresponding Common Server application group's Query Restriction field, a new OS/400 and OnDemand System Administrator user (one that is not specified in the key security files) should be created. This user should be used to run the migration program calls. See Creating an OnDemand Administrator user to run the migration program calls for more details on this topic. The reason this needs to be done before you even migrate your other Common Server users is rather simple: You must be recognized as a valid Common Server System Administrator in order to create additional Common Server users. So remember to create this first System Administrator user manually before running the user migration step for your other users, and then continue to run the remainder of the migration steps with this same System Administrator user. Remember, too, that this OS/400 user must have *ALLOBJ special authority to run the migration program calls, as stated in the requirements for running the migration steps. After completing "Phase 2: Definition conversion step" this special Common Server user can be deleted from OnDemand (using the OnDemand Administrator Client). The OS/400 user profile can also be deleted if it is no longer needed. Additional notes 1. Key security is migrated for individual reports. If you are using Spool File Archive report groups, note that key security at the report group level is not migrated. 2. Spool File Archive reports with key security defined are not accessible when combining Spool File Archive and Common Server folders on a single OnDemand client hit list (known as Coexistence (Phase 0)). It is recommended that report definitions that use key security be some of the first definitions that you migrate if you plan to use the coexistence function. 3. See Reports with key security that have *ALL for Low and High key values in this document for information regarding a change that should be made to certain report definitions before migrating. 4. Query restrictions use the field names from the application group, not the field names from the folder. The application group field names created by migration are always in the form of F_01, F_02, F_03, and so forth. For example, if you create a query restriction in the form of OFFICE = MEDI, an error 13 is sent to the system log with the text DB Error: Column OFFICE not in specified tables. To correct this error you should specify the query restriction as F_03 = MEDI. 6/7/2013 Page 97

98 How application groups are created Overview Migration of report definitions from the SFA environment creates certain fields in the application group. The particular fields created are based on the type of report definition. The following tables list those fields. All migrated report definitions have the following fields Application Group Field Name Application Load ID Name Field Type Data Type Length F_00 Application Identifier (version) Filter String 2 F_01 Report Date Index Date - F_02 Sequence Number Filter String 16 DOC or ANYS migrated definitions have the following fields Application Group Field Name Application Load ID Name Field Type Data Type Length F_03 SFA key 1 Index String (Note 1) F_04 SFA key 2 (optional) Index String (Note 1) F_05 SFA key 3 (optional) Index String (Note 1) F_06 SFA key 4 (optional) Filter String (Note 1) F_07 SFA key 5 (optional) Filter String (Note 1) PAGE migrated definitions have the following fields Application Group Field Name F_03 F_04 F_05 Application Load ID Name SFA key 1 - group range field Low value for segment SFA key 2 - group range field High value for segment SFA key 3 - group range field Starting page number for segment Field Type Index Index Data Type String (Note 2) String (Note 2) Index Integer - Length (Note 1) (Note 1) F_06 SFA key 4 (optional) Filter String (Note 1) F_07 SFA key 5 (optional) Filter String (Note 1) F_08 SFA key 3 - group range field Ending page number for segment Index Integer - NODX migrated definitions have the following fields Application Group Field Name Application Load ID Name Field Type Data Type Length F_03 SFA key 1 - segment number Index Integer - F_04 SFA key 2 - segment date Index Date - From the object name ( ONAME ) F_05 SFA key 3 - group range field Starting page number for segment Index Integer - F_06 SFA key 4 (optional) Filter String (Note 1) 6/7/2013 Page 98

99 F_07 SFA key 5 (optional) Filter String (Note 1) F_08 SFA key 3 - group range field Ending page number for segment Index Integer - Note 1: Field length is determined by the maximum length of index data stored in SFA Note 2: See How PAGE reports are migrated on page 81 for information about when PAGE report fields are migrated as Decimal or Integer. If you do not want to use these field names you should pick a different migration strategy, for example re-spooling your reports, creating new definitions in Common Server, and then re-archiving your reports. 6/7/2013 Page 99

100 Part 3: Appendices 6/7/2013 Page 100

101 Appendix A: Reading reports from Report Definition migration Overview This appendix explains the reports produced by the analyze and migrate functions of the migration process from Spool File Archive report definitions to Common Server applications, application groups, and folders. How to read the Migrate Report Definitions report for *ANZDFN The analyze definitions run type (*ANZDFN) creates a spooled file, QPRLRMGRD, that contains many sections. There is one section that shows the parameters used to run the program, one section for each report definition, one section for each report group, and one section containing a summary. Program Parameters section This section of the report lists the migration run type (analyze definitions in this example), the Common Server instance into which the reports are being migrated (QUSROND), and the report definitions to be included in this run of the migration program (all report groups and all report definitions) RD1 V5R4M OnDemand for eserver i5 Migrate Report Definitions Part 1 - Create Common Server Definitions Report name : *ALL Instance : QUSROND Run type : *ANZDFN 6/7/2013 Page 101

102 Report Definition section This section of the report lists details about every report definition included in this migration run. Each report definition is listed on a separate page RD1 V5R4M OnDemand for eserver i5 Migrate Report Definitions Part 1 - Create Common Server Definitions Report name : ANYSLWP No errors or warnings detected The following field names have been assigned: Folder Application Identifier Date Sequence number File Name Created By Location Application Definitions ANYSLWP-01 ANYSLWP Application Group Definitions ANYSLWP Folder Definitions ANYSLWP Application Group F_00 F_01 F_02 F_03 F_04 F_06 6/7/2013 Page 102

103 Report Group section This section is present only if a report group and its member report definitions are being migrated. The report group section of the report contains a list of the application groups that are included in the folder that is created to represent the report group. There is also a listing of the mapping of fields from the application group to the folder RD1 V5R4M OnDemand for eserver i5 Migrate Report Definitions Part 1 - Create Common Server Definitions Report Group name : ANYSPDFGRP Application groups in folder ANYSLWP ANYSPDF Folder Definitions ANYSPDFGRP Folder to Application Group Field Mapping: Folder fields: File Name Created By Location FileType Application Group fields: ANYSLWP File Name Created By Location FileType ANYSPDF File Name Created By Location FileType Summary section The summary section of the report contains statistics on how many report definitions were migrated, and how many Common Server definitions (application groups, applications, and folders) were created RD1 V5R4M OnDemand for eserver i5 Migrate Report Definitions Part 1 - Create Common Server Definitions Report definition migration statistics Report definitions processed : 15 Applications generated : 20 Application Groups generated : 10 Folders generated : 15 Export failures : 0 YOU MUST REVIEW THIS REPORT AND THE JOB LOG TO BE SURE THE ENTIRE PROCESS COMPLETED SUCCESSFULLY. 6/7/2013 Page 103

104 Error and warning messages Migration Reference Error and warning messages are listed in the sections of the report for individual report and group definitions. Some of the possible error and warning messages are shown in the following figures. Migrate Report Definitions Part 1 - Create Common Server Definitions Report name : RD4575 Errors and warnings: Type Version Message Warning 80 - Keys vary among versions Migrate Report Definitions Part 1 - Create Common Server Definitions Report name : RD5577 Errors and warnings: Type Version Message Error POLICY NOT FOUND - NOT MIGRATED Errors detected in one or more versions. Report not migrated. Migrate Report Definitions Part 1 - Create Common Server Definitions Report name : LATECHARGE Errors and warnings: Type Version Message Error 95 - Folder will contain duplicate key names Key 1 Key 2 Version Length Name Length Name Patient Name 20 Patient Name The following field names have been assigned: Folder Application Group Application Identifier F_00 Date F_01 Sequence number F_02 Patient Name_ F_03 Patient Name F_04 Page number_ F_05 Patient Name F_03,F_04 Page number F_05,F_08 Errors detected in one or more versions. Report not migrated. 6/7/2013 Page 104

105 Migrate Report Definitions Part 1 - Create Common Server Definitions Report name : LABOR No errors or warnings detected The following field names have been assigned: Folder Application Identifier Date Sequence number Employee Name Employee # Badge # Order # Op # Application Group F_00 F_01 F_02 F_03 F_04 F_05 F_06 F_07 Application Definitions LABOR-01 LABOR Application Group Definitions LABOR 63 - Already exists in specified instance Folder Definitions LABOR 63 - Already exists in specified instance How to read the Migrate Report Definitions report for *MGRDFN The migrate definitions run type (*MGRDFN) creates a spooled file named QPRLRMGRD. The USERDATA of the spooled file is set to the report name specified when running the *MGRDFN program. The contents of that spooled file are shown below. Program Parameter section This section of the report lists the migration run type (migrate definitions in this example), the Common Server instance into which the reports are being migrated (QUSROND), and the report definitions to be included in this run of the migration program (FUELMAN report definition). Migrate Report Definitions Part 1 - Create Common Server Definitions Report name : FUELMAN Instance : ONDMIGR3 Run type : *MGRDFN 6/7/2013 Page 105

106 Report Definition section This section of the report lists details about every report definition included in this migration run. Each report definition is listed on a separate page. Migrate Report Definitions Part 1 - Create Common Server Definitions Report name : FUELMAN No errors or warnings detected The following field names have been assigned: Folder Application Group Application Identifier Date Sequence number Customer Name Account # Card # Vehicle Desc Vehicle # F_00 F_01 F_02 F_03 F_04 F_05 F_06 F_07 Application Definitions FUELMAN-01 FUELMAN Application Group Definitions FUELMAN 6/7/2013 Page 106

107 Report Group section This section is present only if a report group and its member report definitions are being migrated. The report group section of the report contains a list of the application groups that are included in the folder that is created to represent the report group. There is also a listing of the mapping of fields from the application group to the folder. Migrate Report Definitions Part 1 - Create Common Server Definitions Report Group name : ANYSPDFGRP Application groups in folder ANYSLWP ANYSPDF Folder Definitions ANYSPDFGRP Folder to Application Group Field Mapping: Folder fields: File Name Created By Location FileType Application Group fields: ANYSLWP File Name Created By Location FileType ANYSPDF File Name Created By Location FileType Export section The export section of the report contains a list of application groups and folders that were created. Migrate Report Definitions Part 2 - Export Common Server Definitions Application Group Definitions FUELMAN 62 - Exported successfully Folder Definitions FUELMAN 62 - Exported successfully 6/7/2013 Page 107

108 Summary section The summary section of the report contains statistics on how many report definitions were migrated, and how many Common Server definitions (application groups, applications, and folders) were created. Migrate Report Definitions Part 2 - Export Common Server Definitions Report definition migration statistics Report definitions processed : 1 Applications generated : 2 Application Groups generated : 1 Folders generated : 1 Export failures : 0 YOU MUST REVIEW THIS REPORT AND THE JOB LOG TO BE SURE THE ENTIRE PROCESS COMPLETED SUCCESSFULLY. 6/7/2013 Page 108

109 Appendix B: Differences between the Spool File Archive indexer and the OS/400 indexer Blank value bypass The Common Server OS/400 indexer treats spaces differently from the Spool File Archive Indexer. The OS/400 Indexer treats spaces as a change in value. The Spool File Archive Indexer does not. This behavior can be referred to as Blank Value Bypass. To illustrate this behavior, the Flash Sales Report is defined with the segmentation condition specified based on a change in the value of the department number. This definition results in 28 segments being created when the report is loaded into OnDemand. The report definition was then migrated to Common Server. No changes were made to the migrated indexer parameters. This migrated definition results in 29 segments being created during the index and load process. Illustration of differences The figures and tables below illustrate the differences between the indexers. Figure B-1 shows the Flash Sales Report with the department number highlighted. Figure B-2 shows the Spool File Archive segmentation condition. Table B-1 shows the Common Server indexer parameters. Figure B-3 shows the first page of the new segment, which reveals that there is no department number on this page. Figure B-1: Sample Data from Flash Sales Report 6/7/2013 Page 109

110 Figure B-2: Spool File Archive segmentation condition Table B-1: Common Server Indexer Parameters from Migration CCTYPE=A CONVERT=NO FILEFORMAT=RECORD GROUPMAXPAGES=100 DOCTYPE=SCS CPGID=37 TRIGGER1=*,1,X'F1',(TYPE=GROUP) /* 1 */ FIELD1=3,13,25,(TRIGGER=1,BASE=0) FIELD6=X'6D5CD9E4D5C4C1E3C55C6D' /* _*RUNDATE*_ */ FIELD7=X'6D5CE2C5D8D5E4D45C6D' /* _*SEQNUM*_ */ FIELD29=X'6D5CC3C8C1D5C7C55C6D' /* _*CHANGE*_ */ FIELD30=3,13,5,(TRIGGER=1,BASE=0) INDEX1=X'C A A3',FIELD1,(TYPE=GROUP,BREAK=NO) /* Department */ INDEX6=X'C481A385',FIELD6,(TYPE=GROUP,BREAK=NO) /* Date */ INDEX7=X'E28598A A ',FIELD7,(TYPE=GROUP,BREAK=NO) /* Sequence number */ Figure B-3: First page of new segment Impact In some ways the Common Server result is better, at least for this report, because the summary section of the Flash Sales Report is now retrievable as a separate segment. However, the results are now different than what the end users expect to see, based on what they have seen in the past in Spool File Archive. Other reports defined in Spool File Archive that depend upon blank values NOT being treated as a change in value also produce different results after migration. You might need to run other reports through the migration process and verify that blanks are being treated correctly and that the migrated definitions produce the required results. This is yet another reason why careful verification of migrated reports is critical for a successful migration to Common Server. 6/7/2013 Page 110

111 Circumvention If the use of blank value bypass is critical for correct segmentation, a mask should be defined for the Common Server field used to determine segmentation. In the previous example, this is field 30. The recommended mask for this example would be #####, where each # represents a required numeric. The resulting indexer parameter would be: FIELD30=3,13,5,(TRIGGER=1,BASE=0,MASK= ##### ) Posting Date New Method Effective with PTF SI40481, the SFA posting date field is now migrated using the Common Server indexer parameter DEFAULT='_*FINDONCE*_'. This means that, in Common Server, the date is now located only in the first document. The date located in that document is now carried forward and used in all documents in that particular input file. If you have migrated any report definitions before the installation of PTF SI40481, the following information might be helpful. Old Method Before PTF SI40481, if the posting date (segment date in Common Server) is taken directly from the contents of the report itself, then the Spool File Archive Indexer and the Common Server OS/400 Indexer handle the posting date differently. The Spool File Archive Indexer determines the posting date one time, from the first document of the report. The Common Server OS/400 Indexer determines the segment date for each document of the report, which means that it expects to find a valid date on each individual document of the spooled file. If you need the Common Server to capture the date one time, the same way that Spool File Archive did, you can specify the _*FINDONCE*_ default value for the indexer parameter of the date field in the migrated Common Server application definition. If _*FINDONCE*_ is specified as the default for the date field, then Common Server finds the date in the first document, and uses that date for all remaining documents. An example of this indexer parameter is: FIELD6=0,39,8,(TRIGGER=7,BASE=0,DEFAULT='_*FINDONCE*_') This special default value must be entered manually for the date field using the Keyboard modify option on the Indexer Information tab of the application definition. You would modify the migrated definition that does not have a -xx suffix, such as CHECKSTMTS (rather than CHECKSTMTS-01, for example). Note that if the posting date is set to the date the report is archived, then Spool File Archive and Common Server work identically. Dates containing month and year only If the posting date (segment date in Common Server) contains only the month and year, for example, 08/2009, then Spool File Archive and Common Server handle the posting date differently. 6/7/2013 Page 111

112 In Spool File Archive, a date of the format mm/yyyy is stored as the last day of the month, for example 08/2009 is stored as 08/31/2009. In Common Server, a date of the format mm/yyyy is stored as the first day of the month, for example 08/2009 is stored as 08/01/2009. If you require that the date be stored as the last day of the month in Common Server and you are using the migrated definition, no changes are required. The migrated definition uses Spool File Archive date routines. If you require that the date be stored as the last day of the month in Common Server, and you created a new definition (for example if you re-spooled all of the reports) instead of using a migrated definition, the new definition must be changed to use Spool File Archive date routines. Illustration of differences The figures below illustrate the differences between Spool File Archive and Common Server. Start with the following physical file containing simple test data. Table B-2: Sample Data ************Beginning of data************** 1 MONTH END REPORT 08/2009 ************End of Data******************** Using the OnDemand Administrator client Report Wizard, create a set of definitions which have these indexer parameters: Table B-3: Indexer Parameters from the Graphical Indexer TRIGGER1=*,72,X'61',(TYPE=GROUP) /* / */ FIELD1=0,14,25,(TRIGGER=1,BASE=0) FIELD2=0,70,7,(TRIGGER=1,BASE=0) INDEX1=X' A ',FIELD1,(TYPE=GROUP,BREAK=YES) /* reportname */ INDEX2=X' A385',FIELD2,(TYPE=GROUP,BREAK=NO) /* medate */ Change the load information for the date field to use format %m/%y, then archive the report. The date is stored as 08/01/2009. Now create a new version of the application. Use these indexer parameters. Table B-4: Revised Indexer Parameters TRIGGER1=*,72,X'61',(TYPE=GROUP) /* / */ FIELD1=0,14,25,(TRIGGER=1,BASE=0) FIELD6=0,70,7,(TRIGGER=1,BASE=0) FIELD28=X'6D5CC4C1E3C5C6D4E35C6DF240' /* _*DATEFMT*_2 */ INDEX1=X' A ',FIELD1,(TYPE=GROUP,BREAK=YES) /* reportname */ INDEX6=X' A385',FIELD6,(TYPE=GROUP,BREAK=NO) /* medate */ Change the load information for the date field to use format %Y-%m-%d and archive the report. The date is stored as 08/31/2009. Note that the Spool File Archive date routines are supported by Common Server. However, the OnDemand Administrator client graphical indexer and wizard do not support the Spool File Archive date routines. Therefore, to use the Spool File Archive data routines, the required information must be entered manually using the Keyboard modify option on the Indexer Information tab of the application definition. 6/7/2013 Page 112

113 Appendix C: Difference between Spool File Archive and Common Server dates used for expiration In Spool File Archive, both a posting date and a run date are captured when data is archived. The posting date represents the date printed on the report itself (if you choose to extract it) or the job date of the job that archived the report (if you do not choose to extract the date from the printed page). The run date is the job date of the job that archived the report. If the archive job's job date is captured as the posting date for the report (instead of extracting the posting date from the printed page), then the run date and posting date will be the same date. The posting date is used in combination with the report sequence number to identify the stored object. In Spool File Archive, it is the run date that is used to determine when data is migrated to other media, such as from disk to optical. It is also the run date that is used to determine when the data is eligible to be expired by the Report Management Cycle (RMC) job. In Common Server, when data is archived, one date is extracted for each document within the load. Depending on the data being loaded, the date for the first document in the load might be different than the date for the last document in the load. If no date field is defined in the OnDemand application group definition to be extracted from the printed page, the system date is used for the date of each document in the load. Regardless of whether the date was extracted from the printed page of each document in the load or the system date was used for each document in the load, the date associated with the last document of the load is used to determine when the data is eligible to be expired from OnDemand. This date is also used to determine the length of time the data will reside in Cache as well as when it is sent to the Archive Storage Manager (ASM), based on the settings on the Storage Management tabs in the OnDemand application group definition. It is important to understand the difference between how Spool File Archive and Common Server determine the eligibility of data to be expired, because data might expire sooner than expected in Common Server if you are loading data with a date older than today's date. For example, if a spooled file is archived into Spool File Archive on 2012/08/01 and the date captured from the spooled file is a date older than today's date (such as 2011/01/01), the posting date associated with the archived report is 2011/01/01. However, the run date associated with the archived report is the job's date which was 2012/08/01 in our example. Based on the migration policy named in the OnDemand report definition, the data should be expired after 1 year. Because the run date is 2012/08/01, the data will be expired the first time RMC is run after 2013/08/01. If this same spooled file is loaded into Common Server on 2012/08/01 and the date is captured from the spooled file being loaded, the date associated with the last document is 2011/01/01. If the Life of Data and Indexes for the OnDemand application group to which the spooled file was loaded specifies 365 days, then the first time the Disk Storage Manager (DSM) is run after the spooled file is loaded into OnDemand, the data will be expired. This is because Common Server uses the date associated with the last document in the load to determine eligibility for expiration rather than the actual date the data was loaded. In our example, this date was 2011/01/01 and this is the date used to compare to today's date to determine if the data should be expired. Since today's date is 2012/08/01, the data is older than 365 days and therefore will be expired. 6/7/2013 Page 113

114 Appendix D: Tips for customizing the XML files created by migration Effective with PTF SI40481, you can customize the XML files created by migration and use those customized files to add users, groups, and report definitions to Common Server. The use of one or more test instances is highly recommended when customizing your migration XML. Note that if you change the XML created by migration, you are responsible for the results. Do not call software support for assistance if you customize the XML created by migration. When customizing the XML files created by migration, you should either move the customized files to another directory, or rename the customized files so that the names are different from the names of any files created by migration. This will help prevent your customized files from being overwritten accidentally. Users Some possible changes that you can make by editing the USERS.XML file created by migration are: Insert or delete users Change user type Add individual user authorities Change membership in user groups Customization Steps To customize the XML file before adding the users and groups to a production instance: Migrate users to a test instance to create the XML file. Edit the XML file and make the required changes. Add to the production instance, or another test instance, using arsxml, for example: arsxml add -v -x -e c -h instance_name -i /QIBM/USERDATA/ONDEMAND/instance_name/SFAMIGR/USERS.XML Insert or Delete Users You can insert additional users into the XML file, or delete users that are in the XML file. Change User Type By default, all users are added with a user type of User. User types that can be specified in XML are: usertype="user (this is the default user type if a user type is not specified in the file) usertype="user Admin" usertype="ag/folder/cabinet Admin" 6/7/2013 Page 114

115 usertype="system Admin" Migration Reference To change the user type of a user before adding users to a production instance, insert the user type between the user and /user tags. For example, to add user DBRYANT with a type of system administrator, change this: <user name="dbryant" > </user> to this: <user name="dbryant" usertype="system Admin" > </user> Add User Authorities Individual user authorities that can be specified in XML are: createusersauth="true" creategroupsauth="true" createappgroupsauth="true" createfoldersauth="true" createcabinetsauth="true" To add individual user authorities before adding the users to a production instance, insert the authorities between the user and /user tags. For example, to add user TSTAMP as a user with authority to create folders and cabinets, change this: <user name="tstamp" > </user> to this: <user name="tstamp" usertype="user" createfoldersauth="true" createcabinetsauth="true" > </user> Change membership in user groups To change membership in user groups before adding the groups to a production instance, insert or delete users between the group and /group tags. For example: <group name="acctgroup" > <user name="dbryant" /> <user name="tstamp" /> add more users here </group> 6/7/2013 Page 115

116 Report Definitions Note that you must understand your report definitions, application groups, applications, and folders before making any changes to the XML created by migration. Do not call software support for assistance if you customize the XML created by migration. Some possible changes that you can make to your application groups, applications, and folders by editing the XML file created by migration are: Change field lengths Change field type from index to filter Change string type from fixed to variable Change folder field information Add full text search to your folders Change field data type Change field names Customization Steps To customize the XML file before adding the application groups, applications, and folders to a production instance: Migrate the report definition to a test instance to create the XML file. Edit the XML file and make the changes you need. Add the definitions to the production instance, or another test instance, using arsxml. For example: arsxml add -v -x -e c -h instance_name -i /QIBM/USERDATA/ONDEMAND/instance_name/SFAMIGR/report_name.XML Change field length Changing field lengths in the application group also requires changing application indexer parameters. For example, the customer name field is defined as 25 characters in SFA, which is the maximum field length for key 1. You can edit the XML file to increase the field length in the application group and application indexer parameters. The maximum field length for a string field is 254 bytes. In the following example we increase the length of field F_03 from 25 to 40 characters. In the application group change this: <field name="f_03" type="index" stringlength="25" > </field> to this: <field name="f_03" type="index" stringlength="40" > </field> 6/7/2013 Page 116

117 In the application change this: Migration Reference FIELD1=0,37,25,(TRIGGER=1,BASE=0) to this: FIELD1=0,37,40,(TRIGGER=1,BASE=0) Change field type Changing the field type from index to filter can be done by editing the XML created by migration. The following items should be considered when determining which fields should be indexes in Common Server. All SFA keys that are searchable are migrated as indexes. However, not all searchable SFA keys need to be indexes. Migration has no method to determine which searchable keys should become indexes in Common Server. Through your knowledge of your reports, you can make this determination. A filter field is a field not used to identify a document or a field that is always used with an index field to refine the results of a query. A filter field does not create a logical view (access path) over the index data. In the application group change this: <field name="f_03" type="index" stringlength="40" > </field> to this: <field name="f_03" type="filter" stringlength="40" > </field> Change string type You can change the string type from fixed to variable. This change can potentially save space in the index files, if most of your indexes are shorter than the default length. Note also that fixed length is the default setting. In the following example we change field F_03 from string type Fixed (the default setting) to string type Variable. In the application group change this: <field name="f_03" type="index" stringlength="40" > </field> to this: <field name="f_03" type="index" stringlength="40" stringtype="variable" > </field> When you define a field as variable length, make sure that you remove trailing spaces in the Application > Load information tab. Otherwise, the spaces will be archived as valid characters. In the following example we change field F_03 to remove both leading and trailing blanks. In the application change this: <preprocessparm dbname="f_03" loadidname="cust Name" leading=" " /> 6/7/2013 Page 117

118 to this: <preprocessparm dbname="f_03" loadidname="cust Name" leading=" " trailing=" " /> Note that variable length string fields only save space if the field is longer than 24 characters. Note also that every variable length field has an overhead of 2 bytes. The default allocation used by OnDemand for variable length fields is as follows: Length (L) L < 25 OnDemand Defaults Allocated Length 24 < L < 50 3 / 4 * L 49 < L < / 3 * L L > 99 L 1 / 2 * L The allocated length is rounded down to the nearest integer. Change Folder Field Information Document List Order You can change the document list (also known as the hit list) order to 0 for Application ID and Sequence Number. The Application ID and Sequence Number do not need to appear in the document list for most reports. In the folder change this: <field name="application Identifier" > <mapping dbname="f_00" appgroup="checkstmts" /> <fieldinfo group="*public" displayorder="7" queryorder="0" </field> to this: <field name="application Identifier" > <mapping dbname="f_00" appgroup="checkstmts" /> <fieldinfo group="*public" displayorder="0" queryorder="0" </field> and change this: <field name="sequence number" > <mapping dbname="f_02" appgroup="checkstmts" /> <fieldinfo group="*public" displayorder="6" queryorder="0" </field> to this: <field name="sequence number" > <mapping dbname="f_02" appgroup="checkstmts" /> <fieldinfo group="*public" displayorder="0" queryorder="0" </field> 6/7/2013 Page 118

119 Default date range You can change the default date range from 30 days (the default value in SFA) to 60 days (the default value in Common Server) or another value. In the folder change this (you will need to window to the right to change this value): <field name="date" fieldtype="date" > <mapping dbname="f_01" appgroup="checkstmts" /> <fieldinfo group="*public"... dateinttype="days" dateintlength="30" /> </field> to this: <field name="date" fieldtype="date" > <mapping dbname="f_01" appgroup="checkstmts" /> <fieldinfo group="*public"... dateinttype="days" dateintlength="60" /> </field> Valid date interval types for date fields are Days, Months, and Years dateinttype="days" dateinttype="months" dateinttype="years" Add full text search to your folders You can add a full text search field to your folders. This feature can always be added by creating new folders after migration, but now you can add this feature during migration. After migrating to a test instance, you can add this field to the folder definition: <field name="text Search" fieldtype="text Search" > <fieldinfo group="*public" queryorder="6" /> </field> Set the value for the queryorder to a higher number than that used by the other fields. For example, if the highest value for queryorder is 5, when you add the text search field, set the queryorder to 6. For more information about full text search, see the OnDemand Windows Client help. Search for Text search field. Change Field Names Note that you must understand your report definitions, application groups, applications, and folders before changing the field names in the XML created by migration. IBM support will not assist you in making these changes. You should use a test instance to verify the correctness of your changes, including testing index migration, and testing archival of new reports, before using them in production. You can change the field names used in migration from the generic F_00, F_01, etc. to something meaningful, such as VERSION, PATIENTNAME, etc. You must be consistent, changing the existing field name in the XML, F_03 for example, to the same field name every time, PATIENTNAME for example. 6/7/2013 Page 119

120 In this example, the following field names are being changed: F_00 is changed to VERSION F_01 is changed to RDATE F_02 is changed to SEQNUMBER F_03 is changed to PATIENTNAME F_04 is changed to PATIENTNUMBER For example, you can change this XML for the application group: <field name="f_00" type="filter" appidfield="true" stringcase="mixed"... <mapping dbvalue="01" displayedvalue="01" /> <mapping dbvalue="a1" displayedvalue="a1" /> </field> <field name="f_01" type="index" datatype="date" segment="true" > </field> <field name="f_02" type="filter" stringcase="mixed" stringlength="16" > </field> <field name="f_03" type="index" stringlength="25" > </field> <field name="f_04" type="index" stringlength="7" > </field> to this XML: <field name="version" type="filter" appidfield="true" stringcase="mixed"... <mapping dbvalue="01" displayedvalue="01" /> <mapping dbvalue="a1" displayedvalue="a1" /> </field> <field name="rdate" type="index" datatype="date" segment="true" > </field> <field name="seqnumber" type="filter" stringcase="mixed" stringlength="16" > </field> <field name="patientname" type="index" stringlength="25" > </field> <field name="patientnumber" type="index" stringlength="7" > </field> And if the report uses key security, change this XML: <permission user="*public" adminauthority="false" lvauthority="false" accessauthority="false" queryres="f_00 = 'XX' AND NOT F_00 = 'XX'" /> <permission user="tbrown" adminauthority="false" lvauthority="false" accessauthority="true" queryres="(f_04 BETWEEN ' ' AND ' ')" /> <permission group="sfagroup2" adminauthority="false" lvauthority="false" accessauthority="true" queryres="(f_04 BETWEEN ' ' AND ' ')" /> 6/7/2013 Page 120

121 to this XML: Migration Reference <permission user="*public" adminauthority="false" lvauthority="false" accessauthority="false" queryres="version = 'XX' AND NOT VERSION = 'XX'" /> <permission user="tbrown" adminauthority="false" lvauthority="false" accessauthority="true" queryres="(patientnumber BETWEEN ' ' AND ' ')" /> <permission group="sfagroup2" adminauthority="false" lvauthority="false" accessauthority="true" queryres="(patientnumber BETWEEN ' ' AND ' ')" /> For example, you can change this XML for the application: <preprocessparm dbname="f_03" loadidname="patient Name" leading=" " /> <preprocessparm dbname="f_04" loadidname="patient #" leading=" " /> <preprocessparm dbname="f_01" loadidname="date" defaultvalue="t" format="%y-%m-%d" /> <preprocessparm dbname="f_02" loadidname="sequence number" /> to this XML: <preprocessparm dbname="patientname" loadidname="patient Name" leading=" " /> <preprocessparm dbname="patientnumber" loadidname="patient #" leading=" " /> <preprocessparm dbname="rdate" loadidname="date" defaultvalue="t" format="%y-%m-%d" /> <preprocessparm dbname="seqnumber" loadidname="sequence number" /> For example, you can change this XML for the folder: <field name="application Identifier" > <mapping dbname="f_00" appgroup="hcfaovl" /> <field name="date" fieldtype="date" > <mapping dbname="f_01" appgroup="hcfaovl" /> <field name="sequence number" > <mapping dbname="f_02" appgroup="hcfaovl" /> <field name="patient Name" > <mapping dbname="f_03" appgroup="hcfaovl" /> <field name="patient #" > <mapping dbname="f_04" appgroup="hcfaovl" /> to this XML: <field name="application Identifier" > <mapping dbname=" VERSION" appgroup="hcfaovl" /> <field name="date" fieldtype="date" > <mapping dbname="rdate" appgroup="hcfaovl" /> <field name="sequence number" > <mapping dbname="seqnumber" appgroup="hcfaovl" /> <field name="patient Name" > <mapping dbname="patientname" appgroup="hcfaovl" /> <field name="patient #" > <mapping dbname="patientnum" appgroup="hcfaovl" /> 6/7/2013 Page 121

122 Change Field Data Type Note that you must understand your report data, SFA indexes, application groups, applications, and folders before changing the field data type in the XML created by migration. IBM support will not assist you in making these changes. You should use a test instance to verify the correctness of your changes, including testing index migration and testing archival of new reports, before using them in production. By default all Spool File Archive indexes are migrated as string fields, with certain exceptions for page reports. If all the indexes stored in Spool File Archive for a particular key in a particular report are integer or decimal, you can change the XML created by migration to make that particular key integer or decimal. Understanding your report data and indexes is critical to successfully making this type of change. This document provides sample SQL statements and XML to assist in this process. Integer Data To determine if a specific key for a particular report contains integer data, you can run an SQL statement similar to the following. In this example the SQL statement is checking values in Key 1 (KY1VAL), for report MOBILEFOCA, which is not in a report group and which uses QARLR000PF to store the index records. SELECT CDTYPE, DVERSION, KY1VAL, ONAME FROM QUSRRDARS/QARLR000PF WHERE CDTYPE = 'MOBILEFOCA' AND TRANSLATE(KY1VAL,' ',' ')<>' ' The output will look similar to the following if all the index records contain only numbers or spaces. Report Version Key1 Value Object Type ******** End of data ******** No data selected for output. Name The output will look similar to the following if there are index records where the values contain characters other than numbers or spaces. Report Version Key1 Value Object Type Name MOBILEFOCA A If there are index records where the key contains characters other than numbers or spaces, you must determine if this is incorrect data that can be fixed before migration, or if this key sometimes contains alphabetic characters and must remain as a string type field. If you determine that the key contains only integer data, and will continue to contain only integer data, you can change the XML created by migration to create the field as integer in Common Server. You can also create the field as small integer (datatype="small Int") or big integer (datatype="big Int"), depending on the length of data contained in that key. See the OnDemand Administrator Client help text for more information on database field data types. For example, you can change this XML for the application group: <field name="f_03" type="index" stringlength="12" > </field> 6/7/2013 Page 122

123 to this XML: <field name="f_03" type="index" datatype="integer" > </field> For example, you can change this XML for the application: <preprocessparm dbname="f_03" loadidname="mobile Number" leading=" " /> to this XML: <preprocessparm dbname="f_03" loadidname="mobile Number" leading=" " trailing=" " /> For example, you can change this XML for the folder: <field name="mobile Number" > <mapping dbname="f_03" appgroup="mobilefoca" /> <fieldinfo group="*public" displayorder="2" queryorder="1" sortorder="2" viewtitle="true" equal="default" notequal="true" lessthan="true" ltorequal="true" greaterthan="true" gtorequal="true" in="true" notin="true" between="true" notbetween="true" like="true" notlike="true" wildcard="none" /> </field> to this XML: <field name="mobile Number" fieldtype="integer" > <mapping dbname="f_03" appgroup="mobilefoca" /> <fieldinfo group="*public" displayorder="2" queryorder="1" sortorder="2" viewtitle="true" equal="default" notequal="true" lessthan="true" ltorequal="true" greaterthan="true" gtorequal="true" in="true" notin="true" between="true" notbetween="true" /> </field> Decimal Data To determine if a specific key for a particular report contains decimal data, you can run an SQL statement similar to the following. In this example the SQL statement is checking values in Key 3 (KY3VAL), for report PATCHECKS, which is in report group PATINFO and uses QARLRPATPF to store the index records. SELECT CDTYPE, DVERSION, KY3VAL, ONAME FROM QUSRRDARS/QARLRPATYPF WHERE CDTYPE = 'PATCHECKS' AND TRANSLATE(KY3VAL,' ',' ,.')<>' ' The output will look similar to the following if all the index records contain only numbers, periods, commas, or spaces. Report Version Key3 Value Object Type ******** End of data ******** No data selected for output. Name The output will look similar to the following if there are index records where the values contain characters other than numbers, periods, commas, or spaces. 6/7/2013 Page 123

124 Report Version Key3 Value Object Type Name PATCHECKS 01 VOID If you determine the key contains only decimal data, and will continue to contain only decimal data, you can change the XML created by migration to create the field as decimal in Common Server. See the OnDemand Administrator Client help text for more information on database field data types. For example, you can change this XML for the application group: <field name="f_05" type="index" stringlength="10" > </field> to this XML: <field name="f_05" type="index" datatype="decimal" > </field> For example, you can change this XML for the application: <preprocessparm dbname="f_05" loadidname="check Amount" leading=" " /> to this XML: <preprocessparm dbname="f_05" loadidname="check Amount" leading=" " trailing=" " embedded=,. /> For example, you can change this XML for the folder: <field name="check Amount" > <mapping dbname="f_05" appgroup="patchecks" /> <fieldinfo group="*public" displayorder="4" queryorder="3" sortorder="4" viewtitle="true" equal="default" notequal="true" lessthan="true" ltorequal="true" greaterthan="true" gtorequal="true" in="true" notin="true" between="true" notbetween="true" like="true" notlike="true" wildcard="none" /> </field> to this XML: <field name="check Amount" fieldtype="decimal" > <mapping dbname="f_05" appgroup="patchecks" /> <fieldinfo group="*public" displayorder="4" queryorder="3" sortorder="4" viewtitle="true" equal="default" notequal="true" lessthan="true" ltorequal="true" greaterthan="true" gtorequal="true" in="true" notin="true" between="true" notbetween="true" /> </field> Monetary Amounts To determine if a specific key for a particular report contains monetary amounts, which can be stored as decimal values (in our example we use dollar amounts), you can run an SQL statement similar to the following. In this example the SQL is checking values in Key 4 (FL1VAL), for report CHECKSTMTS, which is not in a report group and uses QARLR000PF to store the index records. SELECT CDTYPE, DVERSION, FL1VAL, ONAME FROM QUSRRDARS/QARLR000PF WHERE CDTYPE = 'CHECKSTMTS' AND 6/7/2013 Page 124

125 TRANSLATE(FL1VAL,' ',' ,$.')<>' ' The output will look similar to the following if all the index records contain only numbers, commas, periods, dollar signs, or spaces. Report Version Field1 Value Object Type ******** End of data ******** No data selected for output. Name The output will look similar to the following if there are index records where the values contain characters other than numbers, commas, periods, dollar signs, or spaces. Report Version Field1 Value Object Type Name CHECKSTMTS 01 $ 1,275/ If there are index records where the key contains characters other than numbers, commas, periods, dollar signs, or spaces, you must determine if this is incorrect data that can be fixed, or if this key sometimes contains alphabetic characters and must remain as a string type field. If you determine that the key contains only monetary values, and will continue to contain only monetary values, you can change the XML created by migration to create the field as decimal in Common Server. See the OnDemand Administrator Client help text for more information on database field data types. Note that after making this change, the dollar sign will no longer be displayed in the OnDemand Client. For example, you can change this XML for the application group: <field name="f_06" type="filter" stringlength="15" > </field> to this XML: <field name="f_06" type="filter" datatype="decimal" > </field> For example, you can change this XML for the application: <preprocessparm dbname="f_06" loadidname="ending Balance" leading=" " /> to this XML: <preprocessparm dbname="f_06" loadidname="ending Balance" embedded=",." leading=" $" trailing=" " /> For example, you can change this XML for the folder: <field name="ending Balance" > <mapping dbname="f_06" appgroup="checkstmts" /> <fieldinfo group="*public" displayorder="5" queryorder="0" sortorder="5" viewtitle="true" equal="default" notequal="true" lessthan="true" ltorequal="true" greaterthan="true" gtorequal="true" in="true" notin="true" between="true" notbetween="true" like="true" notlike="true" wildcard="none" /> </field> 6/7/2013 Page 125

126 to this XML: Date Migration Reference <field name="ending Balance" fieldtype="decimal" > <mapping dbname="f_06" appgroup="checkstmts" /> <fieldinfo group="*public" displayorder="5" queryorder="0" sortorder="5" viewtitle="true" equal="default" notequal="true" lessthan="true" ltorequal="true" greaterthan="true" gtorequal="true" in="true" notin="true" between="true" notbetween="true" /> </field> To determine if a specific key for a particular report contains dates with the forward slash character ( / ) as the separator, which can be stored as date values, you can run an SQL statement similar to the following. In this example the SQL statement is checking values in Key 5 (FL2VAL), for report HCFA1500, which is in report group PATINFO and uses QARLRPATPF to store the index records. SELECT CDTYPE, DVERSION, FL2VAL, ONAME FROM QUSRRDARS/QARLRPATPF WHERE CDTYPE = 'HCFA1500' AND TRANSLATE(FL2VAL,' ',' /')<>' ' The output will look similar to the following if all the index records contain only numbers, date separators, or spaces. Report Version Field2 Value Object Type Name ******** End of data ******** No data selected for output. The output will look similar to the following if there are index records where the values contain characters other than numbers, date separators, or spaces. Report Version Field2 Value Object Type Name HCFA DATE OF SERVICE If there are index records where the key contains characters other than numbers, date separators, or spaces, you must determine if this is incorrect data that can be fixed, or if this key sometimes contains alphabetic characters and must remain as a string type field. If you determine that the key contains only date values, and will continue to contain only date values, you can change the XML created by migration to create the field as date in Common Server. See the OnDemand Administrator Client help text for more information on database field data types. Note that all dates stored in this field must be in the same format. For example, you can change this XML for the application group: <field name="f_07" type="filter" stringlength="8" > </field> to this XML: <field name="f_07" type="filter" datatype="date" > </field> 6/7/2013 Page 126

127 For example, you can change this XML for the application: <preprocessparm dbname="f_07" loadidname="doc Date" leading=" " /> to this XML: <preprocessparm dbname="f_07" loadidname="doc Date" leading=" " format="%m/%d/%y" /> For example, you can change this XML for the folder: <field name="doc Date" > <mapping dbname="f_07" appgroup="hcfa1500" /> <fieldinfo group="*public" displayorder="4" queryorder="0" sortorder="4" viewtitle="true" equal="default" notequal="true" lessthan="true" ltorequal="true" greaterthan="true" gtorequal="true" in="true" notin="true" between="true" notbetween="true" like="true" notlike="true" wildcard="none" /> </field> to this XML: <field name="doc Date" fieldtype="date" > <mapping dbname="f_07" appgroup="hcfa1500" /> <fieldinfo group="*public" displayorder="4" queryorder="0" sortorder="4" sorttype="descending" viewtitle="true" equal="default" between="true" notbetween="true" /> </field> 6/7/2013 Page 127

128 Appendix E: Migrating only certain users to Common Server If you have hundreds of user profiles on your system and do not want or need to migrate all of them to Common Server, there are a couple of alternatives to using the *MGRUSR option. As described in Appendix C, you can edit the xml file created by running *MGRUSR to your test instance and delete or add specific users and groups. Then use arsxml to add the users and groups to your production instance. Another alternative to the migration of all users is to write a program to add selected users to the Common Server instance. The following two programs are examples of how to do this using CL. Example 1 This program uses the DSPUSRPRF command to place a list of all user profiles on the system into an output file (QTEMP/USERS). The program then uses the following logic to determine the settings for each user added: Do not add any user profiles starting with the letter Q If the group profile is QONDADM, then make the user's type s system administrator If the user class is *SECOFR, then make the user's type g user administrator If the limit capability value is *YES, then make the user's type u user Otherwise make the user's type b create appl/appl grps & folders Code for sample program 1 /* DISCLAIMER OF WARRANTIES. The following code is sample code */ /* created by IBM Corporation. IBM grants you a nonexclusive */ /* copyright license to use this sample code example to generate */ /* similar function tailored to your own specific needs. This sample */ /* code is not part of any standard IBM product and is provided to */ /* you solely for the purpose of assisting you in the development of */ /* your applications. This example has not been thoroughly tested */ /* under all conditions. IBM, therefore cannot guarantee nor may you */ /* imply reliability, serviceability, or function of these programs. */ /* The code is provided "AS IS", without warranty of any kind. IBM */ /* shall not be liable for any damages arising out of your or any */ /* other parties use of the sample code, even if IBM has been */ 6/7/2013 Page 128

129 /* advised of the possibility of such damages. If you do not agree */ /* with these terms, do not use the sample code. */ /* */ PGM DCL VAR(&COMMAND) TYPE(*CHAR) LEN(512) DCL VAR(&TYPE) TYPE(*CHAR) LEN(1) DCLF FILE(QADSPUPB) DSPUSRPRF USRPRF(*ALL) OUTPUT(*OUTFILE) + OUTFILE(QTEMP/USERS) OVRDBF FILE(QADSPUPB) TOFILE(QTEMP/USERS) LOOP: RCVF MONMSG MSGID(CPF0864) EXEC(GOTO CMDLBL(RETURN)) IF COND(%SST(&UPUPRF 1 1) *EQ Q) THEN(GOTO + CMDLBL(LOOP)) CHGVAR VAR(&TYPE) VALUE('b') IF COND(&UPLTCP *EQ *YES) THEN(CHGVAR + VAR(&TYPE) VALUE('u')) IF COND(&UPUSCL *EQ *SECOFR) THEN(CHGVAR + VAR(&TYPE) VALUE('g')) IF COND(&UPGRPF *EQ QONDADM) THEN(CHGVAR + VAR(&TYPE) VALUE('s')) CHGVAR VAR(&COMMAND) VALUE('arsadm user -v -h + ondlab -a a -i' *BCAT &UPUPRF *BCAT '-t ' + *BCAT &TYPE *BCAT ' -N ''' *BCAT &UPTEXT + *BCAT ''' -u qondadm -p qondadm1') QSH CMD(&COMMAND) GOTO CMDLBL(LOOP) RETURN: RETURN ENDPGM Example 2 This program creates all the user IDs in Common Server as 'Users', with no additional authority. However, it does add each user to a Common Server group. Manually create the desired user groups in Common Server. The Common Server groups should have the same names as the OS/400 group profiles the users belong to. Use the DSPUSRPRF command to create a list of user profiles in an output file. DSPUSRPRF USRPRF(*ALL) OUTPUT(*OUTFILE) OUTFILE(QGPL/ONDUSERS) Review the contents of the output file. Delete the records of user profiles that you do not need or want to migrate. Run the following program to add the remaining users to the OnDemand instance and to add them to a group at the same time. 6/7/2013 Page 129

130 Code for sample program 2 Migration Reference /* DISCLAIMER OF WARRANTIES. The following code is sample code */ /* created by IBM Corporation. IBM grants you a nonexclusive */ /* copyright license to use this sample code example to generate */ /* similar function tailored to your own specific needs. This sample */ /* code is not part of any standard IBM product and is provided to */ /* you solely for the purpose of assisting you in the development of */ /* your applications. This example has not been thoroughly tested */ /* under all conditions. IBM, therefore cannot guarantee nor may you */ /* imply reliability, serviceability, or function of these programs. */ /* The code is provided "AS IS", without warranty of any kind. IBM */ /* shall not be liable for any damages arising out of your or any */ /* other parties use of the sample code, even if IBM has been */ /* advised of the possibility of such damages. If you do not agree */ /* with these terms, do not use the sample code. */ /* */ /* THIS PROGRAM SHOULD BE COMPILED IN LIBRARY QUSRRDARS */ /* THIS PROGRAM SHOULD BE SUBMITTED WITH THE COMMAND */ /* SBMJOB CMD(CALL PGM(QUSRRDARS/ADDONDUSRS)) JOBD(QRDARS/QOND400) */ PGM DCL VAR(&COMMAND) TYPE(*CHAR) LEN(512) DCL VAR(&GRPPRF) TYPE(*CHAR) LEN(11) /* */ /* DECLARE FILE CONTAINING A LIST OF USER PROFILES */ DCLF FILE(QGPL/ONDUSERS) LOOP: RCVF MONMSG MSGID(CPF0864) EXEC(GOTO CMDLBL(END)) /* */ /* PREFIX GROUP PROFILE NAME WITH A PLUS SIGN */ CHGVAR VAR(&GRPPRF) VALUE('+' *CAT &UPGRPF) /* */ /* BUILD THE QSH COMMAND STRING */ CHGVAR VAR(&COMMAND) VALUE('arsadm user -v -h + QUSROND -t u -a a -i ' *CAT &UPUPRF + *CAT ' -d ''' *CAT &UPTEXT *CAT + ''' -g ' *CAT &GRPPRF) /* */ /* EXECUTE THE QSH COMMAND */ QSH CMD(&COMMAND) GOTO CMDLBL(LOOP) END: RETURN ENDPGM 6/7/2013 Page 130

131 Appendix F: Automating the re-spooling of Spool File Archive reports The following steps show one method of reprinting (re-spooling) all reports from a single Spool File Archive report definition and then storing the reprinted reports into a new Common Server definition. This method uses some sample SQL statements and CL programs which are included below. It is important to note that re-spooling and then storing data to migrated definitions is not supported. The -xx version(s) are only used for migrating index data in Phase 3, and for viewing migrated reports using that index data. The non-dash version may not be correct for some of your older data that might have changed format or content. If you choose the method described in this appendix to re-spool your Spool File Archive data, you must store the re-spooled data into Common Server application groups/applications that have been created by using the OnDemand Administrator client or the OnDemand ARSXML batch administration API. 1. Use an SQL statement to select the reports you want to reprint from SFA and rearchive into Common Server. Output of the SQL statement should be sent to a database file. (Use F13, option 1 to specify the output option and file name. In this example, the output file name is QGPL/SRTRCDS.) We use the report definition named CHECKSTMTS in the following example. Change 'CHECKSTMTS' to the name of the report definition you are re-spooling. SELECT CDTYPE, "VERSION", SUBSTR(ONAME,1,8) AS ODATE, SUBSTR(ONAME,10,3) AS OSEQ FROM QUSRRDARS/QARLRSRT WHERE CDTYPE = 'CHECKSTMTS' 2. Run this CL program to re-spool all the reports listed in the output file specified in SQL. In this example, the output file name is QGPL/SRTRCDS. You can also change the output queue name on the PRTRPTRDAR command as needed. This CL program should be submitted to batch with the SBMJOB command. /* DISCLAIMER OF WARRANTIES. The following code is sample code */ /* created by IBM Corporation. IBM grants you a nonexclusive */ /* copyright license to use this sample code example to generate */ /* similar function tailored to your own specific needs. This sample */ /* code is not part of any standard IBM product and is provided to */ /* you solely for the purpose of assisting you in the development of */ /* your applications. This example has not been thoroughly tested */ /* under all conditions. IBM, therefore cannot guarantee nor may you */ /* imply reliability, serviceability, or function of these programs. */ /* The code is provided "AS IS", without warranty of any kind. IBM */ /* shall not be liable for any damages arising out of your or any */ /* other parties use of the sample code, even if IBM has been */ /* advised of the possibility of such damages. If you do not agree */ /* with these terms, do not use the sample code. */ /* */ PGM DCLF FILE(QGPL/SRTRCDS) LOOP: RCVF 6/7/2013 Page 131

132 END: MONMSG MSGID(CPF0864) EXEC(GOTO CMDLBL(END)) PRTRPTRDAR REPORT(&CDTYPE) VERSION(&VERSION) + RPTDATE(&ODATE) RPTSEQ(&OSEQ) + PRINTER(*OUTQ) OUTQ(QRDARS/QRDARS400) + SBMJOB(*NO) GOTO CMDLBL(LOOP) ENDPGM After all the reports have been re-spooled, and after the Common Server application group, application, and folder definitions have been created, continue with the following steps: 3. Move one of the re-spooled reports into an output queue monitored by OnDemand Common Server (which is a monitor started with the STRMONOND TYPE(*OUTQ) command). 4. Verify that the report archived correctly and that the report has the keys and segmentation you expect. 5. Move all of the remaining re-spooled reports into an output queue monitored by OnDemand Common Server (which is a monitor started with the STRMONOND TYPE(*OUTQ) command). 6. Verify with your users that the reports are segmented and indexed as expected. 7. Delete the re-spooled reports from your system. 6/7/2013 Page 132

133 Appendix G: Changing the Spool File Archive server port number The Spool File Archive server can be changed to a port other than 0. This allows the Common Server server to use port 0. When used with coexistence, this prevents you from having to change the server port specified on the end user's PC. If you are performing a Spool File Archive to Common Server migration and have many workstations with the OnDemand Windows Client installed, the following procedure may be helpful. It allows users to start using Common Server (in coexistence mode) without having to change their client from their default port 0. Below are the steps for allowing the OnDemand Client port 0 to access Common Server. (Port 0 is currently defined on the OnDemand Client PCs to use SFA.) This change will affect ALL users. 1. End the Spool File Archive server and Common Server server jobs: ENDTCPSVR *ONDMD 2. Change the Spool File Archive server port from the default 1445 to an unused port: ADDSRVTBLE SERVICE('ondemand') PORT(1465) PROTOCOL('tcp') TEXT('OnDemand Server') where port 1465 can be ANY unused socket port on the IBM i system. 3. Update the Common Server file ARS.INI to use Port 1445 (This is default for port 0 from the client) EDTF '/QIBM/UserData/OnDemand/CONFIG/ARS.INI' Locate the stanza for your Common Server instance, QUSROND in our example, and change the port number to [@SRV@_QUSROND] PORT= Restart the Common Server server job using the command to start a single instance. This command must be used instead of the command STRTCPSVR to ensure that Common Server starts first and allocates port If the Spool File Archive server starts first it will allocate port 1445 and ignore the service table entry. CALL QRDARS/QRLMCTL PARM(*STRTCPSVRQUSROND) 5. Restart the Spool File Archive server job (now accessible on port 1465): CALL QRDARS/QRLGCTL PARM(*STRTCPSVR) ALL users who currently have their OnDemand Windows Client server defined on port 0 will be now accessing Common Server and NOT Spool File Archive. There are no changes required to the configuration of OnDemand Windows Client for the end user workstations. If you are also using coexistence mode, you must first change the SFA port referenced in the coexistence entry in the instance ARS.CFG file to the new SFA port so that it matches the port used in the ADDSRVTBLE command. 6/7/2013 Page 133

134 Appendix H: Archiving migration reports To keep a long term record of your migration activity you can archive your migration reports and migration job logs in Common Server. The following table lists the spooled files created by migration. Spooled File Name User Data QPRLRCVUG *MGRUSR *MGRUSR QPRLRCVPCY *MGRPCY *MGRPCY QPRLRMGRD QPRLRCVRD QPRLRMIDX QPJOBLOG Report name, generic name, or report group name Report name, generic name, or report group name Report name, generic name, or report group name Created by migration program option *ANZDFN, *MGRDFN *CVTDFN *ANZIDX, *RCLIDX, *MGRIDX, *CVTIDX, *RMVIDX Can be created by each step. If you run the migration jobs interactively, using *SIGNOFF *LIST will ensure that a job log is created. See the following pages for indexing hints for each spooled file name. 6/7/2013 Page 134

135 QPRLRCVUG Migrate Users and Groups Migration Reference The following figure shows the suggested trigger (red box) and fields (blue boxes) for page 1 of the QPRLRCVUG spooled file. All the fields can be located using trigger 1. The following figure shows the suggested trigger (red box) and fields (blue boxes) for the last page of the QPRLRCVUG spooled file. The trigger on this page is a float trigger. Your trigger on this page will vary depending on the language used for your migration jobs. 6/7/2013 Page 135

136 QPRLRCVPCY Migrate Migration Policies The following figure shows the suggested trigger (red box) and fields (blue boxes) for page 1 of the QPRLRCVPCY spooled file. All the fields can be located using trigger 1. The following figure shows the suggested trigger (red box) and field (blue box) for the last page of the QPRLRCVPCY spooled file. The trigger on this page is a float trigger. Your trigger on this page will vary depending on the language used for your migration jobs. 6/7/2013 Page 136

137 QPRLRMGRD Migrate Report Definitions The following figure shows the suggested trigger (red box) and fields (blue boxes) for page 1 of the QPRLRMGRD spooled file. All the fields can be located using trigger 1. The following figure shows the suggested trigger (red box) and field (blue box) for the last page of the QPRLRMGRD spooled file. The trigger on this page is a float trigger. Your trigger on this page will vary depending on the language used for your migration jobs. 6/7/2013 Page 137

138 QPRLRCVRD Convert Report Definitions The following figure shows the suggested trigger (red box) and fields (blue boxes) for page 1 of the QPRLRCVRD spooled file. All the fields can be located using trigger 1. The following figure shows the suggested trigger (red box) and field (blue box) for the last page of the QPRLRCVRD spooled file. The trigger on this page is a float trigger. Your trigger on this page will vary depending on the language used for your migration jobs. 6/7/2013 Page 138

139 QPRLRMIDX Migrate Indexes The following figure shows the suggested trigger (red box) and fields (blue boxes) for page 1 of the QPRLRMIDX spooled file. All the fields can be located using trigger 1. 6/7/2013 Page 139

140 QPJOBLOG Job Log The following figure shows the suggested trigger (red box) and fields (blue boxes) for page 1 of the QPJOBLOG spooled file. All the fields can be located using trigger 1. 6/7/2013 Page 140