User's Guide c-treertg COBOL Edition

Transcription

1 User's Guide c-treertg COBOL Edition

2 User's Guide c-treertg COBOL Edition V2

3 Copyright Notice Copyright FairCom Corporation. All rights reserved. No part of this publication may be stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise without the prior written permission of FairCom Corporation. Printed in the United States of America. Information in this document is subject to change without notice. Trademarks c-treeace, c-treertg, c-treeams, c-tree Plus, c-tree, r-tree, FairCom and FairCom s circular disc logo are trademarks of FairCom, registered in the United States and other countries. The following are third-party trademarks: AMD and AMD Opteron are trademarks of Advanced Micro Devices, Inc. Macintosh, Mac, Mac OS, and Xcode are trademarks of Apple Inc., registered in the U.S. and other countries. Embarcadero, the Embarcadero Technologies logos and all other Embarcadero Technologies product or service names are trademarks, service marks, and/or registered trademarks of Embarcadero Technologies, Inc. and are protected by the laws of the United States and other countries. Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company. HP and HP-UX are registered trademarks of the Hewlett-Packard Company. AIX, IBM, POWER6, POWER7, and pseries are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Intel, Intel Core, Itanium, Pentium and Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Microsoft, the.net logo, the Windows logo, Access, Excel, SQL Server, Visual Basic, Visual C++, Visual C#, Visual Studio, Windows, Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Novell and SUSE are registered trademarks of Novell, Inc. in the United States and other countries. Oracle and Java are registered trademarks of Oracle and/or its affiliates. QNX and Neutrino are registered trademarks of QNX Software Systems Ltd. in certain jurisdictions. CentOS, Red Hat, and the Shadow Man logo are registered trademarks of Red Hat, Inc. in the United States and other countries, used with permission. UNIX and UnixWare are registered trademarks of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Python and PyCon are trademarks or registered trademarks of the Python Software Foundation. OpenServer is a trademark or registered trademark of Xinuos, Inc. in the U.S.A. and other countries. Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the United States and other countries. Btrieve is a registered trademark of Actian Corporation. ACUCOBOL-GT, MICRO FOCUS, RM/COBOL, and Visual COBOL are trademarks or registered trademarks of Micro Focus (IP) Limited or its subsidiaries in the United Kingdom, United States and other countries. iscobol and Veryant are trademarks or registered trademarks of Veryant in the United States and other countries. All other trademarks, trade names, company names, product names, and registered trademarks are the property of their respective holders. Portions Copyright Unicode, Inc. All rights reserved. Portions Copyright The OpenSSL Project. All rights reserved. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit ( Portions Copyright Eric Young (eay@cryptsoft.com). All rights reserved. This product includes cryptographic software written by Eric Young (eay@cryptsoft.com). This product includes software written by Tim Hudson (tjh@cryptsoft.com). Portions Dharma Systems, Inc. All rights reserved. This software or web site utilizes or contains material that is DUNDAS DATA VISUALIZATION, INC. and its licensors, all rights reserved. Portions Copyright Jean-loup Gailly and Mark Adler. 12/13/2017

4 Contents 1. c-treertg Ready-to-Go Products Documentation Overview Key Benefits of c-treertg COBOL Edition c-treertg COBOL Edition Quick Start ACUCOBOL-GT Setup Recompiling the Runtime... 5 Recompiling the Windows Runtime... 5 Recompiling the Unix Runtime... 6 Adding Support for --setenv Command-Line Argument to Runtime... 8 Troubleshooting Configuring the Runtime for ACUCOBOL-GT CTREE_LIB Environment Variable ACUCOBOL-GT Environment Variables Enabling XDDOPEN, XDDCHECK, and XDDCLOSE for Programmatic SQL Conversion Error Checking Converting ACUCOBOL Data to c-treertg COBOL Edition Data Micro Focus COBOL Setup Configuring the Runtime for Micro Focus Dynamic Redirection Recompiling Your Application (Optional) Using the CALLFH Compiler Directive Specifying c-tree as Indexed File Handler at Link Time Configuration Note for Micro Focus on 64-bit AIX iscobol Setup Configuring the Runtime for iscobol Troubleshooting RM/COBOL Setup Installing the RM/COBOL Driver Installing the RM/COBOL Driver on Windows Installing the RM/COBOL Driver on Linux Adjusting the RM/COBOL Configuration File Copying the RM Library to the Local Folder All Rights Reserved iv

5 c-treertg Ready-to-Go Products 6.4 Adjusting the Paths Multiple File Systems with RM/COBOL Additional Documentation c-treertg Server Setup c-treertg for Windows c-treertg for Unix/Linux Shared Memory for c-treertg Runtime Configuration Configure the c-treertg Server c-treertg Configuration c-treertg Configuration Tool - RTG Config Creating a New File (Basic) Creating a New File (Advanced) Editing a Configuration File Support for Encrypting the Configuration File CTREE_CONF Environment Variable - COBOL CTREE_CONF_DUMP environment variable to specify configuration dump file Data Conversion RTG Migrate ctmigra Using the ctmigra utility Micro Focus COBOL / BTRV Migration Example c-treertg SQL Access Creating an XDD from an XFD Creating an XDD from the COBOL Source Creating an XDD Manually Storing the XDD in the Data File and Linking to the SQL Dictionary - COBOL Defining External Rules <XFDrules> root element <rule> XFDRules element <when> rule element <[Condition]> when elements <do> rule element All Rights Reserved v

6 c-treertg Ready-to-Go Products <[Action]> do elements <[Target]> action element Rule Examples Type Mapping Table Variable-length fields mapped into LONGVAR* SQL field Troubleshooting Data Conversion Errors Viewing Sqlized Tables in c-treeace SQL Explorer Adding SQL Indices to Sqlized Files API for SQL Conversion Error Checking SQL Conversion for ACUCOBOL Users XDDCHECK Errors Background Information about Sqlizing Record Structure Definition: The XDD Data Conversion Considerations SQL Considerations Tips for Advanced Sqlizing Step-by-Step Sqlizing Instructions xddgen Techniques Using Group Names Splitting an OCCURR Combining Multiple XDD Directives Name Conflicts HIDDEN Directive Multi-Record Example Source Code SQLIZEEXAMPLE.CBL CARDFILE.FD CARDFILE.SL rules.xml Utilities: ctutil, ctcbtran, xddgen, cttrnmod, ctmigra, ctfileid ctutil ctutil Notes ctutil Commands addimg check checkimg clone compact compress All Rights Reserved vi

7 c-treertg Ready-to-Go Products -conv copy cryptconf filecopy getimg info make makeimg load loadtext profile rblimg rebuild remove rename run 130 -segment setpath sign sqlcheck sqlinfo sqllink sqlunlink sqlize test tron uncompress unload unloadtext upgrade xfd2xdd ctcbtran xddgen XDD Directives Syntax for WITH DUPLICATES on RECORD KEY xddgen Configuration File Suppress dash or replace with underscore Configuration files directory cttrnmod - Change Transaction Mode Utility ctfileid - Update File IDs Copying Server-Controlled Files ctclntrn and cthghtrn utilities for managing transaction mark numbers All Rights Reserved vii

8 c-treertg Ready-to-Go Products 12.6 ct_tpc and cttpca TPC A Test Additional Command-Line Tools A. Details about the File System and SQL A.1. c-treertg File System Details A.2. The SQL Challenge B. Configuration File Elements B.1. c-treertg Configuration File B.2. Structure Elements <config> <instance> <redirinstance> <file> B.3. Settings Elements <automkdir> <batchaddition> <bulkaddition> <ctfixed> <datacompress> <datafilesuffix> <detectlock> <encrypt> <filecopy> <filepool> <hugefile> <ignorelock> <indexfilesuffix> <keycheck> <keycompress> <log> <locktimeout> <locktype> <maxlencheck> <map> <memoryfile> <normalize> <optimisticadd> <prefetch> <retrylock> <runitlockdetect> <skiplock> All Rights Reserved viii

9 c-treertg Ready-to-Go Products <smartcopy> <sqlize> <temporary> <transaction> <trxholdslocks> <writethru> B.4. Substitution Specifiers C. XDD File Schema C.1. <table> root element C.2. <key> table element C.3. <part> key element C.4. <segment> key element C.5. <filters> table element C.6. <field> filters element C.7. <filter> filters element C.8. <[Operator]> filter elements C.9. <field> operator element C.10. <value> operator element C.11. <schema> table element C.12. <field> schema element D. c-treertg COBOL Edition Directories E. Sample Programs E.1. The Samples Directory E.2. iscobol Samples F. Connecting to the c-treertg Server G. Transaction Processing Notes H. Performance Tuning Performance Monitoring with ctstat Additional Monitoring Tools Performance Tips I. Logging, Error Codes, and Troubleshooting I.1. Improved log output with file names on ERROR entries All Rights Reserved ix

10 c-treertg Ready-to-Go Products I.2. Driver Error Codes Error Codes I.3. c-treertg SQL Access Errors I.4. Troubleshooting Error Error 408 / Client/Server Incompatibility File Matching Rules in ctree.conf iscobol Fails to Run cobol_tutorial READ NEXT at End of File Index All Rights Reserved x

11 FairCom Typographical Conventions Before you begin using this guide, be sure to review the relevant terms and typographical conventions used in the documentation. The following formatted items identify special information. Formatting convention Bold CAPITALS FairCom Terminology FunctionName() Parameter Code Example utility filename CONFIGURATION KEYWORD CTREE_ERR Type of Information Used to emphasize a point or for variable expressions such as parameters Names of keys on the keyboard. For example, SHIFT, CTRL, or ALT+F4 FairCom technology term c-treeace Function name c-treeace Function Parameter Code example or Command line usage c-treeace executable or utility c-treeace file or path name c-treeace Configuration Keyword c-treeace Error Code

12 All Rights Reserved xii

13 1. c-treertg Ready-to-Go Products The c-treertg product line leverages the technology of the c-treeace Advanced Core Engine. Before explaining the details of installation and configuration, this guide will describe the features common to the c-treertg products. Because the c-treertg products are designed as direct replacements for the native file systems, they require little or no modifications to your application. Note: c-treertg V2 is based on the latest version of the powerful c-treeace Server. The version numbers returned from utilities may reflect the underlying core version number of this server, which is currently V11.x. How c-treertg Empowers Your Applications The c-treertg products allow you to enhance your existing COBOL applications with little or no development. The c-treertg file system allows direct access to your files. By replacing the native COBOL file system with a specialized version of the c-treeace Advanced Core Engine, c-treertg brings many of the benefits of c-treeace to your COBOL applications: Transaction Processing Client/Server Architecture, Sophisticated Caching & Index Compression Multi-User Performance Cross Platform/Multi-Platform Portability All Rights Reserved 1

14 c-treertg Ready-to-Go Products Dynamic Backups Granular Cache Support Memory Files Minimal Resource Requirements Developer to Developer Support SQL Access to Your Data To take advantage of the complete set of features available to c-treeace developers, you may need to use the c-treeace Professional SDK in addition to your COBOL application development. This allows you to use industry-standard APIs such as.net, Java or C/C++. Contact FairCom to explore using c-treeace Professional. 1.1 Documentation Overview This manual contains the following chapters to help you get up and running with your c-treertg product: Begin by performing the setup procedures for the COBOL compiler you are using: ACUCOBOL-GT Setup (page 5) Micro Focus COBOL Setup (page 14) iscobol Setup (page 17) RM/COBOL Setup (page 21) Read the following chapters to get your server set up and configured: c-treeace Server Setup (page 27) c-treertg Configuration (page 30) To migrate your data into c-treertg, read this chapter: Data Conversion (page 42) If you are providing SQL access, read these chapters: c-treertg Data: SQL Access (page 52) Tips for Advanced Sqlizing (page 89) A variety of utilities for migrating and other functions are found here: Utilities: ctutil, ctcbtran, xddgen, cttrnmod, ctmigra, ctfileid (page 103) Useful samples are found here: Samples (page 246) Additional useful information can be found in these chapters: Troubleshooting (page 259) Logging & Error Codes (page 255) All Rights Reserved 2

15 c-treertg Ready-to-Go Products 1.2 Key Benefits of c-treertg COBOL Edition c-treertg provides many features for the application developer. The actual features available may be limited based on the particular license you have purchased. The complete list includes: Support for ACUCOBOL-GT Version and later and iscobol, including the "I$IO" call. Support for Micro Focus COBOL and other COBOL dialects through the standard Callable File Handler interface (ExtFH). No need to recompile your application. Integrated as an additional file system. The file and path management is transparent for both Vision and c-treertg files. Details for handling c-treertg files are specified in the c-treertg configuration file. See c-treertg File System Details (page 162) in the c-treertg User's Guide. c-treertg and other file handlers may co-exist in the same application allowing on-the-fly data conversion from one file system to another. Provides a ctutil utility that implements the same functionality as Vision's vutil utility as well as importing/exporting ASCII files created with the Btrieve butil utility. Native COBOL record support with no SQL record buffer conversion which can slow down your applications. SQL support for native data records, including tables that have multiple schema definitions (redefined fields) in the same table. See Multiple Record Virtual Tables in the c-treedb Developer's Guide. The cross-platform nature of the c-treertg Server, based on the c-treeace engine, makes it easy to provide client/server support across multiple platforms including Windows, Linux, Solaris, Mac OS X, AIX, HP-UX and others. The c-treertg Server includes industrial-quality on-line transaction processing ( (OLTP) features that guarantee ACID (atomicity, consistency, isolation, durability) properties of transactions that are transparently (to the application) activated on any table. All Rights Reserved 3

16 2. c-treertg COBOL Edition Quick Start This section is designed to get you up and running in a hurry. The first steps establish c-treertg as your file handler. After completing these steps, your applications will have the benefits of performance and stability provided by c-treertg; the final step adds SQL access to your data: 1. File Handler Installation c-treertg Server Setup (page 27) c-treertg Configuration (page 30) Data Conversion (page 42) SQL Setup (page 52) The final step is optional. It is needed if you want SQL access to your COBOL data. The first step is to install the file handler for the type of COBOL you are using: ACUCOBOL-GT Setup (page 5) Micro Focus COBOL Setup (page 14) iscobol Setup (page 17) RM/COBOL Setup (page 21) The next step is to install the c-treertg server (page 27). Now it is time to configure c-treertg. The c-treertg configuration file allows considerable flexibility to handle even complex environments, so a graphical Configuration Tool (page 31) is provided to simplify the process. Before your application can use c-treertg, the data must be converted. c-treertg provides GUI and command-line tools for data conversion (page 42). Define the record schema of your data in an XDD file and store it in the data file. See c-treertg SQL Access (page 52). All Rights Reserved 4

17 3. ACUCOBOL-GT Setup This chapter describes how to configure your ACUCOBOL-GT installation to use c-treertg COBOL Edition as file handler. 3.1 Recompiling the Runtime ACUCOBOL-GT requires the runtime to be compiled with the c-treeace configuration enabled. The following steps will guide you through this task. Recompiling the Windows Runtime Linking the ACUCOBOL-GT runtime with the c-treeace client libraries requires a suitable compiler installed: ACUCOBOL-GT 6.X requires Microsoft Visual Studio 6. ACUCOBOL-GT 7.X and V8X require Microsoft Visual Studio ACUCOBOL-GT 9.X requires Microsoft Visual Studio Follow these procedures: 1. Copy the c-tree module ctreeacu.c into the directory that contains the ACUCOBOL-GT libraries, for example: copy win32\driver\ctree.cobol\acucobol\ctreeacu.c C:\AcuGT\lib 2. Copy the c-tree utilities and DLLs ctutil.exe and mtclient.dll into the directory that contains the ACUCOBOL-GT binaries, for example: copy win32\driver\ctree.cobol\acucobol\mtclient.dll C:\AcuGT\bin copy win32\driver\ctree.cobol\acucobol\ctutil.exe C:\AcuGT\bin 3. Open the VC++ project located in the ACUCOBOL-GT lib directory and edit the wrundll project by adding ctreeacu.c. To add the files, click Project from the menu, select Add to Project and select item Files. Locate the ACUCOBOL-GT lib directory and select ctreeacu.c. ACUCOBOL-GT 6.X: Open the Visual Studio 6 workspace: wrun32.dsw ACUCOBOL-GT 8.X: Open the Visual Studio 2005 solution: wrun32.sln ACUCOBOL-GT 9.X: Open the Visual Studio 2008 solution: wrun32.sln 4. Edit the ACUCOBOL-GT file system configuration file filetbl.c located in the ACUCOBOL-GT lib directory, for example: C:\AcuGT\lib. All Rights Reserved 5

18 ACUCOBOL-GT Setup There are three entries to modify: a. The original filetbl.c contains the entry: #ifndef USE_VISION #define USE_VISION 1 #endif Add the following additional define section: #ifndef USE_CTREE #define USE_CTREE 1 #endif b. The original filetbl.c contains the entry: extern DISPATCH_TBL v5_dispatch,...; Add the following extern definition: extern DISPATCH_TBL ct_dispatch; c. The original filetbl.c contains the entry: TABLE_ENTRY file_table[] = { #if USE_VISION { &v5_dispatch, "VISIO" }, #endif /* USE_VISION */ Add the following additional define section: #if USE_CTREE { &ct_dispatch, "CTREE" }, #endif /* USE_CTREE */ 5. Re-build the ACUCOBOL-GT runtime by building the wrun32.dll library. To build this library, choose Build from the menu and select Build wrun32.dll. 6. Copy the new wrun32.dll to the directory that contains the ACUCOBOL-GT binaries, for example C:\AcuGT\bin. 7. Verify the link by running wrun32 vv. This will return version information on all of the products linked into your runtime system. Ensure it reports the version of the c-tree interface. For example: FairCom c-treeace version 11.0.xxxxx-yymmdd file system (interface v11.0.xxxxx-yymmdd) Recompiling the Unix Runtime 1. Copy the c-tree module ctreeacu.c into the directory that contains the ACUCOBOL-GT libraries, for example: cp linux.v2.6.x86.32bit.cobol/driver/ctree.cobol/acucobol/ctreeacu.c /usr/acucbl810/acugt/lib 2. Copy the c-tree utility ctutil and the libmtclient.so library into the directory that contains the ACUCOBOL-GT binaries: All Rights Reserved 6

19 ACUCOBOL-GT Setup cp linux.v2.6.x86.32bit.cobol/driver/ctree.cobol/acucobol/libmtclient.s o /usr/acucbl810/acugt/bin cp linux.v2.6.x86.32bit.cobol/driver/ctree.cobol/acucobol/ctutil /usr/acucbl810/acugt/bin 3. Edit the ACUCOBOL-GT file system configuration file filetbl.c located in the ACUCOBOL-GT lib directory, for example: /usr/acucbl810/acugt/lib. There are three entries to modify: a. The original filetbl.c contains the entry: #ifndef USE_VISION #define USE_VISION 1 #endif Add the following additional define section: #ifndef USE_CTREE #define USE_CTREE 1 #endif b. The original filetbl.c contains the entry: extern DISPATCH_TBL v5_dispatch,...; Add the following extern definition: extern DISPATCH_TBL ct_dispatch; c. The original filetbl.c contains the entry: TABLE_ENTRY file_table[] = { #if USE_VISION { &v5_dispatch, "VISIO" }, #endif /* USE_VISION */ Add the following additional define section: #if USE_CTREE { &ct_dispatch, "CTREE" }, #endif /* USE_CTREE */ NOTE: The c-tree entry must be added below the Vision entry so as to leave the Vision entry the first one in the file_table[] array. 4. Open the file Makefile located in the ACUCOBOL-GT lib, for example: /usr/acucbl810/acugt/lib. Add the c-treertg ACUCOBOL source module to the line FSI_SUBS=, for example: FSI_SUBS= <... > ctreeacu.c Note: There may be other items already listed in FSI_SUBS. Leave those in place and only add ctreeacu.c. 5. Re-build the ACUCOBOL-GT runtime by running make f Makefile. All Rights Reserved 7

20 ACUCOBOL-GT Setup 6. Copy the new runcbl file to directory that contains the ACUCOBOL-GT binaries, for example /usr/acucbl810/acugt/bin. This file needs to have the execute permission set for everyone who will be using the runtime system. 7. Verify the link by running runcbl vv. This will return version information on all of the products linked into your runtime system. Ensure it reports the version of the c-tree interface. For example: FairCom c-treeace version 11.x.xxxxx-yymmdd file system (interface v11.x.xxxxx-yymmdd) Adding Support for --setenv Command-Line Argument to Runtime The ACUCOBOL-GT product allows the user to introduce new command line arguments that can be passed to the ACUCOBOL-GT runtime. The function exam_args() defined in ACUCOBOL-GT's interface to C module sub.c is called immediately upon startup and is passed the command line arguments that were passed to the runtime. The c-tree module ctreeacu.c contains the function ct_exam_args() that provides support for --setenv command-line argument. The --setenv command-line argument can be used to set environment variables once the ACUCOBOL-GT runtime has already been started. To add support for --setenv command-line argument, edit the sub.c module located in the ACUCOBOL-GT lib directory, for example: /usr/acucbl810/acugt/lib. 1. Add the ct_exam_args() prototype declaration before exam_args() definition as follows: int ct_exam_args(int argc, char *argv[]); int exam_args(int argc, char *argv[]) { return 0; } /* exam_args */ 2. Change exam_args() to call ct_exam_args() function as follows: int exam_args(int argc, char *argv[]) { return ct_exam_args(argc, argv); } /* exam_args */ 3. Re-compile the ACUCOBOL-GT runtime as described in Recompiling the Windows Runtime (page 5) or Recompiling the Unix Runtime (page 6). Once the ACUCOBOL-GT runtime supports the --setenv command line argument, you can call the runtime passing --setenv:variable=value, for example: All Rights Reserved 8

21 ACUCOBOL-GT Setup Windows wrun32.exe cobol_tutorial1.acu --setenv:default_host=ctree Unix runcbl cobol_tutorial1.acu --setenv:default_host=ctree Troubleshooting Common errors and relative solutions are described in the following sections. Error message: "libctclient.so: cannot open shared object file" If you receive an error such as: runcbl: error while loading shared libraries: libctree.so: cannot open shared object file: No such file or directory Make sure you have the environment variable LD_LIBRARY_PATH (platform specific) set appropriately to access the c-treeace library, for example: LD_LIBRARY_PATH=/usr/acucbl610/acugt/bin; export LD_LIBRARY_PATH All Rights Reserved 9

22 ACUCOBOL-GT Setup 3.2 Configuring the Runtime for ACUCOBOL-GT This section describes how to configure the recompiled ACUCOBOL-GT runtime to use c-treertg. CTREE_LIB Environment Variable Specifies the full path of the c-tree library that ACUCOBOL-GT attempts to load. If CTREE_LIB is not defined then a library named mtclient.dll (Windows) or libmtclient.so (Unix) is searched in the path. Windows set CTREE_LIB=C:\FairCom\V11.x.x.COBOL\win32\Driver\ctree.cobol\ACUCOBOL\mtclient.dll Unix CTREE_LIB=/FairCom/V11.x.x.COBOL/linux.v2.6.x86.32bit/Driver/ctree.cobol/ACUCOBOL/libmtclient.so export CTREE_LIB ACUCOBOL-GT Environment Variables ACUCOBOL-GT allows a program to interface with more than one external file system in the same program. To define a file system for use with a particular file, you need to set the filename_host configuration variable. To specify the indexed file system that the program will use by default, you need to set the DEFAULT_HOST configuration variable. For an introduction to ACUCOBOL-GT runtime configuration variables and the configuration file, see section 2.7, Runtime Configuration, in Book 1 of the ACUCOBOL-GT documentation set. DEFAULT_HOST DEFAULT_HOST specifies the default file system to be used for all file I/O. Setting DEFAULT_HOST to CTREE indicates usage of c-tree while setting it to VISION or leaving it unset indicates usage of Vision. For example, to make c-tree the default file system, set the environment variable DEFAULT_HOST as follows: Windows set DEFAULT_HOST=CTREE Unix DEFAULT_HOST=CTREE; export DEFAULT_HOST All Rights Reserved 10

23 ACUCOBOL-GT Setup filename_host This variable specifies the file system to use for a particular file. For example, if DEFAULT_HOST is set to VISION, and the file CUSTOMERS is a c-tree file, you could set the environment variable CUSTOMERS_HOST as follows: Windows set CUSTOMERS_HOST=CTREE Unix CUSTOMERS_HOST=CTREE; export CUSTOMERS_HOST This definition directs the runtime to treat CUSTOMERS as a c-tree file. Note that the file name may not include any path or directory name and should not include the file extension. DEFAULT_HOST and filename_host are described in detail in Appendix H, Configuration File Entries, in Book 4 of the ACUCOBOL-GT documentation set. SET ENVIRONMENT ACUCOBOL-GT Verb When your program executes, each time a file is opened, the ACUCOBOL-GT runtime checks filename_host and DEFAULT_HOST to determine which file system to use. You can change the value of these variables, just before you open the file, by including the following: SET ENVIRONMENT "DEFAULT_HOST" TO value or SET ENVIRONMENT "filename_host" TO value 3.3 Enabling XDDOPEN, XDDCHECK, and XDDCLOSE for Programmatic SQL Conversion Error Checking The XDDOPEN, XDDCHECK, and XDDCLOSE commands make it possible to programmatically check if there will be any conversion errors accessing a table from SQL (see API for SQL Conversion Error Checking (page 76) for details). This allows them to be called directly from ACUCOBOL without the necessity to load the mtclient.dll. To allow these commands to be called from ACUCOBOL, the following modifications are necessary: 1. Edit the ACUCOBOL-GT file system configuration file direct.c located in the ACUCOBOL-GT lib directory, for example: /usr/acucbl810/acugt/lib. 2. Search for the struct DIRECTTABLE LIBDIRECT[] array. 3. Just before the array found in step 1, add these three lines: extern void* ct_xddopen(); extern long ct_xddcheck(); extern void ct_xddclose(); 4. Within the array elements add the following: All Rights Reserved 11

24 ACUCOBOL-GT Setup { ""XDDOPEN"", FUNC ct_xddopen, C_pointer}, { ""XDDCHECK"", FUNC ct_xddcheck, C_long}, { ""XDDCLOSE"", FUNC ct_xddclose, C_void}, 5. Make sure the following line (which is already part of the array) is the last one: { NULL, NULL, 0 } After performing the above steps, XDDOPEN, XDDCHECK, and XDDCLOSE will be directly callable from the COBOL application. See Also API for SQL Conversion Error Checking (page 76) 3.4 Converting ACUCOBOL Data to c-treertg COBOL Edition Data This walkthrough describes how to convert existing COBOL data created with the ACUCOBOL Vision file system to c-treertg COBOL Edition data. If your COBOL program performs OPEN OUTPUT operations, set the file system to CTREE (see Runtime Configuration (page 28)) before running your program to create files in c-tree format. If you cannot create files with your COBOL application, ctutil can use your XFD files to create a new empty file having the same structure described by the XFD files. The following steps are used to convert ACUCOBOL Vision files into c-tree files. If you have already created the c-tree files you can jump directly to step 2: 1. To create an empty file using c-treertg COBOL Edition, execute ctutil -make passing the new file name and the XFD file containing its definition: ctutil -make myctdata mydata.xfd This command will create a file named myctdata using the image string obtained from the mydata.xfd. 2. Dump the existing data from the existing mydata data file created using Vision by using the vutil32 -unload functionality: vutil32 -unload mydata mydata.dmp This command will dump the entire contents of the mydata file to a plain text file called mydata.dmp 3. Load the dumped data into the file created in step 1 using the ctutil -load functionality: ctutil -load myctdata mydata.dmp A This command loads the data from the plain text file called mydata.dmp and writes it to the mydata c-treertg COBOL Edition data file. 4. Now the new c-treertg COBOL Edition data file mydata can be read through the c-treertg COBOL Edition SQL engine by running the following command: ctutil -sqlize myctdata mydata.xfd ADMIN ctreesql All Rights Reserved 12

25 ACUCOBOL-GT Setup This command parses the mydata.xfd file and generates an XDD schema definition on the fly to be used by the c-treertg COBOL Edition engine to import the data file as part of a SQL database. All Rights Reserved 13

26 4. Micro Focus COBOL Setup c-treertg COBOL Edition supports Micro Focus COBOL and other dialects through the standard Callable File Handler interface, ExtFH. Please refer to your COBOL runtime documentation to know how to configure your application to use external file handlers. 4.1 Configuring the Runtime for Micro Focus This section describes how to instruct the Micro Focus File Handler to dynamically load (page 14) and use the c-treertg COBOL Edition file handler. Alternatively, you can force programs to always use the c-treertg COBOL Edition file handler for processing all COBOL I/O by compiling your application as described in the section titled Compiling Your Application (Optional) (page 15). Dynamic Redirection At runtime set the DYNREDIR environment variable to point to the c-treertg COBOL Edition driver library as follows: Windows set DYNREDIR=CTEXTFH.dll!CTEXTFH Unix export DYNREDIR=CTEXTFH.so When using Micro Focus Visual COBOL for Linux a path can be specified for the DYNREDIR variable. export DYNREDIR=/usr/local/lib/CTEXTFH.so When using DYNREDIR with Micro Focus Visual COBOL it is possible to use redirinstance (page 169) without specifying the library and the func. In this case, the c-treertg interface will signal to the COBOL runtime that it does not handle the files and the runtime falls back to the next filesystem specified by DYNREDIR. The client library for Micro Focus is available inside the Driver\ctree.cobol\EXTFH\ folder. All Rights Reserved 14

27 Micro Focus COBOL Setup 4.2 Recompiling Your Application (Optional) There are two methods to instruct Micro Focus COBOL to use the c-treertg COBOL Edition file handler: Using the CALLFH Compiler Directive (page 15) Specifying c-tree as Indexed File Handler at Link Time (page 15) Using the CALLFH Compiler Directive Compile your programs with the directive: $set callfh"ctextfh" Link the c-treeace CTEXTFH.lib library to your program objects. For example: COBOL d g myprogram.obj _CLASS+CTEXTFH+COBINTFN This causes all COBOL I/O operations to be compiled into calls to c-treertg COBOL Edition driver. Please be aware that the c-treertg COBOL Edition driver implements support only for indexed files. If your application uses other types of files, this approach is not viable. Please consider using the alternative Dynamic Redirection (page 14) approach instead. Specifying c-tree as Indexed File Handler at Link Time To link c-tree as indexed file handler with a COBOL program, you must specify the following flags on the cob command line: -m ixfile=ctextfh to specify ctree as file handler for fixed record length indexed files -m ixfilev=ctextfh to specify ctree as file handler for variable record length indexed files This ensures that the necessary external references to the ctree library are included in the resulting executable file, and that they are used instead of the default Micro Focus File Handler. Examples cob -x prog.cbl -m ixfile=ctextfh -m ixfilev=ctextfh CTEXTFH.so 4.3 Configuration Note for Micro Focus on 64-bit AIX When configuring c-treertg for Micro Focus on 64-bit AIX systems, you will need to use the following settings: The <redirinstance> (page 169) properties in the c-treertg configuration file, ctree.conf (page 163), should be set as follows: lib="libcobrts64.so" func="extfh" All Rights Reserved 15

28 Micro Focus COBOL Setup For example: <redirinstance lib="libcobrts64.so" func="extfh">... </redirinstance> An AIX environment variable must be set as follows: LDR_PRELOAD64=$COBDIR/lib/libcobrts64.so All Rights Reserved 16

29 5. iscobol Setup Veryant iscobol ships with a version of the c-treertg file system called c-treeace for iscobol (previously known as iscobol ISAM Server ). This file system can be upgraded to the full version of c-treertg COBOL Edition. c-treertg provides a dynamic link library for supporting iscobol. This library allows the c-treertg database engine to handle your program s files. This chapter describes how to configure your iscobol installation to use c-treertg. 5.1 Configuring the Runtime for iscobol Upgrading to c-treertg c-treertg includes a client DLL and server software that correspond to the c-tree client and server that originally shipped with iscobol. Replace the corresponding iscobol programs with the c-treertg programs listed below: Windows Linux/Unix Folder (in the c-treertg distributable) Client (see note below) ctree.dll libctree.so Driver\ctree.cobol\iscobol\ Server Executable (see note below) ctreesql.exe ctreesql Server\sql Server Library ctreedbs.dll ctreedbs.so Server\sql Note: If you are using iscobol ACE Bound Server configuration in conjunction with the iscobol Application Server, you do not need to replace the Client or the Server Executable. Setting the Configuration File The iscobol.properties file (located in the etc folder) configures iscobol. In addition to setting iscobol properties, this file allows you to configure a considerable number of c-treertg properties. As an alternative to using iscobol.properties to configure c-treertg, you can use an external ctree.conf (page 163) file, which provides more configuration options. The iscobol.properties file contains an entry, iscobol.ctree.new_config, which allows you to specify which configuration file to use for c-treertg: iscobol.ctree.new_config=true - (Default) Use the c-treertg properties specified in the iscobol.properties file. This mode is required if you are using the file system that ships with iscobol. Note: With this setting, you will not use a ctree.conf file for configuration. All Rights Reserved 17

30 iscobol Setup iscobol.ctree.new_config=false - Use the c-treertg properties specified outside of the iscobol.properties file in ctree.conf. This mode is used if you are using the c-treertg Server (page 27). Note: The c-treertg properties specified in the iscobol.properties file will not be used for configuration. Note: The c-treertg Configuration Tool (page 31) provides an easy, graphical interface for configuring c-treertg. The Configuration Tool uses the ctree.conf file, so you must set iscobol.ctree.new_config=false to use the tool. See the section titled c-treertg Configuration (page 30) for more about using ctree.conf. Setting the Default File System iscobol allows a program to interface with more than one external file system. It provides a mechanism for specifying a default file system and a way to override it on a file-by-file basis so you can specify a different file system for each file. Using Configuration Variables Two configuration variables are provided for configuring the file system: iscobol.file.index - To specify the indexed file system that the program will use by default. iscobol.file.index.filename - To define a file system for use with a particular file (.filename). This setting will override the default file system for the file specified. For an introduction to iscobol runtime configuration variables and the configuration file, see Configuration in User s Guide of the iscobol documentation set. iscobol.file.index The iscobol.file.index configuration variable specifies the default file system to be used for file I/O. Setting this variable to ctree2 sets the file system to c-tree; setting it to jisam (or leaving it unset) sets the file system to JISAM. For example, to make c-tree the default file system, set the environment variable iscobol.file.index in one of the following ways: In the iscobol configuration file (usually iscobol.properties): iscobol.file.index=ctree2 In the iscobol runtime command line: iscrun J-Discobol.file.index=ctree2 PROGRAMNAME In the system environment on Windows: SET file.index=ctree2 In the system environment on Unix: file.index=ctree2 export file.index All Rights Reserved 18

31 iscobol Setup iscobol.file.index.filename The iscobol.file.index.filename configuration variable specifies the file system to use for a specified file. It overrides the default file system for the filename indicated. For example, if the default file system is jisam and the file customers is a c-tree file, you would set the environment variable iscobol.file.index.customers as follows: In iscobol configuration file (usually iscobol.properties): iscobol.file.index.customers=ctree2 In iscobol runtime command line: iscrun J-Discobol.file.index.customers=ctree2 PROGRAMNAME In system environment, Windows: SET file.index.customers=ctree2 In system environment, Unix: file.index.customers=ctree2 export file.index Using the SET ENVIRONMENT Verb When your program executes, each time a file is opened, the iscobol runtime checks iscobol.file.index.filename and iscobol.file.index to determine which file system to use. You can change the value of these variables before you open the file by including the following: SET ENVIRONMENT "file.index" TO value or SET ENVIRONMENT "file.index.filename" TO value Using the CLASS Clause iscobol allows you to associate a file handler in the logical file declaration in the program source code. The specified file handler will be used for that logical file regardless of the physical name of the file. The following file declaration makes FILE1 a c-tree file regardless of the value you may assign to FILE1-NAME before opening FILE1. SELECT FILE1 ASSIGN TO FILE1-NAME ORGANIZATION INDEXED CLASS "com.iscobol.io.dynamicctree2" ACCESS DYNAMIC RECORD KEY FILE1-KEY. Setting iscobol to Use c-treertg To set iscobol to use c-treertg: 1. Replace the client library as described earlier. 2. Add the following line to the iscobol.properties file (located in etc folder) to force it to use an external configuration file (ctree.conf): All Rights Reserved 19

32 iscobol Setup iscobol.ctree.new_config=false False allows ctree.conf to be used; all c-treertg properties in the iscobol.properties file will be ignored. 3. Put the iscobol.properties file in the same directory as the COBOL program, or set the PATH environment variable to include it, so the iscobol runtime can access it. 4. Make sure the iscobol runtime is not controlling versions between client and server. Add versioncheck="no" in ctree.conf as a property: <instance server="faircoms@ " versioncheck="no"> 5. To test if iscobol is properly set, run the following: iscrun -J-Discobol.file.index=ctree2 com.iscobol.rts.version You will see a message that references the version of FairCom being used: iscobol release 2012 R2 build# DB FairCom c-treeace;x.x.x.xxxx-xxxxxx;x.x.xxxxx-xxxxxxc/s Version 0020 License info VERYANT##303564/ Troubleshooting Common errors and their solutions are described below. Error message: java.lang.unsatisfiedlinkerror If you receive an error such as: Exception in thread "main" java.lang.unsatisfiedlinkerror: no ctree in java.library.path make sure you have the environment variable LD_LIBRARY_PATH (platform specific) set appropriately to access the c-treeace library, for example: LD_LIBRARY_PATH=/opt/isCOBOL/native/lib; export LD_LIBRARY_PATH iscobol Fails to Run cobol_tutorial1 The tutorial will fail if you have the following setting in iscobol.properties: iscobol.io_creates=1 If you ran the tutorial with iscobol.io_creates=1 enabled, comment out that setting and delete the custmast.dat and custmast.idx files. All Rights Reserved 20

33 6. RM/COBOL Setup c-treertg COBOL Edition supports COBOL runtimes that use Btrieve as their underlying file system by including support for Btrieve commands in the c-treertg COBOL Edition driver. This support allows programs written in RM/COBOL to handle files using c-treertg. This chapter describes how to configure your RM/COBOL installation to use c-treertg COBOL Edition as the file handler. 6.1 Installing the RM/COBOL Driver RM/COBOL typically calls a Btrieve library. To use the new c-treertg Btrieve functions, you will need to replace the Btrieve library with the c-treertg COBOL Edition library, as described below. Installing the RM/COBOL Driver on Windows The Windows driver is located at: Driver\ctree.cobol\rm-cobol\mtclient.dll To install the driver for RM/COBOL on systems running Windows: 1. Make a backup copy of the original Btrieve library WBTRV32.DLL and save it for safekeeping. 2. Rename the FairCom c-treertg library with the same name as the original Btrieve library (typically WBTRV32.DLL). 3. Be sure the FairCom c-treertg library has precedence over the original Btrieve library in the PATH environment variable. Installing the RM/COBOL Driver on Linux The Linux driver is located at: driver\ctree.cobol\rm-cobol\libmtclient.so To install the RM/COBOL driver on systems running Linux: 1. Make a backup copy of the original Btrieve library libpsqlmif.so.8 and save it for safekeeping. Hint: This may be found in two locations. Look in /usr/local/psql/lib as well as /usr/rmcobol. 2. Rename the c-treertg library with the same name as the original Btrieve library (libpsqlmif.so.8). 3. Be sure the c-treertg library has precedence over the original Btrieve library in the LD_LIBRARY_PATH environment variable. All Rights Reserved 21

34 RM/COBOL Setup 6.2 Adjusting the RM/COBOL Configuration File The RM/COBOL-to-Btrieve Adapter program for Linux, librmbtrv.so (Linux) or WBTRV32.DLL (Windows), is initiated by placing the following configuration record in the RM/COBOL configuration file (see the EXTERNAL-ACCESS-METHOD configuration record for more information on specifying keywords) and starting the RM/COBOL runtime system: EXTERNAL-ACCESS-METHOD NAME=RMBTRV Notes: 1) Version 7.1 and later of RM/COBOL for Unix and version 7.5 and later of RM/COBOL for Windows allow a configuration file to be located automatically by the RM/COBOL runtime system, the compiler, and the recovery utility. If the Automatic Configuration File module, librmconfig.so (UNIX) or librmcfg.dll (Windows), is present in the RM/COBOL execution directory, it will be loaded and it will attempt to locate an automatic configuration file. The execution directory on UNIX is normally /usr/bin; on Windows it is normally C:\Program Files\RMCOBOL. 2) If you want a configuration file to by loaded automatically, you need a file named runcobol.cfg (for the runtime system), rmcobol.cfg (for the compiler), or recover1.cfg (for the recovery utility) in the execution directory. If the appropriate file is present, the records in the file will be used to configure the component being executed. 3) The Windows version of RM/COBOL allows configuration files to be physically attached to rmcobol.exe, runcobol.exe, and recover1.exe using the Attach Configuration utility (rmattach.exe). The attached configuration will be processed prior to a configuration file specified by a command-line option. If automatic configuration finds a configuration file, an attached configuration is ignored. Configuration Options The following option designates a file to be used as the complete runtime configuration or as a supplement to it and allow suppression of the banner and STOP RUN messages. Use the C Option to designate a file to be used as the runtime configuration. If the C Option is specified, any attached or automatic configuration is ignored (not processed). The C Option has the following format: C=pathname 6.3 Copying the RM Library to the Local Folder The RM library must be copied to the local folder where you are running the RM/COBOL runtime. Place the RM library (for Linux: librmbtrv.so) in the same directory as the RM/COBOL runtime system (runcobol). Typically, the support module is copied into the execution directory (for Linux: /usr/rmcobol, /usr/local/bin, or /usr/bin). All Rights Reserved 22

35 RM/COBOL Setup 6.4 Adjusting the Paths Linux must be able to locate the various executable files that are required. For this support module to be loaded properly, you must make sure that you have set the LD_LIBRARY_PATH environment variable. Add the directory that contains the Pervasive libraries, DSOs (dynamic shared objects), to LD_LIBRARY_PATH. For example: export LD_LIBRARY_PATH=/usr/local/psql/lib:/usr/lib Note that if you logged into the system as "psql" these paths will have already been set. To verify that the shared object, librmbtrv.so, is being loaded properly by the RM/COBOL runtime, type the following from the shell command line (for more information about the V Option, see Configuration Runtime Command Options): runcobol xxx -v If the following line is displayed in the RM/COBOL runtime banner, the RM/COBOL-to-Btrieve Adapter for Linux has been loaded correctly: $EXEDIR/librmbtrv.so - RM/COBOL Btrieve Adapter - Version n.nn.nn. 6.5 Multiple File Systems with RM/COBOL The demonstration below shows that it is possible to have an RM/COBOL application that opens some files with the default file system and some files with c-treertg WITHOUT any special configuration of c-treertg. The attached sample program creates two files: custmast and itemmast. Run it with RM/COBOL to create these files. $ runcobol cobol_tutorial2 INIT DEFINE Create table CustomerMaster... Add records in table CustomerMaster... Create table ItemMaster... Add records in table ItemMaster... MANAGE Display records Bryan Williams 1001 Michael Jordan 1002 Joshua Brown 1003 Keyon Dooling 1 Hammer 2 Wrench 3 Saw All Rights Reserved 23

36 RM/COBOL Setup 4 Pliers DONE Press <ENTER> key to exit... ls *mast* custmast itemmast At this point you have an RM/COBOL application that opens multiple files, however you can use c-treertg only on one file. Create a runcobol.cfg file in the execution directory as follows: echo EXTERNAL-ACCESS-METHOD NAME=RMBTRV32 CREATE-FILES=YES OPTIONS='T=RMBTRV32.log' >runcobol.cfg Install the c-treertg library: cp whatever/libmtclient.so./libpsqlmif.so.8 export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH Configure c-treertg to enable logging so we can see that it is working: echo \<config\>\<log/\>\</config\> >ctree.conf $ cat ctree.conf <config><log/></config> Start the c-treertg server and then re-run the program to check that c-treertg is called by noticing the log output with the "missing file" errors (15:12:2): $ runcobol cobol_tutorial2 INIT DEFINEF7FDFAC0> T api:0324:ctl_init INFO client version: id:34 F7FDFAC0> T api:4473:ctl_setins INFO server version: id:34 F7FDFAC0> T core:3091:cts_open ERROR 15:12:2 OPNRFIL(/home/fctech/rtg.rm/160707/bin.mtc32bitdynamic/custmast.dat,129) F7FDFAC0> T core:3091:cts_open ERROR 15:12:2 OPNRFIL(/home/fctech/rtg.rm/160707/bin.mtc32bitdynamic/itemmast.dat,129) MANAGE Display records Bryan Williams 1001 Michael Jordan 1002 Joshua Brown 1003 Keyon Dooling 1 Hammer 2 Wrench All Rights Reserved 24

37 RM/COBOL Setup 3 Saw 4 Pliers DONE Press <ENTER> key to exit... Now remove the custmast file and re-run the program to re-create the custmast.dat file with c-treertg (notice the "Create table CustomerMaster..." message): $ rm custmast $ runcobol cobol_tutorial2 INIT DEFINEF7F0DAC0> T api:0324:ctl_init INFO client version: id:34 F7F0DAC0> T api:4473:ctl_setins INFO server version: id:34 F7F0DAC0> T core:3091:cts_open ERROR 15:12:2 OPNRFIL(/home/fctech/rtg.rm/160707/bin.mtc32bitdynamic/custmast.dat,129) Create table CustomerMaster...F7F0DAC0> T core:3091:cts_open ERROR 15:12:2 OPNRFIL(/home/fctech/rtg.rm/160707/bin.mtc32bitdynamic/custmast.dat,129) F7F0DAC0> T core:3091:cts_open ERROR 15:12:2 OPNRFIL(/home/fctech/rtg.rm/160707/bin.mtc32bitdynamic/custmast.dat,129) Add records in table CustomerMaster...F7F0DAC0> T core:3091:cts_open 15:12:2 OPNRFIL(/home/fctech/rtg.rm/160707/bin.mtc32bitdynamic/itemmast.dat,129) ERROR MANAGE Display records Bryan Williams 1001 Michael Jordan 1002 Joshua Brown 1003 Keyon Dooling 1 Hammer 2 Wrench 3 Saw 4 Pliers DONE Press <ENTER> key to exit... Verify that the file is created with c-treertg (file name ends with.dat and.idx extension): $ ls custmast* custmast.dat custmast.idx Now re-run the program to check that it works with both the default file system and c-treertg: All Rights Reserved 25

38 RM/COBOL Setup $ runcobol cobol_tutorial2 INIT DEFINEF7EF3AC0> T api:0324:ctl_init INFO client version: id:34 F7EF3AC0> T api:4473:ctl_setins INFO server version: id:34 F7EF3AC0> T core:3091:cts_open ERROR 15:12:2 OPNRFIL(/home/fctech/rtg.rm/160707/bin.mtc32bitdynamic/itemmast.dat,129) MANAGE Display records Bryan Williams 1001 Michael Jordan 1002 Joshua Brown 1003 Keyon Dooling 1 Hammer 2 Wrench 3 Saw 4 Pliers DONE Press <ENTER> key to exit... Please notice above that c-treertg attempts to open itemmast.dat but it fails with a "missing file" error (15:12:2) so RM/COBOL falls back to use the default file system. You can see that itemmast was opened because the itemmast records are read and displayed (Hammer, Wrench, etc.) 6.6 Additional Documentation RM/COBOL User s Guide V7.5 for UNIX and Windows: Additional Notes on new editions of RM/COBOL V10.0: All Rights Reserved 26

39 7. c-treertg Server Setup 7.1 c-treertg for Windows The c-treertg installer for Windows creates a Microsoft Windows service that automatically starts the database engine at Windows startup. This allows for seamless and unattended operation. Use the standard Windows service control panels to start/stop and configure various service options. VSS Backup - c-treertg for Windows supports the Windows Volume Shadow Copy Service (VSS) Writer for backups. This support is supplied as a dynamic link library (c-treeacevsswriter.dll) that allows Windows VSS Backup to be used. See VSS Integration in the Technical White Papers on the FairCom website. 7.2 c-treertg for Unix/Linux The c-treertg tar archive for Unix/Linux contains a ready-to-run c-treeace database engine: cd /FairCom/Vx.x.x.RTG/<platform>/server/sql./ctreesql Feel free to include it along with your /etc/init.d startup scripts for fully unattended operation. Consult your local system administrator for details about specific machine configurations. 7.3 Shared Memory for c-treertg c-treertg supports the shared memory communication protocol between clients and servers residing on the same machine. Shared memory communication generally provides much better performance for locally running applications. Note: Shared memory support does not include Linux Kernels 2.4 or Mac OS X systems. The c-treeace shared memory communication protocol creates a file used by clients to find the shared memory identifier for its shared memory logon region, and creates a Unix domain socket as a file for initial communication between a client and server. Configuration Include the following server configuration in ctsrvr.cfg to enable this support: COMM_PROTOCOL FSHAREMM All Rights Reserved 27

40 c-treertg Server Setup c-treeace creates the directory /tmp/ctreedbs and the file /tmp/ctreedbs/<servername>.logon. This file name is determined by the value specified with the SERVER_NAME configuration option. This file contains an identifier of a shared memory region used for clients to connect. The following configuration option allows this directory to be directly specified: SHMEM_DIRECTORY (/doc/ctserver/#57538.htm) <directory_name> c-treeace must have sufficient read, write, create, and delete permissions with this directory. The following server keyword sets the shared memory resource permissions: SHMEM_PERMISSIONS <permissions> The default is 660. Caution: A setting of 666 allows access to c-treeace by any user account, permitting any user to attach to a shared memory segment and read or write to it, which may cause a security breach. By default, a client application must belong to the server owner s primary group to use shared memory. This is configurable with the SHMEM_GROUP keyword. SHMEM_GROUP <group> Possible errors indicating problems: FSHAREMM: Could not get group ID for group <group> for shared memory FSHAREMM: Failed to set group for LQMSG shared memory region: X Client Configuration On Unix/Linux system it is necessary to set the CTREE_SHMEM_DIRECTORY environment variable. This allows the directory to be dynamically overridden without having to recompile the client code. 7.4 Runtime Configuration ACUCOBOL-GT allows a program to interface with more than one external file system in the same program. To specify the indexed file system that the program will use, you must set the DEFAULT_HOST configuration variable. For example, to define a file system for use with a particular file, you set the filename_host configuration variable. For an introduction to ACUCOBOL-GT runtime configuration variables and the configuration file, see section 2.7, "Runtime Configuration," in Book 1 of the ACUCOBOL-GT documentation set. Micro Focus COBOL can be configured to use external file handlers in two different ways: you can insert the CALLFH directive inside your COBOL program and recompile it, or you can use dynamic redirection. Please refer to you Micro Focus COBOL documentation. All Rights Reserved 28

41 c-treertg Server Setup 7.5 Configure the c-treertg Server Note: The c-treertg server (a specialized version of c-treeace) is configured separately from the c-treertg client. Client configuration is discussed in the next section, c-treertg Configuration (page 30). For information about configuring the c-treertg Client, see the c-treertg Configuration (page 30) chapter in this manual. For information about configuring the c-treertg Server, refer to the Configuring the c-treeace Server chapter of the c-treeace Server Administrator's Guide ( The c-treertg Server has a multitude of options for configuring many of the various database subsystems including: Data and index cache control Transaction logging features Memory files Backup scripts Diagnostic options Operational statistics logging Unless otherwise instructed, the c-treertg Server starts with default settings for all configurable parameters. The c-treertg Server takes configuration instructions from a configuration file, ctsrvr.cfg, placed in the c-treertg Server directory. When the c-treertg Server finds a ctsrvr.cfg file, it uses all the specified configuration values. c-treertg is delivered with appropriate settings for the server. If you want to change these settings, the ctsrvr.cfg file can be edited using the c-treeace Config Manager, located in: FairCom\vX.X.X.RTG\<win*>\Tools\gui\c-treeConfigManager.exe (where FairCom\vX.X.X.RTG\<win*>\ is the installation directory) for Windows installations, or simply edit this file using any text editor on Unix/Linux systems. For more information, refer to Configuring the c-treeace Server in the c-treeace Server Administrator's Guide ( All Rights Reserved 29

42 8. c-treertg Configuration The c-treertg file system is designed for easy configuration and requires little if any ongoing maintenance. The c-treertg database engine uses a client/server architecture. The c-treertg Server is a specialized version of c-treeace, which is configured separately from the c-treertg client. Client configuration is discussed in this section. A graphical Configuration Tool (page 31) is provided to simplify the process. Information about configuring the c-treertg Client is provided in this chapter in this manual. For information about configuring the c-treertg Server, refer to Configure the c-treertg Server (page 29) in this book and the Configuring the c-treeace Server chapter of the c-treeace Server Administrator's Guide ( Configure while You Migrate If you migrate your data using the RTG Migrate graphical tool, you can create a basic configuration file in the final step of the wizard, as described in RTG Migrate (page 43). Much of the information you need for configuring your system is entered when you migrate your data. If your system has additional considerations that dictate a more complex configuration file (such as files that need to be treated specially, multiple clients, servers, etc.), this chapter will explain how to edit the configuration file to make adjustments. If you create a configuration file when you migrate your data, you can skip the Basic Configuration wizard and edit the resulting file as described in Editing a Configuration File (page 39). All Rights Reserved 30

43 c-treertg Configuration 8.1 c-treertg Configuration Tool - RTG Config The c-treertg Configuration tool, RTG Config, provides a graphical interface for configuring the c-treertg products. This tool helps you to edit the XML file, ctree.conf, which is used to configure these products. It also tests ctree.conf for correctness, so you can use it to find any problems with a ctree.conf you have edited by hand. The RTG Config tool, RTGConfig.jar, is located in the Tools\guitools.java\ directory. The ctree.conf file consists of a collection of elements nested in a hierarchy. The <config> element is the root of the hierarchy. It will have one or more <instance> elements below it. Both <config> and <instance> can have other elements beneath them (e.g., <file>). Options (such as data compression) can be applied to elements in the tree. The tool displays descriptions of the selected elements and options. (For more complete definitions of the elements, see Configuration File Elements (page 163). For more about the file format see c-treertg Configuration File (page 163)). The Configuration Tool depicts the XML hierarchy as a tree. The tree is composed of: Elements (e.g., config, instance, file, etc.) are depicted as folders. These elements define the basic structure you will configure. Each configuration file has a single <config> element with at least one <instance> under it. See Structure Elements (page 165). Attributes (e.g., name) are depicted as red dots (a summary of attributes also appears after the name of each element). They are details that further describe an element (e.g., a file element is specified with a name or directory attribute). Options (e.g., datacompression) are depicted as blue dots. They are optional settings that can be applied to an element. See Settings Elements (page 175). All Rights Reserved 31

44 c-treertg Configuration You can click the + or - buttons next to each branch to expand or collapse it. The tool bar provides Expand Tree and Collapse Tree buttons to quickly hide or show all details. The Show Attributes button allows you to hide or display attributes to simplify viewing the tree. The image above depicts the following XML configuration file, ctree.conf (data compression will be active for all the specified instances but not for the third instance ("FAIRCOM3") which specifies its own datacompress setting): <?xml version="1.0" standalone="yes"?> <config> <datacompress>yes</datacompress> <instance server="faircoms@ "> <file name="custordr"/> </instance> XML header. This is the root element; all of the other elements and settings are subordinate to it. This option element is immediately under the root level, so it applies compression to all elements (unless it is overridden at a lower level). This structure element specifies an instance of a connection to a c-treertg server, called FAIRCOMS, specified by the server= attribute. This structure element uses the name= attribute to specify the CUSTORDR file, which is on the FAIRCOMS c-treertg server because this element is a child of that instance. The end of the FAIRCOMS instance. <instance server="faircom2@ "> <file name="itemmast"/> An instance of a connection to a c-treertg server called FAIRCOM2. This structure element specifies the ITEMMAST file on the FAIRCOM2 c-treertg server. All Rights Reserved 32

45 c-treertg Configuration </instance> The end of the FAIRCOM2 instance. <instance <datacompress>no</datacompress> <file name="custmast"/> </instance> This structure element specifies an instance of a connection to a c-treertg server called FAIRCOM3. This settings element turns off compression for its parent instance, FAIRCOM3. This structure element specifies the CUSTMAST file on the FAIRCOM3 c-treertg server. The end of the FAIRCOM3 instance. </config> The end of this configuration file. For more about the file format, see c-treertg Configuration File (page 163). Configuration Tool Menus The following menus are provided: File New (Basic) - Create a simple configuration file. New (Advanced) - Create a new configuration file (see below). Open - Opens a configuration file. Save - Saves the current configuration file. Save As - Allows the current configuration file to be saved under a new name. Exit - Closes the tool and exits. Actions Remove Item - Removes the selected item from the configuration file. Help About - Displays information about the version of the tool. Tool Bar The following controls are provided in the tool bar: New File (Basic) - See Creating a New File (Basic) (page 35). Create a basic configuration file. The Basic Configuration window will appear so you can create a new configuration file. New File (Advanced) - See Creating a New File (Advanced) (page 37). Create a new configuration file. Open a File - Opens a configuration file. Save Current File - Saves the current configuration file. Expand Tree - Expands the display of the tree to show all branches so that the full contents of the file can be seen. You can click the - buttons next to each branch to collapse it. All Rights Reserved 33

46 c-treertg Configuration Collapse Tree - Collapses the display of the tree to show all branches so that only the top level of the file can be seen. You can click the + buttons next to each branch to expand it. - Use this button to display or hide the attributes of the elements. Showing attributes depicts attributes as separate items in the tree; hiding them can make it easier to see the entire tree. Creating/Editing Your Configuration File The RTG Config tool provides two ways to create your configuration file: Basic Configuration - This simple, two-step wizard guides you through the configuration process to get started in a hurry. Fill in the fields to create a configuration file that connects to a single c-treertg server. If you require a more advanced configuration, you can start with the Basic Configuration wizard and then use the RTG Config to edit the file. See Creating a New File (Basic) (page 35). Advanced Configuration - The RTG Config allows you to create and edit a configuration file. Selecting File > New (Advanced) (or clicking on the corresponding hot button), clears any entries shown in the RTG Config tree view (and prompts you to save changes, if any) and starts a new configuration. You can then add the elements required for your environment. See Creating a New File (Advanced) (page 37). See also Editing a Configuration File (page 39) All Rights Reserved 34

47 c-treertg Configuration Creating a New File (Basic) The Basic Configuration wizard allows you to configure your system in a hurry. This wizard will help you create a configuration file for a system that connects to a single c-treertg server. It offers a rich set of options, so it may be all you need to get going. If your system is more advanced, for example if it connects to multiple c-treertg servers, you can start with the Basic Configuration wizard and then edit the results in the RTG Config tree window (or you can simply start with New File (Advanced)). If you created a basic configuration file when you migrated your data with the RTG Migrate tool, you can skip the Basic Configuration wizard and edit the resulting file as described in Editing a Configuration File (page 39). To create a new configuration file using the Basic Configuration wizard: 1. Select File > New (Basic) or click the New File (Basic) hot button: If you have unsaved changes in the configuration shown in the tree, you will be prompted to save them. The Basic Configuration wizard will appear: 2. Fill in the fields listed in this dialog. The first tab, called Create Instance, is where define your connection to the c-treertg server. Instance - These fields define how you log onto the c-treertg server: Server - Specifies the server name and the host name of the c-treertg to connect to. The format can be one of the following syntaxes: servername servername@hostname All Rights Reserved 35

48 c-treertg Configuration If the host name or the IP address is omitted, host name defaults to localhost. User - Specifies the c-treertg user name. Password - Specifies the c-treertg user password. Instance Global Options - These fields define attributes that apply to this entire connection to the c-treertg server. Many options are available, including transaction processing, encryption, compression, logging, and file name suffixes. The options provided here correspond to the configuration Settings Elements (page 175) defined later in this manual. 3. The Files / Directories Maps tab allows you to enter more information about the location of your files. Notice that these fields accept wildcards. See Wildcard File Matching Rules (page 173) and File Matching Precedence (page 174). 4. Click Create to save your file. A file dialog will allow you to select a folder and file name for the configuration file. The new configuration file will be displayed in the RTG Config showing the instance you created. You may now add elements as described in the later sections, Editing a Configuration File (page 39). All Rights Reserved 36

49 c-treertg Configuration Creating a New File (Advanced) To create a new configuration file: 1. Select File > New (Advanced) or click the New File (Advanced) hot button: If you have unsaved changes in the configuration shown in the tree, you will be prompted to save them. A new configuration will be created in the tree with a single root element, <config>. 2. Each configuration file requires at least one <instance> element. You will need one instance for each connection to a c-treertg server. To add a new instance, right-click on the <config> root element and select Add New Element > Instance. The New Instance dialog appears so you can enter the basic information needed for specifying the c-treertg server. Note: If your environment includes more than one c-treertg server, you will need to create an instance for each one. Each instance will be directly under the root <config> element. To add another instance, right-click the <config> element and select Add New Element > Instance. All Rights Reserved 37

50 c-treertg Configuration 3. Fill in the fields listed in this dialog: Server - Specifies the server name and the host name of the c-treertg to connect to. The format can be one of the following syntaxes: servername servername@hostname servername@ipaddress If the host name or the IP address is omitted, host name defaults to localhost. User - Specifies the c-treertg user name. Password - Specifies the c-treertg user password. Connect - Choose the desired setting to indicate whether to connect to c-treertg at runtime initialization or wait for the first OPEN operation. Not Set - Use the default value. Yes - Connect at runtime initialization. No - Connect during the first OPEN operation. This is the default value. Versioncheck - Choose the desired setting to indicate whether to check that the c-treertg version matches the c-treertg version. Not Set - Use the default value. Yes - Turn on version matching check. Versions must match exactly otherwise an error is returned at runtime initialization. No - Turn off version matching check. This is the default value. 4. A file dialog will allow you to select a folder and file name for the configuration file. The new configuration file will be displayed in the RTG Config showing the instance you created. You may now add elements as described in the subsequent sections, Editing a Configuration File (page 39). All Rights Reserved 38

51 c-treertg Configuration Editing a Configuration File The configuration file consists of a collection of elements and attributes nested in a hierarchy. The hierarchy is depicted by the tree displayed in the Configuration Tool. The tool allows you to edit the tree by adding and removing elements and attributes. The tool displays information about the selected elements and attributes. To add an element to the file: 1. Select an existing element in the file. The new element will be placed under the selected element (e.g., the selected element will be the parent of the new element: if you select <config>, the new element will be a child of <config>; if you select <instance>, the new element will be a child of <instance>). 2. Right-click on the selected element and select Add New Element from the menu that appears. 3. Select the desired element from the list of valid elements that appears. The list shows only elements that can be used at the selected location in the tree. This implies that the list will be different depending on the element selected in step The new element will be placed in the tree. 5. The new element will be given a default value. A list labeled Name and Value is shown in the right side of the tool. Edit the value in that list to set the desired value. All Rights Reserved 39

52 c-treertg Configuration To add an attribute to an element: 1. Select an existing element in the file. The new attribute apply to the selected element (e.g., the selected element will be the parent of the new attribute). 2. Right-click on the selected element and select Add New Attribute from the menu that appears. 3. Select the desired attribute from the list of valid attributes that appears. The list shows only those attributes that can be used at the selected location in the tree, so the list will be different depending on the element selected in step The new attribute will be placed in the tree under the selected element. 5. The new attribute will be given a default value. A list labeled Name and Value is shown in the right side of the tool. Edit the value in that list to set the desired value. To remove an element from the file: 1. Right-click on the element. 2. Select Remove Item from the menu that appears. COBOL Users: To check the syntax of your configuration file and see if it can connect to the server, see ctutil -test (page 139). All Rights Reserved 40

53 c-treertg Configuration 8.2 Support for Encrypting the Configuration File It is possible to prevent the user from seeing the user password in ctree.conf by using configuration files that are encrypted. The typical configuration files in XML format can be encrypted using the ctutil command -cryptconf (page 118). Usage: ctutil -cryptconf config_file output_file Notice: For security reasons, FairCom does not provide a way to decrypt an encrypted configuration file. Hence, it is advisable to maintain a clear text version of ctree.conf in a safe place for future reference. See -cryptconf (page 118) 8.3 CTREE_CONF Environment Variable - COBOL The CTREE_CONF environment variable specifies the full path to the configuration file. If CTREE_CONF is not defined, c-treertg searches for a file named ctree.conf in the current directory (the directory of execution of the user application that loads c-treertg). Windows set CTREE_CONF=C:\MyConfigFiles\my.ctree.cobol.conf Unix CTREE_CONF=/etc/MyConfig/my.ctree.cobol.conf export CTREE_CONF 8.4 CTREE_CONF_DUMP environment variable to specify configuration dump file The CTREE_CONF_DUMP environment variable allows the c-treertg configuration to be saved to a specified file in XML format. If the value of this environment variable is a file name, c-treertg dumps its configuration in XML format to the specified file after c-treertg has been initialized with a ctree.conf (or iscobol.properties) configuration. For iscobol: This feature can be used as a way to migrate the iscobol configuration options related to c-treertg from iscobol.properties format to ctree.conf format. All Rights Reserved 41

54 9. Data Conversion Migrating Your Data to c-treertg COBOL Edition Before your application is ready to use c-treertg COBOL Edition, the data must be converted to a format that can be used by c-treertg. This is a one-time process that must be done before you can use c-treertg. c-treertg provides a GUI tool to aid in migration, RTG Migrate (page 43), as well as a command-line utility, ctmigra (page 48). SQL Access If you will be accessing your data using SQL, you will need to see c-treertg SQL Access (page 52). Configuring Your System You can create a basic configuration file when you migrate your data with the RTG Migrate tool. This is done in the final step of the wizard, as described in RTG Migrate (page 43). Much of the information you need for configuring your system is entered when you migrate your data, so the configuration file you create may handle your environment with no changes. If you have additional considerations that dictate a more complex configuration file, such as files that need to be treated specially, multiple clients, servers, etc., you will need to edit the resulting configuration file as described in Editing a Configuration File (page 39). All Rights Reserved 42

55 Data Conversion 9.1 RTG Migrate The RTG Migrate tool provides a graphical interface to help you migrate data into c-treertg applications by reading records from an external database and writing them to c-treertg files. This tool provides functionality similar to the command-line version, ctmigra (page 48). The RTG Migrate tool, RTGMigrate.jar, is located in the Tools\guitools.java\ directory. Use the RTGMigrate.bat batch file to run this utility. Note: This utility requires a c-treertg DLL located in the driver directories (for example, Driver\ctree.cobol\extfh for COBOL or Driver\ctree.btrv for BTRV). If you know the path to this directory, you can add it to your PATH environment variable. On Windows platforms, the RTGMigrate.bat batch file correctly sets the path for you and runs the utility. You will see four tabs in the RTG Migrate tool, which correspond to the four steps in migration: One - Source Environment - Allows you to prepare for the migration. Two - Source Files - Indicates the source of the data you will be migrating to c-treertg. Three - Destination - Indicates the destination of the migrated data, which must be accessible to the c-treertg system. Four - Migrate - Begins the migration process. All Rights Reserved 43

56 Data Conversion One - Source Environment The first tab of the RTG Migrate tool (shown above) is where you prepare for the migration. Migration Origin Select original environment - Select ExtFH for Micro Focus and other COBOL compilers that use this file system or select PSQL / BTRV for Btrieve files. The Check Library button tests the presence of the native library to convert data from the original format to the c-treertg format. Select the source base directory of the source files - Enter the directory that contains the source files or click the three dots to navigate to it. Optional Btrieve owner name - If you are migrating from Btrieve files (PSQL / BTRV) to indicate the owner. After entering the information in the fields, click the Next button to advance to the next tab. Two - Source Files Use this tab to select the data files to be migrated. The data will be read out of these files and copied to the destination you specify in the next tab. The original files will not be altered in the process. Do not specify index files because they will be created as part of the migration. The file system is shown in the tree view on the left. Click to the notes to expand them and click on the files to select the ones to be migrated. All Rights Reserved 44

57 Data Conversion File Filter The field labeled File Filter allows you to specify a mask using wildcards. Click Apply to apply changes to the mask. Click the Next button to advance to the next tab. Three - Destination Destination Index Filename Extension - Enter the suffix of the destination index file name (default:.idx). Index Filename Extension Mode - Select Append to if you want the extension entered to be appended to the file name or select Replace to have it replace an extension already in the file name. Replace Existing Files - Select whether or not to overwrite existing files. Destination Base Path - Enter the path to the destination root directory. Migrated files and directories will be placed under this directory on the destination host. This path can be relative to the c-treertg LOCAL_DIRECTORY or it can be an absolute path. Encryption, Compression, Transaction - You can also set these parameters. All Rights Reserved 45

58 Data Conversion Resulting Directory The Resulting Directory layout shows indices with their names and it highlights conflicts. c-treertg Server Connection Information The destination server is the server to which the migrated files will be copied. Server Name, User Name, Password - Enter the login information for the destination server. Test Connection - The Test Connection button in the c-treertg RTG Migrate tool allows you to check the configuration by pinging the server. Note: If you see an error message "Could not find RTG library" it is because the proper driver directory (e.g., Driver\ctree.cobol\extfh for COBOL or Driver\ctree.btrv for BTRV) is not in the path. A batch file, RTGMigrate.bat, is provided so you run the utility without having to set the PATH environment variable in Windows environments. Click the Next button to advance to the next tab. Four - Migrate Migration Batch Size - Enter the number of records to read/write in batches. This setting may affect the speed of the conversion by grouping the records into batches that will be inserted into the All Rights Reserved 46

59 Data Conversion database together. Increasing the size of the batches reduces the number of inserts and speeds performance, as long as system resources (e.g., memory) permit. Log File - Enter the name of the file to log additional information. To redirect log to stderr, enter a dash (-). Script Name - If you want to create a script that can be run later, enter a file name for the script here and click the Create Script button. Create Script - Click this button if you want to create a file containing a script that can be run later. The file name will be the name entered in Script Name. The script will contain the ctmigra command with the appropriate command-line parameters based on the options you have entered in this tool. Configuration Filename - If you want to create a basic configuration file containing the information you have entered in this wizard, enter a file name in this field and click the Create Conf. File button. Create Conf. File - Click this button if you want to create a basic configuration file containing the information you have entered in this wizard. The file name will be the name entered in Configuration Filename. This configuration file can be edited in the RTG Config tool if you need to further refine your configuration for your environment. Click Migrate to begin migration. Alternatively, you can use the Create Script button to create a script that can be used later with the ctmigra utility. You can click Stop Migration at any time to halt the operation. All Rights Reserved 47

60 Data Conversion 9.2 ctmigra ctmigra is a general purpose data migration utility for converting non-c-tree data for use with c-treertg. It can be used with multiple external data types, including COBOL data files. For certain data types, such as Btrieve, this also requires access to your original external libraries used to access incoming source files in their native format, and ctmigra has ability to link with these libraries as required. The ctmigra utility migrates data by reading records from existing external tables and writing them to c-treertg files. Supported interfaces include ExtFH (for access to Micro Focus COBOL files) and BTRV (for access to Btrieve files). It also supports other file handlers. ctmigra is located in the tools\cmdline directory of your c-treertg installation. A graphical interface version RTG Migrate (page 43) is also available providing similar functionality in an easy to view window. You'll find this in your tools/guitools.java of your c-treertg installation. The next sections provide a description of the command and its usage. Using the ctmigra utility Usage of c-treertg ctmigra depends on your native data file types and your platform. Usage ctmigra btrv extfh [OPTIONS] SOURCE DEST where: SOURCE - name of the file to be copied. DEST - name of the file to which it will be copied. Options for c-treertg Files -n SERVERNAME or --dest-server=servername where SERVERNAME is the name of the destination c-treeace Server -u USERID or --dest-user=userid where USERID is the user name of the destination c-treeace Server -p PASSWORD or --dest-password=password where PASSWORD is the user password of the destination c-treeace Server -I STRING or --dest-idx-suffix=string where STRING is the suffix of destination index file name -N SERVERNAME or --source-server=servername where SERVERNAME is the name of the source c-treeace Server -U USERID or --source-user=userid where USERID is the user name of the source c-treeace Server -P PASSWORD or --source-password=password where PASSWORD is the user password of the source c-treeace Server -i STRING or --source-idx-suffix=string where STRING is the suffix of source index file name All Rights Reserved 48

61 Data Conversion -l FILE or --log=file where FILE is the name of the file to log additional information. To redirect log to stderr use - as FILE -b RECORDS or --batch-size=records where RECORDS is the number of records to read / write in batches -a (--append-ext) appends index extension instead of replacing data extension. -o OWNER (--owner=owner) specifies the BTRV file owner (can be used if you encountered an error 51 when attempting to migrate). -r (--replace) instructs ctmigra to replace the existing destination file (V11 and later). -e, --encrypt=cipher - Encrypt destination file with CIPHER algorithm. -z, --datacompress=type[;lev][;str] - Compress destination data file with TYPE algorithm. -t, --transaction=no yes logging - Create destination data file with or without transaction support. The last three options enable the following configuration options on the destination file (corresponding to -e, -z, and -t respectively): <encrypt type="cipher"> <datacompress type="type"= level="lev" strategy="str"> <transaction logging="no yes"> See usage examples in the next topics. Notice that the Micro Focus COBOL Migration Example (page 50) can be useful for Btrieve users as well and COBOL users. External Library Configuration for Native File Access For certain data types, such as BTRV, original external libraries are required to access native data formats. -s LIBRARY!FUNCTION or --source-lib=library!function where LIBRARY is the external dynamically loadable library and FUNCTION is the interface entry-point function to handle the source file -d LIBRARY!FUNCTION or --dest-lib=library!function where LIBRARY is the external dynamically loadable library and FUNCTION is the interface entry-point function to handle the destination file Note: Current usage options are always available when no command-line options are supplied. ctmigra --quiet and --verbose options to select output information These options for the ctmigra command-line utility suppress all output (--quiet or -q) or select the information to be sent to stdout (--verbose=level or -v). The --verbose parameter is a bitmask which can combine the following values: 1 - show message about final result 2 - show percentage progress 4 - show time spent in migration phases (read, write, finalize) For example, to display everything, use --verbose=7, which is If not specified, the --verbose level is 3 (1+2). The --quiet option is identical to --verbose=0. All Rights Reserved 49

62 Data Conversion Alternative Usage In some cases, ctmigra can be used with existing local c-treertg configuration files. For example: ctmigra btrv extfh -c CONFIG_FILE SOURCE DEST where: SOURCE - name of the file to be copied DEST - name of the file to which it will be copied -c CONFIG_FILE or --config=config_file where CONFIG_FILE is a valid c-treertg configuration file. Potential Errors and Troubleshooting BTRV Error: -7 BTRV Error: 53 Your native source library is likely not available. Check the path to your native data file handling library and be sure it is specified with the --source-lib= option. Attempted to open a non-btrv file via a BTRV interface. This can possibly happen when you load a 32-bit BTRV DLL with a 64-bit version of the tool, or vice versa. In c-treertg V2, error messages displayed by the ctmigra utility have been enhanced as follows: It now displays actual file names instead of $SOURCE$ and $DEST$. It now displays an error message if the source or destination c-tree Servers are not running (c-tree error 133). It now displays an error message if the user/password is not correct (c-tree error 450/451) or GUEST logon is disabled (c-tree error 470). It now displays an error message if c-tree Server is not valid due to OEM version incompatibility (c-tree error 530). It now displays an error message if the specified external library cannot be loaded. Behavior Change: This modification changes the behavior of the ctmigra tool. Micro Focus COBOL / BTRV Migration Example The Micro Focus COBOL compiler uses the Btrieve database, therefore even though this example is for COBOL, it is a useful example of how to migrate Btrieve data to c-tree. To copy and covert data from Micro Focus COBOL tables into c-treertg, use ctmigra as follows: ctmigra extfh -s C:\MICROFOCUS\lib\MFFH.DLL!MFFH S:\Data.MF\MYFILE D:\Data.RTG\MYFILE.dat All Rights Reserved 50

63 Data Conversion To use ctmigra with Unix-based Micro Focus libraries be sure and pre-load them with the LD_PRELOAD environment variable or an alternative mechanism. For example, on an AIX 64-bit platform (this is all on a single command line): LDR_PRELOAD64=libcobrts64.3.so:libcobcrtn64.3.so:libcobmisc64.3.so ctmigra extfh -s libcobrts64.3.so!extfh /data/mf/myfile /data/rtg/myfile.dat For example, on a Linux 32-bit platform (this is all on a single command line): LD_PRELOAD=libcobrts.so:libcobcrtn.so:libcobmisc.so ctmigra extfh -s libcobrts.so!extfh /data/mf/myfile /data/rtg/myfile.dat All Rights Reserved 51

64 10. c-treertg SQL Access c-treertg offers more than just a high-performance file system. The c-treertg SQL database engine opens a new and efficient method to access your data from many other applications via SQL. By avoiding the overhead imposed by many hybrid SQL implementations, c-treertg gives you the most direct access available. COBOL and Btrieve data files do not explicitly define a record schema, which is mandatory for SQL. If you want to provide SQL access to this data, you will need to define the record schema through an XDD (external Data Definition) file. The XDD is an external XML file that is stored as a special resource within the data file created through the processes described in this chapter. FYI: If your data file contains more than one record schema (REDEFINES), c-treertg provides an ability to define multiple record schemas, each of which will appear to SQL as a virtual table. See the c-treedb Virtual Tables ( technical white paper on the FairCom website. The procedures in this chapter are optional: they are needed only if you want SQL access to your COBOL data. There are two main parts to this process: 1) Create an XDD First, you will define the record schema of your data with an XDD file. The method you will use depends on the compiler you are using: Creating an XDD from an XFD (page 53): If your compiler can create an XDD, you can generate the XDD from it. Creating an XDD from the COBOL Source (page 54): If your COBOL compiler does not provide an XFD, you can generate the XDD from COBOL source. Creating an XDD Manually (page 55): If you do not have an XFD or source, you can manually create an XDD. Users of c-treertg BTRV Edition who desire SQL access to their data will need to use these procedures. 2) Store the XDD in the data file (page 56) Next, you will store the XDD in the file and link it to the c-treeace SQL dictionaries using ctutil -sqlize. Note: In some cases, these procedures will not produce the desired results because your data may be structured in a way that cannot be interpret correctly. In those situations, you will need to add directives to your copybook to handle the data. If the procedures in this chapter are not able to sqlize your data as expected, see Tips for Advanced Sqlizing (page 89). All Rights Reserved 52

65 c-treertg SQL Access 10.1 Creating an XDD from an XFD If your COBOL compiler can provide an XFD, it can be used to create the XDD required for sqlizing your data. If your COBOL compiler cannot provide an XFD, see the section Creating an XDD from COBOL Source (page 54). If your compiler can create an XFD, you can generate the XDD from it using ctutil -xfd2xdd (page 145) or ctutil -sqlize (page 138). Use the ctutil utility with the -xfd2xdd (page 145) parameter to create a valid XDD file starting from an XFD file, which can the generated by COBOL compilers such as ACUCOBOL. You can specify a rules file to fine-tune the creation of the XDD, as described in Defining External Rules (page 57). If you need to further fine-tune the XFD, it can be edited with an XML editor before it is stored in the data file. If you use ctutil -xfd2xdd (page 145) to create the XDD, follow the procedures in the section titled Storing the XDD in the Data File (page 56) to prepare your data file for SQL access. All Rights Reserved 53

66 c-treertg SQL Access 10.2 Creating an XDD from the COBOL Source If you do not have an XFD, c-treertg allows you to create the XDD from your COBOL source code. The XDD is generated from COBOL source code using xddgen (page 147). This command-line utility analyzes the COBOL program to determine the schema. You will specify the COBOL program file as input. A variety of other parameters allow you to specify the dialect of COBOL (ACUCOBOL, Micro Focus, IBM, etc.), source format (free or fixed), directories, and other options. The section titled Tips for Advanced Sqlizing (page 89) explains the process using a tutorial provided with c-treertg. It provides step-by-step instructions to guide you through the process. After creating the XDD, use the procedures in the section titled Storing the XDD in the Data File (page 56) to prepare your data file for SQL access. All Rights Reserved 54

67 c-treertg SQL Access 10.3 Creating an XDD Manually If you do not have access to a DDF or to your COBOL program source code, c-treertg allows you to create an XDD manually. Users of c-treertg BTRV Edition who do not have a DDF can use these procedures if they desire SQL access to their data. The XDD file can be created using any XML editor. To manually create an XDD, you will need to know the structure of your data. You will then need to translate that structure into an XDD. The XDD is explained in the section titled XDD File Schema (page 226). After creating the XDD, use the procedures in the section titled Storing the XDD in the Data File (page 56) to prepare the data file for SQL access. All Rights Reserved 55

68 c-treertg SQL Access 10.4 Storing the XDD in the Data File and Linking to the SQL Dictionary - COBOL After creating an XDD, the information in that file will need to be stored in the data file and it must be linked to the c-treeace SQL dictionary. Use the ctutil -sqlize (page 138) command to make an existing data file accessible from SQL by storing the XDD in the file and linking the file name to the c-treeace SQL dictionaries. After You Have Sqlized After you have stored the XDD in the data file, you have finished sqlizing and you are ready to take it for a test spin. Experiment with some sample data to see if the process yielded the expected results. Be aware of two factors that may affect the results: Some COBOL fields do not translate well to SQL. For example a text field may be used to store a date in a human-readable format that has no meaning to SQL. Some COBOL programing practices, such as REDEFINES, do not lend themselves to relational access. c-treertg provides tools to help you fine-tune your installation to overcome these problems. In particular, review these sections: Troubleshooting Data Conversion Errors (page 69) Tips for Advanced Sqlizing (page 89) You will find an abundance of useful information in this guide, such as c-treertg Errors (page 258), XDD File Schema (page 226), and Background Information about Sqlizing (page 81). Advanced users may want to consult API for SQL Conversion Error Checking (page 76). All Rights Reserved 56

69 c-treertg SQL Access 10.5 Defining External Rules When generating a XDD file from an XFD file, it is useful to have an automated method for altering the resulting XDD to better fit user requirements (e.g., to add defaults, hide fields, etc.). The "rules file" provides an automated method to control the process using rules saved in XML format. The rules file must be specified when using the ctutil -sqlize option or the ctutil -xfd2xdd option to generate an XDD. XFDRules XML rules files have the following structure: <XFDrules> <rule> <when> OPTIONAL <"condition"> </"condition">... More <"condition"> elements... </when> <do> <"action"> <"target"> </"target">... More <"target"> elements... </"action">... More <"action"> elements... </do> </rule>... More <rule> elements... </XFDrules> Example 1 Here is an example demonstrating all hidden fields, as it deletes all the attributes hidden="true". <rule sequence="1"> <when> <field hidden="true"/> </when> <do> <delete> <field hidden="true"/> </delete> </do> </rule> All Rights Reserved 57

70 c-treertg SQL Access Example 2 COBOL types mapped to date, datetime, or time may cause conversion issues if the COBOL data cannot be properly represented by a type of date, datetime, or time. Hence it may be useful to have a rule to set fields to NULL when a conversion error occurs on the schemas having a field mapped to a date, datetime, or time field. Here is a rule to do that for date fields: <rule sequence="2"> <when> <field dbtype="date"/> </when> <do> <set> <schema OnConvertError="null"/> </set> </do> </rule> All Rights Reserved 58

71 c-treertg SQL Access <XFDrules> root element This is the root element used to contain all the rules to apply to the XDD file. <XFDrules> </XFDrules> Elements Element <rule> (page 60) Description Define one rule, this element can be repeated to define multiple rules. All Rights Reserved 59

72 c-treertg SQL Access <rule> XFDRules element The rule tag defines a rule to be applied to the XDD file. There must be at least 1 rule element. In case of multiple rule elements, they define multiple rules. All of the rules defined will be checked, and, if the <when> condition is satisfied, applied in defined ascending numeric sequence. <XFDrules> <rule> </rule> <XFDrules> Elements Element <when> (page 61) <do> (page 63) Description Defines the condition when to apply the current rule. This element is optional and if missing the rule will always be applied. This can only be specified once. Defines the action to apply to the XDD file. This element can only be defined once. Mandatory Attributes Attribute sequence Description Defines the numerical order of the defined rules. The rules will be evaluated and applied in the specified order. Optional Attributes Attribute debug Description The name of the file containing information about rule matching and execution. If the file exists, the output will be appended. The file will be created in the current directory if the path is not specified along with this attribute. See Also <XFDrules> (page 59) All Rights Reserved 60

73 c-treertg SQL Access <when> rule element The optional <when> element is used to specify the criteria identifying if the rule should be applied or not. If there is no when tag then the rule always applies. <XFDrules> <rule> <when> </when> </rule> </XFDrules> Elements Element Condition (page 62) Description Defines the condition when to apply the current rule. This element can be specified only once for each rule. See Also <XFDrules> (page 59) <rule> (page 60) All Rights Reserved 61

74 c-treertg SQL Access <[Condition]> when elements The condition elements are used by the <when> element to identify when the current rule will be applied. A condition rule is satisfied if all the specified attributes match the matching element of the XDD file: <table> condition is satisfied if all attributes specified in the rule condition matches the <table> tag of the XDD file. <schema> condition is satisfied if all attributes specified in the rule condition matches the <schema> tag of the XDD file. <field> condition is satisfied if all attributes specified in the rule condition matches the <field> tag of the XDD file. The condition elements are mutually exclusive and contain the attributes of the element with the same name as the XDD file. If more condition elements are specified the current rule will be applied if all the conditions are satisfied: <table> used to describe a condition to verify on the <table> element (page 228) of the XDD file. <schema> used to describe a condition to verify on all the <schema> elements (page 239) of the XDD file. <field> used to describe a condition to verify on all the <field> elements (page 240) (children of the <schema> element (page 239) element) of the XDD file. <XFDrules> <rule> <when> <"Condition"> </"Condition"> </when> </rule> </XFDrules> Attibutes For <table> condition element attributes refer to the attributes list of the <table> element (page 228) of the XDD file. For <schema> condition element attributes refer to the attributes list of the <schema> element (page 239) of the XDD file. For <field> condition element attributes refer to the attributes list of the <field> element (page 240) (children of the <schema> element (page 239) element) of the XDD file. See Also <XFDrules> (page 59) <rule> (page 60) <when> (page 61) All Rights Reserved 62

75 c-treertg SQL Access <do> rule element The <do> element describes the action to apply by this rule. If a <when> element (page 61) is also specified the action will be applied only if all the Condition elements (page 62) are verified. <XFDrules> <rule> <do> </do> </rule> </XFDrules> Elements Element Action (page 64) Description Defines which action the rule will execute. This element can be defined multiple times to specify different actions. See Also <XFDrules> (page 59) <rule> (page 60) All Rights Reserved 63

76 c-treertg SQL Access <[Action]> do elements The action elements define which action to execute for the current rule. If a <when> element (page 61) is also defined, the actions will be executed only if all conditions are verified. There are 4 different types of action elements: <add> This action adds new attributes to the matching element in the XDD file. The attributes will be added only if they do not exist. <set> This action changes the value of the attributes to the matching element in the XDD file. If the specified attributes do not exist they will be added. <modify> This action changes the value of the attributes to the matching element in the XDD file. The attributes will be changed only if they exist. <delete> This action removes an existing attribute to the matching element in the XDD file. <XFDrules> <rule> <do> </do> </rule> </XFDrules> Elements <"Action"> </"Action"> Element Target Description Defines the target to identify which element the rule touches. This element can be specified multiple times to apply more changes in the XDD file. See Also <XFDrules> (page 59) <rule> (page 60) <do> (page 63) All Rights Reserved 64

77 c-treertg SQL Access <[Target]> action element Target elements are used by the <do> elements to identify which element the rule touches. The target elements are mutually exclusive and contain the attributes of the element with the same name of the XDD file. More target elements can be specified: <table> used to describe a change to make on the <table> element (page 228) of the XDD file. <schema> used to describe a change to make on one of the <schema> element (page 239) of the XDD file. <field> used to describe a change to make on one of the <field> element (page 240) (children of the <schema> element (page 239) element) of the XDD file. <XFDrules> <rule> <do> </do> </rule> </XFDrules> Attibutes <"Action"> <"Target"> </"Target"> </"Action"> For <table> target element attributes please refer to the attributes list of the <table> element (page 228) of the XDD file. For <schema> target element attributes please refer to the attributes list of the <schema> element (page 239) of the XDD file. For <field> target element attributes please refer to the attributes list of the <field> element (page 240) (children of the <schema> element (page 239) element) of the XDD file. See Also <XFDrules> (page 59) <rule> (page 60) <do> (page 63) <"Action"> (page 64) All Rights Reserved 65

78 c-treertg SQL Access Rule Examples The example below shows the following rules, which will be applied in the order shown below (based on "sequence="): When the field type="numunsigned", add bindefault="all-spaces". When the field dbtype="date", add field cbdefault="0". When the field name="customer_id", set field onconverterror="null". When the field dbtype="date", add field cbdefault="0" onconverterror="null" and add field format="yyyymmdd" cbdefault="0" onconverterror="error". <rule sequence="100"> <when> <field type="numunsigned"/> </when> <do> <add> <field bindefault="all-spaces"/> </add> </do> </rule> <rule sequence="110"> <when> <field dbtype="date"/> </when> <do> <add> <field cbdefault="0"/> </add> </do> </rule> <rule sequence="900"> <when> <field name="customer_id"/> </when> <do> <set> <field onconverterror="null"/> </set> </do> </rule> <rule sequence="1000"> <when> <field dbtype="date"/> </when> <do> <add> <field cbdefault="0" onconverterror="null"/> </add> <add> <field format="yyyymmdd" cbdefault="0" onconverterror="error"/> </add> </do> </rule> All Rights Reserved 66

79 c-treertg SQL Access All Rights Reserved 67

80 c-treertg SQL Access 10.6 Type Mapping Table To define an XDD file requires defining the data type of each column of the original data file. The following table describes the accepted XDD data types: Type to be used in the XDD field specification NumUnsigned NumSignSep NumSigned NumSepLead NumLeading Alphanum Float CompSigned CompUnsigned PackedPositive PackedSigned PackedUnsigned BinarySigned BinaryUnsigned NativeSigned NativeUnsigned Type description ASCII string representing an unsigned number ASCII string representing a signed number with trailing separate sign. ASCII string representing a signed number with trailing sign ASCII string representing a signed number with leading separate sign ASCII string representing a signed number with leading sign ASCII string Float or Double values Signed computational Unsigned computational Packed string representing a positive number Packed string representing a signed number Packed string representing an Unsigned number Integer number represented in binary signed format (big endian) Integer number represented in binary unsigned format (big endian) Integer number represented in native O/S binary signed format Integer number represented in native O/S binary unsigned format See also <schema> table element (page 239). Variable-length fields mapped into LONGVAR* SQL field Support has been added for mapping fields into LONGVAR* SQL fields. The XDD has been expanded as follows: 1. New dbtype possible values: BLOB: Indicates a variable-length binary object with length depending on a field value. CLOB: Indicates a variable-length text object with length depending on a field value. All Rights Reserved 68

81 c-treertg SQL Access 2. New <field> attribute sizefield to be used in conjunction with dbtype BLOB or CLOB and having size = "0" For an XDD field to be mapped into a LONG VARCHAR or LONG VARBINARY, the following conditions must be met: 1. The field definition must have: size="0" sizefield="x" where "X" is a valid field containing the number of bytes (we suggest this field to be hidden, but this is not mandatory) dbtype="clob" or dbtype="blob" 2. At maximum, 1 field mapped to a BLOB or CLOB type. 3. It must be the last field in the record buffer. If one (or more) of the above condition is not met, an error CTDBRET_CALLBACK_11 ("Unsupported clob/blob definition") is returned Troubleshooting Data Conversion Errors The fact that the XDD initially does not contain any error handling information immediately exposes data conversion errors to a SQL user. This provides the way to begin troubleshooting data conversion errors and to identify the proper settings to specify in your XDD file. Checking in c-treeace SQL Explorer c-treeace SQL Explorer and c-treeace Explorer include a button to simplify the process of checking for bad records. To check for bad records in c-treeace SQL Explorer or c-treeace Explorer: 1. Select a sqlized table, such as custmast in the image above. 2. Click the Table Records tab. 3. A button labeled Check Bad Records appears at the right of the row of buttons (the image above shows the Java version of c-treeace Explorer where the buttons are at the top of the tab; the.net version, called c-treeace SQL Explorer, shows this row of buttons at the bottom of the tab). 4. Click the button to execute a SQL query to find records that did not sqlize properly. All Rights Reserved 69

82 c-treertg SQL Access Checking with a SQL Query You can use the procedures in this section to identify data-conversion errors and use that information to fine-tune your XDD files. If a query fails, it is possible the failure is due to a problem with SQL data conversion. Troubleshooting this type of error is quite easy with the following steps: 1. Identify the table (in case of a complex query) on which the conversion error occurs by running the following SQL statement on each table involved in the query: SELECT * FROM <table> If none of the queries fail, the original query failure is not due to a conversion problem. 2. Run the following SQL statement to select only the records that do not properly convert: SELECT * FROM <table> ctoption(badrec) ctoption(badrec) is a c-treeace extension to SQL indicating the query should return only records having conversion errors and expose values that do not properly convert as NULL. 3. Look for NULL values returned from the query in step 2. These are the fields that do not properly convert. The remaining record values should be sufficient to identify the record that requires investigation. The ctoption(badrec) command generates a log file, ctsqlcbk.fcs, in the c-treeace SQL Server directory that can be used to determine the exact conversion error and the data causing it. This file lists all the fields that caused a conversion error exposed in SQL along with the value of the data that could not be converted. Note that the log contains information for the fields that result in propagating the conversion error to SQL. It does not log conversion errors that result in a SQL value because they were already handled successfully following the settings in the XDD. Each log entry is made of three lines: 1. The first line is similar to the following: Convert error XXXX on field[yyyy] Where XXXX is the error code indicating the cause of the conversion error, YYYY is the field on which the conversion error occurred. 2. The second line contains a message which gives internal information for FairCom technicians to identify where the error occurs in the code, as well as a message explaining the problem. 3. The third line is a hexadecimal dump of the field content. For example: Convert error 4028 on field[field1] {ctdbstringformattodatetime} on failed Convert error 4028 on field[field2] {ctdbstringformattodate} on failed Convert error 4118 on field[field3] [NormalizeCobolStr] ascii value is not a digit Convert error 4118 on field[field4] [NormalizeCobolStr] ascii value is not a digit All Rights Reserved 70

83 c-treertg SQL Access All Rights Reserved 71

84 c-treertg SQL Access 10.8 Viewing Sqlized Tables in c-treeace SQL Explorer c-treertg opens your COBOL data to the world of SQL access. FairCom knows you will want to monitor and manage this data and that means seeing it the way it appears to SQL applications. FairCom's c-treeace SQL Explorer and c-treeace Explorer are graphical tools that allow you to view and manipulate your data, providing comprehensive management capabilities. c-treeace SQL Explorer and c-treeace Explorer are client tools designed to interact with c-treertg Servers through the TCP/IP SQL port. Using the SQL language, they give you access to all the items in your c-treeace database. They also provide the ability to execute single SQL statements and SQL scripts including options for viewing the execution plans. c-treeace SQL Explorer and c-treeace Explorer show linked tables in a different color (a reddish orange). Tables that were also sqlized have a jagged "S" (or lightning bolt) to indicate they were sqlized and linked. The custmast table in the image below is a sqlized and linked table: When you right-click a table, the context menu will display only the options that are available to that table (all other options are dimmed), as shown in the image above. Some options available to regular tables, such as Alter Table, Clone, and Constraints, are disabled because they are not available for sqlized tables. All Rights Reserved 72

85 c-treertg SQL Access 10.9 Adding SQL Indices to Sqlized Files In release V2, c-treertg takes a giant stride in its SQL capabilities by allowing you to create a SQL index on your COBOL tables. You can execute CREATE INDEX on imported tables or you can use the graphical tools, c-treeace SQL Explorer and c-treeace Explorer. Performance of SQL Queries A practical use of this feature is in handling COBOL indices that do not translate well to SQL. In some cases, the index is imported into SQL by sqlize with limited functionality, so it can be used only to perform searches using the "=" and the "<>" (not equal) operators. In a small number of cases, the index cannot be sqlized at all. These cases may impact the performance of SQL queries on those fields. This issue is now easily addressed: simply create SQL indices where needed to speed up queries. It is not necessary to replace every COBOL index, simply replace those that are needed to speed up your SQL queries. Note: A SQL index cannot be created on certain COBOL data types: date, time, datetime (timestamp), and bit (Boolean). The presence of a SQL index will better optimize some queries. As a side effect, the index returns rows sorted in logical order as opposed to binary order as is the case for COBOL indices. All Rights Reserved 73

86 c-treertg SQL Access To create a new index for a table, click the table name to see the group labeled Indexes, right-click on the group, and select Create from the context menu. The following window will appear for creating the new index definition including defining index columns (segments): Create Definition Controls UNIQUE - Check this option if you don't want your index to contain duplicate values. Index Name - Enter the name to be assigned to the new index. Table Name - This box will display the table name receiving the new index. If you access this dialog from a specific table, the table name will be displayed read-only. If you access this dialog from the Index group under a user, select the table from the drop-down list. Storage_Attributes Partition - This check box adds STORAGE_ATTRIBUTES "PARTITION" to the SQL statement. This creates the index as the partition index to the table enabling multiple table partitions. Column Definitions Use the controls in the Columns group to define the columns composing your new index. New lines can be added by filling the line marked with the asterisk: Column - Click inside the Column drop-down list to select the column name to be included in your new index. If your index will be built over multiple columns, continue this process until all columns are listed. Desc - Check this box if you want the column to be sorted in descending order. Otherwise an ascending sort is the default. - Use this button to move a column up the index column list. Note, the column in the top of this list will appear first (to the left) within the index. - Use this button to move a column down the index column list. All Rights Reserved 74

87 c-treertg SQL Access To delete an index column: Select the row header that contains the column to be deleted and then press the Delete key on your keyboard. Resulting Statement The Resulting Statement window will show the CREATE statement to be executed for building the new index. Once your index is completely defined, press the Create button to create your index and remember to check Result in the left corner of the status bar at the bottom of the window for either Success or an error message. Once you see Success in the status window, click on exit to return to the main window. You can save the CREATE statement shown in the Resulting Statement window by clicking Save Statement or using the File menu or pressing CTRL+S. Finishing Up Save Statement - Click this button if you want to save the SQL statement so you can execute it at a later time. Create - Click this button to create the index. Exit - Click this button to close this dialog. Alter Table Callbacks Allow Better Control The following callbacks allow better control during Alter Table operations: 1. A new field callback was added. This callback is called by ctdbaddsegment for those fields that have been remapped to a different field type using a field callback. 2. Alter Table logic has been enhanced to call the "alter table callback" at the beginning of the Alter Table and at the end and once the action to be carried out is determined. The "time" when the callback is called can be determined by looking at the new CTDBTABLE alter_time member. The action to be performed is accessible and modifiable in the callback code by using the alter_action member of CTDBTABLE. 3. During the Alter Table execution it is possible that the table will be closed and re-opened. In such cases the following callbacks are called when needed: _OnTableGetSchema _OnTableGetDoda _OnTableGetExtInfo All Rights Reserved 75

88 c-treertg SQL Access API for SQL Conversion Error Checking It is possible to programmatically check if there will be any conversion errors accessing a table from SQL. Three functions, exported by mtclient.dll, provide functionality to verify if a record buffer will generate any conversion error when accessed through SQL. This checking capability requires an XDD file. It verifies if a record buffer generates any errors when converted into SQL. It considers all the error handling specified in the XDD (as opposed to the SQL "badrec" approach that logs any error, even the ones that are handled). SQL Conversion for ACUCOBOL Users ACUCOBOL users can call these commands from their programs. See Enabling XDDOPEN, XDDCHECK, and XDDCLOSE for Programmatic SQL Conversion Error Checking (page 11). A sample program using ACUCOBOL is provided (samples\acucobol\xddcheck\xddchk.cbl). It shows how to write a simple COBOL program that takes advantage of this new functionality by performing a table scan to verify that all records can be converted to SQL. The sample is for ACUCOBOL only. It assumes that SQL conversion error checking has been enabled in the runtime as explained in Enabling XDDOPEN, XDDCHECK, and XDDCLOSE for Programmatic SQL Conversion Error Checking (page 11). All Rights Reserved 76

89 c-treertg SQL Access ct_xddopen Declaration cbdllexport pvoid ct_xddopen(count signconv, TEXT text[ct_path_len]) This function accepts: signconv: the numeric format to be used; the value must be one of the value specified as sign conventions defined in ctcbxdd.h text: the name of XDD file to open and use Description This function returns a VOID pointer holding to be used on calls to other functions, NULL in case of error. The function opens and parses the XDD file and prepares all the information required to perform the buffer checking. Return Values See XDDCHECK Errors (page 80) for a complete listing of valid c-treertg XDDCHECK API error values. See Also API for SQL Conversion Error Checking (page 76) ACUCOBOL Users: Enabling XDDOPEN, XDDCHECK, and XDDCLOSE for Programmatic SQL Conversion Error Checking (page 11) All Rights Reserved 77

90 c-treertg SQL Access ct_xddcheck Declaration cbdllexport LONG ct_xddcheck( pvoid hdl, putext buffer, LONG buflen, ptext msg, plong offset, plong size, plong type) This function accepts: hdl: the handle returned by a ct_xddopen call buffer: pointer to the buffer to be analized buflen: significant len of the buffer msg: pointer to a 256 bytes buffer for error message description offset: pointer to a 4 bytes variable where the logic stores the offset of the first field having a problem size: pointer to a 4 bytes variable where the logic stores the size of the first field having a problem type: pointer to a 4 bytes variable where the logic stores the type of the first field having a problem (value as defined by the enum in ctcbxdd.h) Description This function returns 0 if no error is encountered. It returns an error code (enum XDD_CONV_RET value) in case of an error. It also returns a message (the msg parameter is a pointer to this message). The function checks the buffer for conversion from COBOL to SQL based on the XDD specifications. It stops at the first error and returns information about the error. In case of multi-record table, it scans through all the possible schema that apply to the buffer. Return Values See XDDCHECK Errors (page 80) for a complete listing of valid c-treertg XDDCHECK API error values. See Also API for SQL Conversion Error Checking (page 76) ACUCOBOL Users: Enabling XDDOPEN, XDDCHECK, and XDDCLOSE for Programmatic SQL Conversion Error Checking (page 11) All Rights Reserved 78

91 c-treertg SQL Access ct_xddclose Declaration cbdllexport VOID ct_xddclose( pvoid hdl) This function accepts: hdl: the handle returned by a ct_xddopen call Description This function frees the memory allocated to the XDD handler and "closes" the XDD checking session for the specific XDD. Return Values See XDDCHECK Errors (page 80) for a complete listing of valid c-treertg XDDCHECK API error values. See Also API for SQL Conversion Error Checking (page 76) ACUCOBOL Users: Enabling XDDOPEN, XDDCHECK, and XDDCLOSE for Programmatic SQL Conversion Error Checking (page 11) All Rights Reserved 79

92 c-treertg SQL Access XDDCHECK Errors The table below lists the error codes returned by the XDDCHECK API: Symbolic Error Code Description XDD_CONV_NOERR 0 No error. XDD_CONV_NOMRTMATCH 1 Record does not match filter definition. XDD_CONV_NOMRTFILTERERR 2 Filter definition evaluation error. Contact FairCom. XDD_CONV_NOMEM 3 No memory. XDD_CONV_UNDERLEN 4 Record length smaller than defined minimum record length. XDD_CONV_OVERLEN 5 Record length larger than defined maximum record length. XDD_CONV_INTERNAL 1000 Internal error. Contact FairCom. XDD_CONV_INVTYP 1001 Unexpected Field type. Contact FairCom. XDD_CONV_NOTSUPPORTED 1002 Conversion not supported. XDD_CONV_INVBOOL 1003 Invalid mapping to Boolean value. XDD_CONV_UNDERFLOW 1004 Conversion causes an underflow error. XDD_CONV_OVERFLOW 1005 Conversion causes an overflow error. XDD_CONV_INVNUMBER 1006 Value is not a valid number. XDD_CONV_INVDATETIMEFORMAT 1007 Invalid date/time format specification. XDD_CONV_INVHOUR 1008 Invalid Hour value. XDD_CONV_INVMINUTE 1009 Invalid Minute value. XDD_CONV_INVSECOND 1010 Invalid Second value. XDD_CONV_INVDATE 1011 Invalid Date value. XDD_CONV_INVYEAR 1012 Invalid Year Value. XDD_CONV_INVMONTH 1013 Invalid Month Value. XDD_CONV_INVDAY 1014 Invalid Day Value. XDD_CONV_ARGSMALL 1015 Conversion buffer too small. Contact FairCom. XDD_CONV_NOTADIGIT 1016 COBOL buffer contains a value that cannot be interpreted as a digit. XDD_CONV_BADSIGN 1017 COBOL buffer contains a value that cannot be interpreted as sign for numeric types XDD_CONV_INVINTSIZE 1018 Integer value of an unsupported size. Contact FairCom. XDD_CONV_DIGITOVERFLOW 1019 Converted value has more digit than defined. All Rights Reserved 80

93 c-treertg SQL Access Background Information about Sqlizing Record Structure Definition: The XDD To provide simultaneous access to the data through COBOL and SQL, c-treertg maps COBOL types into SQL types (and vice versa). c-treertg must know the structure of the records so it can provide the SQL schema required for this mapping. COBOL and Btrieve do not provide information about the record structure required by SQL. However, some COBOL compilers can generate XFD files which define this record structure. When an XFD is Available c-treertg provides a command-line switch to ctutil (page 106) utility to convert XFD files into an XDD (extended Data Definition) file. This file is in XML format and contains information about the data and index structures including: index definitions, record schemas, default field values, null field handling (an undefined concept in COBOL), behavior of data conversion errors, etc. During the conversion from XFD to XDD, ctutil also accepts a "rule file" that contains instructions to customize the generated XDD file for specific needs (such as default values for fields). When an XFD is Not Available: xddgen For those compilers that do not provide a means to generate an XFD file, we provide the OpenCOBOL-based xddgen, which can create an XDD from the COBOL source code. See the xddgen (page 147) chapter for more information. Merging the XDD with the Data File The information contained in the XDD is stored directly in the data file itself with no effect on COBOL access to the data. The XDD file is embedded into the data file using the ctutil -sqlinfo (page 135) switch. After adding the XDD information to the data file, the ctutil -sqllink (page 136) switch is used to make it immediately accessible through SQL. Indices Any operation performed through SQL or from the application uses and maintains existing indices for the best performance. Because of the nature of some COBOL types encoding, an index may All Rights Reserved 81

94 c-treertg SQL Access not sort as an end user expects. c-treeace SQL can still take advantage of these indices to retrieve records while not using them for sorting. This architectural limitation does not have significant impact in practice because the SQL engine is able to build temporary index files on the fly when necessary and use dynamic index techniques. Multiple Record Types An interesting feature available with c-treertg is the ability to have multiple record schemas in the same data file. A common habit used by programmers when memory and storage was at a premium was to combine multiple schemas in the same data file or table. Depending on some criteria, each record was interpreted using a particular schema. This technique was widespread in COBOL using redefines. The XDD file contains information about the different available schemas and their criteria. SQL takes advantage of this information by presenting each schema as a separate table (named as defined in the XDD file). Select statements on one of these tables display only records matching the selected criteria. Inserts into these tables are checked for matching criteria. See the c-treedb Virtual Tables ( technical white paper on the FairCom website. All Rights Reserved 82

95 c-treertg SQL Access Data Conversion Considerations As described earlier, data is stored in its native (COBOL) format. To share data between COBOL and SQL, the SQL engine performs data conversion between COBOL and SQL data types on-the-fly as the fields are accessed. The XDD file is the source of information about all the operations that need to be performed when bridging SQL and COBOL data. There are two sides of conversion: From SQL to COBOL when writing (e.g. INSERT INTO, UPDATE statements) From COBOL to SQL when reading (e.g. SELECT). We will begin by describing the SQL-to-COBOL conversion because there are significant concepts to understand that affect the COBOL-to-SQL side. SQL to COBOL Converting from SQL to COBOL, there are a few situations where determining what the COBOL record buffer should contain is difficult if not impossible. Typical situations are the following: Null Values In SQL it is possible to set fields to NULL indicating missing information or inapplicable information or lack of a value. The same concept does not exist in COBOL where any field has a value. When a field is set to NULL by SQL, the conversion logic needs to know what value to store in the field. In COBOL there is no standard discipline for uninitialized fields (for example, in a table of people, the date of death is unknown while the person is alive). The field may contain a high-value, a low-value, spaces, zeros or even garbage; every application has its own discipline, which may be different on different fields. To address this issue, the XDD file allows you to specify a bindefault or a cbdefault attribute for the field, which will be written to disk in place of a SQL NULL. Essentially, bindefault is a binary value to replace a SQL NULL and cbdefault is a COBOL value to replace a SQL NULL (see <field> schema element (page 240)). If the XDD file DOES NOT specify a bindefault or a cbdefault attribute for the field, it is linked in SQL as NOT NULL, meaning that SQL will refuse any statement setting the field to NULL. If the XDD file DOES specify a bindefault or a cbdefault attribute for the field, the field is linked as nullable, SQL will allow setting it to NULL and conversion logic stores the specified default value in the COBOL record. In this second case, it is worth noticing that if the default value set in the XDD is a valid value for the field type (e.g., 0 for numeric types) when reading the record, it will be converted into that value in SQL and not NULL as it was set. If the value is not a valid value (e.g., spaces for numeric types), a conversion error will occur when reading the record but the engine automatically handles it (unless the XDD specifies otherwise, as explained later) and exposes the value as NULL in SQL, as expected. All Rights Reserved 83

96 c-treertg SQL Access Hidden Fields The XDD file may specify that some of the fields are hidden to SQL, which is to say that SQL is not aware of them. This occurs for filler fields or for redefines where a portion of the record is not redefined or simply because the person who generated the XDD decided not to expose some field (and so the information it contains) to SQL. When updating an existing record, hidden fields are not a problem, because when performing an update, the engine will convert and store only fields that are set by the update; the portion of the record belonging to hidden fields is left untouched. When inserting a new record, hidden fields are a problem: SQL does not pass any value to store in these fields, because the SQL engine does not know they exist. However, the portion of the record belonging to these fields must be set. The engine sets it to the default value specified in the XDD for the specific hidden field which is consistent with the approach taken for NULL values. However, if the default value is not specified you may encounter error (SQL) or possibly 4126, CTDBRET_CALLBACK_18 (page 258) (Missing default for null field). c-treeace SQL Explorer will display the above SQL error if you attempt to insert a record into a table that has hidden fields without default values set in the rules file. To prevent this, add the following to your rules file (page 57): <rule sequence="1002"> <when> <field hidden="true"/> </when> <do> <add> <field cbdefault="0" onconverterror="null"/> </add> </do> </rule> Unsigned Fields Unsigned COBOL types are mapped into SQL types, which are signed. SQL itself does not check for negative values, which are not acceptable values for the COBOL data types. If the XDD has defined a field as unsigned, attempting to set it to a negative value causes a CTDBRET_UNDERFLOW error, which is a common situation of out-of-the-box. Inline Redefines It is possible for the XDD file to specify that the redefines are expanded as further fields in the same table instead of creating a separate SQL table. In this case, attempting to add a new record (i.e., INSERT INTO) fails with error CTDBRET_NOTYET (Not yet implemented). COBOL to SQL When converting data from COBOL to SQL there may be situations where the content of the record cannot be interpreted and/or cannot be mapped into a SQL value. One typical situation is a PIC 9(3) containing three spaces. The conversion does not know what the space is or how to interpret it. It cannot tell if the content is garbage or if it was set on purpose. All Rights Reserved 84

97 c-treertg SQL Access Another common situation occurs with dates, where it happens that although the field content is clear it does not map into a valid SQL value. For instance a PIC 9(10) mapped into a date value may have its value set to 0, which is a good numeric value, but not a valid date! A third common scenario is due to the fact that in COBOL there are different conventions on how to represent numeric data in the record on disk. In particular, programs have different conventions on how the sign for signed numeric values is encoded. The SQL engine must know which convention is in use and the data on disk must adhere to that convention, otherwise a conversion error occurs. The XDD file contains information on how to handle conversion errors at the schema level (for the whole table) or at field level (just for a single field) by setting the onconverterror attribute. onconverterror When a conversion error occurs the engine determines what to do based on the onconverterror setting for that specific field. If the onconverterror for the field missing, it uses the one set on the schema or, when that is missing, it uses the default. The possible behaviors are: "strict" - No action is taken, the error is propagated to SQL and the query fails. This behavior is very strict but very safe. It is good for fields where the value is expected to always be a valid value, such as the fields composing the main index of the table. "null" - The values that cannot be converted are exposed in SQL as NULL. This is very convenient in those cases where the field contains garbage for whatever reason. It is also convenient when there is no way to handle the conversion error differently without touching the data. However this setting makes it impossible for SQL to use any index on the fields to which it applies. "error" - The error is propagated to SQL and the query fails unless the field content matches the bindefault or cbdefault (see <field> schema element) in which case the value will be exposed in SQL as NULL. This is the default behavior when onconverterror is not specified. It is very convenient as is it a good compromise between the strict and the null approaches. It is similar to strict with only one precise exception for values that SQL would store in the record when setting the field to NULL. "value:?" (where? is the wanted value in SQL) - The values that cannot be converted are exposed in SQL as the specified value. This is similar to the null approach but it exposes the content as the specified value instead of as NULL. It has the same pros and cons as the null approach. XDD files that are automatically generated do not contain any onconversionerror setting and do not contain any default value. Hence human intervention is required, at least initially, to properly set it up. Once properly set up and verified that conversion errors are properly handled as desired, it is possible to automate the XDD modification using the rule file, see Define External Rules (page 57). All Rights Reserved 85

98 c-treertg SQL Access SQL Considerations FairCom c-treertg allows the same data files to be accessed by both SQL and ISAM interfaces. This provides a truly flexible development environment allowing you to choose exactly the techniques that best suit their needs for performance and data integrity. The SQL support provided by c-treertg is targeted at the most important SQL functionality, such as reading and writing records using SQL queries, user-defined functions, stored procedures & triggers, views, and joins (including joining a SQL-created table with an ISAM table). There are multiple considerations on SQL tables that are important for you to keep in mind. Please note c-treeace provides full, industry-standard SQL. You can create a database that is accessed exclusively through SQL and it will not experience the limitations listed below. The limitations discussed below arise when accessing data from both SQL and ISAM interfaces on files created through ISAM. Due to the high flexibility of ISAM contrasted with the discipline of SQL, some SQL features cannot be expected to work in this environment. The next sections list important considerations to keep in mind when developing applications using both ISAM and SQL to access the same data. Note: Additional considerations apply if you use the c-treeace Professional Developer's Kit to expose your ISAM data through SQL, giving you full read, write, insert, and delete access to the data. Please contact FairCom ( for a complete list of supported functionality. All Rights Reserved 86

99 c-treertg SQL Access Common Issues The following issues are important to keep in mind when developing c-treeace or c-treertg applications that use both ISAM and SQL to access the same data files created through the ISAM interface. Limitation Table definitions are done through ISAM; any changes to the table definitions must be done through ISAM. This implies: You cannot execute Alter Table over an ISAM data file. Referential Integrity cannot be defined across ISAM tables. Consideration Not all ISAM indices are fully usable in SQL. In particular, some indices do not collate as SQL mandates and therefore cannot be used by SQL for sorting or to perform searches depending on the collation. This does not limit functionality, however it may impact performance. In such a case it is now possible create a new index from SQL that satisfies all SQL needs and apply to both SQL and COBOL access. Has Workaround ISAM files are not portable among different endianess, even if you are using portable data types. You can use the FairCom ctunf1 utility to convert endianess before moving between a big-endian and a little-endian system. In addition, keep in mind that SQL triggers are applicable at the SQL level only. NoSQL APIs (e.g., ISAM, c-treedb, COBOL, c-treedb Java, etc.) cannot invoke a SQL trigger. This is not a limitation specific to c-treeace and c-treertg: a trigger is a SQL concept that is not supported by ISAM. c-treertg SQL Support The c-treertg products allow your COBOL data to be exposed through SQL, giving you full access to read, write, insert, and delete records. This support is targeted at the most important SQL functionality, which includes: Reading and writing records using SQL queries User-defined functions Stored procedures Triggers Views Joins (including joining a SQL-created table with an ISAM table). The topics listed in Common Issues (page 87) are important to remember when developing c-treertg applications that use both ISAM and SQL to access the same data files. In addition, keep the following in mind: A table created through SQL cannot be accessed using c-treertg ISAM drivers for COBOL. SQL triggers are applicable at the SQL level only; your COBOL program cannot invoke a SQL trigger. All Rights Reserved 87

100 c-treertg SQL Access Referential integrity is applicable at the SQL level only; no check is performed to ensure referential integrity. If transactions are disabled on COBOL files, this creates additional considerations to accessing these files at the SQL level: rollbacks do not affect these files; isolation of transaction is not provided; etc. All Rights Reserved 88

101 11. Tips for Advanced Sqlizing This section examines directives you may need to place in the copybook to handle certain situations. Depending on the structure of your data, you may need to use some of these techniques to sqlize your files. The xddgen utility prepares your COBOL data for SQL access by reading your program to determine relationships in the data (similar to a SQL schema). Using this information, c-treertg maps the data for SQL access on-the-fly. Your COBOL program will continue to access the same data, with no changes. The xddgen utility may be able to sqlize your data without any modifications to your copybook. In certain situations, your data may be structured in a way that xddgen cannot interpret correctly. In those situations, you will need to add directives to your copybook to tell xddgen how to handle the data. You will not need to restructure your data or rewrite your program. Adding these directives does not cause any compilation issue to your application because they are comments for the COBOL language. The code used in these examples is provided in the Sqlize Tutorial: <install directory>\<platform>\driver\ctree.cobol\tutorials\sqlize\ SQLIZEEXAMPLE.CBL - An example COBOL program Tutorial.sql - A SQL file for creating and populating the CARDFILE database used by this tutorial. CARDFILE.FD - The file descriptor used by the tutorial. CARDFILE.SL - A select statement used by the tutorial. rules.xml - An XML file containing the rules for sqlizing. Note: The Sqlize Tutorial COBOL source code is compatible with the Micro Focus ACUCOBOL compiler, Veryant iscobol, and Micro Focus starting with Visual COBOL 2010 release. Please check with FairCom regarding future support for other COBOL compilers. See also xddgen (page 147) 11.1 Step-by-Step Sqlizing Instructions This section provides step-by-step instructions for sqlizing your data based on the Sqlize Tutorial (Driver\ctree.cobol\tutorials\sqlize\). It assumes you have already set up the environment where you will run the COBOL application using c-treertg. The process of sqlizing consists of three major steps for the end user (the tutorial below includes extra steps for clarity): All Rights Reserved 89

102 Tips for Advanced Sqlizing 1. Extract the XDD from COBOL (step 2 in the tutorial below). 2. Sqlize it using ctutil (step 4 in the tutorial below). 3. Use SQL to access the database (step 6 in the tutorial below). The SQLIZEEXAMPLE program prints out information about membership cards satisfying a partial card number request. The program itself can create the required file if the file does not exist, however it does not populate the file. 1. Compile the SQLIZEEXAMPLE.CBL program: iscc SQLIZEEXAMPLE.CBL 2. Extract the XDD from SQLIZEEXAMPLE.CBL by running xddgen: C:\FairCom\vx.x.x.RTG\winX64\Tools\cmdline\XDDGEN\xddgen.exe SQLIZEEXAMPLE.CBL 3. Create empty c-treertg files using either the COBOL program or the XDD. a. You can execute the COBOL program one time to create the COBOL files (.DAT and.idx): iscc -ze SQLIZEEXAMPLE.CBL b. Or you can use the XDD to create the file, using ctutil -make: ctutil.exe -make CARDFILE CARDFILE.xdd 4. Sqlize it, using ctutil sqlize and the XDD: ctutil.exe -sqlize CARDFILE CARDFILE.xdd ADMIN ctreesql -rule=rules.xml 5. Populate the table using the SQL script Tutorial.sql: isql.exe -s Tutorial.sql -u ADMIN -a ADMIN ctreesql All Rights Reserved 90