Copyright by Connectivity Systems, Inc. All Rights Reserved

Transcription

1

2 All Rights Reserved RESTRICTED RIGHTS LEGEND Use, duplication, or disclosure by the Government is subject to the restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS This material contains confidential and proprietary material of Connectivity Systems, Inc. (hereafter also referred to as CSI International and CSI) and may not be used in any way without written authorization from Connectivity Systems, Inc. This material may not be reproduced, in whole or in part, in any way, without prior written permission from Connectivity Systems, Inc. Permission is hereby granted to copy and distribute this document as follows: Each copy must be a complete and accurate copy. All copyright notices must be retained. No modifications may be made. Use of each copy is restricted to the evaluation and/or promotion of Connectivity Systems, Inc. s Data-Miner/VSE product or in accordance with a license agreement. DATA-MINER/VSE VSAM/SAM Data Manipulation Tool Version 5 Release 1 First Edition: August 2005 Published by Connectivity Systems, Inc State Route 138, Williamsport OH Phone: Fax: tcpinfo@e-vse.com Internet:

3 DATA-MINER/VSE Version 5 Release 1 Table of Contents Table of Contents DATA-MINER Introduction...1 Batch and Online...1 DATA-MINER Batch...1 DATA-MINER Online...1 Uses...1 Function...1 Note to Users of CSI-DATA/QUICK...2 VSAM/Easy...2 What DATA-MINER Does...3 Features and Capabilities...4 Some Practical Examples...5 Execution JCL...6 Telling DATA-MINER about Your Files...6 Describing the Input File...6 INPUT=type...6 Describing the Output File...7 Defining the Files...7 DATA-MINER Commands...10 Command Coding Rules...10 DATA-MINER Functions...12 Major Functions...12 COPY...12 UPDATE...12 EXTRACT...12 PRINT...13 DUMP...13 INSERT...13 DELETE...13 DATACARDS...13 Field Definitions...13 Naming Fields in Records...14 Relative Fields...15 Literals...15 Creating work fields...17 Record Selectors...18 FIRSTKEY...19 START...19 STOP...19 FIRST...20 LAST...20 SKIP...20 ONLY...20 FREQ...21 KEEP...21 DROP...21 Record Selector Rules...21 Selector Parameters...22 Data Manipulators...24 MOVE...24 i

4 DATA-MINER/VSE Version 5 Release 1 Table of Contents PACK...25 UNPACK...25 SET Command...26 Arithmetic commands...27 ADD...27 ADD literal TO field...27 ADD field1 TO field SUBTRACT...27 SUBTRACT literal FROM field...28 SUBTRACT field1 FROM field MULTIPLY...28 MULTIPLY field BY literal or MULTIPLY field BY field DIVIDE...28 DIVIDE field BY literal...28 DIVIDE field1 BY field More complicated examples...29 IF Commands...30 IF syntax...30 GOTO command...31 EXIT Command...32 Multi-Field Commands...33 INSERT Command...33 VALUES Command...33 SELECT Command...34 WHERE Command...34 Output Commands...36 REUSE...36 MAXLN...36 NONUM...36 DBLSP...36 MAXRCDS...36 CANCEL...37 ABEND...37 TRLRNEW...37 NOHD...37 HD...37 VALIDATE...37 SPACE...38 TOT...38 Formatting Masks...38 Changing the starting location...39 OFFSET=n...39 Parameter Element Descriptions...40 field, field1, field n...40 outloc...40 search...40 Sequence of Execution...42 Sample Print...43 Messages...43 ii

5 DATA-MINER/VSE Version 5 Release 1 Table of Contents Summary messages...43 Error messages...44 Examples of Use...47 Typical JCL Streams...47 Example 1:...47 Example 2:...47 Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example DATA-MINER ONLINE...56 Introduction...56 Features and Capabilities...57 Supported Environments...57 System Requirements...57 How DATA-MINER ONLINE Works...57 A Note about Browsing...58 Sample Screen...58 Screen Description...58 RECORD NUMBER (RCD xxx OF yyy)...58 FILEID...59 OPER...59 VSAM Commands...59 Temporary Storage commands...60 LRECL...60 KEY...61 THRU KEY...61 REMOTE KEYPOS...61 DATA-AREA...61 PF KEYS...61 Data-Miner Online Error Messages...62 Data-Miner Installation...67 VSAM-EASY Overview...69 Introduction...69 The 4GL Connection...70 The COBOL and PL/I Connection...71 iii

6 DATA-MINER/VSE Version 5 Release 1 Table of Contents VSAM Access Options...72 Features and Capabilities...73 How CSI-VSAM/EASY Works...74 How to Use CSI-VSAM/EASY...75 Supported Environments...76 System Requirements...76 CICS Users:...76 COM-PLETE Users:...76 Executing CSI-VSAM/EASY...78 Online CICS Access...79 Introduction...79 CONTROL-BLOCK-CICS...80 RECORD-AREA...85 Batch Access...86 Introduction...86 CONTROL-BLOCK-Batch...86 COBOL Example...86 Assembler example:...86 RECORD-AREA...91 Return Codes...92 Code...92 Explanation...92 Examples of Use...96 Online Command Level COBOL Program...96 Online NATURAL Program (Single File Access)...97 Online NATURAL Program (Multiple File Access)...98 Online Assembler Program...99 Batch NATURAL Program Batch COBOL Program Dynamic Call to VNATB Static Call to VNATB Batch Assembler Program Installation CSI-VSAM/EASY Online Security Security Checking Options Inclusive Security Exclusive Security Security Check Methodology Adding or Changing Security Entries Security Module Source VSAM/EASY Control Table Appendix A: VSAM/EASY Programming Tips CICS Pseudo-Conversational EOT with NATURAL: Avoiding Lockouts: Using CICS Temp Storage for Scrolling: iv

7 DATA-MINER/VSE Version 5 Release 1 Table of Contents Accessing Multiple Files: Alternate Indexes with Non-Unique Keys: NATURAL Packed Fields in Keys: NATURAL Calls: Appendix B: Trouble Shooting VSAM/EASY Appendix C: VSAM Basics File Types Key Sequenced Dataset (KSDS) Entry Sequenced Dataset (ESDS) Relative Record Dataset (RRDS) Reusable Dataset Record Retrieval KSDS (Keyed Sequential Dataset) ESDS (Entry Sequenced Dataset) RRDS (Relative Record Dataset) Appendix D: Reserved Words INDEX v

8 DATA-MINER/VSE Version 5 Release 1 Table of Contents vi

9 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Introduction DATA-MINER Introduction CSI-DATA-MINER/VSE (hereafter DATA-MINER ) is a collection of powerful data manipulation tools. These tools provide easy access to your files without the user needing any programming. Because DATA-MINER can be used by both business users and computer programmers, some areas of this manual are more technical and are addressed specifically to computer programmers. These areas are marked as such, and you can skip them if you are a business user. This manual uses some computer-related technical terms that business users do not need to know to use DATA-MINER. Files can be browsed, changed, copied, or printed. Data can be moved from a file of one type to a file of another type and reformatted at the same time. VSAM and sequential files can be accessed in batch mode. Batch and Online DATA-MINER reduces costs associated with learning and using many complex and unrelated utilities. DATA-MINER consolidates many functions into one simple system, giving you one common method of accomplishing most of your day-to-day data handling and reporting tasks. A popular use for DATA-MINER is to fix records that have become corrupted in some way. It is also a good way to search for records that meet criteria in which you are interested. DATA-MINER works in both batch and online environments. Uses Function DATA-MINER Batch Making a copy of a file Making large numbers of alterations Printing results Changing record values in a large file Change, copy, and print records using just a few commands Create selection criteria and data manipulation commands using IF, AND, OR & NOTT Produce simple reports and lists in minutes Accomplish file conversions and mass changes quickly and accurately Correcting corrupted records Using DATA-MINER Batch allows you set a long-running script in progress and then let the script run by itself. DATA-MINER BATCH can be used by non-technical and technical people to selectively search and retrieve records by record key, record location, or record content. DATA-MINER Online Making changes to a small number of easily identified records Making changes in a small file or with a file in which you know the keys of the records you want to change Debugging and testing VSAM programs Looking at and modifying temporary storage queues Correcting corrupted records DATA-MINER ONLINE is an online VSAM editor that non-technical users can use to display or change parts of VSAM files. 1

10 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Introduction Note to Users of CSI-DATA/QUICK DATA-MINER supports all the commands used in CSI-DATA/QUICK, as well as a large number of commands that are new in DATA-MINER. Although DATA-MINER is capable of running a script containing both the old and the new format commands, we very strongly recommend that you do not mix the commands in the same script. This manual covers the new format commands used by DATA-MINER and the old format of some commands used by DATA/QUICK. To T maintain or understand the commands used by CSI-DATA/QUICK only, see a CSI DATA/QUICK manual. Note There is no need to tell DATA-MINER that you are using DATA/QUICK commands. You can mix DATA/QUICK commands with DATA-MINER commands. However, if you have parameter elements in your DATA/Quick commands that DATA-MINER thinks are DATA-MINER commands, you could get an error message from DATA-MINER VSAM/Easy DATA-MINER uses its own highly efficient VSAM I/O driver called VSAM/EASY. VSAM/EASY is also used by DATA-MINER-ONLINE to access CICS temporary storage files. You can use VSAM/EASY in your own online and batch programs to access VSAM files and temporary storage queues. Complete details on how to use VSAM/EASY with your own COBOL, Assembler, PL/I, and 4GL (such as NATURAL) programs are to be found at the end of this manual, beginning on page

11 DATA-MINER/VSE Version 5 Release 1 What DATA-MINER Does What DATA-MINER Does DATA-MINER is a high performance software utility package that permits quick, easy access to SAM and VSAM files for selective searching, printing, extracting, updating, and copying of file data. DATA-MINER needs no programming and is designed for non-technical computer users, perhaps with a little initial help from the IT department. DATA-MINER has several basic functions: COPY one file to another. The output file can be a different format from the input file. For example, you can change the blocksize or copy a VSAM file to a sequential file on disk or tape. You can also make modifications to the records in the file, such as setting the account balance in every record to zero. UPDATE one or more records in a VSAM KSDS or sequential file. You can choose which records to change by matching, or not matching, criteria that you specify. You do not need to know the key of a record or anything about the file s internal characteristics, other than the record layout. For example, you can change the account balance of every overdrawn account to zero, change the surname of customer from Jones to Brown, or correct a spelling mistake and change Simth to Smith. DUMP a file in hexadecimal to list the internal contents of a file. This function is primarily for programmers. You can select which records to dump. For example, you can dump every record where the zip code is less than DUMP is a very useful function for problem determination. PRINT T one or more records from a file. PRINT is easier to read than DUMP because it formats the data to make it readable and understandable. You can quickly produce a report that shows every overdrawn account and ignores all those that are in credit. PRINT can produce a formatted report with headers, formatted data elements, literals, and accumulated counts and totals. Reports can be configured for hard copy printing or for viewing on a screen. EXTRACT parts of records from a file into another file. For example, you can create a new file from your customer detail file that contains only account numbers, names, and addresses. INSERT new records into a VSAM file. For example, you can insert a description of a new company branch into a VSAM file. DELETE records from a VSAM file that meets a criterion that you specify. For example, you can delete all records in a VSAM file that have not been active in five years. DATA-MINER can be used in both production and development environments. DATA-MINER can copy, dump, or print all or part of a file. Both sequential (SAM) and VSAM files are supported for both input and output. All three types of VSAM files, keyed sequence (KSDS), relative record (RRDS), and entry sequence (ESDS), are supported. DATA-MINER can selectively retrieve records by full key, generic key, relative record number, or record number. DATA-MINER accesses a VSAM file s catalog to establish whether it is KSDS, RRDS, or ESDS. Commands that select records can be used to retrieve specific records or groups of records based on where the records are in the file, on the contents of a record, or a combination of both. For example, records can be selected if they are in the first 1000 records on file or if a given field or combination of fields in a record contains a specific value or values. Logical AND and OR commands can be used with all filtering arguments to allow the most complex filtering criteria. After records have been selected, DATA-MINER can be used to modify the selected records. The modification process can be performed on all or some of the selected records. Records to be modified can be selected using the same types of commands used to select the records. 3

12 DATA-MINER/VSE Version 5 Release 1 Features and Capabilities Features and Capabilities This section lists the features and capabilities of DATA-MINER. DATA-MINER is command driven. All functions can be accomplished without program coding, testing, or cataloging. The commands you give to DATA-MINER are carried out in the same order as you give them to DATA-MINER. For more information on command execution order, see the section Sequence of Execution, 42. DATA-MINER lets you easily access SAM, QSAM, and VSAM files of any organization or record type. VSAM files can be accessed through either primary or alternate index. DATA-MINER examines the VSAM catalog to determine file attributes automatically. DATA-MINER can start a search at any point in the file. A search can be started at any point based on key, record position, record content, or a combination. DATA-MINER can select every n th record so that you can create a smaller version of a production file for testing. For example, you can tell it to pick every 20th record. DATA-MINER s free-form commands can be entered anywhere from position 1-80, with no limit to the number of commands entered. DATA-MINER automatically calculates field lengths and aligns decimal points. DATA-MINER can examine and select records based on position within the file or on record contents. Additionally, DATA-MINER can scan and select records based on a global examination of their contents. DATA-MINER provides flexible report generation capabilities including headers, data formatting, literal insertion, hexadecimal data display, data conversion, and totaling. DATA-MINER can be used to accomplish many useful tasks: Search files for specified character strings Change or delete records on a file Selectively change or reformat SAM or VSAM files Copy or back up files with or without changes Create representative samples of production files Create a selective backup of a file (for example, including only active accounts) Create a copy of a production file with sensitive information blanked out Create and modify comprehensive test files Verify test results Print selected records from any file Print formatted reports with headers and totals Print edited hexadecimal and/or character dumps Accumulate and print totals and record counts Sequence check files DATA-MINER includes the VSAM/EASY tool for making access to VSAM files and temporary storage queues much simpler and faster. 4

13 DATA-MINER/VSE Version 5 Release 1 Features and Capabilities Some Practical Examples Full examples are at the end of the manual, but here are just a few things you can do with DATA-MINER: Create a test data file containing every 100th record from a production file. Create a test data file from a production file but set every customer name to XXX. Remove every record with a negative balance from a file. Create a report showing every customer who owes more than $20,000. Print a hexadecimal dump of every record in a file with a negative balance. This is primarily for programmers use. Increase the length of a field in a file to allow for larger numbers. 5

14 DATA-MINER/VSE Version 5 Release 1 Features and Capabilities Execution JCL DATA-MINER runs as an ordinary batch job. You will need JCL that looks something like this example, though you have a huge amount of flexibility, as you will see later in the manual. // JOB COPY A FILE // DLBL INFILE,'ACCTS.MASTER',,VSAM // DLBL SEQFILE,'ACCTS.MASTER.COPY' // EXTENT SYS020,TST001,,,150,200 // ASSGN SYS020,DISK,VOL=TST001,SHR // EXEC CSIDMBMD,SIZE=500K COPY INPUT=VSAM FILENAME=INFILE OUTPUT=DISK FILENAME=SEQFILE BLKSIZE=5000 VARBLK LRECL=200 BGNFIELDS STATUS 12,4,C ENDFIELDS MOVE 'XXXX' TO STATUS /* This is a basic example. The lines starting with a slash (/) are typical JCL statements needed to run DATA-MINER. The commands, starting with COPY, tell DATA-MINER what you want to do and what files you want to use. In this case, the script copies INFILE to SEQFILE, creating a variable blocked record file from a VSAM input file. The script also puts XXXX in a field called STATUS that occupies positions of each record. Note: The first position in each record is referred to as position 1 unless you override that with an OFFSET=n command. Note: A DLBL or TLBL for the input file is mandatory, and the name on it (the word after DLBL) must be the same as the FILENAME on the INPUT command. A DLBL or TLBL for the output file is mandatory only if you are creating an output file, such as with COPY or EXTRACT functions. The name for it must match the FILENAME on the OUTPUT T command. If you are doing an UPDATE, you only need to specify an input file, because the updated record is written back to the input file. UPDATE does not create a new file. Telling DATA-MINER about Your Files DATA-MINER needs information about the files you want to use, such as what they are called, what type of device they are on, and, possibly, how big the records are. Describing the Input File Every run of DATA-MINER has an input file. DATA-MINER gathers as much information as it can from other places in your system, such as the VSAM catalog. However, the following parameters give DATA-MINER a place to start searching. For example: INPUT=DISK FILENAME=MASTIN BLKSIZE=20000 VARBLK LRECL=500 tells DATA-MINER that the input file is on disk, its DLBL statement refers to it as MASTIN, it contains variable length blocked records, up to 500 bytes long, and the blocksize is 20,000 bytes. INPUT=type This parameter tells DATA-MINER what type of file is to be used for input. This parameter is mandatory. 6

15 DATA-MINER/VSE Version 5 Release 1 Features and Capabilities Example: INPUT=VSAM Type must be CARD/CARDS, TAPE, VSAM, or DISK. VSE/VSAM managed SAM files must be defined to DATA-MINER as DISK, not as VSAM. Card input must immediately follow the DATACARDS command that marks the start of the card data in your script. The DATCARDS command follows the last command in your script. Anything that follows the DATACARDS command is treated as data. Describing the Output File OUTPUT=type This parameter tells DATA-MINER what type of output file to produce. OUTPUT is not needed for an UPDATE run. Example: OUTPUT=TAPE Type must be CARD/CARDS, TAPE, VSAM, or DISK. VSE/VSAM managed SAM files must be defined to DATA-MINER as DISK, not as VSAM. Defining the Files The following commands apply to the INPUT= T or OUTPUTT= command that precedes them. FILENAME=filename This parameter specifies a DLBL or TLBL filename for the file. The filename can be from 1 to 7 characters long. Default is INFILE for input and OUTFILE for output. SYSNO=SYSnnn This parameter specifies a SYSNO, (system number) for the file. This is very useful when opening a tape file that is already assigned. Default is SYS010 for input and SYS012 for output. LRECL=n Note: SYSnnn can be either a valid programmer SYS number (SYS000 - SYS255) or one of the following: SYSPCH, SYSLST, SYSLNK, SYSRES, SYSREC, SYSVIS, or SYSCAT. This parameter specifies the maximum record length for the file. It is mandatory for fixed length records, since it tells DATA-MINER the length of each record It is optional for variable length record files and for VSAM Note: You must specify the LRECL exactly for fixed length blocked records. If BLKSIZE is not an exact multiple of LRECL, Data-Miner will flag it as a serious error and terminate the job. For variable length records, you can either omit LRECL or you can specify a number that is at least as large as the number of data bytes in longest record in the file. If you specify an LRECL that is less than the longest record in your file, long output records will be truncated to this size. Any input records longer than this size will either cause the operating system to abend your job or, under certain conditions, cause the operating system to overlay some of DATA-MINER's working storage. For this reason, we recommend that you specify an LRECL for a variable length file that is somewhat larger than the actual longest record that you expect to process. Specifying an LRECL much larger than the actual longest record will result in a minor increase in CPU time. The default values used when LRECL= is not entered are as follows: 7

16 DATA-MINER/VSE Version 5 Release 1 Features and Capabilities 1. For card input, DATA-MINER assumes that the LRECL is 80 bytes. 2. For fixed length input files, DATA-MINER sets the LRECL to the input BLKSIZE, (unblocked). 3. DATA-MINER creates variable length records with a record length of: BLKSIZE= a. The length generated by any Data Manipulator commands (such as MOVE) if the command makes the record longer b. The length of the LRECL of the input, if there are no Data Manipulators that make the record longer c. In an EXTRACT run, the size of the smallest record needed to accommodate all the selected fields and literals. This parameter specifies the block size for the file. It is not used by VSAM input and output files. We strongly recommend that you specify BLKSIZE for VSE input files because this reduces the amount of memory and the amount of CPU time DATA-MINER uses. The default values used when BLKSIZE is not entered are: 1. For card input, DATA-MINER assumes that the BLKSIZE is 80 bytes. 2. Input files, other than card, use 32K, because DATA-MINER cannot determine the blocksize of an input file without pre-reading it. The file will be processed properly even if the actual blocksize is not 32K. 3. Fixed length output BLKSIZE is equal to: FIXED a. Output LRECL, if entered, (unblocked). b. Input BLKSIZE, if input is fixed length or VSAM. This parameter specifies that the tape or disk file contains fixed length blocked or unblocked records. This is the default for CARD, TAPE, or DISK type files. VARBLK This parameter specifies that a tape or disk is a variable blocked file. VARUNB This parameter specifies that a tape or disk is a variable unblocked file. UNDEF or UNDEFINED This parameter specifies that a tape or disk has an undefined file format. Each physical block in the file is processed as if it were a single logical record. LABEL=NO/STD or LABELS=NO/STD This parameter specifies that a tape is either unlabeled or labeled. Default is standard label. 8

17 DATA-MINER/VSE Version 5 Release 1 Features and Capabilities FSF=n or FSPACE=n This parameter forward spaces a tape n files before open. NOREW or NOREWIND This parameter causes a TAPE file to stay positioned where it is at end of file. This is the default for TAPE files. REW or REWIND This parameter rewinds a tape after a close. Default is norewind. UNLD or UNLOAD This parameter rewinds and unloads a tape after a close, unless unload is controlled by a tape management system. The default is no-rewind. REUSE This parameter resets the end pointer to zero for VSAM output files only. The file must be defined as reusable. Writes to the file start at the beginning of the file, even if data is present. Examples: Note Variable length input records are padded on the right with binary zeroes to assure predictable record selection. INPUT=TAPE FILENAME=TAPEIN BLKSIZE=3000 LRECL=100 REWIND OUTPUT=VSAM FILENAME=MYKSDS OUTPUT=DISK FILENAME=OUTDISK BLKSIZE=6000 VARBLK Note: If you are using disk files, Data-Miner ignores any parameters that are relevant to tape files only. Note: Users of tape and disk management systems such as BIM-Epiccan use those systems to override any parameters specified on the INPUT and OUTPUT commands. 9

18 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Commands DATA-MINER Commands Commands are the means by which you tell DATA-MINER what you want to do with the data. They include commands for record selection, data manipulation, and output specifications. Parameters give more detail of what the command is to do. Parameter elements contain information such as input location, numeric literals, and logical relationships. Collectively, the commands, the parameters, and their associated parameter elements contain instructions for What function is performed How a record search is started, Which records are selected or skipped Any changes to make to the selected records prior to writing them to the output file Where the records are written Performing arithmetic on record and work fields Command Coding Rules DATA-MINER commands must conform to the following rules: 1. Any command or parameter shown in capital letters must be spelled exactly as shown. Example: MAXRCDS= Any command or parameter shown in lower case letters must be entered with your own values replacing the lower case words. Example: MOVE field1 TO field2 You will need to replace "field1" and "field2" with the names of your own data fields. 3. The word script used in DATA-MINER implies a group of commands that tell DATA-MINER what to do with the files named in the INPUT and OUTPUT commands. 4. Commands are free form and can be anywhere from position Commands can be continued across cards and must be split at the end of the parameter, not in the middle of a word. Field definition commands cannot be split across cards. There is no limit to the number of commands per card. There is no need for a continuation character in ca command that is to be continued. 5. Commands must be separated by at least one blank. 6. You can enter as many sets of parameters for a command as will fit on one line. For example, you can enter MOVE AMOUNT TO OUTAMT COST TO OUTCOST to move two fields to the output file with one MOVE command. 7. You can put spaces anywhere in a command to make it more readable: MOVE AMOUNT TO OUTAMT or BALANCE 53, 6, P, 2 10

19 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Commands 8. There is no limit to the number of commands in a script, provided there is enough memory available. Commands require up to a maximum of 128 bytes of memory each. 9. Comments can be placed between the commands by putting an asterisk anywhere before the first non-blank character on a card. After DATA-MINER encounters an asterisk, it ignores the rest of that card. This is a good way to remove a command from a script temporarily. 10. You can put a sequence number on the right end of the card, in columns DATA-MINER ignores columns if they contain a sequence number. 11. You are not allowed to put a single byte containing X'FF' anywhere in an input command because DATA-MINER uses this as an internal flag. It is not possible to put X'FF' into a command using a standard keyboard. 12. DATA-MINER refers to the first byte of a record as byte 1. You can override this with the OFFSET=n command. 13. You cannot use a DATA-MINER command as a fieldname. DATA-MINER s commands are reserved words. For example, START 15,6,C is not a valid field definition and will lead to a Field already defined message. The configuration of the commands, parameters, and elements, which make up these and other commands, is explained in the sections that follow. Every command in your script is executed for each record read from an INPUT file unless the command is jumped over by a GOTO command or an IF command. You can enter commands in any combination of upper and lower case. DATA-MINER is not case sensitive for commands and field names. Literals, however, are processed in the case in which they are entered. For example: Move Name To OUTname is the same as MOVE NAME TO OUTNAME but Move 'John' TO NAME is not the same as Move 'JOHN' TO NAME 11

20 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Functions DATA-MINER Functions Major Functions Function commands say what is done with the input file. You can COPY it, UPDATE it, EXTRACT from it, PRINT it, DUMP it, DELETE records from it, and INSERT records into it. You can do only one of COPY, UPDATE, and EXTRACT. If you do not specify any of these then no output file is produced. If you have entered a PRINT or DUMP command, DATA-MINER prints or dumps the output file or, if there is no output file, it prints or dumps the input file. You can do only one of PRINT and DUMP. If you specify both PRINT and DUMP, you receive DUMP output and no PRINT output. COPY COPY copies the input file to the output file. You can select which records to copy or not to copy using the Record Selectors described later. You can change the records as they are copied and the changes are applied to the output file. You can also change the file format or recording medium during the copy using the INPUT T and OUTPUTT commands. For example, you can copy a VSAM KSDS to a variable blocked tape file. Each record in the output file is a replica of the matching record of the input file unless you make changes to it. If you use COPY to change the format of records from variable length to fixed length, every record in the output file becomes the same size and is either truncated or extended with low values to make it the right size as specified in the LRECL of the output file. If the output record is longer than the LRECL of the output file defined in the OUTPUT command, DATA-MINER truncates the record to match the LRECL parameter. UPDATE Note: DATA-MINER copies the entire input record to the output record before applying any updates, so there is no need for MOVE commands to move individual fields to the output record. If you want to copy only selected fields, use EXTRACT instead of COPY. If you are used to working with DATA/QUICK, notice that DATA-MINER performs copies differently from DATA/QUICK. UPDATE changes records in their original location on a VSAM KSDS or in a sequential file. You can change fields or put extra fields on the end of a KSDS record, for example. You are not allowed to change the length of a record being updated in place in a sequential file. This is a restriction imposed by the operating system. If you try to change the length of a record in a sequential file, DATA-MINER issues an error message telling you which record it has not updated. It applies any updates that do fit inside the original record length and continues processing the remainder of the file. You can update as many records as you like, selecting which records you want to update using the Record Selectors described later. For example, you can set a flag in every record with an account balance of $10,000 or more and leave all the other records alone. To update the file called FILEA, specify a command saying "UPDATE FILEA" in your script after the "INPUT=DISK FILENAME=FILEA" command. EXTRACT EXTRACT is similar to COPY except that the output file contains only fields that you specifically ask for with MOVE or SELECT commands. It provides a useful tool for creating files to send to external service providers without sending them all the information about your customers. For example, a credit card company can use EXTRACT to create a file containing customer name, card number, and expiry date to send to the company that 12

21 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Functions embosses the credit cards. You can also use EXTRACT to change the order of fields in a record, insert a new field, insert a literal into a record, or extend a field. PRINT The PRINT command prints records from the output file, if there is one, or the input file if you have not specified COPY or EXTRACT. Specify one or more SELECT commands to select the fields that you want to print. You can include literals in your print line. The length of the print line depends entirely on the fields that you select to print. DATA-MINER automatically spaces the fields across the print line and, if you do not create your own headings, it puts a heading at the top of each page containing the names of the fields in the report. If you ask to have more fields printed than will fit in the print line, DATA-MINER prints only the fields that do fit. DATA-MINER prints as much of the last field as will fit on the line. DUMP The DUMP command is for programmers and other technical staff. It dumps the output file if there is one or the input file on the print file in hexadecimal and character form. It is formatted in the same way as a program dump with an offset in the left hand column, then 32 bytes (X'20') of hexadecimal data and then a character translation of the hexadecimal. Each dumped record is preceded by a heading line showing its length, its position in the input file and its position in the output file. Note: INSERT Even if there is no output file, you can use PRINT or DUMP, and modify the input records with MOVE and other field manipulator commands. This lets you see what a file will look like if certain changes are made to it. The INSERT command, along with the VALUES command, inserts one or more records into a KSDS. For example, you can use the INSERT command to insert new records describing a new company branch. DATA-MINER inserts records after all the other processing is done to the file. DELETE The DELETE command deletes one or more records from a KSDS. DATA-MINER selects records according to the contents of any record in the field, not just the key.. For example, you can use the DELETE command to delete all records that have not been active in five years. As a safety precaution, DATA-MINER requires that you provide a range or some selection parameters to limit the scope of a DELETE script. This reduces the likelihood that you will delete more records than you intend. DELETE deletes records one at a time. For a large dataset, it is far slower to delete records one at a time than using IDCAMS to clear a dataset by deleting and defining it. DATACARDS The DATACARDS command comes after all the other commands in a script that is using card data input. It is followed by the data cards which, in turn, are followed by a /*. Sample job number 11 in the section Examples of Use, page 47, has an example of card input. Field Definitions There are two ways to define the fields you want to use with DATA-MINER. The first way, using named fields, is the preferred way and will be used in all the examples in this manual. For examples using positional parameters refer to 13

22 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Functions the Data-Quick manual. Whichever type you use, the first character of a record is referred to as location 1 unless you override it with an OFFSET command. Note: Positional parameters are not supported by all the DataMiner commands and are included solely for backward compatibility with Data-Quick. Technical Note You do not need to allow for the record descriptor word of variable length records. DATA-MINER looks after that for you, so byte 1 is the first data byte of a record. There are more details under the OFFSET= command. Naming Fields in Records There are three types of fields available to you in DATA-MINER. Record fields are fields in the records being processed. Work fields are fields you define with the DEFINE command. You can use them for your own calculations and they are carried across from record to record. System work fields are counters used by DATA- MINER for its own internal purposes. They include such things are input and output record numbers and the number of records you want to process. You give a name and format for every field you want to work with so that you can then refer to the field by name. To name record fields, you should include a BGNFIELDS / ENDFIELDS group before any data manipulation commands that refer to the fields. For each record field, you need to specify a name, a starting location in the record, a length, and a format; C for character, B for binary, P for packed decimal, Z for zoned decimal and X for hexadecimal. For numeric fields, (binary, packed, or zoned) you can also specify how many decimal places the field has and which formatting mask you want to use to print that field. For example: Notes: BGNFIELDS ACC_NO 1,8,C BALANCE 32,7,P,2,M5 FLAGS 41,1,X NEW_BAL 50,9,P,2,M3 ADDRESS 120,400,C ENDFIELDS Field names can be up to 32 characters long and contain letters, numbers, and the underscore character ( _ ). Field names must start with a letter. Do not use a name that is the same as the name of a DATA-MINER command; for example, do not name a field MOVE. Do not user a name that is the same as one of the internal field names, such as STOP or START, or you will get a Fieldname not found error message. See Appendix D at the end of this manual for a complete list of reserved words. Fieldnames are not case sensitive. For example, Amount, amount, and AMOUNT are all treated as the same fieldname You do not need to give a name to every field in the record, only those fields that you are going to use in this script. Fields can overlap or have gaps between them. The first byte of a record is byte 1. 14

23 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Functions If your script has an input and an output file, you can use the same fieldnames for both files. Do not try to define the same fieldname twice. You can define the fields in any order. You can combine several fields into a single field; for example, you could combine the NAME, ADDRESS, and CITY fields into one field called ADDRALL. Character fields can be up to 32,767 bytes long, packed fields can be up to 16 bytes long, binary fields can be up to 4 bytes long,, hex fields can be up to 8 bytes long, and zoned fields can be up to 16 bytes long. Technical Note DATA-MINER does not do propagating moves or destructive overlap when moving overlapping fields. There is more information about this in the paragraph on the MOVE command. Note Make sure that you do not try to define a field whose name is the same as any DATA-MINER command or parameter name. For example, do not call a field MOVE, INPUT, RCDNO, or SKIP. If you define a field as packed or zoned decimal and one or more records contain invalid packed or zoned data at that location, DATA-MINER ignores the field and any operations you try to perform on it for that record only. If you refer to a field containing invalid data in an IF or SELECT statement, DATA-MINER will always treat that statement as if the condition had not been satisfied. All other fields are processed normally. Division by zero is recognized as an error DATA-MINER ignores any attempt to divide by zero and the fields involved remains unchanged. Field validation can be turned off using the VALIDATE command. This will lead to slightly better performance by DATA- MINER but DATA-MINER will program check if any invalid data is encountered. Technical Note DATA/QUICK used to cause a data exception if it hit an invalid decimal field. DATA- MINER does not. Relative Fields Relative fields are specific to DATA/QUICK versions of the commands Relative fields do not use field names but use the location of the field in the record, its length, and its type. Using the field names in the Named Fields example above, the following two commands are equivalent: or MOVE BALANCE TO NEW_BAL MOVE 15,6,30 You can mix named fields and relative fields in your script if you must, but we do not advise mixing named fields and relative fields in the same command. Commands containing both named and relative fields make a script harder to understand. Do not split commands that use relative fields across cards anywhere except at the end of a group of parameters; for example, after the final 30 in the MOVE command above. Future releases of DATA-MINER may not support relative fields, and any new facilities added to DATA-MINER will not support relative fields. For example, the MULTIPLY and DIVIDE commands do not support relative fields and there is no support for relative fields with decimal places. Literals A literal is a group of characters or a number that you want to move into a record or compare with a field in a record. You can use either single quotation marks (') or double quotation marks (") to enclose a literal. Numeric 15

24 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Functions literals can be specified simply as numbers without quotes, such as 12 or -15. They do not need to have the same number of decimal places as the field with which they are being used. A non-numeric literal must start and end with the same type of quotation mark. There are several types of literal. In this manual, wherever a reference is made to literal, it means that you can use any type of literal. Character literals Character literals are used for words, names, and account numbers for example. You specify a character literal by putting its value in quotes such as 'ABC'. The literal can contain any characters you want. You can put a quote inside a literal by enclosing the literal in the other kind of quotes. For example, "SUZIE'S DINER" or LOUIS SATCHMO ARMSTRONG. Packed-decimal literals Packed decimal is the most common way to keep numeric data on a file. You specify a packed decimal literal as nnn or P'nnn' where nnn is any decimal number up to 15 digits long. Example: or P or The literal does not need to be the same length or have the same number of decimal places as the other fields with which it is being used. You can move a 3-digit number into an 8-byte field and it arrives with all the leading zeroes and decimal places in place. Packed decimal literals can have a plus or minus sign in front of them, such as -23. If you put in a - sign, DATA-MINER puts the sign X'D' on the end of the number. If you put in a + sign or do not put in a sign, DATA-MINER will make the number positive with either a C or an F sign. You can put a decimal point in a numeric literal. The decimal point in the literal will be aligned with the decimal point in the field definition. For example, if you define WEIGHT to have three decimal places and MOVE 12.5 to WEIGHT, the value of WEIGHT is If you try to compare a packed decimal literal with a field that does not contain a valid packed decimal number, DATA-MINER treats the comparison as FALSE. The hardware regards any string of numeric digits with A, B, C, D, E or F in the low order half-byte as valid. Even though A, B, and E signs cannot occur as the result of an arithmetic instruction, DATA-MINER also regards them as valid. However, DATA-MINER does not generate A, B or E signs itself. Zoned decimal literals Zoned decimal literals are similar to packed decimal literals except for the way the number is stored in the file. They are most commonly used for numbers that never, or rarely, change such as account numbers. You specify a zoned decimal literal as Z'nnn' where nnn is any decimal number up to 15 digits long. All notes about packed decimal literals apply to zoned decimal literals, too. Hexadecimal literals Hexadecimal literals are used for flags and other technical pieces in records. They are specified as X'nnnn' where nnnn is any even number of hexadecimal digits (0-9, A-F). Hexadecimal literals cannot contain decimal places. Example:X'1FA2' Binary literals Binary literals are used for fixed-point binary numbers. You specify the value as B'nnn,' where nnn is any decimal number that DATA-MINER will convert to binary for you. Binary literals can contain signs and/or decimal places. Example: B 3214 or B

25 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Functions Numeric literals Numeric literals are numbers specified with or without signs or decimal places. They can be used with named fields only. A numeric literal can be between minus and plus 2, 000,000,000. Example: MULTIPLY PRICE BY Creating work fields You may want to create a work field to contain such things as running totals, a value from a header record to compare with a value from a trailer record etc. This is done with the DEFINE command. DEFINE command The format of the DEFINE command is: DEFINE fld,length,type[,decpl][,mask] Each work field is defined separately. They can be defined anywhere in the script before they are first used, although where they are defined makes no difference to how the script runs. The field name follows the same rules as record fields. Decpl is optional and is valid only for packed, binary and zoned fields. The default value is 0. Mask is also optional and is valid only for packed, binary and zoned fields. The default is M0 meaning that the field will be printed with leading zeroes suppressed and with the same number of decimal places for which the field is defined. Examples:- DEFINE Counter,8,P Packed number, 8 bytes long, no decimal places DEFINE COMPANY,32,C Character field, 32 characters long DEFINE Difference, 4,B,5,M5 4 byte binary number with 5 decimal places, printed as a currency amount, with two decimal places. A work field can be used anywhere that a record field can be used - you can add to it, add it to another field, move it, compare it etc. Numeric and hex work fields are set to zero at the start of a script while character work fields are filled with spaces. Work fields are not reset for the duration of the script so that you can use them to keep running totals or to remember the contents of fields from one record to another. System work fields DATA-MINER makes a number of its internal counters available for you to use in any way you like in your own script. You can query them and you can update them (although updating some of them may not be a particularly meaningful thing to do). Some system work fields have commands to set their initial value (e.g. MAXRCDS=200). If you want to change the value of these work fields as your script is running, you should use the Data Manipulator commands (MOVE, ADD etc) to change them. Example: IF INPUT_RECORD > 500 says that the following things are to be done only on the 501st and later input records. 17

26 DATA-MINER/VSE Version 5 Release 1 DATA-MINER Functions The system work fields are :- INPUT_RECORD - the number of the record from the input file currently being processed. The first record is record number 1. Skipping records in your script does not stop INPUT_RECORD from being updated. For example, if you skip record number 1000, the next record you read is record number OUTPUT_RECORD - the number of the next record that will be written to the output file. If you skip writing a record to the output file, OUTPUT_RECORD is not updated. For example, if you skip writing record number 1000, the next record is also record For this reason, OUTPUT_RECORD may not be the best system work field to use to decide on which records to skip. MAX_RECORDS - the maximum number of records you want to write to the output file, set by the MAXRCDS command FIRST_RECORD - the first record from the input file that you want to process, set by the FIRST command LAST_RECORD - the last record from the input file that you want to process, set by the LAST command RETURN_CODE - allows you to set your own return code from your script. That means you can identify your own errors or suppress non-zero return codes that DATA-MINER generates. VALIDATION_ERRORS - this is a count of the number of field validation errors that have occurred in your script. DATA-MINER validates every numeric field it uses (unless you tell it not to with the VALIDATE=NO command) and keeps a total of how many errors it finds. It may find more than one error in any given record and so VALIDATION_ERRORS may exceed the number of records in a file. T0 - T9 - the internal total fields are 8 byte packed decimal fields with no decimal places that are included mainly for compatibility with older versions of DATA-MINER. You can use them to keep running totals of whatever you like. System constants DATA-MINER has a number of system constants that you can refer to in your script. SPACES - a string of 256 spaces that you can use in MOVE, IF, and selector commands NULLS - a string of 256 nulls (X'00') that you can use in MOVE, IF, and selector commands. Do not modify these system constants since they are used internally by DATA-MINER and the effect of changing them will be unpredictable. Record Selectors After you have said what you want to do with the file, such as COPY or UPDATE, you can select records on which to do that. Record selection commands let you control where in the file a search starts and/or ends. You can specify that a search is to begin at a specific key, or is to start at a specific record, based on either relative record number or specific record content. You can likewise end a search based on either relative record number or specific record content. The commands execute as shown in Sequence of Execution. Each type of record selector command can be used any number of times in a script and must be followed by one or more selector parameters. You can use several record selectors of different or the same type in your script. See the Record Selector Rules section below for more information. Record selector commands are executed in the order in which they are coded. A SKIP or ONLY command executed before a START T or STOP command can cause some records to skip the START or STOP compares. After a record 18