IBM Student Mainframe Challenge Part Two Time to complete about six to eight hours The company that you work for acts as a broker for various suppliers, and provides a website that customers can use to purchase products that they sell. In this part of the Mainframe Challenge, we will use CICS Transaction Server to update order information, create a batch job to extract sales data from a DB2 database and then generate a sales report from it. Before we do this though, let's have a brief look at some system settings to introduce you to how a z/os system can be configured by its administrators. Help You may find the following references useful when completing the tasks: z/os v1.11 Information Center: http://publib.boulder.ibm.com/infocenter/zos/v1r11/index.jsp CICS Transaction Server v3.2 Information Center: http://publib.boulder.ibm.com/infocenter/cicsts/v3r2/index.jsp
GETTING STARTED To test out your new z/os skills, I'd like you to work with the order processing team, but first Gemma will hand you over to Carlos, one of our system programmers. To check your progress and understanding, I have some questions for you to answer along the way. Before you can start this part of the contest you'll need to obtain the questions that you need to answer along the way. These are available in a question and answer sheet in the sequential data set called ZOS.CONTEST1.PART2.ANSWER.SHEET. Using ISPF take a copy of this data set, which you can complete as you progress through the tasks. To copy a data set, use utility 3.3 from the ISPF primary panel. (Type =3.3 at the command line in ISPF.) You can also copy PDSE members from this panel, by putting the member name in brackets after the data set name. Call your sequential data set UKxxxxx.UKxxxxx.ANSWERS It should be allocated in tracks (TRKS) It should have a primary quantity of 1 and secondary quantity of 1 It should have 0 directory blocks The record format should be FB (fixed block) and the record length should be 80 The block size should be 32000 The data set name type should be blank Ensure that the first TWO qualifiers of your data set are your user id otherwise your answers may not be marked correctly.
If it all goes wrong, don't worry! Delete the dataset by finding it in utility 3.4, and typing D in the command column on the left. Then try again! You will be advised when you should be able to answer each of the questions, but please read the comments at the top of the answer sheet carefully before proceeding. Fill in your z/os user id (i.e. UKxxxxx) in the space provided above question 1.
INTRODUCTION TO SYSTEM PROGRAMMING Mainframe systems are designed to be used by large numbers of people. Most of those who interact with mainframes are end users - people who use the applications that are hosted on the system. However, because of the complexity of the mainframe software that supports the large number of users and applications, a variety of roles are needed to operate the mainframe itself. One of those roles is System Programmer that's me! I'm Carlos; I'm responsible for the installation, customisation and maintenance of z/os on our mainframe. z/os provides many commands and facilities to help the System Programmer achieve this and we will now look at a few of these. When answering the following questions you might find the MVS book within the z/os Information Center useful to understand a command's syntax, what it is used for, and the output that it produces. You can find the Information Center at: http://publib.boulder.ibm.com/infocenter/zos/v1r11/index.jsp
USING SDSF In this section, you may find yourself having to repeatedly swap between two different applications, for example, between the answer sheet data set in the editor, and SDSF to find out the answers to the questions. It can get frustrating to keep having to go in and out of different applications and back again, so there is a better way to do it: Pressing F2 will cause the screen to "split" at your cursor, showing the two screens separated by a horizontal dotted line. You can then press F9 to swap between the two screens. To maximise the available screen size, put your cursor on the very top line of the screen before pressing F2. You'll never know how you lived without it! SDSF Primer System Display and Search Facility (SDSF) provides information that you can use to monitor, manage and control a z/os system. You can use SDSF to view the system log and the status of jobs that are running in the system or have finished running, including any held output of jobs that have not yet been purged (discarded). You access SDSF from the ISPF primary panel with the command S. From there, DA can be used to view the active (running) jobs in the system, including any started tasks, which are a special type of job. ST can be used to view the status of jobs, those that are active, those yet to be processed, and those that have completed.
Once you are viewing a list of jobs, at the command input prompt you can filter the list of jobs to just those that you own by entering: OWNER UKxxxxx To revert back to showing all jobs enter: OWNER * Similarly you can filter the list of jobs by their name by entering: PREFIX ABC* which would show only those jobs whose names begin with the letters ABC. To remove this filter enter: PREFIX * You can use both the owner and the prefix filters at the same time. These are particularly useful because they hide jobs on the system that you are either not interested in or not authorized to look at. Be aware that SDSF remembers the filters that you are using. If you are not seeing a job that you expect to see, set both filters to a single asterisk (*) so that all jobs are displayed. In the DA and ST panels you are presented with a list of jobs. Use the PF7 and PF8 keys to page up and down. To view more information about a particular job you can enter S in the left-hand column (NP) and press Ctrl. This can provide a lot of data and so alternatively you can enter a question mark (?) instead, which will give a listing of the various sections of the output. You can then use S to select these sections individually. Use the PF3 key to exit the screen that you are on and return to the previous one.
The prefix and owner filters aren't displayed anywhere by default, so if you leave the filters on you might end up with a blank job list! ISPF remembers your filters between sessions, so look in the Options menu at the top of the screen and choose option 5 to display them.
INTRODUCTION TO IPL When you turn on your desktop or laptop at home the operating system starts via a process known as booting. This process uses a boot record on disk to locate the operating system and initiate the startup process. On z/os this is known as IPLing, and the equivalent of a boot record is known as a load member. IPL stands for Initial Program Load, but people use it as a verb: to IPL meaning to start the mainframe. It's a good place to start learning about system programming, so we'll have a look at the info provided to our z/os system when it was IPL'd. From the z/os Primary Option Menu in ISPF select the option for SDSF and then the ULOG option that opens the User Session Log, which can be used to look at just the output for commands that you issue. This saves having to scroll up and down the main system log, which can have lots of entries, to locate the appropriate lines. To look at the IPL information for the z/os system issue the /DISPLAY IPLINFO command, or if you prefer, /D IPLINFO instead. The response will be output to your session log, take a moment to have a read of it. Now answer question 1.
In addition to the load member, the system also requires one or more data sets that are collectively known as the System Parameter Library (PARMLIB). These data sets contain various members that detail additional configuration information for the different aspects of the system. If you're more familiar with Linux or UNIX, the PARMLIB on z/os is a bit like the /etc/ directory. Now answer question 2. Display commands are listed under MVS System Commands in the z/os Infocenter
The PARMLIB configuration settings on z/os are divided into different members for different aspects of the system. z/os builds the names of these members using specific prefixes combined with customizable suffixes. Most of these suffixes are defined in the system parameter list, which can be found in one or more members whose names start with IEASYS. The use of suffixes such as this allows an operator to IPL a z/os system with different settings simply by specifying different suffixes for the IEASYS members. The default IEASYS suffixes are generally specified in the load member but an operator may override this when an IPL is started. The DISPLAY IPLINFO command you issued a minute ago displayed the list of suffixes this system used. Look at the top of the ULOG for the output you should see a line for IEASYS LIST You should see that this list contains one entry for the suffix 00; the (OP) afterwards, if present, indicates that it was specified by the operator, not by the load member. From this we can determine that when the system was IPLed it read a member called IEASYS00. Locate and browse this member in the PARMLIB data set. You should be able to see how the suffixes for the other configuration settings are defined. Now answer question 3.
System resources When going to buy a personal computer most people like to look at the specifications, such as how much memory it has. z/os refers to memory (RAM) as real or central storage. The physical System z machine can run a large number of z/os systems, known as logical partitions (LPARs). Each LPAR has a subset of the physical resources allocated to it, either exclusively or shared with other LPARs. To display the amount of real storage that has been made available to the LPAR for the z/os system you are using you can use the DISPLAY M=STOR command. Issue this command and then answer question 4. z/os also provides a UNIX environment, known as UNIX System Services (USS) or OMVS (it used to be called OpenEdition MVS). You will get a chance to use USS in part 3 of the contest. Just like UNIX systems you can configure many options, such as how many processes can run concurrently, which is not related to the number of concurrent jobs that you identified earlier. These can be found in one of the PARMLIB members, but it is much simpler to issue the DISPLAY OMVS,L command to view the configured limits. Issue this command and then answer question 5.
BATCH JOBS In this part of the contest you will be using batch jobs to perform work. To run a batch job you have to submit it and it then gets queued up to be executed until an initiator (a process that runs jobs) becomes free. To help the system balance and prioritise the various jobs that might be submitted each job is assigned a classification (class) and each initiator is configured to run jobs of one or more classes. This allows the System Programmer to easily configure the system to allow, for example, twice as many jobs of one class to be run at the same time as that of another, less important classification. If a job is submitted for a class for which there is no initiator it remains in the queue indefinitely until it is either purged or an initiator for it is made available.
You can see the various initiators that have been configured on our z/os system using another SDSF option. See if you can guess which one! Exit out of the user session log and select this option from the SDSF Primary Option Menu. From this screen you can see the initiators, their status and what job classes they have been configured to run. An active initiator is currently running a job, an inactive one is currently idle (waiting for work), and a drained initiator is currently disabled. Now answer question 6. There's plenty more to system programming than what you've seen here, but I think that's enough for now. I'll hand you back to Gemma, and you can run some jobs of your own! Don't break anything!
PART 2 - TASKS Welcome to the order processing team!...or, as I like to call it, the awesome processing team! Our company's strategy is to provide a website that allows customers to place orders for products sold by several other retailers. The order processing team can review recent orders and change their state as they are processed. Information about the products that are sold by each retailer and the orders that customers have placed are stored in a DB2 database. The order processing team uses a CICS transaction which connects to DB2 to extract and update the necessary data.
INTRODUCTION TO CICS CICS (Customer Information Control System) Transaction Server is IBM's general-purpose transaction processing software for z/os. It is a powerful application server that meets the transaction-processing needs of both large and small enterprises. It builds on z/os and System z facilities to provide high availability and scalability at a low cost per transaction; it supports large transaction volumes with a fast and consistent response time. Transaction processing is a style of computing, typically performed by large server computers, that supports interactive applications. In transaction processing, work is divided into individual, indivisible operations, called transactions. For example, withdrawing cash from an ATM and updating the balance in a database. By contrast, batch processing is a style of computing in which one or more programs processes a series of records (a batch) with little or no action from the user or operator. For example, printing out bank statements for every account holder and sending them in the mail. Transaction processing middleware such as CICS allows application programmers to concentrate on writing code that supports the business, by shielding application programs from the details of transaction management: It manages the concurrent processing of transactions. It enables the sharing of data. It ensures the integrity of data It manages the prioritization of transaction execution. When a transaction starts processing, CICS runs a program that is associated with the transaction. That program can transfer control to other programs in the course of the transaction, making it possible to assemble modular applications consisting of many CICS programs. At any time, in a CICS system, many instances of a transaction can run at the same time. It would be terrible to have to wait for other people to finish paying before you could buy something online.
CICS Primer Now's a good time to give you some guidance on using CICS. You start a CICS transaction by pressing the CLEAR* key to clear the screen, and entering the transaction identifier, either by itself or followed by data, on the command line of the screen. The command line is a single line, usually at the top of the screen. You can type the transaction identifier by itself and follow the prompts until a complete transaction command is built up, or you can type the complete transaction command on the command line. If you do not enter enough information, or if the information you enter is wrong, you are prompted to complete or correct your input. For example, in the following transaction, CESF is the transaction identifier and the additional data is GOODNIGHT. CESF GOODNIGHT This transaction is used to log off CICS.
When the transaction starts, it processes the additional data. At the completion of this transaction, you get the following message: STATUS: SESSION ENDED After a transaction has completed, press the CLEAR* key to get a blank screen in readiness for the next transaction. To end a transaction use the PF3 key. * The special 3270 CLEAR key is not on most keyboards. However, 3270 emulators either provide this via a context (right-click) menu or allow you to map it to a key of your choice, often Pause/Break by default. You can find more information on CICS in the CICS Transaction Server v3.2 Information Center.
Using CICS Open a second 3270 session but instead of entering TSO at the SELECT APPLICATION prompt enter LOGON APPLID(CICSZ057) instead. You should then be presented with the CICS sign-on screen: Enter your user id and password and press enter (remember this is the Ctrl key). The screen should clear except for the following message near the bottom: DFHCE3549 Sign-on is complete (Language ENU).
Viewing and updating orders in CICS Once you have successfully logged on to CICS, run a transaction called ORDS. This is the transaction we use to display recent customer orders. We'll change the state of a couple of orders, then I have some questions for you to answer! Item TN032 from Camping Stuff has been on order for S Carter for quite a long time and she has now decided to give up and cancel the order. Change the status of this order from On Order to Cancelled. The order for item GF157 from Sally's Sports by J Bloggs has been accidentally cancelled by one of your colleagues. Change the status of this item from Cancelled to Packing. Now answer questions 7 to 9.
INTRODUCTION TO JCL How would you achieve a task that was given to you? I'd break it down into one or more steps, each completing a part of the bigger picture. Sometimes I'll need different tools and resources for each step. Job Control Language (JCL) is how you describe this information to z/os, whereby a job represents a task. For every job that you submit, you need to tell z/os where to find the appropriate input, how to process that input (that is, what program or programs to run), and what to do with the resulting output. You use JCL to convey this information to z/os through a set of statements known as job control statements. JCL's set of job control statements is quite large, enabling you to provide a great deal of information to z/os. Most jobs, however, can be run using a very small subset of these control statements. Within each job, the control statements are grouped into job steps. A job step consists of all the control statements needed to run one program. If a job needs to run more than one program, the job would contain a different job step for each of those programs. Job Control Statements Every job must contain: - A JOB statement to mark the beginning of a job and assign it a name. This statement is also used to provide certain administrative information, including security, accounting, and identification information. Every job has one and only one JOB statement. - At least one EXEC (execute) statement, marking the beginning of a job step, a name for the job step, and the program or procedure to execute. You can add various parameters to the EXEC statement to customize the way the program executes and the conditions under which the step should be run.
Most programs require some input or generate some output and so usually also contain: - One or more DD (data definition) statements, to identify and describe the input and output data to be used in the step. This statement may be used to request a previously created data set, to define a new data set, to define a temporary data set, or to define and specify the characteristics of the output. Each job control statement has five fields: 1. An identifier field, which is typically two forward slashes (//). The entire line is treated as a comment if the two slashes are followed by an asterisk (//*). 2. A name field, identifying the statement so that it can be referred to later 3. An operation field identifying the type of statement, i.e. JOB, EXEC, DD. 4. A parameter field 5. Anything following the parameter field is treated as a comment and ignored. JCL statements can be coded only up to column 71. You may question why JCL has this format but it was historically input using punch cards that had 80 columns, the last 8 of which were used for line numbers. JCL's layout can look a bit confusing when you see it for the first time. Luckily the ISPF editor can display JCL syntax highlighting if you type HILITE JCL (or HILITE AUTO) when editing (not browsing) a JCL member. For more information on JCL you can look in the JCL Reference section of the MVS book in the z/os Information Center.
GETTING STARTED For Part 2 we're going to be using both JCL and REXX, so we can create datasets to store both of them now. We won't put anything in the REXX dataset until later. If you've forgotten how, go back to Part 1! Use ISPF to create two new PDSEs called UKxxxxx.PARTTWO.JCL and UKxxxxx.PARTTWO.REXX They should be allocated in tracks (TRKS) They should have a primary quantity of 1 and secondary quantity of 1 The record format should be FB (fixed block) and the record length should be 80 The block size should be 32000 The data set name type should be LIBRARY Extract the order data from DB2 The head of Sales and Marketing is asking me for a report of the total sales to date this year for each retailer. She's also interested in which items the retailers are selling most and least of, so that marketing campaigns can be targeted. We did something similar to this last year......aha! We used an existing application to extract sales data from the database, then ran another program to produce the report from that extracted data. They're both run within batch jobs using JCL. Let's get them working again... The existing program to extract all sales data since the beginning of the year from the DB2 database is called DB2EXTR and is stored in ZOS.CONTEST1.PART2.LOAD. In order to execute this program you will need to submit a JCL job.
There isn't any JCL to run the first program, so you'll need to write some yourself. Don't worry, hardly anyone writes JCL from scratch these days it's much easier to modify a job that already exists! ZOS.CONTEST1.PART2.JCL(RUNDB2AP) is an example of a job to run a DB2 application, so it's a good starting point for your job. Copy this into your JCL library, and rename the member to DB2EXTR. You can rename the new copied member either by giving it a different name in the copy dataset ISPF panel, or by browsing the dataset it has been copied to and putting r next to the member. If you choose the second option, you'll have to type REFRESH on the command line to update the list with the member's new name.
or
The first line in the job contains a JOB statement that assigns a name to the job. Change the name of the job to DB2EXTR. The next statement in the job is an EXEC statement, which marks the beginning of a step in the job and causes a program to be executed. In this case, the program being executed is IKJEFT01, the TSO Terminal Monitor Program. This program will execute the commands specified in the SYSTSIN data definition statement. The DSN command connects the application to a particular DB2 subsystem. Change the name of the subsystem to DB1S. The RUN command runs the application program. Change both the name of the program and the DB2 plan name to DB2EXTR, and the library where the program can be found to ZOS.CONTEST1.PART2.LOAD. Submit the DB2EXTR job and check the output. There doesn't seem to be any output from the program execution, and it executed for zero time. I don't think the program was executed by the job... It turns out that this job contains a parameter that requests that the system validates the syntax of the JCL you have written without proceeding to execute the job. This parameter can be useful for checking for errors when writing or modifying jobs, but you will need to remove it and submit the job once again in order to run the program. Check the job output in SDSF to see if it completed successfully. Hmm, more problems. Your job isn't complete yet!
The DB2EXTR program outputs the data it extracts from the database to a file called ORDERDAT. However, this isn't the name of the data set because z/os uses symbolic file names to provide naming redirection between the file name used within the program and the actual data set used during the execution of the program. This indirection allows the same program to be run against different data without needing to change it or add support for passing in many parameters. The JCL data definition (DD) statement allows you to define a file in a job step that will be available to the program being executed, and specify the data set that this file refers to. Your job was unable to open the output file as there was no corresponding DD statement in the job step. Therefore, you need to add a DD statement named to allocate a data set that the program can use for output. Call new new DD statement ORDERDAT, and specify the necessary parameters so that the data set will be allocated with the following attributes: Use the DSN parameter to specify a data set name of UKxxxxx.HIST.ORDS Use the SPACE parameter to specify a primary and secondary space allocation of 1 cylinder. Use the DISP parameter to allocate a new data set when the job step is executed and catalog the data set when the step has completed successfully Now would be a good time to look in the z/os Infocenter if you haven't already, under JCL reference in the MVS section. Here's a hint, the DISP parameter will need two subparameters! While you're doing this, answer questions 10 and 11.
A brief explanation of storage management When you look at the details of the DISP parameter of the data definition statement in the MVS JCL Reference book, you will see references to SMS-managed data sets. Data sets on z/os are allocated on volumes, in the same way that files on your PC are allocated on your hard disk. However, a z/os system will have tens, or even hundreds of volumes available. This makes it difficult for users to know which volume to use when allocating new data sets, and for storage administrators to balance free space across volumes. SMS (System Managed Storage) solves this problem by enabling the system to choose which volume a new data set should be allocated on using various criteria set by the storage administrator. Data sets allocated in this manner are called SMS-managed data sets. Most data sets on modern systems, including any that you allocate, will be SMS-managed. You will also see references to cataloging data sets. The catalog is the z/os component that allows you to locate which volume a data set with a particular name resides on. This allows you to refer to the data set in a job using only its name. It is possible to allocate a data set without adding it to the catalog, but in order to reference an uncataloged data set you would need to specify both the data set name and the volume where it resides. Now that you've made those changes, submit the DB2EXTR job again. It should with a return code of zero, which indicates that it completed successfully. Browse the output data set to confirm that the program has successfully extracted the sales data from the database, and to familiarise yourself with the format of the data. Note: If you run the job more than once, you will need to delete the data set it creates first. If you do not do this, you will get a JCL Error and a message in the job output indicating that the new data set couldn't be allocated due to a duplicate data set name.
INTRODUCTION TO REXX REXX has been around for a while, but it's still widely used on the mainframe. I like it because it's pretty laid-back about syntax rules variables are case-insensitive, don't have fixed datatypes and don't have to be defined before they're used! It's also really handy for parsing text data, there are plenty of built in functions for that. REXX (which stands for REstructured extended executor) is an interpreted programming language that was designed by IBM to be easy to read and learn while also providing powerful capabilities. As it is an interpreted language there is no need to compile the program before it is executed the REXX command processor directly processes each language statement as it is encountered during the execution of the program. However, there is a REXX compiler available should it be desirable to improve the program's performance or source code security. For more information on the structure and syntax of the REXX language, and descriptions of the operators and built-in functions provided, refer to the TSO/E REXX User's Guide and the TSO/E REXX Reference sections of the z/os Information Center. Some useful REXX tips: Issue the HILITE REXX command in the ISPF editor to get syntax highlighting when editing REXX code. A comma at the end of a line indicates that the current line of REXX code is continued on the next line. Variables of the format x.y in REXX are called compound variables, and are similar to arrays. Check the REXX User's Guide and REXX Reference for details of how to use these variables. Produce the report from the extracted data Now you've extracted the sales data from the database, you need to run another program to generate the report. I can give you the program and the JCL job this time! REXX is a language that is very well suited to the task of producing reports like these due to its character manipulation capabilities.
Copy the job from ZOS.CONTEST1.PART2.JCL(GENREPRT) to your JCL library, and the program source from ZOS.CONTEST1.PART2.REXX(GENREPRT) to your REXX library. You'll notice that this job has two steps. The first step deletes the report data set if it exists already, which is very useful if you need to submit the job more than once! Replace the ++XXXX++ placeholders appropriately in the job, calling your report data set UKxxxxx.SALES.REPORT and submit it.
I said I'd give you the JCL job, I never said it would work first time! Correct the error, and submit the job again. It looks like the REXX program didn't work first time either. We didn't pick up on this earlier because REXX is interpreted it hadn't reached this point when we found the first error. If we'd compiled it we'd have found out. Correct the error in the program, and any others you may encounter. Your report says that the best selling items have fewer sales than the worst selling ones! This report is sorted backwards! Please fix it. This time it should execute without any errors, and a sales report should be generated. However, it doesn't look as though the report of the best and worst selling items has been sorted correctly. Correct this problem in the REXX code, and resubmit the job.
Improve your JCL Great job! You fixed up the JCL and the REXX, and managed to generate the report the boss wanted! But I reckon you can improve the JCL even further can you combine this job and the DB2EXTR job we used earlier? Instead of having to submit two separate jobs to extract the sales data from DB2 and generate the report, this can all be done in one job that has two steps (three steps if you include the first step that deletes the current report data set). Change your GENREPRT job so that it performs both tasks. You might find it helpful to also modify the first step of the job so that it deletes both the old report and extracted sales data sets. The dataset which has sales data extracted from the database is passed to the second step which generates the report. It doesn't need to be kept after the job has finished. A special type of data set called a temporary data set can be used for this purpose. Temporary data sets are created and deleted within a single job. Change the data set used to store the sales data extracted from the database to be a temporary data set. While you're doing this, answer questions 12 and 13. Hint: You will need to change the name of the data set to follow a certain format as this is what tells the system that the data set is temporary, and also change the data set disposition such that the data set allocated in the first step is passed on for use in the second step.
Submit your answers Review your answers and then submit them so that they can be marked. Your report will also be checked but you do not have to submit it in the same way. Do however ensure that it contains data for the whole year and not just for a single month. To do this go to option 6 (Command - Enter TSO or Workstation commands) and enter: XMIT ZOS19.MATTK DA('UKxxxxx.UKxxxxx.ANSWERS') Congratulations! You've finished all the steps for Part Two! Now we can target our advertising towards the most popular products that we sell. If you're one of the first to complete the task and answer the questions correctly, I'll send you a prize in due course! You've achieved a lot so far, but there's still plenty more to do! Part Three builds on what you've been learning, and gets even more challenging! You'll be adding functionality to the customer website, and using another of IBM's world-famous products! I'm recommending you to our events team to carry on to Part Three of the contest you'll need access to their mainframe systems. Please send an email to your contest task ID to request a new user ID, including in your email your current user ID. Good luck!