JCL for installing FTE/MFT on z/os. Colin Paice

Transcription

1 JCL for installing FTE/MFT on z/os Colin Paice September 17th 2014

2 Note Property of IBM Take Note! Before using this User s Guide and the product it supports, be sure to read the general information under Notices. V1 Edition, September Printed September 17, 2014 Sending your comments to IBM. You can send your comments electronically to idrcf@uk.ibm.com. When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you. You may continue to use the information that you supply. Copyright International Business Machines Corporation All rights reserved. Note to US 77 Government Users Documentation related to restricted rights Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule contract with IBM Corp. The following terms, used in this publication, are trademarks or registered trademarks of the IBM Corporation in the United States or other countries or both: IBM z/os MQSeries WebSphere Java, and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both EXPORT REGULATIONS You agree to comply with all applicable export and import laws and regulations. Notices This report is intended to give guidance on the configuration of WebSphere File Transfer Edition V7, and WebSphere Managed File Transfer V8. References in this report to IBM products or programs do not imply that IBM intends to make these available in all countries in which IBM operates. Information contained in this report has not been submitted to any formal IBM test and is distributed as is. The use of this information, and the implementation of any of the techniques, is the responsibility of the customer, and depends on the customer s ability to evaluate and integrate them into their operational environment. The following terms, used in this document, are trademarks of the IBM Corporation in the United States or other countries or both: 1

3 Contents Note Notices What is FTE and MFT Overview of configuration Before you start Check the queue managers have connectivity to each other How to configure FTE Check you have enough disk space Create and check the directory for storing FTE data Review FTE configuration Create a credential file for MFT V Editing the JCL Customising your FTE Collect the information needed for configuration What is in the JCL Tailor the JCL file Create an agent Test the file transfer Migrating to MFT V

4 What is FTE and MFT In WebSphere MQ V7 there was File Transfer Edition(FTE) which could be used to do FTP like activity over WebSphere MQ. In WebSphere MQ V8 this was renamed as Managed File Transfer(MFT). This document uses the term FTE to mean either. FTE runs as java programs in Unix System Services(USS) on z/os. This document uses the JZOS facility to run these java programs. For information on JZOS see JZOS Batch Launcher and Toolkit function in IBM SDK for z/os, Java Technology Edition, Installation and User s Guide. SA See Overview of configuration The key stages in configuring FTE are Edit a PDSE member to specify configuration data. Define the coordination queue manager. Define the command queue manager. Configure one or more agents. Optional: configure a logger task to store data in DB2. MFT requires a queue manager to act as a coordinator, and a queue manager to act as command queue manager. These can be the same queue manager. If you are adding a z/os queue manager to an existing FTE environment you need to define connectivity between the z/os queue manager and the coordinator and command queue manager(s). You need a directory to contain the configuration files such as /u/fte. You can have one or more FTE environments configured (each with their own coordination queue managers). Each environment configuration is stored in a directory with the name of the coordination queue manager. So if the coordination queue manager is called MQPH, The environment name is configured with the -p.... parameter, and the files for -p MQPH would be stored under /u/fte/mqph. This can be a QSG name, but it must be a valid queue manager name, so All characters must be alphanumeric (uppercase A through Z, 0 through 9) and it must start with an alphabetic character. Before you start Check the queue managers have connectivity to each other Ensure you have connectivity between the queue manager in the FTE environment. This will require channels, and these channels may be triggered when messages arrive on the transmission queues. How to configure FTE FTE configuration uses files in USS and PDSE datasets. Most of the configuration and operation is done using JCL from a PDSE. You need to be familiar with working in an USS environment. You can access OMVS from ISPF and use oedit and obrowse the standard ISPF editor and browse; or you can use a Telnet type session using commands on your workstation such as Telnet Putty or SSH, and using editors like vi. 3

5 You need to be familiar with the following USS commands:- Command Function ls -ltr path Lists information about the files in path ls -ltrd directory Lists information about the specified directory rather than the files in the directory. find path -name xxx Search for file named xxxx in the path path. xxx is case senstive and can be like *zzz. chmod xxx file Change the access permissions of a file. chmod -R directory The -R is recursive, and changes all of the files in the directory path chown xxx file Changes the owner. This can be used to change which group of users can access the file. chown -R directory Changes the owner. The -R is recursive, and changes all of the files in the directory path df -k path Reports how much free space remains in the file system containing path. The -k reports it in KB. du -kt path Reports the sizes of directories under path. The size is reported in KB oedit filename Use the ISPF editor on filename from OMVS obrowse filename Use ISPF browse on filename from OMVS rm file Remove (delete) file rm -r directory Removes the specified directory and any subdirectories. If you get USS messages saying you do not have the necessary permission, use the ls -ltr command to check that the files or directories have the correct group access specified, and the permissions (read and write) have been specified. Check you have enough disk space Check you have enough disk space available on the file system where you are going to store the configuration specific files. If an agent trace is enabled then, by default, it can use 100MB of disk space. The configuration files are small, a few KB in size. If you are planning to using two agents and a logger then you need at least 300MB. The z/os systems programmer can configure the file system containing your USS directories to expand when the file system fills up. If so you do not need to worry about having enough space as it will expand as needed. You can use the USS command df -k path where path is the location of the installation specific files. Create and check the directory for storing FTE data You need a directory for storing the FTE data. If have not created it, use the mkdir command; for example mkdir /u/fte mkdir /u/fte/mqph Check you have enough space in the file system using a command like df -k /u/fte On a test system this gave Mounted on Filesystem Avail/Total Files Status /u/fte (OMVS.PLEXCA.FTE) / Available This showere there are K pages available. If the file system fills up, then, depending on the configuration, it may automaticall expand in size. So insufficent space may not be a problem. Display what permissions people have on this directory using the command 4

6 ls -ltrd /u/fte/mqph if the owner or group is not correct use the command chown owner:group /u/fte/mqph For example chown paice:mqprogs /u/fte/mqph. If the read, write and execute permissions for the group are not correct use the following command to set the permissions. chmon 770 /u/fte/mqph Check files exist and you have access to them. customisation for example the location of java. Use the ls -ltr for the files you will be using during ls -ltrd /java/java71 bit64 GA/J7.1 64/bin gives drwxr-xr-x 4 SYSTASK TSOUSER 8192 Nov For /java/java71 bit64 GA/J7.1 64/bin the drwxr-xr-x means d this is a directory rwx the owner SYSTASK has read, write and execute access to the directory r-x people in the group TSOUSER can read and execute files in the directory r-x universal access; anyone can read or execute files in the directory Review FTE configuration There are three common FTE configurations which can overlap A single queue manager with local connections. The agent runs on the z/os queue manager. This might be used to put the contents of a data set into WebSphere MQ queues. A single queue manager with a FTE client on a distributed machine. A local queue manager and a remote queue manager with a client into this remote queue manager. FTE can use multiple queue managers One or more to transfer the data, for example on each machine. Commands queue manager. Requests such as to start a transfer get sent to this machine, and the commands are routed to the agents Coordination queue manager. This manages the work. You can use the same queue manager for transferring data, commands and coordination. You need a coordination queue manager. If you use only a single queue manager, you can use the queue manager to run agents, to do coordination and to process commands. If will be using multiple queue managers then you need to pick one to act as the coordinator. There is not much message traffic associated with the coordination and command queue processing. 5

7 Create a credential file for MFT V8 For MFT Version 8 you need a credentials file. This file is used to hold userid and password information. It is optional, but it is easier to define this before you customise the environment, and if you have the file, you get fewer messages. This file can be in USS, but you can make this more secure by using a member in a dataset. You can then use your security manager to protect the data set. Create a PDSE with format VB and logical record length(lrecl) 200. Create a member BFGCRED within the data set and add <?xml version="1.0" encoding="ibm-1047"?> <tns:mqmftcredentials xmlns:tns=" xmlns:xsi=" xsi:schemalocation=" MQMFTCredentials.xsd"> </tns:mqmftcredentials> There is an sample file MQMFTCredentials.xml in the./mqft/samples/credentials/ directory. If you edit the member of a dataset, you can use the copy edit command to copy from the USS file. You can update this member at any time. Any job or service using this will need to be restarted to pick up any changes. You can use the same dataset(member) for all agents and logger tasks, or each task can have its own dataset(member). Editing the JCL Most of the configuration is done using JCL, but you may need to update the configuration files in USS for some advanced options. You can use the same JCL for V7 and V8 but with some minor changes. A typical file in the JCL library looks like // INCLUDE MEMBER=BFGJCL // DD *. $FTE_PROD/bin/fteBatch SetupCoordination //MAINARGS DD * -p MQPH -f //* V7 Optional -default -coordinationqmgr MQPH -coordinationqmgrhost WINMVSCA -coordinationqmgrport coordinationqmgrchannel SYSTEM.DEF.SVRCONN //* V8 required -credentialsfile filepath //* V8 Optional -userid username -newmquserid user -newmqpassword password -obfuscate The statements following //MAINARGS DD * are the parameters to the command. 6

8 The statements following //* V7 optional can be specified for V7. To configure the JCL for V8, the statements following //* V8 required must be copied after the contents of //MAINARGS statement. For V8 the statements following //* V8 optional can be specified for V8 and are not valid for V7. For example for V8 the above JCL would look like // INCLUDE MEMBER=BFGJCL // DD *. $FTE_PROD/bin/fteBatch SetupCoordination //MAINARGS DD * -coordinationqmgr MQPH -p MQPH -f -credentialsfile // PAICE.FTE.CREDENTIALS.VB //* V7 Optional -default -coordinationqmgrhost WINMVSCA -coordinationqmgrport coordinationqmgrchannel SYSTEM.DEF.SVRCONN //* V8 required -credentialsfile filepath //* V8 Optional -userid username -newmquserid user -newmqpassword password -obfuscate Customising your FTE Collect the information needed for configuration The name of the queue manager used for coordination. This is specified in the -p commands. option of the The name of the queue manager used for processing commands. The name of the queue manager(s) used for the agent(s). (Optional) the name of the queue manager used for logging. These can all be the same queue manager. The name of the USS directory containing the FTE product code. /opt/ibm/mftv800. (FTE PROD). You can use the For example /HMF8800 or jar tvf /HMF7704/lib/com.ibm.wmqfte.bootstrap.jar command to display the dates from the FTE jar files. Check the the dates to make sure they are at the right level (eg this year). The FTE configuration files will be stored in USS (FTE CONFIG). The location in USS of the Java files such as /java/java71 bit64 sr1/j This will have a bin sub-directory. 7

9 The location of the WebSphere MQ USS files. Such as /mqm/v7r1m0/java/lib/. You can use the following jar command to display date information from the jar files. jar tvf /mqm/v7r1m0/java/lib/com.ibm.mq.jar The High Level Qualifier for the WebSphere MQ libraries, such as MQM.V800.SCSQAUTH. What is in the JCL The JCL library contains all the JCL and definitions to run the FTE commands. The members have the following usage BFG1* Setting up the coordination queue manager BFG2* Setting up the command queue manager BFG3* Setting up an agent. BFG4* Commands for all/any agents BFG5* Creating File transfer requests BFG6* Miscellenous, such as using ANT and using templates BFG7* Setting up a logging task. BFG9* Migrating from FTE V7 to MFT B8 Tailor the JCL file. Use a PDSE for each FTE environment, so you could use a PDSE with name MQM.FTE.TEST to store the JCL for your TEST FTE environment. Create a member called AAA and insert JCL for your environment //MYJOB2 JOB 1,MSGCLASS=H // SET JCLPDSE=MQM.FTE.TEST // JCLLIB ORDER=(&JCLPDSE) Create the job card and other information your environment needs. Change JCLPDSE to the name of the PDSE. This information is used when the FTE jobs include JCL and definitions. Edit member BFG1COR and type COPY AAA BEFORE.zfirst;save;exit copy this string to your clipboard. Press enter. This will add the contents of member AAA to the top of each of the member, save it and exit. Edit the member again to check the changes were successful. Edit the BFGn* members in turn, where n is a number. Paste the string from your clipboard and press enter. Repeat this with member BFGMQUT. Edit member BFGZENV to specify your environment. This is a shell script. The directory in FTE PROD=..., will have several directories including bin. So if FTE PROD is /HMF8800, the USS command 8

10 ls -ltr /HMF8800/bin/fteBatch will show the file ftebatch exists. Edit member BFGJCL. Change the libraries for WebSphere MQ. You can add DB2 libraries at a later date if you decide to use a DB2 logger. If you are setting up a coordination queue manager Review member BFGMQCO and change any WebSphere MQ object definition parameters. The default definitions will work unchanged for most people. If you want to use FTE in a QSG, add a valid CFSTRUCT() and QSGDISP(SHARED) to each queue definition. You can specify a STGCLASS if you want to put the queue on a particular page set. The WebSphere MQ definitions change the namelist SYSTEM.QPUBSUB.QUEUE.NAMELIST. Display the namelist SYSTEM.QPUBSUB.QUEUE.NAMELIST and note the entries. If you have any entries, add them to the ALTER NAMELIST(SYSTEM.QPUBSUB.QUEUE.NAMELIST) + NAMES(SYSTEM.BROKER.DEFAULT.STREAM+,SYSTEM.BROKER.ADMIN.STREAM,SYSTEM.FTE) % If using a QSG make this change on all queue managers in the QSG. statement. Edit member BFGMQUT, specify the coordination queue manager name, Uncomment the statement with BFGMQCO. For example //CSQUCMD //*SQUCMD //*CSQCMD DD DISP=SHR,DSN=&JCLPDSE(BFGMQCO) DD DISP=SHR,DSN=&JCLPDSE(BFGMQAG) DD DISP=SHR,DSN=&JCLPDSE(BSGMQLO) Submit the job, check the output and correct if required. If using a QSG make run this on all queue managers in the QSG. Submit member BFG1COR. Check the output files and resolve any problems. You should get return code 0. The output in SYSOUT should contain a lines JVMJZBL2999T Found environment start string.... JVMJZBL2999T Found environment stop string. The lines between them this will be the variables exported from the shell script. You can check these variables to make sure they are correct. Ignore statements about WebSphere MQ definitions. This produces verbose output in SYSOUT. Once you have the JCL running successfully, edit BFGJCL and remove,parm= +T to get less information. Submit member BFG2CMD. This sets up the configuration file for the command queue manager. Resolve any problem 9

11 Create an agent Each agent has a unique name. In the JCL examples it uses a name of MYAGENT. You can change this name, and you can have multiple agents. Repeat this section for each agent you create. Copy the members BFG3* to be your agent name such as Required BFG3CA to MYAG1CA for Create Agent BFG3SA to MYAG1SA for Start Agent BFGMQAG to MYAG1MQ for the WebSphere MQ definitions Optional: you can edit the JCL now, or when you need to use the members. BFG3CLE to MYAG1CLE for Clean Agent BFG3DA to MYAG1DA for Delete Agent BFG3PI to MYAG1PI for Ping Agent BFG3SAT to MYAG1SAT for Set Agent Trace Level BFG3SHOA to MYAG1SHO for Show Agent Details Edit these members and change MYAGENT to your agent name, if required. Review member MYAG1MQ and change MYAGENT to your agent name. If you want to specify which pageset to use, add a STGCLASS entry to each definition. Edit member BFGMQUT. Change the input file to use the WebSphere MQ definitions from the previous step, and comment out the BFGMQCO statement. For example //*SQUCMD //CSQUCMD //*CSQCMD DD DISP=SHR,DSN=&JCLPDSE(BFGMQCO) DD DISP=SHR,DSN=&JCLPDSE(MYAG1MQ) DD DISP=SHR,DSN=&JCLPDSE(BSGMQLO) Change the queue manager name to the where the agent will connect to. Submit the job and check the return code is 0. Submit MYAG1CA to create the agent, Edit MYAG1SA, check the jobname is unique. The JCL to start an agent needs a different job name to the other jobs because if there are two jobs with the same job name, the second job will wait until the first one has finished. If the agent is running, then other jobs with the same name will not run. Submit the job. This job is long running it should start and stay running. You stop the agent using the MVS command P jobname. Submit MYAG1LI. This will list the agent status. Test the file transfer Edit BFG5FT to create a file transfer specify your agent name in -sa, for example MYAGENT specify your agent name in -da, for example MYAGENT check the directory name in -dd for example /tmp/jcl. The files get stored here. If your userid does not have access to /tmp/jcl you can use your home dirctory. Remember to delete the files once the transfer has finished. 10

12 change // MQM.V800.SCSQLOAD to a (small) data set Check the agent is running, and you have connectivity between the queue managers. Submit the BFG5FT job If the agent job is using no CPU, check that the channels are started between the queue managers. Check the files are in the output path /tmp/jcl If you want to use an WebSphere MQ Queue as a source or destination, you will need to configure the agent you want to use WebSphere MQ Queues as source or destinations. See enablequeueinputoutput in the info centre. Migrating to MFT V8 Create a copy of the JCL library so you have a copy before you make any changes. Check the product pre-reqs. Java and WebSphere MQ. Edit member BFGZENV and specify the location of the V8 libraries. For example export FTE_PROD="/HMF8800" Create a VB PDSE and create a member BFGCRED. This member is needed for migration. You can specify a minimum configuration and change it later. For example <?xml version="1.0" encoding="ibm-1047"?> <tns:mqmftcredentials xmlns:tns=" xmlns:xsi=" xsi:schemalocation=" MQMFTCredentials.xsd"> </tns:mqmftcredentials> Edit member BFG9MCOR. Specify the values for your environment. Submit the job. STDOUT should have BFGCL0478I: Migration of xxx has been completed. Copy BFGM9MAG to be agent specific, for example MYAG1MIG. Configure the parameters. Submit the job, and check the output. Start the agent 11