My Java App Called Your COBOL DB2 Stored Procedure

Transcription

1 Session: F13 My Java App Called Your COBOL DB2 Stored Procedure John Mallonee Highmark Inc. May 22, :00 a.m. 9:00 a.m. Platform: Cross-platform Abstract See the experiences of a Java development team that had the guts to call DB2 for z/os stored procedures written in COBOL. See how the application was developed, challenges faced ranging from Workload Manager to developer diversity, and tips for support. Java experience is preferred, but not a must. 1

2 Objectives Objective 1: Learn why and where a COBOL stored procedure makes sense for Java applications. Objective 2: Learn the basics of building a COBOL stored procedure. Objective 3: Learn the basics of calling a DB2 stored procedure from a Java application. Objective 4: Understand the potential challenges. Objective 5: Develop a methodology for ongoing support What are COBOL SP's and Why use them - Explain what a COBOL stored procedure is and why we choose this solution for Java/web application. 2. The COBOL/DB2 side - Look at the definition of the stored procedure in DB2 and how the COBOL application was designed. 3. The Java side - Look at how the Java application was designed, how the JDBC API was used, how transaction management was implemented, and how problems were resolved. 4. Challenges Encountered - Review the challenges and issues that came up and how they were solved. Look at how to avoid some of these challenges. 5. Support Hints - Discuss how to support these cross-platform applications once they're in production and some tools that can help. 2

3 Progress Why, What DB2 COBOL Java Challenges Support 3 Objective 1 Let s start by taking a look at what DB2 Stored Procedures (on z/os) are and why we choose them for our project. 3

4 What and Why Review DB2 Stored Procedures on z/os Business Needs of our project Project Overview Our Environment Decision Process Why to use DB2 Stored Procedures Why to not use DB2 Stored Procedures 4 4

5 Review COBOL DB2 SP External stored procedure definition and program code are separate CREATE PROCEDURE SYSIBM.SYSROUTINES, SYSIBM.SYSPARMS WLM Workload Manager Address space for stored procedures COBOL code SQL, packages Created (global) temporary tables result sets SQL CALL from client application See IBM Redbook SG Review COBOL Stored Procedures on z/os Our use of stored procedures was for external stored procedures indicating that we coded the stored procedure in a language other than SQL, in our case COBOL. The CREATE PROCEDURE statement is used to inform the system of the name of the load module and what parameters are expected when the procedure is called, as well as other execution and environment options. As a reminder, the DB2 catalog tables used to store the details of the declared stored procedure are SYSIBM.SYSROUTINES and SYSIBM.SYSPARMS We are currently at DB2 version 8, so we use DB2 s Workload Manager (WLM) instead of the older SPAS (Stored Procedure Address Space) Our SQL was coded in COBOL programs and we bound packages for each program containing SQL. We used created/global temporary tables to return result sets from the COBOL program to the calling Java program. The SQL CALL command is used to invoke the stored procedure from WLM within a DB2 connection in Java. 5

6 Business Need New application Reuse mainframe COBOL SQL on Java/web application Existing subprograms, DB2 packages Avoid duplication Development Maintenance Business risk Performance 6 Business Need Our business need was to create an online application for a web browser which is our standard for new applications. However, there were existing COBOL subprograms that contained SQL needed by the new application. Our choice was to make that SQL reusable by both COBOL components and Java components. This minimized duplication of effort both at development time and for ongoing maintenance in the future. Additionally, we wanted to minimize risk to the business functionality so that COBOL code and Java code didn t function differently as the application changed over time. Performance is always a consideration. Since we had existing SQL on the mainframe, we didn t want to lose the performance that already existed, especially given the added costs and risks of duplication. 6

7 Project Overview Internal customer billing application Web user interface Shared SQL batch COBOL z/os online Java/HTML Linux Reuse existing business/database logic 16 DB2 Stored Procedures 8 read 8 insert/update Mainframe and Java developers 7 Project Overview The application to be built was an extension to an existing application. The new functionality revolved around tracking customer invoices through a browser-based interface and was to be used specifically for internal users (Intranet). There were also to be batch components to the application, so there was a need to share DB2 SQL between the batch processes (COBOL on z/os) and online processes (Java on Linux). Some of these processes already existed, and we didn t want to duplicate the code between COBOL and Java. Ultimately, we ended up with 16 stored procedures, half of which were SELECT statement based, and half of which were INSERT or UPDATE statement based. The project team was composed of developers that were mixed between COBOL development and Java development. 7

8 Our Environment Mainframe z/os version 1.8 Test and production LPAR s DB2 version 8 full function mode LE COBOL Open Systems DB2 Connect version 8.2 on zlinux Red Hat Linux version WebSphere Application Server 6.0 Java JDK 1.4 JDBC Java EE Environment Our environment is obviously mixed between mainframe and open systems. Our mainframe environment is a multiple LPAR z/os configuration where our COBOL applications access DB2 version 8 using separate sets of packages for each collection (multiple collections for test, one for production). Our open systems environment includes a DB2 Connect server to communicate between WebSphere and DB2 on z/os. Our open systems applications are Java EE applications running in a WebSphere Application Server using the JDBC API. 8

9 Why use DB2 SP s SP s do the job Uses existing skills DB2, COBOL, Java, SQL Minimize middleware use Minimize COBOL coding Intra-application communication not shared across apps Reusable for standards-based SOA with wrappers See Update at end Reusable by any application or tool using DB2/SQL Application Query Tool Reporting Tool 9 Why to use DB2 SP s We had a lot of discussion about whether to use Stored Procedures as compared to our standard component methodology for this application. Some of the pros to using Stored Procedures were that they matched our current skill set and that the team saw them as being less complicated than using messaging middleware. Since the database processes were cross-platform, but not crossapplications, there was not a pressing need to support the messaging approach. Additionally, even though the Stored Procedures are not reusable by other applications as is, they can be adapted to be shared if the need arises in the future through a SOA approach. They also have the added benefit of being used for query and report tools that support SQL CALL commands. 9

10 Why to Not use DB2 SP s Tied to DB2 as a DBMS Not our standard methodology use of WebSphere MQ Not exactly standard for SOA (but pluggable to SOA standards) We chose To use DB2 Stored Procedures 10 Why to not use DB2 SP s A drawback from an architecture perspective is that the application is more tightly coupled to a particular DBMS (DB2). However, since we re not a product vendor, there is less risk in needing to deploy our application on a different DBMS than DB2. As mentioned in the previous slide, our standard methodology is to use messaging middleware to loosely couple applications, so this approach deviates. Additionally, this approach doesn t immediately support typical SOA standards such as web services, but they can be extended to fit into that approach. Ultimately, the choice was made to move forward with them, so let s move on 10

11 Progress Why, What DB2 COBOL Java Challenges Support 11 Objective 2 Let s now take a look at the mainframe side and how the stored procedures were defined in DB2 and how the programs were coded in COBOL. 11

12 The COBOL/DB2 Side Defining the Stored Procedure to DB2 (v. 8) Designing the COBOL application Dynamically called subroutines Different collection id's Interface input/output, messages, errors, result sets Coding the COBOL application PROCEDURE DIVISION USING statement DECLARE Temporary Table DECLARE cursor Data retrieval Data updates 12 COBOL/DB2 This section will discuss what we did on the mainframe side (COBOL and DB2) to configure and build the stored procedures to be later called from the Java application. Defining the stored procedure to DB2 using the CREATE PROCEDURE command Designing the COBOL application that will return results (parameters and result sets) including how we used dynamically called subprograms and how we designed the interface to the procedure The actual COBOL program code with examples from the passing of parameters to the SQL to retrieve and update data. 12

13 DB2 Define Stored Proc CREATE PROCEDURE TST1.GET_LIST ( IN COL_1 CHAR(9), IN COL_2 CHAR(16), OUT RETURN_STATUS CHAR(8), OUT RETURN_MESSAGES VARCHAR(1002), OUT COL_5 DATE ) DYNAMIC RESULT SET 1 LANGUAGE COBOL EXTERNAL NAME MYPROG PARAMETER STYLE GENERAL NOT DETERMINISTIC FENCED 13 CREATE PROCEDURE CREATE PROCEDURE stores the procedure definition in the DB2 catalog tables. In this case, the owner is the TST1 collection and the procedure name is GET_LIST. There are 2 input parameters and 3 output parameters with varying field types. DYNAMIC RESULT SET 1 this indicates that one result set will be returned by the COBOL program, meaning that a cursor will be open to a global temporary table which can then be read from the Java application (estimated upper bound) LANGUAGE COBOL COBOL is the programming language for the stored procedure EXTERNAL NAME MYPROG MYPROG is the name of the program/load module that the Workload Manager will call when the stored procedure is invoked PARAMETER STYLE GENERAL parameters are 13

14 DB2 Define Stored Proc # 2 READS SQL DATA NO DBINFO COLLID XXX WLM ENVIRONMENT XXXXXX ASUTIME LIMIT STAY RESIDENT YES PROGRAM TYPE SUB SECURITY DB2 COMMIT ON RETURN NO We varied between test and production COBOL doesn t manage transaction RUN OPTIONS 'MSGFILE(SYSPRINT,,,,ENQ)'; 14 CREATE PROCEDURE - continued READS SQL DATA program within procedure reads data through SQL (use MODIFIES SQL DATA for INSERTs/UPDATEs) NO DBINFO no database info such as database name, application id, version are passed to procedure COLLID XXX collection name WLM ENVIRONMENT XXXXXX - WLM environment name ASUTIME LIMIT STAY RESIDENT YES program will not be loaded for every call which should perform better in higher volume situations when used in conjunction with PROGRAM TYPE SUB PROGRAM TYPE SUB indicates the program can be called as a subprogram (vs. main program) see STAY RESIDENT SECURITY DB2 authentication to stored procedure call is using DB2 security COMMIT ON RETURN NO transaction is not managed by the stored procedure (managed by caller) RUN OPTIONS 'MSGFILE(SYSPRINT,,,,ENQ) specific runtime options 14

15 COBOL Design Application Load module per stored procedure Dynamically called subprograms Dynamic load module SQL Packages Different collection id's One per test environment One for Production Application Interface Transaction Management calling application 15 Application Design Each of our stored procedures consists of an entry level load module to process the SQL CALL and multiple subprograms which are called dynamically. DB2 is accessed via SQL through packages at the subprogram level. Collections are pervasive in our test and production environments and were tied to our stored procedure definitions. We had to be careful especially when switching environments, or supporting multiple environments. We use a distinct collection per test environment and a different one for production. We maintain separate load libraries per environment and have separate stored procedure definitions per environment. The application interface is described on the next slide. To be reusable, our stored procedures don t maintain transaction state. In other words, the stored procedures don t commit updates, but allow for transaction management to occur in the program or application that calls the stored procedure. In our application, that means that we do the commits from the Java application using container management from within a session bean. 15

16 COBOL Design Application Application Interface Input parameters Output parameters Business data Status OKAY, NOTFOUND, WARN, ERROR Error Handling/Messages (multiple occurrences) Error messages include embedded DB2 messages Result set(s) 16 Application Design Interface The interface from a calling program to our stored procedures included input parameters, output parameters, and optional returned result sets. Standard output parameters for all of our stored procedures were a status code and set of error messages The status code includes common values to indicate that a call was successful, that a requested key wasn t found from the stored procedure, or a value indicating a warning or error in which case a message(s) is included in the error message section Error messages might also include other values for business conditions or errors vs. infrastructure errors. Error messages include SQL error codes that occur from SQL issued by the stored procedure. In this case, the SQL code and any associated text was included in the error message as compared to being returned as a separate code. The Java application responds to the value of ERROR returned in the status code. 16

17 COBOL Design Application Stored Procedure PROCEDURE DIVISION USING IN, OUT parameters SQL Package Dynamic Subroutine Dynamic Subroutine SQL Package DB2 IMS File 17 This is a high level depiction of the major components of the z/os stored procedures. Stored Procedure this is the load module level program which is triggered by the SQL CALL statement from Java. It accepts the input parameters and returns the output parameters through the PROCEDURE DIVISION USING statement (through LINKAGE SECTION fields). Dynamic Subroutine this is a program that is dynamically called by the Stored Procedure load module program. This means that the subprogram is a separate load module as well as a called subprogram. The SQL coded to read or update a DB2 table is coded in a dynamic subprogram and has a corresponding bound DB2 package. DB2, IMS, File These represent the various data sources that are accessed by the subprograms. In our example, we didn t read IMS or flat files, just DB2, but they are available options. 17

18 COBOL Code Application PROCEDURE DIVISION USING statement DECLARE result set cursor optional DECLARE Temporary Table optional SELECT/INSERT/UPDATE data from/to DB2 or obtain from other source MOVE data to INOUT/OUTPUT parameters INSERT to temporary table (result set) optional 18 COBOL application The following components of the COBOL application are further described with examples in the following slides PROCEDURE DIVISION USING entry to the stored procedure with parameters passed through linkage section fields DECLARE CURSOR for stored procedures returning result sets, the definition of the cursor to be made available to the calling program DECLARE GLOBAL TEMPORARY TABLE for stored procedures returning result sets that are built in the procedure. In other words, the returned data is not a simple SELECT to a table or tables, but involves procedural processing or other translation of the raw data. We tried to avoid these where possible to avoid the overhead of building the temp table(s). SELECT/INSERT/UPDATE subprograms read data from DB2 (or insert or update) and translate it if necessary INPUT/OUTPUT parameters single iteration data was returned to the calling program through output parameters Result set(s) when data to be returned was more than a single iteration, a result set cursor was populated and returned 18

19 COBOL Parameters IDENTIFICATION DIVISION. PROGRAM-ID. MYPROG.... PROCEDURE DIVISION USING VAR-1 VAR-2 RETURN-STATUS RETURN-MSGS VAR-5 Input Output 19 Parameters As indicated, input and output parameters are defined in the PROCEDURE DIVISION USING statement. Each parameters is also defined as a LINKAGE SECTION field. The order of the parameters in the USING statement match the order passed from the SQL CALL statement in the calling program. 19

20 DECLARE Temp Table DECLARE GLOBAL TEMPORARY TABLE SESSION.XXX_RETURN_LIST ( COL_A CHAR(5) NOT NULL, COL_B CHAR(10)NOT NULL, COL_C DATE NOT NULL, )ON COMMIT DROP TABLE Temporary table is dropped when main application commits Declare temporary table to store results set 20 DECLARE GLOBAL TEMPORARY TABLE For stored procedures that in some way translate the data to be returned, global temporary tables are defined to contain the returned data. The temporary table is tied to the current session and contains columns that are defined by the COBOL program, similar to defining other permanent tables to DB2. The ON COMMIT DROP TABLE clause indicates that the temporary table will be dropped once the current transaction is committed (by the Java program). This indicates that there is overhead for declaring, populating, and dropping temporary tables for each stored procedure invocation. 20

21 DECLARE CURSOR DECLARE REVIEWLIST CURSOR WITH RETURN FOR SELECT COL_A, COL_B, COL_C FROM SESSION.XXX_RETURN_LIST EXEC SQL OPEN REVIEWLIST END-EXEC Declare cursor for returned results set 21 DECLARE CURSOR returned data For stored procedures that return a result set(s), a CURSOR is declared per result set. Declaring the cursor is the same as a typical cursor in a COBOL program with the addition of the WITH RETURN clause. Note that this example is returning a cursor that selects data from a global temporary table that has been populated with the result set. 21

22 INSERT to Temporary Table INSERT INTO SESSION.XXX_RETURN_LIST SELECT COL_1, COL_2, RETURN_STATUS, RETURN_MSGS, COL_5 FROM XXX_TABLE_A FROM... WHERE... Populate results set into temporary table 22 INSERT to Temporary Table Data to be returned as a result set is inserted into the declared global temporary table. This example shows a simple case of inserting directly from a SELECT statement against a DB2 table. The SELECT could also be separate with data populated to host variables which can be manipulated or translated with the INSERT referring to a set of host variables. Additionally, the population of the rows can be from other sources such as IMS databases, flat files, or other sources accessible by the COBOL program, again using host variables within the INSERT statement. 22

23 INSERT/UPDATE Same structure as stored procedure for data retrieval Probably don t need returned results set (or temporary table or cursor) INPUT parameters Key(s) for row to UPDATE Column data to UPDATE or INSERT OUTPUT parameters Communicate status (OKAY, WARN, NOTFOUND, ERROR) # Rows inserted/updated 23 INSERT/UPDATE example The previous example showed a stored procedure retrieving data based on a key specified in an input parameter(s). For an INSERT or UPDATE, the process is similar, but it s not likely that there would be any returned result set(s). This implies that there would be no need to declare a temporary table or cursor for the result set. Input parameters would contain any column data to be inserted, and the INSERT statement would refer to the input parameter host variables. For an UPDATE, the key of the row(s) to be updated would also be included as a parameter(s). In most cases, the output parameters would include a status of the call (status code, error messages) as well as a count for the number of rows affected by the call. 23

24 Progress Why, What DB2 COBOL Java Challenges Support 24 Objective 3 This section looks at how a Java application is coded using the JDBC API to CALL the z/os external stored procedure. 24

25 The Java Side Java application design Java API s JDBC, SQLJ, others JDBC Steps JDBC Example SQLJ Example 25 The Java Side This section shows the overall design of our Java application, a quick review of some of the Java API s available. 25

26 Java Application Design Java Enterprise Edition application HTML screens through JSP pages Struts (open source web framework) Session Beans for transaction management (EJB) Handles commits, rollback Data Access Objects (DAO) to invoke stored procedure using JDBC Reusable objects for error handling Debug capabilities for CallableStatement class More in Support Hints 26 Java Application Design Our online application is a Java EE (Java Enterprise Edition) application based on the Struts MVC framework to present data to the user via HTML pages in a web browser. Behind the presentation layer we use EJB Session beans to manage the transactions for commits and rollbacks (the Java EE server manages the transactions instead of application code to do the commits and rollbacks as needed). The session beans provided an interface to business classes which in turn invoke methods in Data Access Objects which contain the database SQL. The SQL (CALL statement) is transmitted to the database through the JDBC API using the CallableStatement class. NOTE: For debug purposes, the CallableStatement class can be extended to show the values of host variables in application logging. This is discussed further in the Support Hints section. 26

27 Java Application Design Action Session Bean Business Object Data Access Open systems z/os JDBC DB2 Connect WLM DB2 27 Java Application Design Our Java application uses Struts for its presentation and control framework. The basic design uses an instance of a Struts Action to process the web request which in turn invokes a business method on an EJB session bean (through an EJB façade class not shown here). Behind the session bean method is a corresponding method on a business object which manages the connection to the database and invokes a database access method in a data access object. The data access object contains the interaction with the JDBC API to issue the CALL statement to DB2. In our environment, we use DB2 Connect as middleware from WebSphere (where the Java application runs) to DB2 on z/os. The stored procedures are running in a Workload Manager (WLM) region. 27

28 Java API s JDBC Dynamic SQL Part of Java Standard Edition specification Dynamic statement caching SQLJ Static SQL ISO Standard not part of base Java specification DB2 packages SQL is in the DB2 catalog We use JDBC Others Hibernate, ibatis, EJB 28 Java API s There are many API s available for issuing SQL statements from Java. Two major API s are JDBC and SQLJ. JDBC is part of the base Java JDK (Java Standard Edition) and issues dynamic SQL. To help in performance, dynamic statement caching is used to improve the throughput of SQL. SQLJ is not part of the JDK, but is an ISO standard and issues static SQL (packages are bound to DB2). Our applications use JDBC (very common). There are also other options available such as Hibernate, ibatis, and the EJB specification (enterprise beans). 28

29 Java JDBC Steps CallableStatement extension of Statement class Step 1 build CALL string and prepare the call Step 2 set input parameters, register output parameters Step 3 execute and retrieve output parameters Step 4 iterate through any result sets NOTE: This example does not show the EJB which manages the transaction, issuing commits or rollbacks in the case of failure. 29 JDBC Steps The steps to invoke a stored procedure using the JDBC API in a Java application are similar to using JDBC for other SQL statements like SELECT, INSERT, and UPDATE. A major difference is the use of the CallableStatement class from the API as compared to the PreparedStatement or Statement class. First, the CALL statement is built and then the CallableStatement is obtained from the Connection object using the preparecall( ) method. Second, any input and output parameters are defined to the CallableStatement by setting the contents of each input parameter and registering the location of each output parameter. Third, the CallableStatement is executed which invokes the stored procedure. On return, the output parameters are retrieved from the CallableStatement. Fourth, if there were any result sets returned, the JDBC API is used to iterate through the rows in the result set(s). 29

30 JDBC Example Part 1 String procname = GET_LIST"; String proccall = "CALL " + proccollection + "." + procname + "(?,?,?,?,?)"; CallableStatement cs = conn.preparecall(proccall); cs.setstring(1,keyobject.getfield1( )); cs.setstring(2,keyobject.getfield2( )); cs.registeroutparameter(3, java.sql.types.char); cs.registeroutparameter(4, java.sql.types.char); cs.registeroutparameter(5, java.sql.types.date); cs.execute( ); someobject.setstatus(cs.getstring(3)); 30 JDBC Example As indicated in the previous slide, the Java code starts with building a CALL statement string. This string is used to build a CallableStatement object from the current Connection object which represents the connection to the database. Any input parameters values are populated in the CALL statement from the local Java fields using the set methods of the CallableStatement (cs is the local instance name in this example for the CallableStatement). Additionally, any output parameters need to be registered with the CallableStatement with the registeroutparameter( ) method. Once the input and output parameters are accounted for, the CallableStatment is executed using the execute( ) method which processes the CALL statement, invoking the remote stored procedure. On return, the output parameters are obtained using the associated get method(s) of the CallableStatement class. 30

31 JDBC Example Part 2 ResultSet rs = cs.getresultset( ); while (rs.next( )) { MyResult result = new MyResult( ); result.setstatus(rs.getstring( RETURN_MSGS")); result.setmessages(rs.getstring( RETURN_MSGS")); result.setdatefield(rs.getdate( COL_5")); } if (rs.getmoreresults( )) { rs = cs.getresultset( ); while (rs.next( )) { } } Iterate through additional result sets if any 31 JDBC Example Part 2 After the CALL is complete and any output parameters obtained (assuming that no error was identified), result sets are processed if there are any. This example shows that the first result set is located using the getresultset( ) method on the CallableStatement object. Similar to how result sets are processed for SELECT statements through a PreparedStatement, the next( ) method of the ResultSet object is used to iterate through the returned rows and the appropriate get( ) methods called to get individual column values. The getmoreresults( ) method (returns true or false) is invoked to determine if more result sets exist to be processed. If there are, the getresultset( ) method is repeated and the returned ResultSet object is processed the same as the first until there are no remaining result sets. 31

32 SQLJ Example #sql [myconnctx] {CALL GET_LIST( :IN field1, :IN field2, :OUT status, :OUT messages, :OUT datefield)}; Multiple result sets: index.jsp?topic=/com.ibm.db2.udb.doc/ad/ cjvsjmlt.htm 32 SQLJ Example This slide is provided mainly to indicate that the SQLJ API supports calls to DB2 Stored Procedures and to give a visual example. We do not use the SQLJ API in our application or within other applications. 32

33 Progress Why, What DB2 COBOL Java Challenges Support 33 Objective 4 We obviously had some challenges in pulling the two sides together. The objective of this section is to communicate some of the challenges we encountered and how we resolved them. 33

34 Challenges Encountered SQL Error Codes there were many Application changes even when there weren t any Distributed development Mainframe COBOL vs. Java WLM environment Miscellaneous Client fields (e.g. use in triggers) Security calling from program vs. testing 34 Challenges Encountered We certainly encountered our share of problems and issues which made our development challenging at times (what else is new). There was a learning curve as we learned how to avoid SQL errors (at least partly avoid) and understand what needed to be done when different types of application changes were made on the mainframe side. Having a mixture of mainframe COBOL developers and Java developers implies that there will be some confusion when it comes to problem resolution, and our project was no different. The WLM environment still remains partially a mystery to the developers, but we have made some progress. Finally, we experienced some miscellaneous challenges, some of which we re still trying to resolve. 34

35 SQL Error Codes -805/SQL0805N Ensure STAY RESIDENT NO (main) STOP/STA procedure for main program change Quiesce/resume WLM pick up load module changes for lower level AI s or wait for WLM recycle Migrating to new test environments 35 SQL Error -805 Stored procedures in COBOL wasn t unlike other mainframe COBOL DB2 applications in that there were plenty of -805 errors to track down. The difference was that the developers couldn t always explain them. An initial learning opportunity was to declare our stored procedures as STAY RESIDENT NO so that changes are more readily effective in the test environment. Unfortunately, if we changed a dynamically called subprogram, the WLM wouldn t know about the change. In this event, we had to contact our administrators to quiesce and resume the WLM environment to pick up the change. Alternatively, we could wait for the WLM to recycle naturally (such as in an overnight cycle), but usually, we couldn t wait. This issue alone could cause much confusion, especially if no one knew what changed when a -805 would occur. Most of the issues revolved around the use of dynamically called subprograms. This problem was also common when moving from one test environment to another. 35

36 SQL Error Codes -471/SQL0471N stored procedure is stopped DIS PROCEDURE(TSTn.MY_PROC_NAME) to view status STA PROCEDURE(TSTn.MY_PROC_NAME) Use SCOPE(GROUP) for data sharing -430/SQL0430N stored procedure abended Correct COBOL program abend Repeated instances may cause SQL Error Codes continued -471 We received this error off and on when a stored procedure would be stopped in DB2. Sometimes the stop would be as a result of repeated program abends, but sometimes it was unclear why the procedure was stopped. The START/-STA command was used to start the procedure in the appropriate test environment This occurred as a result of the COBOL program abending. In the Java application, we handled any ERROR status in the same way and indicated to the user that there was a temporary problem with the system (as compared to displaying a specific error code). Specific error information was logged for developer review. Repeated occurrences of this SQL code resulted in stopping the procedure, so we would have to start the procedure after the error was corrected. 36

37 Application Changes Don t always know application changed stored procedures aren t working Making application changes effective Use STAY RESIDENT NO for test Picks up SP load module changes Use STAY RESIDENT YES for production for performance Must STOP/START procedure on change Dynamic subprograms Not affected by STAY RESIDENT NO/YES Quiesce/refresh WLM environment for changes 37 Application Changes This was one of our biggest challenges for support by the developers. To make COBOL programs reusable (i.e. between multiple stored procedures or between multiple COBOL-only processes), subprograms were called dynamically. Declaring the Stored Procedure as STAY RESIDENT NO accounted for refreshing the version of changed stored procedure main programs. However, this did not account for making changes to dynamically called subprograms effective. This required a refresh of the WLM environment which had cached versions of those subprograms. So, it could be very challenging to keep track of each program change and what was required to make it effective. For production, with STAY RESIDENT YES for better performance. See the article in the reference section to see how STAY RESIDENT YES works with PROGRAM TYPE SUB. 37

38 Distributed Environment COBOL/mainframe vs. Java developers Logon to multiple platforms View debug info Tools on different platforms Unit testing tools Debug tools Release management We had lack of mainframe debug tool Ownership of different application code Same issues as for any cross-platform applications 38 Distributed Environment It was challenging for developers to coordinate problem resolution between the Java and COBOL sides. Bringing the two sides together in a single application was basically the same as having an integration point between two separate applications. To make a problem determination, it was necessary for a collective access to messages (e.g. SYSOUT) on z/os as well as log information on the Java application server. This is more convenient for developers who are familiar with both sides (and we were lucky to have that), but it is even more troublesome when the Java developers don t even know how to logon to the mainframe. As with any cross-platform project, there were challenges in managing all of the tools and release management, but the bigger challenges were with identifying what to fix when something was broken. It s helpful to have a developer(s)/architect(s) that is familiar with both platforms to coordinate and resolve issues between platforms (plentiful). However, it is also helpful to have problem resolution tools which indicate the source of the problem (see Support Hints). 38

39 WLM Environment WLM address space management/monitoring Know the right people to contact WLM administrators didn t have much more experience than the development team Dynamic subroutines changes require WLM quiesce/refresh Consider WLM Refresh Stored Procedure (more in Support Hints) Stopped procedures occurred periodically Difficult to identify results of stored procedure call -STOP/-STA PROC(ALL) admin access required 39 WLM Environment The WLM environment was new to the technical staff on the team, so it was imperative to know who the administrators were and how to contact them. In our case, we had a mailbox which was monitored by the administrators. Even with that, there were situations where the administrators couldn t readily identify why stored procedures weren t running, even if the WLM was up. As indicated earlier, the nature of dynamically called subprograms has a big impact on how applications are managed in a WLM environment. Any change required a WLM refresh which was only in the hands of our administrators. We are currently working on a stored procedure to allow us to do a refresh from the development area. There were many times when the application would receive an error because stored procedures were stopped. We attributed much of this to previous abends, but we weren t always sure of the cause. 39

40 Progress Why, What DB2 COBOL Java Challenges Support 40 Objective 5 As a result of our experiences in developing, testing, implementing, and supporting this application, this section shares some of the hints that we have for supporting DB2 Stored Procedures on z/os that are called by Java. 40

41 Support Hints Testing (including security) Debugging (cross-platform) Environment Developer skills Documentation Security Where we are today over one year later 41 Support Hints This section covers some of the things that we learned from a support perspective through the development, implementation, and support of our application. At the end of a discussion on the various areas of support, there is a summary on the state of the application today, over a year after implementation. 41

42 Support Hints Testing Stored Procedure tester (HTML/Java) Setting input parameters generically Obtaining output parameters generically Returning list of SP s More details on next slide Testing from COBOL Stub module to invoke SP Generic mainframe tool for calling load module Specify parameters/configuration 42 Support Hints Testing One tool that we found to be extremely valuable was an independent tool for invoking a stored procedure from a Java client program. There were numerous times when both the COBOL developers and mainframe developers believed that their components were functioning correctly, but problems were occurring. The tester removed the layer of business code to provide a generic interface to pass input parameters and receive output parameters and result sets through a web browser (HTML) interface. Additionally, mainframe developers used stub programs to call the stored procedure in two different ways. One was to use an in-house driver utility to call the dynamic subprograms. This allowed for unit testing of the called programs. Additionally, stub programs were used to actually perform an SQL CALL from another COBOL program. Even with these tools, there were many times when the Java/HTML tester was still necessary to resolve differences. See the next slide for more details on the Java/HTML stored procedure tester. 42

43 Support Hints SP Tester SELECT NAME, SCHEMA, EXTERNAL_NAME, RESULT_SETS, PARM_COUNT FROM SYSIBM.SYSROUTINES WHERE NAME LIKE? AND SCHEMA =? INPUT and OUTPUT parameters CallableStatement set and get methods Identify parameter field types at input time Convert entered name mask to all CAPS Special characters view HTML Security for SP execution Or third party product 43 Support Hints Stored Procedure Tester Here are some hints about building a custom Java/HTML stored procedure tester. A set of generic HTML pages displays a list of available stored procedures and accepts parameter values to be used in stored procedure execution. The SELECT statement retrieves stored procedure names from the DB2 catalog based on an entered schema and name mask. INPUT parameters and parameter (field) types are entered and set in the CallableStatement through get methods. The specified stored procedure is invoked through the CallableStatement and OUTPUT parameters are obtained through the set methods. Additionally, any result sets are obtained and displayed. We made the decision to automatically convert the entered procedure name mask to all CAPS to avoid confusion for the user. For a given user to be able to run a stored procedure, it was required that EXECUTE be granted to the user in DB2 (such as through as secondary authid). 43

44 Support Hints SP Tester 44 Support Hints Debugging These are two sample pages from the custom stored procedure tester that we created. Left: This shows the entry of a stored procedure mask to display a list of stored procedures matching the mask for a given schema name. Right: After selecting a stored procedure from a list, the user can specify all INPUT parameter values and Execute the procedure. The same page is redisplayed with results at the bottom of the page and with OUTPUT parameter values. 44

45 Support Hints Debugging Trace information DISPLAYs in COBOL log4j in Java (dynamic setting, multi-server) What to DISPLAY/log: All interface fields (input, output) Date/time differentiate between instances Java and COBOL show both sides Tool for stored procedure program tracing (mainframe) Stored Procedure tester Independent verification separate from Java/web application Debug capabilities for CallableStatement 45 Suport Hints Debugging It was difficult at times during testing to know what part of the application had a problem when things weren t working as expected. Trace information became very important to focus on either the Java or COBOL side. Some information that we included in Java logging and COBOL DISPLAY s: All interface fields (INPUT parameter values, OUTPUT parameter values) from the Java CALL and the COBOL SP Stored Procedure Name and date/time to differentiate between different instances of a call In our environment, we did not have support in our third party product to trace in the COBOL programs, so we were hampered with stepping through code as it executed. The stored procedure tester came in very handy to isolate issues between Java and COBOL. We have previously extended the PreparedStatement class to log host variable values when debugging, but we have yet to do so for the CallableStatement. As it is, only parameter markers (? s) show in the log. 45

46 Support Hints - Environment Ability to recycle WLM environment WLM_REFRESH IBM Stored Procedure We use SP Tester to invoke (test only) See STOP/-STA commands above Looking at the catalog (definitions and other) WLM view Need to know started task name(s) Indicates whether regions are running, when they started JESMSGLG error messages SYSOUT DISPLAY messages 46 Support Hints Environment During testing, we found it necessary to repeatedly recycle our WLM environment to pick up changed COBOL dynamically called subprograms. We had to submit requests to our administrators to do this which was cumbersome, and we ve since enabled the process for senior-level architects by configuring the WLM_REFRESH stored procedure from IBM. Our stored procedure tester provided an easy user interface to call this utility procedure. The STOP and START commands in DB2 were very useful, primarily for starting procedures that had reached a threshold for abends. We had to start the procedure after the program was corrected for errors introduced in testing. The ability to see the catalog for stored procedure definitions was important to validate which version of a stored procedure definition was in effect in a given test region. This was more of an issue when the definitions were still changing. The ability to view the WLM was critical. We used it to view DISPLAY s from the COBOL programs, view program dumps, and to determine whether WLM regions were actually running and if they had been restarted since latest changes were introduced into runtime libraries. 46

47 Support Hints Skills Mainframe COBOL WLM viewing started tasks DB2 programming (SQL, binds) DB2 commands (STOP, START, DIS) Open Systems Java/JDBC CallableStatement ResultSet DB2 SQL, error codes 47 Support Hints Skills There were obviously different sets of skills required to make the application work. It was useful to have some team members who had cross-domain skills to resolve differences between the mainframe and open systems environments. Additionally, it was helpful to have developers or architects who were knowledgeable of DB2 commands and who would navigate the WLM environment. As with any DB2 application, an understanding of DB2 SQL, binds, and error codes was essential (from a z/os perspective since DB2 is on z/os in our environment). 47

48 Support Hints Documents DB2 How to create stored procedures How the WLM works including refreshing, viewing current and history regions COBOL What comprises the standard interface and what common fields mean (status, messages) Java How to test stored procedures How to view results on the mainframe Overall Coding practices e.g. DISPLAY and logging 48 Support Hints Documents Our project created a large number of design and support documents to be used by developers during development and support. New developers rely on this documentation to understand the application and how to test and diagnose issues. DB2 We found it helpful to document how stored procedures are created including examples and where to find the DCLGEN card library files. It was also valuable to have information on how to work with and understand our Workload Manager environments, how to view them, how to recycle them, etc. COBOL We documented the standard interfaces, including what common fields mean such as error status codes. Doing things a common way made the application easier to understand. Java The stored procedure tester was used repeatedly, so it was important for developers to know how to use the tool. Sometimes, however, it was still necessary for Java developers to view results in the WLM on the mainframe. Overall Whether in Java or COBOL, we had documented coding practices which included minimum requirements for application logging. 48

49 Support Hints - Security SELECT vs. INSERT, UPDATE, DELETE Application requirements Debugging requirements (e.g. SELECT) Test vs. production Debugging, unit testing Querying Secondary authid Application id Individual users stored procedure testing 49 Support Hints Security In general, we apply DB2 security using secondary authid s with a few exceptions: Unit testing developers have individual access in test since they test with their own id s vs. an application level id. It s possible to include the individual user in the secondary authid, but we try to keep the test privileges equivalent to production privileges for an authid to show that it was adequately tested. Stored procedure testing for SELECT only stored procedures, we have granted access to EXECUTE for developers to validate that stored procedures are functioning properly. This implies that the user has read access to the results of the stored procedure (i.e. would normally have inquiry access in the application). Querying we have separate access for doing native SELECT statements to the database, especially in the test environment. Overall, we had to manage security for the application (application level id) and developer needs. 49

50 Support Hints - Status Application in production for almost a year Running smoothly Maintainable Some functionality exposed as web services from Java to another application Documentation is a must Coding Testing Errors Code changes 50 Support Hints The status of our application After going into production in 2007, the application has functioned normally for almost a year. The response time for the web application is good (generally 1-3 seconds per page). Almost as important as the stability of the runtime environment, the application has been maintainable with multiple subsequence functionality releases. The WLM environment is still a challenge at times and developers still get confused with dynamically-called subroutines, but the application continues to evolve. As an example, a few processes were adapted to function as web services to be called from another web application. Documentation has been a must for reference by developers maintaining the application, even by developers who contributed to the original coding. 50

51 References DB2 Stored Procedures for z/os - html CREATE PROCEDURE topic/com.ibm.db2.udb.doc/admin/r htm CREATE GLOBAL TEMPORARY TABLE index.jsp?topic=/com.ibm.db2.udb.doc/admin/ r htm Java CallableStatement (J2SE 1.4.2) lablestatement.html 51 51

52 References (cont d) SQLJ calls to Stored Procedures - index.jsp?topic=/com.ibm.db2.udb.doc/ad/ tjvsjcal.htm Article on STAY RESIDENT

53 That s a Wrap 53 53

54 Session F13 My Java App Called Your COBOL DB2 Stored Procedure John Mallonee Highmark Inc