Session: E05 Faster FETCH, INSERT, UPDATE (MERGE) DB2 for z/os Multiple Row Processing

Transcription

1 Session: E05 Faster FETCH, INSERT, UPDATE (MERGE) DB2 for z/os Multiple Row Processing Christopher J. Crone IBM DB2 for z/os Development November 9 th, :00 12:00 Platform: DB2 for z/os

2 Presentation Topics Host Variable Arrays Java and ODBC Multiple-Row Insert Multiple-Row Fetch Positioned UPDATE/DELETE MERGE GET DIAGNOSTICS

3 Host Variable Arrays

4 Host Variable Arrays Host variable array is an array in which each element of the array contains a value for the same column Changes have been made to allow host variable arrays in: ƒ COBOL ƒ PL/1 ƒ C++ ƒ NOTE: Assembler support is limited to cases where USING DESCRIPTOR is allowed. Assembler pre-compiler does not recognize declaration of host variable arrays. The programmer is responsible for allocating storage correctly, etc. Can only be referenced in multi-row fetch or insert In general, arrays may not be arrays of structures Some arrays of structures are supported, for instance, DB2 V8 does support arrays of structures for C varying length strings (C varying length strings are structures). DB2 V8 would not support an array of structures for say a structure that was created to fetch an entire row of data.

5 COBOL This is an Example of the changed Syntax for a COBOL Numeric host variable other declarations for things like strings, etc. have similar changes. For COBOL, the specification of the array size is determined by the OCCURS dimension [TIMES] clause.

6 COBOL Example 1: Declare a CURSOR C1 and fetch 10 rows using a multi-row FETCH statement 01 OUTPUT-VARS. 05 NAME OCCURS 10 TIMES. 49 NAME-LEN PIC S9(4)COMP-4 SY C. 49 NAME-DATA PIC X(40). 05 SERIAL-NUMBER PIC S9(9)COMP-4 OCCURS 10 TIMES. PROCEDURE DIVISION. EXEC SQL DECLARE C1 CURSOR WITH ROWSET POSITIONING FOR SELECT NAME, SERIAL# FROM CORPORATE.EMPLOYEE END-EXEC. EXEC SQL OPEN C1 END-EXEC. EXEC SQL FETCH FIRST ROWSET FROM C1 FOR 10 ROWS INTO :NAME, :SERIAL-NUMBER END-EXEC. This example shows a FETCH of 10 rows into two host-variable-arrays, NAME, and SERIAL-NUMBER. NAME is a VARCHAR, and SERIAL NUMBER is an INTEGER.

7 PL/I This is an example of how to declare a character (CHAR) or character varying (VARCHAR) string in PL/I. Again, other variable definitions have similar changes.

8 PL/1 Example 2: You can retrieve 10 rows from the table CORPDATA.DEPARTMENT with: DCL DEPTNO(10) CHAR(3), DEPTNAME(10) CHAR(29) VAR, MGRNO(10) CHAR(6), ADMRDEPT(10) CHAR(3); DCL IND_ARRAY1(10) BIN FIXED(15); DCL IND_ARRAY2(10) BIN FIXED(15);... EXEC SQL DECLARE C1 CURSOR WITH ROWSET POSITIONING FOR SELECT * FROM CORPDATA.DEPARTMENT;... EXEC SQL FETCH FIRST ROWSET FORM C1 FOR 10 ROWS INTO :DEPTNO :IND_ARRAY1, :DEPTNAME :IND_ARRAY2, :MGRNO :IND_ARRAY3, :ADMRDEPT :IND_ARRAY4; In this example, several arrays of size 10 are declared for host variable values. Also, indicator arrays are declared to accept indicator values. Finally 10 rows are fetched into the host-variable-arrays and indicator-arrays.

9 C/C++ This slide show the changes to C/C++ to declare a numeric array.

10 C++ Example 3: Declare an integer and varying character array to hold columns retrieved from a multi-row fetch statement long serial_num[10]; struct { short len; char data [18]; }name [10];... EXEC SQL DECLARE C1 CURSOR FOR SELECT NAME, SERIAL# FROM CORPDATA.EMPLOYEE WITH ROWSET POSITIONING;... EXEC SQL OPEN C1; EXEC SQL FETCH FIRST ROWSET FORM C1 FOR 10 ROWS INTO :NAME, :SERIAL_NUM; In this example, we declare two host-variable-arrays serial_num, and name (which is a C/C++ structure). We then declare a cursor and FETCH 10 rows into the host-variable-arrays.

11 Java and ODBC

12 JDBC try { stmt = con.preparestatement ("insert into T1 values (?,?)"); stmt.setint (1, 1); stmt.setint (2, 1); stmt.addbatch(); stmt.setint (1, 2); stmt.setint (2, 2); stmt.addbatch(); stmt.setint (1, 3); stmt.setint (2, 3); stmt.addbatch(); int[] updatecount = stmt.executebatch(); for (int i = 0; i < updatecount.length; i++) actualresults.println(updatecount[" + i + "] = " + updatecount[i]); } stmt.clearbatch(); con.commit(); stmt.close(); The JDBC driver is being modified to use MRF and MRI on interfaces where it has previously performed batching, the driver will now use MRI under the covers. In this example, a T4 or T2 Drvier (running off z/os using DRDA) would have previously used a Chained Insert flow. Now the driver will use an MRI flow.

13 ODBC #define TC 10; SQLWCHAR SAVE_H1WCHR[TC][1025]; SQLINTEGER LNSAVE_H1WCHR[TC]; /* Main Program */ int main() { /* initialize data */ wcscpy(save_h1wchr[0], (SQLWCHAR *)"abc 1");... wcscpy(save_h1wchr[9], (SQLWCHAR *) abc 10"); hstmt=0; rc=sqlallocstmt(hdbc, &hstmt); rc=sqlbindparameter(hstmt, 1, SQL_PARAM_INPUT, SQL_C_WCHAR, SQL_DBCLOB 1024, 0, SAVE_H1WCHR, (SQLINTEGER) 2050, (SQLINTEGER *) LNSAVE_H1WCHR); /* Set number of rows to insert */ rc=sqlparamoptions(hstmt, TC, NULL); /* Insert rows into DBCLOB column via SQLBindParameter */ rc=sqlpreparew(hstmt,(wchar_t *) "INSERT INTO TABLECU (C1) VALUES (?)",SQL_NTS); rc=sqlexecute(hstmt); rc=sqltransact(henv, hdbc, SQL_COMMIT); } /* End Main */ In this ODBC example, we insert 10 rows into table TABLECU using and ODBC Array Insert API. In particular, note that SQLParamOptions, the number of rows inserted (TC in this case) is specified. Please note that in this example the error checking for each statement has been removed (rc is set as each statement is processed, but the error checking of RC has been omitted for brevity).

14 Multiple-Row INSERT

15 Multiple Row INSERT Different Forms of INSERT ƒ INSERT via VALUES is used to insert a single row into the table or view using values provided or referenced ƒinsert via VALUES FOR "n" ROWS form is used to insert multiple rows into table or view using values provided in host variable array ƒ INSERT via SELECT is used to insert one or more rows into table or view using values from other tables or views FOR "n" ROWS ƒ For static SQL, specify FOR "n" ROWS on INSERT statement ƒ For dynamic SQL, you may also specify FOR "n" ROWS on EXECUTE ƒif FOR n ROWS is specified on EXECUTE, it must not have also been specified on INSERT statement ƒ Maximum value of n is specified as host-variable, parameter marker, or literal value New syntax is added to allow a FOR n ROWS clause to be added to INSERT. This clause allows specification of the number of rows to be inserted. The clause may be specified on the INSERT statement (either static or dynamic SQL), or as part of the EXECUTE statement (dynamic SQL only).

16 Multiple-Row Insert Changed Statements ƒ INSERT ƒ PREPARE ƒ EXECUTE Changes were made to INSERT, PREPARE, and EXECUTE to support MRI.

17 INSERT Here is part of the new INSERT syntax diagram. Note the multiple-row-insert clause

18 INSERT (cont) In this segment of the INSERT syntax diagram, we show changes to the VALUES clause to allow specification of host-variable-arrays, and optionally, the number or rows to be inserted. We ll mostly focus on the FOR n ROWS clause next, We ll talk atomicity later.

19 INSERT Example 1 Insert a variable number of rows using host variable arrays for column values. Assume that the table T1 has one column and that a variable (:hv) number of rows of data are to be inserted into T1 table. EXEC SQL INSERT INTO T1 VALUES (:hva :hvind) FOR :hv ROWS ATOMIC; In this example, :hva represents the host variable array and :hvind represents the array of indicator variables In this example, :hva represents the host variable array and :hvind represents the array of indicator variables

20 INSERT Example 2 Insert 10 rows into a table T2, and return the rows that have been inserted using a multiple row fetch statement. DECLARE CS1 ASENSITIVE SCROLL CURSOR WITH RETURN WITH ROWSET POSITIONING FOR SELECT T2.C1, T2.C2 FROM FINAL TABLE (INSERT INTO T2 VALUES (:hvai1 :hvindi1, :hva2 :hvinid2) FOR 10 ROWS); EXEC SQL OPEN CS1; /* INSERT OCCURS HERE */ EXEC SQL FETCH FIRST ROWSET FROM CS1 FOR 10 ROWS INTO :hvao1 :hvindo1, :hvao2 :hvindo2 In this example, 10 rows of data are inserted into T2 (columns C1 and C2). The rows are inserted as part of a SELECT FROM FINAL TABLE statement. This statement was declared and associated with CURSOR CS1. When CURSOR CS1 is opened, the actual INSERT will occur, and the rows that are inserted will be materialized to a temporary table. The input values are specified in host-variable-arrays hvai1 and hvai2. Indicator arrays are specified by hvindi1 and hvindi2. After the OPEN, the rows that have been inserted are then fetched back by the application using a rowset cursor. These output values are placed in host variable arrays :hvao1 and hvao2. Indicator arrays are specified by hvindo1 and hvindo2.

21 INSERT Example 3 Assume that table T3 has three columns: S1 SMALLINT, I1 INTEGER, and T1 TIMESTAMP The application allocates - :hv1 is a scalar host variable, :hva2 an array with 6 elements, and value for T1 is a special register In this example, hv1, and CURRENT TIMESTAMP are used for each row of data, the values for I1 are obtained from host-variable-array :hva2 :hv1 :hva Result of Insert S1 I1 T INSERT INTO T3 VALUES (:hv1, :hva2, CURRENT TIMESTAMP) FOR 5 ROWS; -- Assume CURRENT TIMESTAMP = In this example, we have an insert into table T3 a scalar host variable hv1, which is used for C1 for every row of the insert, and a host-variable-array with 10 elements for column C2, and a literal value ABCD which is also used for every row of the insert.

22 Local INSERT Flow x1000 x2000 x3000 Internal Buffers Table SQLDAID = SQLDA SQLDABC = 104 SQLN = 2 SQLD = 2 SQLTYPE[1] = 496 SQLTYPE[2] = 496 SQLLEN[1] = 4 SQLLEN[2] = 4 SQLDATA[1] = x SQLDATA[2] = x SQLIND[1] = x SQLIND[2] = x SQLNAME[1] = x SQLNAME[2] =x The host variable array starts at x1000 The indicator array starts at x2000 The value N is locates at x3000 Runtime will first locate the value for N and move it into the DB2 (ADMF) address space Runtime will then move the values for the indicator and column (assuming the indicator is not null) Runtime will then drive DM INSERT Runtime will then move the next value for the indicator and column into the DB2 address space Runtime will then drive DM again. Note that the SQLNAME values for 1 and 2 are set to x 0008 (the length) of information provided in this field. The values x 0000 (in black) represent the CCSID not specified in this case, so the application encoding bind option will be used. Next, for SQLNAME[1], we have x 0001 (in Red) which indicates that the SQLVAR contains an array entry, followed by x 0004 (in Blue) which represents the dimension of the array. Finally for SQLNAME[2], we have x 0002 (in Red) which indicates the SQLVAR contains an entry for N, followed by x 0000 (in Blue) which is default information for this type of entry.

23 Distributed Multiple Row INSERT Flow Application C1 C2 Client Com-Buffer Server Com-Buffer Intermediate Buffer Table Page Distributed Chained INSERT Flow Application Stmt Data Client Com-Buffer Server Com-Buffer Intermediate Buffer Table Page For a distributed flow, the data is blocked up at the requestor and moved to the server. The value for N will flow as part of the statement metadata. Runtime will then move the values for the first row from the com buffer to the intermediate buffer Runtime will then drive DM INSERT Runtime will then move the values for the next row from the com buffer to the intermediate buffer Runtime will then drive DM again.

24 ATOMIC -vs- NOT ATOMIC ATOMIC ƒtraditional behavior ƒall rows being inserted must successfully be inserted NOT ATOMIC CONTINUE ON SQLEXCEPTION ƒinsert rows that are successful ƒreject rows that are not successful GET DIAGNOSTICS can be used to determine which rows were not successful ƒsqlcode will indicate if all failed, all were successful or at least one failed A major consideration of ATOMIC vs NOT ATOMIC is the amount of data you are inserting -- inserting 32K rows into a table whose rows are 32K bytes long (the row is in a 32K page and 1 row/page) consumes 1G of space in the application and would log > 1G of data, so rollback could be painful. NOT ATOMIC CONTINUE ON SQLEXCEPTION allows some rows to be successfully inserted into a table, and failing rows to be indicated for possible further processing (For example, SQLCODE -803). GET DIAGNOSTICS should be used to determine the failing rows.

25 ATOMIC -vs- NOT ATOMIC with Triggers Trigger Behavior on Multiple Row Insert ATOMIC ƒthe inserts are processed as a single statement. ƒany statement level triggers fire once for the statement, and the transition tables will include all of the rows that were inserted. NOT ATOMIC CONTINUE ON SQLEXCEPTION ƒinserts are processed separately. ƒany statement level triggers are processed for each row that is inserted ƒtransition tables include the individual row that is inserted. ƒwhen errors are encountered with this option in effect, processing continues, and some of the specified rows will not be inserted. ƒin this case, if an insert trigger is defined on the underlying base table, the trigger transition table will only include rows that were successfully inserted. A major consideration of ATOMIC vs NOT ATOMIC is the amount of data you are inserting -- inserting 32K rows into a table whose rows are 32K bytes long (the row is in a 32K page and 1 row/page) consumes 1G of space in the application and would log > 1G of data, so rollback could be painful. NOT ATOMIC CONTINUE ON SQLEXCEPTION allows some rows to be successfully inserted into a table, and failing rows to be indicated for possible further processing (For example, SQLCODE -803). GET DIAGNOSTICS should be used to determine the failing rows.

26 PREPARE For Prepare, we have new attributes. FOR MULTIPLE ROWS or FOR SINGLE ROWS and atomicity may be specified on Prepare as attributes.

27 EXECUTE Changes to EXECUTE allow specification of host-variable-arrays and the FOR n ROWS clause. The FOR n ROWS clause may only be specified once, either on the Dynamic INSERT statement, or the EXECUTE statement. If the FOR n ROWS clause is specified in both places, then an error (negative SQLCODE) will occur.

28 PREPARE and EXECUTE - Example Assume that the prog table has 2 columns. Prepare and execute a dynamic INSERT statement which inserts 5 rows of data into the prog table. SQLDAID = 'SQLDA '; SQLN = 3; SQLD = 3; SQLVAR(1).SQLTYPE = 500; /* Integer */ SQLVAR(1).SQLLEN = 4; SQLVAR(1).SQLDATA = ADDR(array_stor1); SQLNAME(1) = x SQLVAR(2).SQLTYPE = 452; /* Char */ SQLVAR(2).SQLLEN = 15; SQLVAR(2).SQLDATA = ADDR(array_stor2); SQLNAME(2) = x stmt = 'INSERT INTO T1 (C1,C2) VALUES (?,?)'; attrvar = FOR MULTIPLE ROWS ; NROWS = 5; EXEC SQL PREPARE ins_stmt ATTRIBUTES :attrvar FROM :stmt; EXEC SQL EXECUTE ins_stmt FOR :NROWS ROWS USING DESCRIPTOR :SQLDA; In this example, we have a statement string stmt which contains an INSERT statement. INSERT attributes are specifications are specified on PREPARE and the EXECUTED statement includes the FOR n ROWS clause.

29 MRI Performance Considerations Up to 30% faster INSERT performance Performance improvement largely due to savings of API costs Savings flattens out quickly, for example, savings (as a percentage) was equal For 100 Rows, 500 Rows, 1000 Rows Reasonable n for MRI is about 200 no additional savings above that, and downside (rollback) increases. Distributed MRI performance up to 70% Elapsed Time and 50% Server CPU time reductions seen Performance variable based on Number of rows INSERTed Number of columns INSERTed Number of INDEXes on table Class 2 accounting (on or off) savings larger is Class 2 is on Note: Don t use MRI to INSERT 1 row due to overhead to set up for MRI Similar improvement with UPDATE and DELETE WHERE CURRENT OF when updating/deleting the entire rowset. This is in addition to the savings provided by MRF As always, when talking about performance, Your Mileage May Vary (YMMV). These are example numbers that we have seen in our testing.

30 Multiple-Row FETCH

31 Multiple-Row Fetch Changed Statements ƒ DECLARE CURSOR ƒ OPEN CURSOR ƒ ALLOCATE CURSOR ƒ FETCH ƒ Positioned UPDATE (UWCO) ƒ Positioned DELETE (DWCO) Changes were made to DECLARE CURSOR, OPEN CURSOR, ALLOCATE CURSOR, DESCRIBE CURSOR, FETCH, UPDATE WHERE CURRENT OF (UWCO), and DELETE WHERE CURRENT OF (DWCO) for MRF. The changes for OPEN and ALLOCATE Cursor are shown in the back of this presentation.

32 DECLARE CURSOR DECLARE CURSOR adds a new WITH ROWSET POSITIONING clause. This clause must be used for MRF cursors, and may be used for non-mrf cursors. Row positioning FETCH statements may be used with cursor declared with the WITH ROWSET POSITIONING clause, or the WITHOUT ROWSET POSITIONING clause. Rowset positioning FETCH statements may only be used with cursors declared with the WITH ROWSET POSITIONING clause.

33 DECLARE CURSOR - Example Declare C1 as the cursor of a query to retrieve a rowset from the table DEPT. The prepared statement is MYCURSOR EXEC SQL DECLARE CURSOR C1 CURSOR WITH ROWSET POSITIONING FOR MYCURSOR; Rowset positioning specifies whether multiple rows of data can be accessed as a rowset on a single FETCH statement default is WITHOUT ROWSET POSITIONING This example demonstrates the new WITH ROWSET POSITIONING clause on DECLARE CURSOR.

34 FETCH In this syntax diagram, you will see that rowset positioned fetches have been added that correspond to the row-positioned fetches.

35 FETCH (cont) Also added to FETCH is the specification of a FOR n ROWS clause, and the ability to fetch data into host-variable-arrays.

36 Rowsets A group of rows for the result table of a query which are returned by a single FETCH statement Program controls how many rows are returned (i.e., size of the rowset) ƒ Can be specified on the FETCH statement (maximum rowset size is 32767) Each group of rows are operated on as a rowset Ability to intermix row positioned and rowset positioned fetches when a cursor is declared WITH ROWSET POSITIONING FETCH FIRST ROWSET STARTING AT ABSOLUTE 10 FROM CURS1 FOR 6 ROWS INTO :hva1, :hva2; A ROWSET is a grouping of rows. When a cursor is positioned on a rowset, all locks (if any) are held on all rows of the rowset. FOR n ROWS with FETCH FIRST n ROWS ONLY these two clauses may be used together. FETCH FIRST n ROWS ONLY dominates. Remote applications with updateable cursors should see an improvement in performance of fetching on these cursors. The functional changes introduced with this line item will allow blocking on cursors where no blocking was allowed before.

37 Determining rowset size If FOR n ROWS is NOT specified and cursor is declared for rowset positioning Size of rowset will be the same as the previous rowset fetch as long as ƒ It was the previous fetch for this cursor ƒ Or the previous fetch was a FETCH BEFORE or FETCH AFTER and the fetch before that was a rowset fetch Else rowset is 1 FETCH FIRST ROWSET FOR 5 ROWS FETCH NEXT ROWSET FETCH NEXT FETCH NEXT ROWSET FETCH NEXT ROWSET FOR 3 ROWS FETCH BEFORE FETCH NEXT ROWSET Returns 5 rows Returns the next 5 rows Returns a single row Returns a single row Returns 3 rows Returns 0 rows Returns 3 rows The size of the rowset is determined explicitly (when specified on the FETCH statement), or implicitly.

38 Rowset Positioned Fetches FETCH FIRST ROWSET FOR 3 ROWS FETCH NEXT ROWSET FETCH NEXT Relative to first row in current rowset FETCH ROWSET STARTING AT ABSOLUTE 8 FETCH NEXT ROWSET FOR 10 ROWS Partial Result Set Result table CUST_NO CUST_TYP CUST_NAME 1 P Ian 2 P Mark 3 P John 4 P Karen 5 P Sarah 6 M Florence 7 M Dylan 8 M Bert 9 M Jo 10 R Karen 11 R Gary 12 R Bill 13 R Geoff 14 R Julia 15 R Sally This example shows how the cursor is positioned after various ROWSET positioned FETCH statements. The last example FETCH NEXT ROWSET FOR 10 ROWS would result in 5 ROWS returned. SQLERRD(3) would be set to 5, and SQLCODE returned. Note : The cursor is positioned on ALL rows in current rowset

39 Partial Result Sets If you fetch beyond the end of the result set, you will receive an end of data condition ƒ i.e., When there are only 5 rows left in result table and you request FETCH NEXT ROWSET FOR 10 ROWS, 5 rows will be returned - SQLCODE +100 ƒ SQLERRD(3) will contain the number or rows returned ƒ This includes where FETCH FIRST n ROWS ONLY has been specified If you fetch beyond the beginning of the result set, you will receive an end of data condition ƒ i.e., if you are positioned on rows 3,4,5,6, and 7, and you request FETCH PRIOR ROWSET FOR 10 ROWS, 2 rows will be returned (Rows 1 and 2) - SQLCODE ƒ SQLERRD(3) will contain the number or rows returned Partial results sets are possible if an application fetches rows beyond the boundaries of the result set. SQLERRD 3 will contain the number of rows returned. SQLCODE +100 will be returned when an application fetches beyond the end of the data. SQLCODE will be returned when an application fetches beyond the beginning of the data.

40 Fetching Beyond the Result Set ABSOLUTE or RELATIVE If you fetch beyond the end of the result set, or beyond the beginning of the result set, you will receive an end of data condition ƒ Assume you are positioned on row 5 in a result set with 10 rows. ƒfetch ROWSET STARTING AT ABSOLUTE 15 ƒfetch ROWSET STARTING AT RELATIVE -7 ƒ No rows will be returned - SQLCODE +100 ƒ SQLERRD(3) will contain 0 ƒ Cursor position will be either BEFORE or AFTER depending on the direction of the FETCH. No results are returned if an application fetches rows beyond the boundaries of the result set using FETCH ABSOLUTE, or FETCH RELATIVE, and the starting position requested would be before the beginning or after the end of the result set. SQLERRD 3 will contain the number of rows returned 0 in all these cases. SQLCODE +100 will be returned. The cursor will be positioned either BEFORE or AFTER depending on the direction of the FETCH.

41 Local FETCH Flow x1000 x2000 Internal Buffers N Table SQLDAID = SQLDA SQLDABC = 104 SQLN = 2 SQLD = 2 SQLTYPE[1] = 496 SQLLEN[1] = 4 SQLDATA[1] = x SQLIND[1] = x SQLNAME[1] = x This example is to demonstrate how the processing of a multi-row FETCH proceeds. The host variable array starts at x1000 The indicator array starts at x2000 The value N is locates at x3000 Runtime will first locate the value for N and move it into the DB2 address space Runtime will then move the values for the indicator and column (assuming the indicator is not null) Runtime will then drive DM INSERT Runtime will then move the next value for the indicator and column into the DB2 address space Runtime will then drive DM again. Note that SQLNAME[1] is set to specify a length of 8 (green), no CCSID override (black x 0000 ), an indication that SQLVAR[1] is an array (red), and the size of the array (blue).

42 Distributed Multiple Row Fetch Flow Application C1 C2 Client Com-Buffer Server Com-Buffer Intermediate Buffer Table Page SQLCA Diagnostic Info Diagnostic Info Limited Block Fetch Flow Application C1 C2 Client Com-Buffer Server Com-Buffer Diagnostic Info Intermediate Buffer Table Page SQLCA SQLCA Diag Info Diag Info Diagnostic Info SQLCA In this flow, data from a table is sent to a client program a row at a time. In this flow, Columns C1, and C2 are placed into the Communications buffer at the server. The buffer then flows as one piece to the client (the blocksize is determined by the client using the FOR n ROWS clause on FETCH). At the client, the data is then returned to the application in column-array format. In this picture, the Blue and Teal blocks represent column information, Red blocks represent diagnostic information, And Green blocks represent control information. Note that there is a 1 byte diagnostic area that flows with each row of data. This diagnostic area is always NULL. Diagnostic information for the statement flows after the data. The application will receive the SQLCA as normal. Extended diagnostics are available at via the GET DIAGNOSTICS command.

43 MRF Performance Considerations Up to 50% faster FETCH performance Performance improvement largely due to savings of API costs Performance variable based on Number of rows fetched Number of columns fetched Class 2 accounting (on or off) savings larger is Class 2 is on Examples DSNTEP4 35% improvement on FETCH of 1000 rows with 5 cols and 20 cols DSNTIAUL 50% improvement on FETCH of rows with 5 cols and 20 cols Up to 50% reduction in CPU cost for LBF vs- V7 As always, when talking about performance, Your Mileage May Vary (YMMV). These are example numbers that we have seen in our testing.

44 Locking and Isolation Levels Cursor will be positioned on all rows in current rowset Locks will be held on all rows in rowset depending on isolation level and whether a result table has been materialized As normal, these factors affect whether you will see changes made to the table following open cursor Also affects whether you refetch the same rows when issuing a FETCH CURRENT to refetch current rowset Isolation levels can greatly affect the results of your processing of rowset cursors, especially if you are using dynamic scrollable cursors. In general, locks will be held on all rows in the rowset if a lock is held on one row, it will be held on all rows. CS current data no, and UR may not hold locks. Isolation levels can affect the results of subsequent fetches. For example, with ISO RR, a FETCH CURRENT ROWSET will always return the same rowset. However with other isolation levels, you may not see the same rows.

45 Using Static Scrollable Cursors When scrolling between rowsets: With insensitive fetches, updates by the application itself could cause changes and holes With sensitive fetches, updates by other applications could cause changes and holes For example, FETCH PRIOR ROWSET may return update or delete holes in place of rows that were fetched before Row contents can change between fetches With ROWSET fetches, you are more likely to encounter some of the issues that may occur if you fetch using a static scrollable cursor. The way an application handles these issues, such as hole rows, is different for a rowset FETCH due to the nature of the way data is returned.

46 If using static scrollable cursors... As holes may occur, ensure at least one indicator variable is defined for a row ƒ If no nullable columns exist add an indicator variable for at least one column ƒ If multiple nullable columns exist each indicator variable will be updated if a hole is found New value of -3 indicates hole SQLCODE +222 will also be returned (via GET DIAGNOSTICS) CUSTNO NULLABLE_COL IND_VAR CUST_TYPE 1000 M P B F P Recommendation always use indicator variables for MRF and Static Scrollable cursors If you fetch from a static scrollable cursor, a new indicator value, -3, is used to reflect holes. With row-positioned FETCH statements, this was not needed because only a single rows was returned, and the SQLCODE +222 was enough information to indicate the hole. With a MRF, -3 is used to indicate which rows have holes. The -3 will be returned for all indicator-variable-arrays that are specified. If at least one indicator is not provided, a negative SQLCODE will result.

47 Positioned DELETE Positioned Update UPDATE WHERE CURRENT OF (UWCO) and DELETE WHERE CURRENT OF (DWCO) have been modified. New syntax has been added to allow you to delete a specific row within a rowset. Use of existing syntax will cause entire rowset to be affected.

48 MERGE

49 MERGE

50 MERGE (Cont) include-columns: INCLUDE Introduces a list of columns that is to be included in the result table of the MERGE statement. The included columns are only available if the MERGE statement is nested in the FROM clause of a SELECT statement or a SELECT INTO statement. source-table:

51 MERGE (Cont) matching-condition: WHEN MATCHED or WHEN NOT MATCHED Specifies the condition under which the modification-operation is run. Each matching condition is evaluated in the specified order. WHEN MATCHED Specifies the operation to perform on the rows where the ON searchcondition is true and the target is not empty. Only UPDATE can be specified after the THEN clause. WHEN MATCHED must not be specified more than one time. WHEN NOT MATCHED Specifies the operation to perform on the rows where the ON search-condition is false or unknown, or the target is empty. Only INSERT can be specified after the THEN clause. WHEN NOT MATCHED must not be specified more than one time. modification-operation:

52 Example: SELECT FROM MERGE SELECT Account, Balance, Action FROM FINAL TABLE (MERGE INTO Customer_Table AS A INCLUDE (Action CHAR(6)) USING (VALUES (:acct_ha :acct_ind, :amount_ha :amount_ind ) FOR :num ROWS ) AS B (Account, Amount) ON (A.Account = B.Account) WHEN MATCHED THEN UPDATE SET A.Balance = A.Balance + B.Amount, Action = Update' WHEN NOT MATCHED THEN INSERT (Account, Balance, Action) VALUES (B.Account, B.Amount, 'Insert') NOT ATOMIC CONTINUE ON SQLEXCEPTION); Example shows SELECT returning data from a MERGE with an INCLUDE column that indicates if a row was an Insert or Update. The first statement does a select from searched update. This will display an athlete and his score after it has been updated in the table. I have decided to select the data from the FINAL TABLE in this case, but could have selected the data from OLD TABLE if I wanted to see the data before the update. This is done in a single operation where this used to take multiple operations to produce the same result. The second statement displays the athlete and score data that is being deleted from a table. This could be very useful for audit trail creation. The INCLUDE column specification allows you to include additional columns, either from other tables or generated, in your nested SELECT or SELECT INTO statement For instance, you may decide you want to include a timestamp with the data you are selecting. SELECT ATHLETE,EVENT,SCORE,DELETE_TS from OLD TABLE (DELETE from EVENTS INCLUDE (DELETE_TS timestamp) set DELETE_ts=CURRENT TIMESTAMP WHERE athlete='forman') ATHLETE EVENT SCORE DELETE_TS

53 Example Application data B.Account B.Amount Customer_Table Before A.Account A.Balance B.Account Returned rows B.Amount Customer_Table After include column Action Insert Insert Update Update Update A.Account A.Balance

54 MRU Multiple Row UPDATE??? MERGE INTO Customer_Table AS A USING (VALUES (:acct_ha :acct_ind, :amount_ha :amount_ind ) FOR :num ROWS ) AS B (Account, Amount) ON (A.Account = B.Account) WHEN MATCHED THEN UPDATE SET A.Balance = A.Balance + B.Amount NOT ATOMIC CONTINUE ON SQLEXCEPTION; You do not have to have both a WHEN MATCHED and WHEN NOT MATCHED clause. In this example, if a ROW is not found, then SQLCODE +100 is returned for that row, and the next row from the input array is processed. The first statement does a select from searched update. This will display an athlete and his score after it has been updated in the table. I have decided to select the data from the FINAL TABLE in this case, but could have selected the data from OLD TABLE if I wanted to see the data before the update. This is done in a single operation where this used to take multiple operations to produce the same result. The second statement displays the athlete and score data that is being deleted from a table. This could be very useful for audit trail creation. The INCLUDE column specification allows you to include additional columns, either from other tables or generated, in your nested SELECT or SELECT INTO statement For instance, you may decide you want to include a timestamp with the data you are selecting. SELECT ATHLETE,EVENT,SCORE,DELETE_TS from OLD TABLE (DELETE from EVENTS INCLUDE (DELETE_TS timestamp) set DELETE_ts=CURRENT TIMESTAMP WHERE athlete='forman') ATHLETE EVENT SCORE DELETE_TS

55 EXPLAIN changes Plan_table New QBLOCK_TYPE : "MERGE" MERGE is QB(1) "Source table" left join "target table" - only nested loop join UPDATE is QB(2) INSERT is QB(3) DSN_Statement_table New STMT_TYPE of "MERGE"

56 Other info on usage of MERGE Source data are piped into target A row inserted into target is immediately available for update A row that is updated is immediately available for further updates in the same statement NOT ATOMIC - operation continues to next input row, even after the merge operation of an input row fails No MERGE trigger UPDATE / INSERT triggers will be fired If target is a view with INSTEAD OF triggers, MERGE is not allowed Multiple-Level-Security Merging into a table with multilevel security with row-level granularity: For the UPDATE if the security label column is not specified in the SET clause as one of the columns to be updated, the security label column is updated with the user's security label. For the INSERT if the security label column is not specified in the column list, or the specified value for the column in the VALUES clause is DEFAULT, the user's security label is the value that is assigned to the security label column. If the security label is specified in the SET clause as one of the columns to be updated, or if the INSERT specified a value other than DEFAULT in the VALUES clause for the security label column, the following rules apply: The value that is specified for the security label column must be assignable to a column that is defined as CHAR(8) FOR SBCS DATA NOT NULL. If the user does not have write-down privilege, the value that is specified is substituted with the user's security label. If the MERGE statement is for a view that is defined with WITH CHECK OPTION, the update or insert fails if the user has write-down privilege and the user's security label does not dominate the security label of the new row. If the user has write-down privilege, but the value that is specified is not a valid security label, an error occurs. The UPDATE fails if the security label of the row to be updated by the MERGE does not dominate the user's security label.

57 GET DIAGNOSTICS Overview

58 Example Get Diagnostics ALL INSERT 10 Rows (not atomic) into a table with a Unique Index, Two rows Fail Eight rows succeed Use GET DIAGNOSTICS to determine which rows failed GET DIAGNOSTICS :hv = ALL STATEMENT, CONDITION; Statement information NUMBER= 3; ROW_COUNT= 8; Note: When multi-row operations are involved, there is a summary SQLCODE when a warning or error occurs. This example returns the number of rows processed by the previous INSERT statement (ROW_COUNT), and the number of diagnostic conditions (NUMBER).

59 Example Condition Number 1 CONDITION_NUMBER=1; DB2_RETURNED_SQLCODE=-253; RETURNED_SQLSTATE=22529; DB2_ERROR_CODE1=-500; DB2_ERROR_CODE2= ; DB2_ERROR_CODE3=8; DB2_ERROR_CODE4= ; DB2_SQLERRD1=-500; DB2_SQLERRD2= ; DB2_SQLERRD3=8; DB2_SQLERRD4= ; DB2_SQLERRD5= ; DB2_INTERNAL_ERROR_POINTER=-500; DB2_MODULE_DETECTING_ERROR=DSNXRINS; MESSAGE_ID=DSN00253E ; SERVER_NAME=STLEC1; MESSAGE_TEXT=A NON-ATOMIC STATEMENT SUCCESSFULLY COMPLETED FOR SOME OF THE REQUESTED ROWS, POSSIBLY WITH WARNINGS, AND ONE OR MORE ERRORS; This example shows how to return all diagnostic information using the ALL STATEMENT clause. This slide shows that the first diagnostic condition contains the information for SQLCODE -253.

60 Example Condition Number 2 CONDITION_NUMBER=2; DB2_RETURNED_SQLCODE=-803; RETURNED_SQLSTATE=23505; DB2_ROW_NUMBER=010; DB2_ERROR_CODE1=-110; DB2_ERROR_CODE2= ; DB2_ERROR_CODE4= ; DB2_SQLERRD1=-500; DB2_SQLERRD2= ; DB2_SQLERRD3=8; DB2_SQLERRD4= ; DB2_SQLERRD5= ; DB2_INTERNAL_ERROR_POINTER=-110; DB2_MODULE_DETECTING_ERROR=DSNXRINS; MESSAGE_ID=DSN00803E ; SERVER_NAME=STLEC1; DB2_ORDINAL_TOKEN_1 =I1; DB2_ORDINAL_TOKEN_2 = A; MESSAGE_TEXT=AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE I1 CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X'' A''; This example shows how to return all diagnostic information using the ALL STATEMENT clause. This slide shows that the second diagnostic condition contains the information for SQLCODE -803.

61 Example Condition Number 3 CONDITION_NUMBER=3; DB2_RETURNED_SQLCODE=-803; RETURNED_SQLSTATE=23505; DB2_ROW_NUMBER=009; DB2_ERROR_CODE1=-110; DB2_ERROR_CODE2= ; DB2_ERROR_CODE4= ; DB2_SQLERRD1=-500; DB2_SQLERRD2= ; DB2_SQLERRD3=8; DB2_SQLERRD4= ; DB2_SQLERRD5= ; DB2_INTERNAL_ERROR_POINTER=-110; DB2_MODULE_DETECTING_ERROR=DSNXRINS; MESSAGE_ID=DSN00803E ; SERVER_NAME=STLEC1; DB2_ORDINAL_TOKEN_1 =I1; DB2_ORDINAL_TOKEN_2 = B; MESSAGE_TEXT=AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE I1 CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X'' B''; This example shows how to return all diagnostic information using the ALL STATEMENT clause. This slide shows that the third diagnostic condition contains the information for SQLCODE -803.

62 Example What you really see :hv = NUMBER=3;ROW_COUNT=08;CONDITION_NUMBER=1;DB2_RETURNED_SQLCODE=- 253;RETURNED_SQLSTATE=22529;DB2_ERROR_CODE1=-500;DB2_ERROR_CODE2= ; DB2_ERROR_CODE3=8;DB2_ERROR_CODE4= ;DB2_SQLERRD1=-500;DB2_SQLERRD2= ;DB2_SQLERRD3=8;DB2_SQLERRD4= ;DB2_SQLERRD5= ; DB2_INTERNAL_ERROR_POINTER=-500;DB2_MODULE_DETECTING_ERROR= DSNXRINS;MESSAGE_ID=DSN00253E ;SERVER_NAME=STLEC1;MESSAGE_TEXT=A NON-ATOMIC STATEMENT SUCCESSFULLY COMPLETED FOR SOME OF THE REQUESTED ROWS, POSSIBLY WITH WARNINGS, AND ONE OR MORE ERRORS;CONDITION_NUMBER=2;DB2_RETURNED_SQLCODE=- 803; RETURNED_SQLSTATE=23505;DB2_ROW_NUMBER=010;DB2_ERROR_CODE1=-110; DB2_ERROR_CODE2= ;DB2_ERROR_CODE4= ;DB2_SQLERRD1= - 500;DB2_SQLERRD2= ;DB2_SQLERRD3=8;DB2_SQLERRD4= ;DB2_SQLERRD5= ;DB2_INTERNAL_ERROR_POINTER=-110;DB2_MODULE_DETECTING_ERROR =DSNXRINS;MESSAGE_ID=DSN00803E ;SERVER_NAME=STLEC1;DB2_ORDINAL_TOKEN_1 =I1;DB2_ORDINAL_TOKEN_2 = A;MESSAGE_TEXT=AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE I1 CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X'' A'';CONDITION_NUMBER=3;DB2_RETURNED_SQLCODE=-803; RETURNED_SQLSTATE=23505;DB2_ROW_NUMBER=009;DB2_ERROR_CODE1=- 110;DB2_ERROR_CODE2= ;DB2_ERROR_CODE4= ; DB2_SQLERRD1=-500;DB2_SQLERRD2= ;DB2_SQLERRD3=8; DB2_SQLERRD4= ;DB2_SQLERRD5= ;DB2_INTERNAL_ERROR_POINTER=-110; DB2_MODULE_DETECTING_ERROR=DSNXRINS;MESSAGE_ID=DSN00803E ; SERVER_NAME=STLEC1;DB2_ORDINAL_TOKEN_1 =I1;DB2_ORDINAL_TOKEN_2 = B;MESSAGE_TEXT=AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE I1 CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X'' B''; This slide shows what you would really see if you issued the GET DIAGNOSTICS ALL statement. The data is returned in a VARCHAR(32672) string.

63 GET DIAGNOSTICS - Performance Local A local application that uses GET DIAGNOSTICS pays an api call to DB2. DSNTIAR is cheaper than calling GET DIAGNOSTICS to get message text. DSNTIAR is NOT an api call to DB2. Remote/DRDA A remote application that uses GET DIAGNOSTICS pays an api call to the client. The server is not involved diagnostic information was sent back with the data.

64 Session: E05 Faster FETCH, INSERT, UPDATE (MERGE) DB2 for z/os Multiple Row Processing Christopher J. Crone IBM DB2 for z/os Development

65 Additional Information

66 INSERT SQLDA Considerations SQLDA must contain a valid description of the host variable arrays or buffers which contain the values to be inserted ƒ Each SQLVAR describes a host variable or host variable array which represents a buffer which contains value(s) for a column of target table ƒ SQLDA must have enough storage to contain SQLVAR for each target column for which values are provided, plus an additional SQLVAR entry for use by DB2 for z/os ƒ Prior to the multi-row insert, the SQLDA fields must be set correctly to include number of SQLVAR occurrences, number of variables used, pointer to arrays, indicator variables etc. If you code statements with USING DESCRIPTOR instead of USING host variables, there are some changes that you might need to make. Please review the SQL Reference, Appendix C, for more information regarding SQLDA fields that must be filled in for MRI. In particular, pay attention to the values that must be set in the SQLNAME field for each SQLVAR.

67 OPEN/ALLOCATE CURSOR Users can use GET DIAGNOSTICS after OPEN or ALLOCATE to determine if a cursor is enabled for ROWSET positioning ƒ DB2_SQL_ATTR_CURSOR_ROWSET N indicates that this cursor only supports row positioned operations Y indicates that this cursor supports rowset positioned operations After OPEN or ALLOCATE CURSOR, new information is returned via GET DIAGNOSTICS that indicates the rowset capabilities of the cursor.

68 Considerations with Dynamic Scrollable Cursors Locks will be held on base table for entire rowset with ISO CS, RR and RS Starting point and contents of rowsets may change when scrolling back and forth Refetching current rowset may return different rows ƒ If ISO(RR), other applications cannot affect your rowset ƒ Your application can still affect your rowset FETCH PRIOR ROWSET will return the previous n rows that qualify from the start of the current cursor position ƒ Therefore n rows will be returned as long as start of the result set is not reached Dynamic scrollable cursors are not supported for queries which result in materialized result tables (i.e., a query containing an ORDER BY, which is not supported by an index). Consequently locks will always be held on the base table according to normal locking rules. The contents of a rowset are a point in time statement. A subsequent FETCH may reflect different rows (Except ISO RR). For example, FETCH NEXT ROWSET, followed by FETCH PRIOR ROWSET may not result in the same data being returned. Even FETCH CURRENT ROWSET may result in different values being returned. If your application cannot handle these sorts of changes, do not use DYNAMIC SCROLLABLE CURSORS, use STATIC SCROLLABLE CURSORS.

69 Multiple Row FETCH/INSERT -- DRDA Considerations For remote client one rowset is returned in a network request this is in effect the blocksize Updateable cursors will still be blocked. Supported by any requester or server that supports DRDA Level 3 DB2 for z/os V8 DB2 LUW V8 FP 4 ODBC client Limit of 10 megabyte buffer for distributed MRF/MRI Distributed support is only in ODBC Embedded support for distributed is not available yet One advantage that rowset cursors will enjoy is that for updateable cursors in V7 and below the protocol is a one row at a time (no blocking) protocol. With rowset cursors, the blocksize will be the rowset size. So blocking is now possible for updateable cursors.

70 Similar to SQLCA EXEC SQL GET DIAGNOSTICS CONDITION 1 :gdsqlcode = DB2_RETURNED_SQLCODE, :gdsqlerrml = :gdsqlerrmc = :gdtokens = DB2_TOKEN_COUNT, :gdtoken1 = DB2_ORDINAL_TOKEN_1, :gdtoken2 = DB2_ORDINAL_TOKEN_2, :gdtoken3 = DB2_ORDINAL_TOKEN_3, :gdtoken4 = DB2_ORDINAL_TOKEN_5, :gdtoken5 = DB2_ORDINAL_TOKEN_5 :gdsqlerrp = DB2_MODULE_DETECTING_ERROR, :dgsqlerrd1 = DB2_ERROR_CODE1, :gdsqlerrd2 = DB2_ERROR_CODE2, :gdsqlerrd3 = DB2_ERROR_CODE3, :gdsqlerrd4 = DB2_CONNECTION_STATUS; :gdsqlerrd5 = DB2_INTERNAL_ERROR_POINTER, :gdsqlerrd6 = DB2_ERROR_CODE4; :gdsqlwarn = :gdsqlstate = RETURNED_SQLSTATE; Information provided in the Diagnostic Area is similar to that provided by the SQLCA. A couple of Key differences: -There is no direct equivalent for SQLERRML and SQLERRMC -SQLWARN info is not included. Even if you wanted to standardize on the Diagnostic area, you need to be aware that there would be a performance cost.

71 Get Diagnostics Syntax

72 Get Diagnostics Syntax

73 Get Diagnostics Syntax

74 Get Diagnostics Syntax pppvvrrm DSN08015

75 Get Diagnostics Syntax

76 Session: E05 Faster FETCH, INSERT, UPDATE (MERGE) DB2 for z/os Multiple Row Processing Christopher J. Crone IBM DB2 for z/os Development