purequery Deep Dive Part 2: Data Access Development Dan Galvin Galvin Consulting, Inc.

Size: px

Start display at page:

Download "purequery Deep Dive Part 2: Data Access Development Dan Galvin Galvin Consulting, Inc."

Douglas Copeland
5 years ago
Views:

1 purequery Deep Dive Part 2: Data Access Development Dan Galvin Galvin Consulting, Inc.

2 Agenda The Problem Data Access in Java What is purequery? How Could purequery Help within My Data Access Architecture? Performance Improvements with purequery Best Practices In Java Data Access Development

4 What Is the Problem? Our Efforts Want to Use Java to Develop Business Critical Applications Access to Operational Data Stores High Volume of Real-Time Transactions The Great Divide Data vs. Application Is Data Access My Responsibility?

5 What Are the Goals? Increase Productivity Developer, Data Specialists, Performance Experts Connect the Dots in Problem Resolution Faster Close the Gap! Deliver More Useful Solutions Improve Performance Significantly Reduce CPU Consumption Improve Access Path Reliability Reduce Costs Reduction in Capacity Requirements Improved Productivity Reduction in Time to Market

6 HOW DO WE ACCESS DATA TODAY?

7 Java Data Access Methods JDBC Frameworks Entity Beans Java Data Objects O/RMs Mappers and Wrappers Hibernate Java Persistence API Toplink ibatis

9 JDBC and Frameworks Pros and Cons JDBC and Frameworks Pros and Cons JDBC is mature Competent resources are available in the workforce Frameworks can help to eliminate much of the tedious coding Tooling exists to help with coding Straight JDBC is tedious and time-consuming Industry Frameworks JDBC is executed as dynamic SQL only, even for SQL that is static in nature.

10 Static vs. Dynamic SQL Execution

11 ORM Pros and Cons Allows Data Model and Object Model to be developed and maintained separately Inherent capability to reduce calls to the database with lazy fetch, managed objects and intelligent updates SQL is Generated at runtime by the implementation Requires developers to learn a new language JPQL, HQL JDBC implementation. Dynamic SQL only. To maintain model independence benefit, you have to use meet in the middle approach which is extremely complex

comprising Persistence Layer Top Down Meet in the Middle Data-Centric Bottom-UP Start with Relational

12 Data Mapping Approaches Application-Centric Top-Down Start with Object Domain Model ORM Mapping Well supported in dynamic languages and frameworks Hybrid Meet in the middle Can be challenging w/o comprising Persistence Layer Top Down Meet in the Middle Data-Centric Bottom-UP Start with Relational Data Model Not well supported in dynamic languages and frameworks Bottom Up 2010 Galvin Consulting, 12 Inc.

JSP Spring XML http SQL mashup JDBC JSON Difference of Concerns QoS goals Runstats Partition Stored strategy

Why do I need to understand how the data is stored? Isn t that the DBA s job? What is an access path?

I don t need to know Java just show me the SQL Here is a long running query who owns it?

13 JSP Spring XML http SQL mashup JDBC JSON Difference of Concerns QoS goals Runstats Partition Stored strategy Procedures Response REORG Time! Developer DBA I need to inflate and persist objects. I know SQL well enough at least. Why do I need to understand how the data is stored? Isn t that the DBA s job? What is an access path? I need to fulfill the business requirements on time. We can tune the data access later. I don t need to know Java just show me the SQL Here is a long running query who owns it? Do they think we just have unlimited capacity? Who is going to prevent SQL injection in the application code? What is J2EE? Isn t this a Java Application? Why don t developers know how to tune?

14 DATA ACCESS METADATA

purequery Components: Optim Development Studio Integrated development environment with Java and SQL support Improve problem

15 Introducing purequery A high-performance, data access platform to simplify developing, managing, securing, and optimizing data access for new and existing applications. purequery Components: Optim Development Studio Integrated development environment with Java and SQL support Improve problem isolation and impact analysis Simple and intuitive API Facilitates JDBC best practices Enables heterogeneous batching Enables SQL access to databases or in-memory Java objects Optim purequery Runtime Flexible static SQL deployment for DB2

16 purequery API Annotated Method Interface with Annotations Generated Implementations Provides Option for Static Packages In-Line Used for Dynamic Natured SQL Coded Directly in the DAO Class

17 purequery Specifies that the method executes a SELECT or SELECT FROM INSERT SQL Specifies that the method runs an UPDATE, INSERT, or DELETE SQL Specifies that the method executes a CALL SQL Specifies specific handler classes to be used for input and Specifies that the String contains a SQL statement to be validated (any Java class).

18 Demo 1: purequery Productivity Using Optim Development Studio

19 Layered Architecture Simplify development, but Complex Problem Resolution Performance Obscure impact analysis Impede capacity planning Layers obscure linkages Where is the problem? Which application is running? Which application does this SQL belong to? Application Layer Database Layer Business Logic Connection Pool EJB Query Language Data Access Logic Persistence Layer DB Java Driver Network A A A

literals with parameter markers at runtime

20 Metadata Provided by purequery Review SQL coming from applications Consolidate SQL by replacing literals with parameter markers at runtime Optimize database resources (e.g. dynamic statement cache) usage

21 Demo 2: Data Access Metadata in Problem Resolution

22 Performance with purequery Static SQL Execution Access Path Stability SQL Operations Cursors versus SELECT INTO Heterogeneous Batching

23 How well does purequery work on z/os? Normalized Throughput by API for JDBC Type 4 Driver % increase/reduction in CPU per transaction compared to JDBC using Type 4 driver 500 Normalized Throughput (ITR) EJB JPA 360 JDBC 420 pq Method Dynamic 446 Client Optimizn Static 485 pq Method Static 524 % increase/reduction in CPU per transn compared to JDBC -50% -14% -35% EJB 2 JPA pq Method Dynamic 15% 25% 6% Client Opt. Static pq Method Static IRWW an OLTP workload, Type 4 driver Cache hit ratio between 70 and 85% 23 % improvement in throughput using purequery over dynamic JDBC 15% - 25% reduction on CPU per transaction over dynamic JDBC

24 purequery Increased Throughput on DB2 LUW On identical hardware, purequery API Static (Method Style) more than doubled the transaction throughput. * See speaker notes

25 Best Practices Configuration Data Access Architecture Scope and Granularity Static SQL Execution Batching Transactions in Data Access

26 Configuration with spring-pdq <beans>  <bean id="customerdao" class="com.inferdata.db.dao.customerdao"> <constructor-arg ref="custdata" /> </bean>   <bean id="custdataclass" class="java.lang.class factorymethod="forname"> <constructor-arg value="com.inferdata.db.dao.pdq.custdata" /> </bean> <bean id="custdata class="org.springmodules.purequery.pdqannotatedmethodfactory factorymethod="getdata"> <constructor-arg ref="custdataclass" /> <constructor-arg ref="datasource" /> <constructor-arg ref="pdqproperties" /> </bean>  </beans>

27 DAO Pattern Implementation Insulate the Service Object from the specific Data Access implementation Provide the SO with a consistent API Group together operations on associated data structures which occur within one Unit of Work (UOW) operation Implement the DAO Pattern with a concrete class that sits in front of the purequery Interfaces.

28 Data Access Architecture

29 Scope and Granularity DAO Objects should encapsulate data access logic for one SO call. One SO may invoke multiple DAOs One DAO may invoke multiple purequery Interface methods across one or more interfaces Interfaces should be generally organized by major data objects Consider limited refactoring if one SO call invokes an excessive number of DB2 packages

30 Static SQL Execution If the statement is static in nature execute it statically Even if a statement is dynamic in nature, if it is executed frequently enough, look for small numbers of patterns which are static Do Not Use Literals in SQL, unless they really never change. Regardless of your Data Access Methodology, consider using Client Optimization to obtain static execution Use wsdb2gen with O/RMs to bind statically your access and allow for easier SQL review

31 Batching Homogeneous Batching allows for one lineturn to the DBMS with the same statement Heterogeneous Batching can be used with multiple statements in one line-turn The larger your batches, the more savings you will realize Heterogeneous Batches can contain 0 to many Homogeneous Batches Code Batch Control in the DAO Returns UpdateManyException if there is an error in the batch. Statements are handled as non-atomic

32 Transactions in Data Access As a general rule, always use an explicitly defined transaction when invoking a DBMS Even for Read Only statements Even if you think you are only going to make one call in the transaction With DB2, an implicit transaction will be created for each call not contained within an existing transaction With Implicit Transactions, you will pick up the default transaction isolation

34 Resources Web page: IBM purequery platform: ibm.com/software/data/optim/purequery-platform/ Community page: Integrated Data Management (links to downloads, forums, articles, and more): ibm.com/developerworks/spaces/optim Redbook: Using Integrated Data Management To Meet Service Level Objectives: Tutorial: Optimize your existing JDBC applications with purequery ibm.com/developerworks/edu/dm-dw-dm-0808titzler-i.html Demo: purequery for DB2 for z/os (Improving ROI for existing applications): ibm.com/developerworks/offers/lp/demos/summary/im-purequery4zos.html Demo: Java acceleration solution demo: ibm.com/developerworks/offers/lp/demos/summary/im-optimsolutionsforjava.html

35 Resources, continues Article: What s new and cool in Optim Development Studio 2.2 ibm.com/developerworks/data/library/techarticle/dm-0906optimdeveloper/ See the related video series on ChannelDB2 Video tutorial series: Develop a Java application using purequery annotated methods ibm.com/developerworks/views/data/libraryview.jsp?type_by=demos&search_by =purequery+annotated+methods Product documentation:

36 Contact Information Web: Blog: LinkedIn:

37 Coming next: What s new in Optim Performance Management Tooling for DB2 for Linux, UNIX, and Windows April 29, 2010 Don t leave yet. Please fill out the survey! Get the latest and greatest news about the IBM Optim performance management solution for DB2 for Linux, UNIX, and Windows to help you resolve emergent performance problems before they affect the business. This session will be held on April 29, Registration available soon at ibm.com/developerworks/data/events/idmbriefings.html

Building and Managing Efficient data access to DB2. Vijay Bommireddipalli, Solutions Architect, Optim

Building and Managing Efficient data access to DB2 Vijay Bommireddipalli, vijayrb@us.ibm.com Solutions Architect, Optim September 16, 2010 Information Management Disclaimer THE INFORMATION CONTAINED IN