1 Copyright 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8

Transcription

1 1 Copyright 2011, Oracle and/or its affiliates. All rights

2 Optimize Java Persistence/Scale Database Access with JDBC and Oracle Universal Connection Pool Ashok Shivarudraiah & Tong Zhou 2 Copyright 2011, Oracle and/or its affiliates. All rights

3 3 Copyright 2011, Oracle and/or its affiliates. All rights

4 Latin America 2011 December 6 8, 2011 Tokyo 2012 April 4 6, Copyright 2011, Oracle and/or its affiliates. All rights

5 Oracle OpenWorld Bookstore Visit the Oracle OpenWorld Bookstore for a fabulous selection of books on many of the conference topics and more! Bookstore located at Moscone West, Level 2 All Books at 20% Discount 5 Copyright 2011, Oracle and/or its affiliates. All rights

6 Agenda Efficient Java Persistence with Oracle JDBC 11g LOB Performance Caching Technologies Optimizing Data Transfer Updating and Patching applications Memory Management and TradeOffs Scale and Failover with UCP Tips on Debugging JDBC and UCP applications Q & A <Insert Picture Here> 6 Copyright 2011, Oracle and/or its affiliates. All rights

7 JDBC in Oracle Database 11g IPV6 & Support for SCAN Support for JDBC 4.0 Specification Built-in Advanced Security in JDBC-Thin New Encryption and Checksum algorithms Third party Authentication: Kerberos, Radius, SSL Support for Query Change Notification Programmatic DB Startup & Shutdown Performance Result Cache (JDBC-OCI): 5-8 x Faster Native AQ interface: % Faster LOB PreFetch Zero Copy SecureFiles LOB 7 Copyright 2011, Oracle and/or its affiliates. All rights

8 Optimize LOB operations LobPrefetch: Faster Lob fetching in a single roundtrip setlobprefetchsize() select name, bio from bios_tab Result Set ResultSet Internal Buffer Parse + Execute + Fetch metadata + Fetch LOB data + Copy 8 Copyright 2011, Oracle and/or its affiliates. All rights

9 LOB PreFetching Performance Throughput (per sec) 9 Copyright 2011, Oracle and/or its affiliates. All rights

10 SecureFiles LOBs: Avoid Internal Memory Copy Large Reads/Writes BASIC LOBs: internal buffer copy are expensive SECUREFILE LOBS: Zero-copy IO or Vectored i/o mechanism setupsecurefile() Blob.getBytes() Result Set Fetch/Stream LOB data directly (bypass internal buffer) 10 Copyright 2011, Oracle and/or its affiliates. All rights

11 Oracle SecureFiles High Performance Large Object Storage Substantial Performance Improvement Performance comparable to or greater than file systems Compression De-duplication Encryption Compatible with existing Oracle BLOB/CLOB syntax Compression and De-Duplication require Advanced Compression Option 11 Copyright 2011, Oracle and/or its affiliates. All rights

12 Oracle SecureFiles Compression and De-Duplication CREATE TABLE images ( image_id NUMBER, image LOB(image) BLOB) STORE AS SECUREFILE ( TABLESPACE lob_tbs COMPRESS); 12 Copyright 2011, Oracle and/or its affiliates. All rights

13 Database Caching Technologies SQL Result Set Cache Cache queries and sub queries Shared by all sessions Full consistency and proper semantics 2x - 100x speedup PL/SQL Function Cache Automatic invalidation Shared by all sessions Use for deterministic functions Client-side Result Set Cache 13 Copyright 2011, Oracle and/or its affiliates. All rights

14 Client-side ResultSet Caching Configure Database (init.ora) client_result_cache_size=200m client_result_cache_lag=5000 Configure Client (sqlnet.ora) OCI_QUERY_CACHE_SIZE=200M OCI_QUERY_CACHE_MAXROWS= : Transparent ResultSet Caching -- No code change alter table emp result_cache; Application Server Database 11.1 : Set hints for Caching the Result Set select /*+ result_cache */ * from employees Available with JDBC-OCI, C, C++, PHP, Ruby, ODP.Net, ODBC 14 Copyright 2011, Oracle and/or its affiliates. All rights

15 Consistent Client-side ResultSet Caching Performance 5-8 x Faster (Query Serviced in Client-Cache) 15 Copyright 2011, Oracle and/or its affiliates. All rights

16 Online Application Upgrade & Patching Requirements & Challenge Application v1 Database objects Application v2 Allow making changes database objects while application is running Make changes transparent to application 16 Copyright 2011, Oracle and/or its affiliates. All rights

17 Online Application Upgrade & Patching Solution: Edition-Based Redefinition 2 Application v1 Application v2 View Procedure View Procedure Base Edition New Edition Table Change objects in new edition w/o affecting online users New Column Deploy new application referencing new edition Current users are not effected, they still reference base edition Phase in new application version over time and drop the old one 17 Copyright 2011, Oracle and/or its affiliates. All rights

18 Upgrading & Patching 24x7 Java Applications App V1 Default EDITION Initial Database schema (tables, PL/SQL packages) V1.java App V2 Create new EDITION Update Database schema (tables, PL/SQL packages) V2.java set session in edition name with conn. property CONNECTION_PROPERTY_EDITION_NAME Update application logic to use new schema. 18 Copyright 2011, Oracle and/or its affiliates. All rights

19 Query Change Notification Automatic Reports Caching/Refreshing a) Execute Query b) Return ResultSet without executing the query as long as ResultSet is valid c) When changes invalidate ResultSet, RDBMS sends Notification Java 1. Register the Query 4.Invalidate cache 5.Repopulate cache Custom cache Callout 2.DML (Change to the result set) 3.Automatic Notification 19 Copyright 2011, Oracle and/or its affiliates. All rights

20 Advanced Security with JDBC For company wide security policies, Oracle JDBC furnishes the following advanced security features: Encryption: AES (128/192/256-bit),3DES (112/168-bit), RC4 Data Integrity: MD5 (16 bytes), SHA1 (20 bytes) Authentication: Radius, Kerberos, SSL, OS, Password Example (Kerberos) create user identified externally; prop.setproperty(oracleconnection.connection_property_thi N_NET_AUTHENTICATION_SERVICES, "( KERBEROS )"); Otherwise, we recommend SSL as single technology for Encryption + Data Integrity + Strong Authentication 20 Copyright 2011, Oracle and/or its affiliates. All rights

21 JDBC Memory Management and Tradeoffs Memory Allocation definebytes and definechars arrays VARCHAR(4000) ->8k bytes, 4k for RAW and LOB columns, and 32 bytes for everything else A Query returning 200 VARCHAR(4000) columns with 100 fetch size will allocate (2 * 4000) * 200 * 100 = 160MB Problem: Zeroing definebytes and definechars array buffers is required by Java lang. spec. but is expensive Solution: Trading Space for Speed or vice versa 21 Copyright 2011, Oracle and/or its affiliates. All rights

22 Memory Allocation and Issues Trading Space for Speed Statement Caching Reuses those buffers Eliminates the overhead of repeated cursor creation Prevents repeated statement parsing and creation Oracle JDBC Drivers furnish two Statement Caching Implicit statement caching Explicit statement caching 22 Copyright 2011, Oracle and/or its affiliates. All rights

23 Memory Allocation and Issues Trading Speed for Space In 10.2 JDBC freememoryonenterimplicitcache connection properties: buffers released when Statement returned to the cache If the application has many open statements at once, this can be a problem. In and up maxcachedbuffersize connection property mitigates this problem Driver more sophisticated Memory Allocation 23 Copyright 2011, Oracle and/or its affiliates. All rights

24 Optimizing Data Transfer (SDU) Controls SQL*Net packet size Default is 2k in pre 11.2 and 8192 from 11.2 Max is 64k For bulk data transfer (LOB, XML), increase to 64k in URL url= (SDU=8192) ) sqlnet.ora: DEFAULT_SDU_SIZE tnsnames.ora: SDU in address Larger SDU gives Better Network throughput Fewer system calls to send and receive data Less CPU usage system and user But Beware Memory allocation 24 Copyright 2011, Oracle and/or its affiliates. All rights

25 Agenda Efficient Java Persistence with Oracle JDBC 11g Scale and Failover with UCP Fast Connection Failover Runtime Connection Load Balancing Web Session Affinity Transaction Affinity Tips on Debugging JDBC and UCP applications Q & A <Insert Picture Here> 25 Copyright 2011, Oracle and/or its affiliates. All rights

26 Universal Connection Pool A Single/Universal Java Connection Pool Supports any type of Java connection (JDBC, XA, JCA, LDAP) Supports stand-alone deployment (BPEL, Toplink, Tomcat) Supports any database (Oracle, non-oracle) HA and Scalability Services Fast Connection Failover (FCF) Runtime Connection Load Balancing (RCLB) Web Session based Affinity to RAC instance Transaction based Affinity to RAC instance 26 Copyright 2011, Oracle and/or its affiliates. All rights

27 Fast Connection Failover (FCF) Rapid database failure detection Aborts and removes invalid connections from the pool Supports unplanned and planned outages Recognizes new nodes that join an Oracle RAC cluster Distributes runtime work requests to all active Oracle RAC instances 27 Copyright 2011, Oracle and/or its affiliates. All rights

28 Implement HA with FCF Fast Connection Failover Failed Database Connections Inst x New Database Connections Inst y 28 Copyright 2011, Oracle and/or its affiliates. All rights

29 Implement HA with FCF Application View PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource(); pds.setonsconfiguration( nodes=racnode1:4200,racnode2:4200 ); pds.setfastconnectionfailoverenabled(true); boolean retry = false; do { try { Connection conn = pds.getconnection("scott", "tiger"); // Necessary recovery if retrying // Data operations using conn object retry = false; } catch (SQLException sqlexc) { if (conn == null!((validconnection)conn).isvalid()) retry = true; } } while (retry); 29 Copyright 2011, Oracle and/or its affiliates. All rights

30 Implement HA with FCF WebLogic config 30 Copyright 2011, Oracle and/or its affiliates. All rights

31 Runtime Connection Load Balancing Manages pooled connections for high performance and scalability Receives continuous recommendations on the percentage of work to route to Database instances Adjusts distribution of work based on different backend node capacities such as CPU capacity or response time Reacts quickly to changes in Cluster Reconfiguration, Application workload, overworked nodes or hangs 31 Copyright 2011, Oracle and/or its affiliates. All rights

32 Runtime Connection Load Balancing 30% connections RAC Database Application UCP I m busy 10% connections I m very busy Instance1 I m idle Instance2 60% connections Instance3 32 Copyright 2011, Oracle and/or its affiliates. All rights

33 Scale Performance with RCLB Client / mid-tier: enable Fast Connection Failover Server: enable RAC load-balancing advisory (LBA) Server: set LBA GOAL for each DB service NONE (default) GOAL_SERVICE_TIME best overall service time GOAL_THROUGHPUT best overall throughput Server: set DB listener CLB_GOAL to SHORT CLB_GOAL_SHORT connection load balancing uses RAC Load Balancing Advisory CLB_GOAL_LONG for applications that have long-lived connections 33 Copyright 2011, Oracle and/or its affiliates. All rights

34 Web-Session Affinity The first connection request uses RCLB to select a connection Subsequent requests enforce Affinity Connection selection falls back to RCLB after Affinity ends Affinity is only a hint Pool resorts to RCLB whenever a desired connection is not found 34 Copyright 2011, Oracle and/or its affiliates. All rights

35 Web-Session Affinity Web Client RAC Database Connect to me Instance1 UCP Instance2 Connection Affinity Context Instance3 35 Copyright 2011, Oracle and/or its affiliates. All rights

36 Implement Web-Session Affinity PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource(); pds.setonsconfiguration( nodes=racnode1:4200,racnode2:4200 ); pds.setfastconnectionfailoverenabled(true); ConnectionAffinityCallback cbk = new MyCallback(); Pds.registerConnectionAffinityCallback(cbk);... class MyCallback implements ConnectionAffinityCallback {... Object appaffinitycontext = null; AffinityPolicy policy = AffinityPolicy.WEBSESSION_AFFINITY; public boolean setconnectionaffinitycontext(object cxt) { appaffinitycontext = cxt; } } public Object getconnectionaffinitycontext() { return appaffinitycontext; } Copyright 2011, Oracle and/or its affiliates. All rights

37 Transaction Based Affinity Enables XA and RAC to work together with optimal performance Eliminates current single DTP service limitation for XA/RAC Transaction affinity is the ability to automatically localize a global transaction to a single RAC instance Transaction Affinity scope is the life of a global transaction first connection request for a global transaction uses RCLB subsequent requests uses affinity and are routed to the same RAC instance when XA first started 37 Copyright 2011, Oracle and/or its affiliates. All rights

38 Transaction Based Affinity TX Client 1 TX Client 2 RAC Database Instance1 UCP Instance2 Connection Affinity Context Instance3 38 Copyright 2011, Oracle and/or its affiliates. All rights

39 Implement Transaction Based Affinity PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource(); pds.setfastconnectionfailoverenabled(true); ConnectionAffinityCallback cbk = new MyCallback(); Pds.registerConnectionAffinityCallback(cbk);... class MyCallback implements ConnectionAffinityCallback {... Object appaffinitycontext = null; AffinityPolicy policy = AffinityPolicy.TRANSACTION_AFFINITY; public boolean setconnectionaffinitycontext(object cxt) { appaffinitycontext = cxt; } } public Object getconnectionaffinitycontext() { return appaffinitycontext; } Copyright 2011, Oracle and/or its affiliates. All rights

40 Q&A 40 Copyright 2011, Oracle and/or its affiliates. All rights

41 Agenda Efficient Java Persistence with Oracle JDBC 11g Scale and Failover with UCP Tips on Debugging JDBC and UCP applications Common JDBC environment issues JDBC Connection performance issues JDBC Statement performance issues Common UCP issues UCP HA and scalability issues Q & A <Insert Picture Here> 41 Copyright 2011, Oracle and/or its affiliates. All rights

42 Debugging Tips: Common Issues Not knowing which Jar file is being used System.out.println( jar file: +oracle.jdbc.oracledriver.class.getresource( )); Multiple JDBC jar files in the classpath You might get java.lang.securityexception: sealing violation: package oracle.jdbc.driver is sealed JDBC driver version Jdbc driver version is included in the Manifest file To figure out the Jdbc jar file version do: unzip c ojdbc5.jar META-INF/MANIFEST.MF java jar ojdbc5.jar version in 11.2.X 42 Copyright 2011, Oracle and/or its affiliates. All rights

43 Debugging Tips: Connect Performance JDBC application slower Check your SDU in the connect string or In sqlnet trace, tune appropriately Frequent connect-disconnect Look at listener.log to figure out, enable Connection Cache Set connect timeout to avoid waiting for unreachable host Connection Property oracle.net.connect_timeout or DriverManager.setLoginTimeout(int) 43 Copyright 2011, Oracle and/or its affiliates. All rights

44 Debugging Tips: Connect Performance The client is stuck on a read Check if the server is still reachable (sqlplus user/passwd) If the host is not reachable, then enable keep-alive (ENABLE=BROKEN) in the connect string Set the tcp_keepalive_time, tcp_keepalive_intvl to detect broken connections faster 44 Copyright 2011, Oracle and/or its affiliates. All rights

45 Debugging Tips: Statement Performance Tip 1: Turn on Statement Caching Enable bind tracing (ALTER SESSION SET EVENTS '10046 trace name context forever, level 8';) Use tkprof to see the number of parse, execute & fetch If parse is more than 1 there is a definite benefit in enabling statement cache Tip 2: Monitor fetch-size and size in registeroutparameter These can be monitored by enabling java.util logging in JDBC The parameter values would be displayed in the generated trace file 45 Copyright 2011, Oracle and/or its affiliates. All rights

46 Debugging Tips: Statement Performance Tip 3: Use dynamic binding instead of static bindings Analyze statements being executed by JDBC logging (oracle.jdbc.driver.level = CONFIG) Change the SQL to dynamic binding to avoid SQL Injection issues and also to achieve better performance Tip 4: Use batching for repeated DML Analyze statement being executed by JDBC logging (oracle.jdbc.driver.level = CONFIG) Enable batching for similar statement execution 46 Copyright 2011, Oracle and/or its affiliates. All rights

47 Debugging Tips: Common Pool Issues UCP logging Standard java.util.logging facility Support for per-package/class logging UCP log formatter UCP exceptions Exception chaining for general pool exceptions Support for JDBC 4.0 SQLRecoverableException UCP version java jar ucp.jar 47 Copyright 2011, Oracle and/or its affiliates. All rights

48 UCP Logging Configuration Example handlers = java.util.logging.consolehandler java.util.logging.consolehandler.level = ALL java.util.logging.consolehandler.formatter = oracle.ucp.util.logging.ucpformatter.level = WARNING oracle.ucp.jdbc.oracle.oraclefailoverhandler.level = FINE oracle.ucp.jdbc.oracle.oraclejdbcconnectionpool.level = ALL java.util.logging.filehandler.pattern = ucp.log java.util.logging.filehandler.formatter = java.util.logging.xmlformatter oracle.ucp.jdbc.level = INFO oracle.ucp.jdbc.handlers = java.util.logging.filehandler oracle.ucp.jdbc.useparenthandlers = false 48 Copyright 2011, Oracle and/or its affiliates. All rights

49 Debugging Tips: HA and Scalability Issues Check HA statistics FCF, RCLB, Web Session and Transaction Affinity Turn on UCP logging Feature-specific logging property files come with demos HA toolkit for troubleshooting setup issues Come with demos and available on OTN for RAC and UCP Turn on WebLogic logging Best using WebLogic Server Administration Console 49 Copyright 2011, Oracle and/or its affiliates. All rights

50 HA Staticstics Example PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();... OracleJDBCConnectionPoolStatistics ostats = (OracleJDBCConnectionPoolStatistics) pds.getstatistics(); String fcfinfo = ostats.getfcfprocessinginfo(); Jan 29, :14:52 SUCCESS <Reason:unplanned> <Type:SERVICE_DOWN> \ <Service:"sn_1"> <Instance:"dsin_1"> <Db:"db_1"> \ Connections:(Available=6 Affected=2 MarkedDown=2 Closed=2) \ (Borrowed=6 Affected=2 MarkedDown=2 Closed=2) Jan 29, :14:53 SUCCESS <Type:HOST_DOWN> <Host:"h_1"> \ Connections:(Available=6 Affected=4 MarkedDown=4 Closed=4) \ (Borrowed=6 Affected=4 MarkedDown=4 Closed=4) Jan 29, :14:54 SUCCESS <Reason:planned> <Type:SERVICE_UP> \ <Service:"sn_1"> <Instance:"dsin_1"> <Db:"db_1"> \ Connections:(Available=6 Affected=2 MarkedDown=2 Closed=2) \ (Borrowed=6 Affected=2 MarkedDown=2 Closed=2) TornDown=2 \ MarkedToClose=2 Cardinality=2 50 Copyright 2011, Oracle and/or its affiliates. All rights

51 WebLogic Logging Example Server Debug Tab 51 Copyright 2011, Oracle and/or its affiliates. All rights

52 52 Copyright 2011, Oracle and/or its affiliates. All rights

53 53 Copyright 2011, Oracle and/or its affiliates. All rights