1 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Transcription

1 1 Copyright 2011, Oracle and/or its affiliates. All rights

2 ORACLE PRODUCT LOGO Advanced Administration and Management of Oracle SOA Suite 11g Samrat Ray, Ramkumar Menon, Srimant Misra Oracle SOA Suite Product Management and Development 2 Copyright 2011, Oracle and/or its affiliates. All rights

3 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 3 Copyright 2011, Oracle and/or its affiliates. All rights

4 Program Agenda Overview Provisioning and Configuration Management Managing a System in Production Transactions and Fault Handling Troubleshooting Q&A 4 Copyright 2011, Oracle and/or its affiliates. All rights

5 Overview Mediation BPEL Rules BPEL orchestration Human Tasks Business Rules Web services and adapters Human Task SCA Composite SOA Composite Editor Policy Management Optimized Service & Eventing Infrastructure BPMN SOAP BPEL B2B Human Task HTTP Policy Manager Service Infrastructure JCA EJB Common Connectivity Infrastructure 5 Copyright 2011, Oracle and/or its affiliates. All rights

6 Program Agenda Overview Provisioning and Configuration Management Managing a System in Production Transactions and Fault Handling Troubleshooting Q&A 6 Copyright 2011, Oracle and/or its affiliates. All rights

7 SOA Domain Overview Simple, one-machine install EM SOA BAM Administration Server Managed Server Managed Server Domain home (configuration) EM SOA BAM Oracle home (binaries) Administration Server Managed Server Cluster 1: SOA Managed Server Typical SOA HA install Managed Server Managed Server SOA_Domain Cluster_Domain Cluster 2: BAM 7 Copyright 2011, Oracle and/or its affiliates. All rights

8 SOA Configuration 3 Types of Properties WLS Properties Persisted in domain configuration files, changes are propagated across the cluster by the Adminserver. Service Infrastructure (soa-infra) Properties 11gR1 PS1: Properties are persisted in MDS (database), changes are automatically propagated across a cluster. Composite Properties Meta-data associated with a composite is stored in MDS (database) These properties can be customized using a Configuration Plan at the time of deployment. Composite properties are automatically propagated across a cluster as part of the composite deployment process. 8 Copyright 2011, Oracle and/or its affiliates. All rights

9 SOA Domain Directory Structure Configuration: Stored under config directory The config directory contains a file called config.xml which is the primary source of all configuration information within a domain. SOA configuration is stored in MDS since PS1. Scripts: Stored under bin directory. Start / stop Admin & Managed server Set environment and JVM properties Logs: Server logs are placed under servers/<server_name>/logs domain home start/stop & env scripts configuration root local cache of deployed composites sub-directory for each managed server (logs / cache) 9 Copyright 2011, Oracle and/or its affiliates. All rights

10 SOA 11g MDS Store Runtime Store Composite meta-data Security policies Error policies B2B Agreements Shared WSDLs and XSDs ADF / Human Workflow Can be accessed from JDeveloper and Ant. Configuration customization is stored in MDS. 10 Copyright 2011, Oracle and/or its affiliates. All rights

11 Configuration Propagation in a Cluster Admin SOA 1 SOA Cluster WLS + Coherence Clusters MDS File based WLS Configuration WLS Configuration AdminServer has gold copy Configuration propagated by WLS JMX framework. During startup, a Managed Server copies configuration from AdminServer and overwrites local copy. SOA 2 SOA Configuration DB (MDS) based. SOA Configuration Database has gold copy Configuration propagated by soa-infra using the Coherence cluster. During startup, a SOA server loads its configuration from MDS. 11 Copyright 2011, Oracle and/or its affiliates. All rights

12 SOA 11g HA Architecture 2 Levels of Clustering WLS Cluster Defined in config.xml Can be viewed from WLS Console and EM SOA Cluster Coherence based cluster Defined as system parameters in startup scripts Can be validated through deployment / configuration changes. 12 Copyright 2011, Oracle and/or its affiliates. All rights

13 Tools for Configuring SOA Applications SOA container configuration done through EM console / WLST Endpoint properties modified during deployment through a Configuration Plan Created Adapter properties WLS Console / WLST for managing WLS configuration Domain creation, adding / changing resources within a domain. 13 Copyright 2011, Oracle and/or its affiliates. All rights

14 Test to Production (T2P) Utility Minimize Manual Steps Primary focus is green field movement. No virtualization required. What is moved? Installed binaries and patches Configuration and metadata Security config e.g. OPSS policies Supports MDS meta-data, JCA connection factories MovePlan Rewiring external references to match target Data-sources, endpoint addresses 14 Copyright 2011, Oracle and/or its affiliates. All rights

15 Program Agenda Overview Provisioning and Configuration Management Managing a System in Production Transactions and Fault Handling Troubleshooting Q&A 15 Copyright 2011, Oracle and/or its affiliates. All rights

16 Production Readiness The Readiness Checklist Sign off on Qualifying Pre-Production Tests Clearly defined and documented processes Trained personnel lined up for Go-live and Post-Production Availability of a Production Replica Environment Identify SLAs and Key Transactions 16 Copyright 2011, Oracle and/or its affiliates. All rights

17 Tools for Middleware Management EM FMW Control EM Grid Control 11g WLS Admin Console JR Mission Control For Farm with a single WLS Domain. Composite lifecycle management, Instance Tracking, Performance Metrics, Fault and Message Recovery Extends FMW Control for multiple farms. SOA/WLS Management Pack. Historical data, Alerts and Notifications Monitor and manage the WLS domain and Managed Servers. Configure JDBC, JMS, JTA, Security, Deploy Applications etc. JVM monitoring and management. Flight recorder for troubleshooting and diagnosis. WLDF WLST BAM Generate and Analyze diagnostic data from WLS instances and your applications. Watches and Notifications. Integrate with JFR for troubleshooting. Create/manage/monitor WLS Domains via Jython based WLI. WLS Admin Console/Configuration Wizard on a command line. Real-Time Business dashboards. Correlate Business Indicators with System Metrics by integrating with EM GC 11g 17 Copyright 2011, Oracle and/or its affiliates. All rights

18 EM FMW Control SOA Composite Lifecycle Management Transaction Search and Drill Down Fault and Message Recovery Service Engine Configuration BPEL External Service Human Workflow Routing Business Rules Drill-down into component s Live In-flight DMS Performance Metrics System MBean Browser for advanced configuration and metrics Use FMW Control for searching and tracking your composite instances 18 Copyright 2011, Oracle and/or its affiliates. All rights

19 FMW Control - Key Performance Tuning Tasks Configure SOA-Infra Properties Audit/Instance Tracking(CPU,DB), Payload Validation (CPU) Configuring Service Engine threads BPEL - Threads(Concurrency), Audit (DB), Validation (CPU) Mediator - Parallel/Resequencer Threads (Concurrency), Audit (DB), Sleep Intervals/Metrics Level (CPU) Configure Adapter Properties MaxRaise/TxnSize(DB), Number of Threads (Concurrency) All this Accessible as MBeans - Update via WLST 19 Copyright 2011, Oracle and/or its affiliates. All rights

20 EM Grid Control - SOA Management Pack EE Central management of multiple SOA domains Historical data and trend analysis of SOA metrics Configure thresholds with alerts Integrated with SNMP traps Scale monitoring and alert configuration using templates. Correlated view between system metrics and SOA composite instances. Extensible through shell / SQL scripts. Monitor custom rroups [apps, DBs] in one place. Use GC11g to track your composite execution flows,monitor your application and system metrics, alert Ops or perform corrective action on critical issues. Detected SOA component name Link to correlated JVM Diagnostics 20 Copyright 2011, Oracle and/or its affiliates. All rights

21 JRockit Mission Control Negligible performance overhead. Comprised of three toolkits. Management Console - for managing and monitoring one or more JVMs. Runtime Analyzer Application and VM Profiler. Record and Playback for Root-cause analysis. Analyze patterns around GC, CPU, Latency, Locks etc Memory Leak Detector Integrates with WLDF for analysis of diagnostic dumps Use JRMC to root-cause performance issues by analyzing memory, CPU, thread contention, locks etc via Flight Recordings. 21 Copyright 2011, Oracle and/or its affiliates. All rights

22 WLST WLST.Jython based CLI Create, monitor and manage WLS Domains Runs in Interactive,Batch or Embedded Mode Online Offline Server Lifecycle DomainMBean Composite Deployment and Lifecycle DomainRunti membean MBean access and Updates ServerRuntim embean WLDF Integration to export Diagnostic Data Custom(For SOA) Create Domain Templates/D omains from Templates, Extend Domains etc Command line automation. Hierarchical navigation via domain and server Mbeans: retrieve and set domain metrics and properties start/stop servers etc 22 Copyright 2011, Oracle and/or its affiliates. All rights

23 Purging Via EM Console [For Targeted Deletion], or via Purge Scripts and/or dropping Partitions[For Bulk] Scripts located at RCU_HOME/rcu/integration/soainfra/sql/soa_purge Looping Useful for small installations [~100k rows] Loops and runs until max_runtime loop parameter is exceeded. Not resource intensive, but slow performance. All purgeable instances are deleted. Parallel Useful for medium [ ~500 GB total to ~10 GB daily] to large installations Resource intensive hence run during off hours.dop indicates number of parallel threads. For large installs [>~500 GB total or > ~10 GB daily], recommended to run a hybrid of parallel purging + dropping partitions, using optimal thread counts. Partitioning Dropping partitions is the fastest way to purge. Time-Range Partitioning Done at the component level. Decide time-range based on average and max composite lifetime. Run this in conjunction with parallel purging for a mixed load. 23 Copyright 2011, Oracle and/or its affiliates. All rights

24 Program Agenda Overview Provisioning and Configuration Management Managing a System in Production Transactions and Fault Handling Troubleshooting Q&A 24 Copyright 2011, Oracle and/or its affiliates. All rights

25 Fault Handling Key Factors Handled Faults Handled programmatically Component / Policy Can specify manual intervention through EM Principal transaction not rolled back Un-handled Faults Not handled programmatically Typically leads to transaction rollback Messages can be re-tried when they are retrievable JMS queue, DB table, Dehydration storeetc. 25 Copyright 2011, Oracle and/or its affiliates. All rights

26 BPEL Service Engine Threads and Persistence Async Process Sync Process Msg persisted (DB) Receive Receive De/hydration (1 st ) process hydration sync process invoke executed by caller thread, e.g. servlet Receive (2 nd ) callback msg persisted (DB), triggers dehydration Reply async process invoke executed by BPEL engine invoker thread After hydration (e.g mid-process receive activity for async callback) process executed by BPEL engine engine thread 26 Copyright 2011, Oracle and/or its affiliates. All rights

27 BPEL Service Engine Faults and Transactions Async Sync Process Process msg persisted (DB) De/hydration (1 st ) process hydration Receive Receive (2 nd ) callback msg persisted (DB), triggers dehydration Receive exception or system crash returns exception (or timeout) to client exception or system crash rolls message back into DB, available to BPEL SE for manual (invoke) recovery. exception or system crash rolls message back into DB, available to BPEL SE for manual (callback) recovery. * A 3 rd type of recovery is Activity recovery. See next slide. composite running but BPEL instance faulted; new instance initiated on recovery. composite and BPEL instance running 27 Copyright 2011, Oracle and/or its affiliates. All rights

28 BPEL Service Engine Faults and Transactions Receive <Activity> Other types of activities can also trigger process dehydration Wait bpelx:exec checkpoint(); onmessage exception or system crash rolls message back into DB, available to BPEL SE for manual (activity) recovery 28 Copyright 2011, Oracle and/or its affiliates. All rights

29 Mediator Service Engine Sequential vs. Parallel Routing (3) separate worker threads process each message M M sequential (aka sync) routing rules are executed by caller thread (1) parallel (aka async) routing rules persist messages into an internal DB table (executed by caller thread) (2) A single locker thread retrieves message batch from DB table into memory Tune the locker thread batch size and # of worker threads for parallel routing rules 29 Copyright 2011, Oracle and/or its affiliates. All rights

30 Mediator Service Engine Sequential vs. Parallel Routing (3) separate worker thread processes each message M M sequential (aka sync) routing rules are executed by caller thread exception or system crash returns exception (or timeout) to caller (1) parallel (aka async) routing rules persist messages into an internal DB table (executed by caller thread) Recover Mediator parallel instances via EM console Faults page for soa-infra, composite or Mediator SE (2) A single locker thread retrieves message batch from DB table into memory Exceptions during processing will cause message to roll back into DB table, available for manual instance recovery 30 Copyright 2011, Oracle and/or its affiliates. All rights

31 Programmatic Fault Handling Fault Policies Fault Policies Fault handling rules centralized in policy files (XML files) Associated with specific a Reference, Component or Composite Can define a conditions chain of recovery actions Recovery Actions Retry Human Intervention recover from EM Abort process Replay scope Re-throw fault Java action custom Java code called 31 Copyright 2011, Oracle and/or its affiliates. All rights

32 Manual / Scripted Fault Handling - Recovery Faults & exceptions Recovery Options Fault policies or BPEL fault handlers for programmatic handling of faults Can delegate to administrator BPEL message recovery for unhandled faults Auto-recovery (recommended) / EM based Recover from BPEL recovery console. Rejected messages from inbound adapters Recovery Action Edit payload Recovery screen log 32 Copyright 2011, Oracle and/or its affiliates. All rights

33 Program Agenda Overview Provisioning and Configuration Management Managing a System in Production Transactions and Fault Handling Troubleshooting Q&A 33 Copyright 2011, Oracle and/or its affiliates. All rights

34 Troubleshooting SOA Oracle Fusion Middleware Diagnostic Framework Diagnostic data Log Files Thread Dumps WLDF kill -3 pid WLST Weblogic Admin Utilility Automatic Diagnostic Repository Command Line Utilities 34 Copyright 2011, Oracle and/or its affiliates. All rights

35 35 Configuring WLDF Watches in WLS Console In-built mapping between Watch code and dumps to create in ADR

36 Troubleshooting SOA Anatomy of a Log File [ T06:50: :00] [AdminServer] [NOTIFICATION] [] 4 [oracle.integration.platform.blocks.deploy] [tid: [ACTIVE].ExecuteThread: '3' for 5 queue: 'weblogic.kernel.default (self-tuning)'] 6 [userid: weblogic] [ecid: 7 3c584fd020bdf1ba:-65df75b4: aa: ,1:23265:2] 8 [APP: soa-infra] 9 [Deployment] Component: SendMail was successfully deployed to service engine:implementation.bpel, for composite:sendmailapp 1. Timestamp 2. Server Name 3. Message Type 4. Component ID 5. Thread ID 9. Message Data 6. User ID 7. Execution Context ID 8. Supplemental Attribute 36 Copyright 2011, Oracle and/or its affiliates. All rights

37 Troubleshooting SOA Editing Log Configuration 37 Copyright 2011, Oracle and/or its affiliates. All rights

38 Troubleshooting SOA Setting Logging Levels Logging Levels Saved Loggers Active Loggers Logger for SOA Adapters Logger for BPEL 38 Copyright 2011, Oracle and/or its affiliates. All rights

39 Troubleshooting SOA Viewing Log Files in EM Message Pane 39 Copyright 2011, Oracle and/or its affiliates. All rights

40 Common Problems and Solutions Resolving common startup problems with SOA server Problem: ClassNotFound exceptions when SOA server is started by a Node Manager Solution: Add property StartScriptEnabled=true to nodemanager.properties. Problem: SOA Server (soa_infra) does not start Solutions: Check port conflicts Make sure MDS schema is reachable Check coherence configuration in clustered setup. Make sure that you re using command line instead of active cache configuration 40 Copyright 2011, Oracle and/or its affiliates. All rights

41 Common Problems and Solutions Resolving too many open files Problem: Too many open files during compilation or runtime. Solution: /etc/security/limits.conf to increase the value of both file descriptors to at least Problem: Transaction or connection timeouts Solution: Increase JTA timeout in weblogic console 41 Copyright 2011, Oracle and/or its affiliates. All rights

42 Common Problems and Solutions Common application-related Problems Problem: Changes to connection factories JNDI (Configuration -> Outbound Connection Pool) for SOA adapters do not take effect. Click Update after selecting the adapter Click Save after the connection factory properties have been updated Hit Enter Key after changing 42 Copyright 2011, Oracle and/or its affiliates. All rights

43 Common Problems and Solutions Common adapter-related Problems Problem: Mis-configured Queues/Datasource in JMSAdapter and DBAdapter. Solution: Make sure that the runtime resources such as JMS connection factories, JMS Queues, DB datasources are configured properly in the WLS console Problem: Duplicate message processing for non-transactional adapters (File/Ftp) in clustered setup. Solution: Use eis/hafileadapter or eis/ftp/haftpadapter in jca configuration. Alternatively configure active-passive failover in the composite.xml file by configuring standalone property with true 43 Copyright 2011, Oracle and/or its affiliates. All rights

44 EM Performance Issue Issue: EM dashboard loads slowly / hangs on login. Step 1: Set default time duration for aggregate queries. Step 2: Disable auto-fetch of aggregated metrics. 44 Copyright 2011, Oracle and/or its affiliates. All rights

45 45 Copyright 2011, Oracle and/or its affiliates. All rights Q&A

46 Latin America 2011 December 6 8, 2011 Tokyo 2012 April 4 6, Copyright 2011, Oracle and/or its affiliates. All rights

47 47 Copyright 2011, Oracle and/or its affiliates. All rights

48 48 Copyright 2011, Oracle and/or its affiliates. All rights

49 49 Copyright 2011, Oracle and/or its affiliates. All rights APPENDIX