OO 10 Database Overview and Maintenance

Transcription

1 OO 10 Database Overview and Maintenance Harar Zafrir, OO DBA June 2015 Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

2 Agenda

3 OO 10 database

4 OO 10 database main characteristics OO 10 has a database-centric design OO uses the database as the system queue, and as a reliable coordinator of distributed actions. OO s cluster nodes are all connected to the same database. Database design is derived from code OO uses Hibernate ORM. Table structure is derived from code objects. Simple, standard, cross-database design Database design is kept simple in order to maintain compatibility with multiple database types and across database releases.

5 RAS central #1 central #2 central N hibernate hibernate Database

6 Differences between OO 9 and 10 A completely new database design 115 tables in 10.x vs. 28 tables in 9.x Support for PostgreSQL was added Flow execution data is saved in the database without truncation Content packs are stored in the database

7 Supported Databases Oracle 11g and 12c ojdbc7 is used as database connector Oracle RAC is supported 12c is supported only in non-cdb/pdb form MS-SQL 2008R2 and 2012 JTDS is used as database connector Always On is supported as legacy client (multi-subnet failover is not supported) MySQL MySQL connector is used (not provided in HP OO s installation) PostgreSQL

8 Key tables in OO 10 Table Name Purpose Comments OO_AUDIT OO_CONTENT_X OO_DEBUGGER_EVENTS OO_DEPLOYED_ENTITIES_X OO_EXECUTION_BOUND_INPUTS OO_EXECUTION_BOUND_OUTPUTS OO_EXECUTION_QUEUES_1 OO_EXECUTION_SUMMARY Keep audit trail Content pack information Studio remote debugging sessions data Deployed flows and actions information Flow level inputs Flow level inputs Queue Executed/executing flow timestamps and main result OO_STEP_LOG_BINDINGS Step-level inputs and outputs Biggest table size and # of rows OO_STEP_LOG_ENDED OO_STEP_LOG_STARTED OO_SYSINFO Ended steps information Started steps information OO release information OO_USERS Internally-configured users Configuration table example

9 Database Structure Comparison Table Name Purpose Somewhat similar in 10.x run Queue, running flows OO_EXECUTION_STATES_N, OO_EXECUTION_QUEUES_1 run_history Flow timestamps and main result OO_EXECUTION_SUMMARY runstep_history Step level main result OO_STEP_LOG_X property_history Step level inputs OO_EXECUTION_BOUND_INPUTS, OO_EXECUTION_BOUND_OUTPUTS log_record Step level outputs OO_STEP_LOG_X flow_metrics Flow result counters

10 Data retention HP OO 10 keeps more data compared to 9, including large flow and step outputs. The data tail (oldest) is not automatically trimmed. Purging old data is important in order to keep database size under control.

11 Key Installation Guidelines

12 Installing OO 10 Database Guide Use it! This document contains vital information: Supported platforms/releases Database sizing and Database server resources Detailed installation instructions Maintenance and data purging guidelines

13 Installing OO 10 (cont.) Provide database admin privileges The resulting database and user / login role are fully compatible with OO s requirements. Admin credentials are not used beyond the installation phase. Pay attention to mandatory requirements There are several mandatory settings per database, on the database level and on the server level.

14 Per-database key installation guidelines MSSQL database level settings Transaction isolation level is crucial. The following must be set ( ON ): ALLOW_SNAPSHOT_ISOLATION READ_COMMITTED_SNAPSHOT Use one of the supported collations: SQL_Latin1_General_CP1_CS_AS Japanese_Unicode_CS_AS Chinese_Simplified_Stroke_Order_100_CS_AS French_100_CS_AS

15 Per-database key installation guidelines (cont.) MySQL server level settings Use InnoDB engine READ-COMMITTED isolation level max_allowed_packet >= 250M PostgreSQL server level settings 'read committed' isolation level autovacuum daemon must be activated

16 Database Maintenance

17 HP OO Database Maintenance Best practices for proper maintenance of HP OO s database: Create a backup plan Regularly perform index and statistics maintenance Index & Statistics Maintenance procedures are available for MSSQL and Oracle Purge unused / irrelevant data Monitor database size and performance

18 Database Size and Purging

19 HP OO Database Size As all flow-level and step-level inputs and outputs are kept in the database, OO s database size increases in accordance with the system throughput, flow complexity, and the amount of data provided to / resulting from the flows and steps. We recommend to have at least 50GB of space allocated for small systems. Enterprise-grade systems are recommended to allocate up to 2TB of database space. Data purging must be used in order to keep the database size below the environment s limit.

20 Purging old data There are two main tools for regularly purging data from the database: Purge Flows: Flow execution data Audit events Studio debugging session data Purge Stored Procedures: Flow execution data

21 General guidelines for Purge Operations Purge data regularly preferably on a daily basis It is better in all respects to purge the data tail in small chunks If possible, schedule purge operations to periods of time in which OO and the database are least active. Use nightly purge for example. Monitor the first purge flow runs and tune for best performance Two most important parameters in any purge operation: Retention period Batch size

22 Batch #4 Batch #3 Batch #2 Amount of Data Purged data Old lastxdays Retention Period New Now

23 Purge Flows ( HP Solutions content pack) Purge Execution Summary Has the biggest impact on database size. Purges flow and step execution data. Purge Debugger events Should be used only in case Studio debugging sessions are commonly used. Purge Audit Records Should have long-enough retention period. Often defined by business restrictions.

24 Purge Stored Procedures Purge stored procedures are provided in HPLN per-database. Installing and scheduling these procedures usually requires DBA s approval and assistance. Purge stored procedures have one main advantage compared to the purge flows, the ability to delete bigger chunks of data at once. Use stored procedures in case you have a high-performance database, and a lot of data to purge every day.

25 Data DB Purge Purge High Water Mark Disk Free Used 28 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

26 Database size behavior Database Filesystem Footprint Following Purge Free space Reclaim MSSQL High Water Mark Free space increases Not recommended Oracle Tablespace Free space increases MySQL Delayed Size on disk decreases (delayed) PostgreSQL Delayed (vacuum) Size on disk decreases (vacuum, delayed) vacuum full

27 Purge Demo

28 Q&A

29 Thank you!