OBIEE Performance Improvement Tips and Techniques

Transcription

1 OBIEE Performance Improvement Tips and Techniques Vivek Jain, Manager Deloitte

2 Speaker Bio Manager with Deloitte Consulting, Information Management (BI/DW) Skills in OBIEE, OLAP, RTD, Spatial / MapViewer, ADF etc. Speaker at IOUG Collaborate, ODTUG KScope, NYOUG etc. Bachelor s from IIT-India, MBA from WashU

3 Agenda Baseline performance considerations when deploying OBIEE Advanced techniques for improving performance Using Exa line of products to improve performance Case Studies Large investment bank Executive Dashboards for daily P&L for previous day trades OBIEE dashboards using Essbase cubes Large investment bank OOTB Spend Analytics Hierarchies from PeopleSoft, Custom Hierarchy for Global Supplier Spend Aggregate tables, database hints

4 Agenda Case Studies Premier National Retailer Custom OBIEE Dashboards Actual and Budget reports for Year Ago comparison Large electric utility Internal management reporting actual vs budget vs forecasts Multiple Hierarchies, Type 2 OBIEE reports using Oracle OLAP Change in Hierarchies and Business Processes to achieve performance goals

5 Objectives Develop thumb rules to achieve optimal performance Share experiences of using these thumb rules and techniques in real life How to first think of a business solution for the performance problem

6 Baseline performance considerations when deploying OBIEE

7 Configuration Settings Functional Area Current Value Suggested Value Description SORT_MEMORY_SIZE 4MB 40 MB Increase max memory to be used for each sort operation SORT_BUFFER_INCREMENT_SIZE 256 KB 512 KB Modify the increment by which the sort memory is increased as more memory is needed VIRTUAL_TABLE_PAGE_SIZE 128 KB 256 KB Increase the size of a memory page for OBI internal processing. SERVER_THREAD_RANGE Increase server thread range DB_GATEWAY_THREAD_RANGE Increase database gateway thread range Default settings are good for development environments These changes show performance improvement when server is under load of many user sessions

8 Database Settings Query Analyzer and Profiles for CBO Take advantage of DB / BI Server partitioning Star Transformation and bitmap index Provide a direct and intuitive mapping between the business entities being analyzed by end users and the schema design. Provide highly optimized performance for typical star queries. A star join is a primary key to foreign key join of the dimension tables to a fact table. Understand the restriction and usage for star transformation

9 Advanced techniques for improving performance

10 Advanced Techniques DB Aggregate Tables Materialized Views DB Parameters and MV refresh options, helps in query rewrite BI Aggregate Tables through Aggregate Persistence Cube based Aggregates OLAP Essbase

11 Multidimensional, OLAP Analysis Why Do We Need It? ROLAP (Relational OLAP) How much profit did each regional division make? MOLAP (Multidimensional OLAP) For calendar quarters this year, show the percentage change in sales for Electronic products sold to customers who live in the USA compared to the same quarters last year. Slice and Dice and refine view of data to uncover trends Difficult to express with relational SQL Simple to describe multi-dimensionally (MDX or DML)

12 Multidimensional, OLAP Analysis Why Do We Need It? Traditional BI: ROLAP (Relational OLAP) Power Highly scalable Level based hierarchies Drill, slice, dice Timely Attribute-based analysis Traditional BI: MOLAP (Multidimensional OLAP) Power Flexible, member-based analytics Parent-child & level-based Expand/collapse outline Fast What-if, plan, consolidate, etc. Pain Parent-child force fit Pain Transaction detail and scale Column-based analytics only Limited attributes

13 OBIEE Aggregation Options Aggregate Tables Oracle OLAP Oracle Essbase Traditional way to improve summary performance in an Oracle Data Warehouse Summary is typically defined at a particular grain Cost based optimizer can automatically rewrite queries to materialized views as appropriate OBIEE can be configured to query aggregate table for the appropriate level Potentially many MVs/aggregate tables to manage Aggregate tables for each combination of dimensions/levels Users expect excellent query response across any summary Practical limitations on number of MVs/aggregate tables one can manage effectively New data source supported for OBIEE 11g Single cube can contain thousands of summary combinations (equivalent to thousands of MVs/aggregate tables) Transparent integration with SQL ideal for relational query tools such as OBIEE OLAP cubes embedded inside Oracle database no separate server needed Full support for text measures ideal for capturing comments for adjustments, variances, etc. Slightly less emphasis on business/finance users, rather more DBA friendly easier to find SQL resources New data source supported for OBIEE 11g Single cube can contain thousands of summary combinations (equivalent to thousands of MVs/aggregate tables) Integrated with OBIEE via MDX Needs standalone Essbase server Does not support write-back for text measures Like Oracle OLAP, provides full OLAP capabilities drill, pivot, select, compare, rank, sort, filter, group, stack, calculate, write-back, excel add-in

14 Spectrum of BI Data Sources Add Diagram

15 Exa Line of Products

16 Heading Add Slides on Exalytics, Exadata and Exalogic Exalytics vs OBIEE on Exalogic with Exadata

17 Case Studies

18 Heading Add Slides for case studies

19 Case Study OBIEE with Essbase 1. OBIEE/ Essbase Integration 2. Essbase a. OBIEE vs. native Excel- Addin retrieval i. Captured and executed OBIEE generated MDX ii. Excel- Addin retrieval of same data is much faster ( 4. 5 vs. 0. 2) b. MDX in itself is quite simple c. MDX is quite sensitive to underlying Essbase design and on the fly calculations a. b. Tuning changes i. Outline Structure Dense/ Sparse settings, Dimensions order ii. Configuration parameters iii. Partitioning configuration Design Changes i. Stored vs. on the fly calculations ii. Alternate calculation methods Transaction Profile Analysis a. Verified application information flow via IP packet capture agents b. Verified time spent on different servers for each dashboard retrieval c. No network delays for local users in NY environment d. Essbase server processing contributed single largest delay on server side e. Many small request/ responses observed on client side that could exasperate response time in higher latency networks i. Communication with file server ( nnyc12p30003) ii. Adobe Flash resource calls over the SMB protocol a.

20 Case Study OBIEE with Oracle OLAP Improvement in retrieval time 4 minutes to 2 seconds All data exists in one cube vs. data in many aggregate tables Much easier to troubleshoot data issues if all data is in one place OLAP model introduces many checks to maintain hierarchical integrity of data Forces optimal use of hierarchies, which further simplifies data model and data analysis Simplified data model and RPD Simplified ETL With only one cube, formulas need to be defined only in one place Simple formula expressions Much easier to integrate a simple data model with other data sources using conformed dimensions Only one cube needs to be loaded in place of multiple aggregate tables 1 hour for full load for 1 scenario, scenarios can be loaded in parallel Out of the box incremental load functionality OLAP/OBIEE Integration Pleasantly surprised to see how easy it was to replicate all existing functionality via OLAP cubes BI Server is good at generating SQL (even cube aware SQL) Attribute reporting / Complex calculations in RPD using attributes 20

21 Case Study Change in Hierarchies and Business Process O&M Filter was applied on all O&M reports Cost CAT L4 is equal to / is in MTCEXP; OPEXP RT L4 is not equal to / is not in DA; TT; TB; TA; PT; PI; PG; PB; DC; PA; FI; IF; FB Business Segment Tree - Corp Code L5 is not equal to / is not in JER; MAS RMC Code L7 is not equal to / is not in INC; REV There were 121 values for RT L4, 38 values for Corp Code L5, and 660 values for RMC Code L7. The definition of this filter at lowest level of these 3 dimensions resulted in retrieval of 10,000 intersections for each intersection of scenarios and other dimensions. 21

22 Case Study Change in Hierarchies and Business Process Better way was to use the following condition Cost CAT L4 is equal to / is in MTCEXP; NONOMEXP Adding a new value for Cost Category, say NONOMEXP, and updating the fact table with the following statement: Update TDW_FINL_BFA_FACT set cost_category = NONOMEXP where resource_type not in ( DA, TT, TB, TA, PT, PI, PG, PB, DC, PA, FI, IF, FB ) and corp_code not in ( JER, MAS ) and rmc_code not in ( INC, REV ) and cost_category in ( MTCEXP, OPEXP ) This update statement would allocate all amounts associated with respective dimension values to a new cost category NONOMEXP. This would eliminate the need to apply detail filters for O&M on dimensions other than Cost Category.. 22

23 Case Study Change in Hierarchies and Business Process Even better way was to use OLAP and change the hierarchies For dimension RMC added subtotal below TOP_RMC which excluded values INC and REV For Corp Code added subtotal below TOP_CorpCode which excluded values JER and MAS For RT added subtotal below TOP_RT which excluded values DA; TT; TB; TA; PT; PI; PG; PB; DC; PA; FI; IF; FB New filter condition: Cost CAT L4 is equal to / is in MTCEXP; OPEXP RT L2 is equal to / is in RT_OM Business Segment Tree - Corp Code L2 is equal to / is in CORP_OM RMC Code L2 is equal to / is in RMC_OM The SQL issued by OBIEE for this filter condition would retrieve only 2 intersections from the cube for each intersection of scenarios and other dimensions. 23

24 Case Study Year Ago analysis with custom ETL code Add Bullet points for logic of custom ETL code to avoid use of OBIEE AGO function Almost no impact to data load times Huge performance gain in OBIEE reports Add metrics 24

25 Case Study Aggregate table with OOTB Spend Analytics RPD Add Content 1 aggregate table for month level summary Parallel hint Connection Scripts (Connection pool settings) Alter session force parallel query parallel 4 25

26 Key Takeaways Need to add content

27

28