<Insert Picture Here> Implementing Efficient Essbase ASO Application

Size: px

Start display at page:

Download "<Insert Picture Here> Implementing Efficient Essbase ASO Application"

Lynn Hensley
6 years ago
Views:

2 <Insert Picture Here> Implementing Efficient Essbase ASO Application Buland Chowdhury & Steve Liebermensch Senior Technical Director

3 Agenda Basic Design Dimension settings and usage Partitioning Formulas MDX Optimization Tuning Calculation Aggregation Materialization Data Load Others

4 The following is intended to outline our general guideline. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.

5 Basic Design An Essbase model needs to have a focus area or needs to be subject oriented. For example if you are creating a customer analysis application stay focused on elements that are relevant to customer analysis. Don t introduce additional elements in to the model just because ASO allows it. Lack of focus or subject orientation can lead to inter dimensional irrelevance. Always avoid it. Inter dimensional irrelevance occurs when many members of a dimension are irrelevant across other dimensions

6 Basic Design Don t design an ASO model as if it is a relational database. ASO s capability has increased significantly but it is still an OLAP application not a Relational Database.

7 Basic Design Theoretically there is no limit on the number of dimensions that an ASO model can have. That should not be an excuse to allow dimension into the model. Don t introduce a dimension just in case some one needs it. Focus should be relevant to subject area and frequency of use. Utilize drill through functionality where elements are not directly tied to subject area but relevant for analysis or less frequently used dimensions.

8 Dimension settings and usage Compression ASO applications can be very large. For this reason, compression becomes very important. Following are the things that can be done to improve compression Make sure one dimension is selected as a compression dimension Select a dimension with a higher probability of data intersection with a lower number of level 0 members If possible change the member ordering so that a member with higher data intersection comes first Round number to 2 decimal points. If data value have more than 2 decimal point values are save as is (no compression) Ideal compression is achieved if the stored leaf level member count is evenly divisible by 16

9 Dimension Settings and Usage Accounts Dynamic dimension Allows for non additive unary operators Required for time balance functionality Expense flags are accomplished through UDAs and member formulae In the case of large GL type accounts dimensions should consider storing the dimension

10 Dimension Settings and Usage Accounts Must be a dynamic dimension due to - unary operator on COGS and Total Expenses

11 Dimension Settings and Usage Accounts Simplest hierarchy and allows for maximum stored aggregates. Leverages load rule setting of sign flip on UDA. Requires the user to accept different signage of values.

12 Dimension Settings and Usage Accounts Preserves original signage and allows for some aggregates to be stored

13 Dimension Settings and Usage Time Can make a good candidate for compression dimension provided no time balance requirements Must be stored for time balance feature Use multi-hierarchy dimension if formulae are necessary To-date formulae Prior to best if performed in Time dimension and forward, most functional to use analytic dimension

14 Dimension Settings and Usage Time Use analytic dimension if you need perform time calculations like period to date sums, period over period comparisons Use AGGREGATE function if your period to date calculations need to take Time Balance tags into account ORACLE CONFIDENTIAL

15 Dimension Settings and Usage Time You can write the AGGREGATE function as follows. It some time improves the retrieval performance YTD: AGGREGATE(PeriodsToDate([Time].Generations(2), [Time].CurrentMember), [Base]) QTD: AGGREGATE(PeriodsToDate([Time].Generations(3), [Time].CurrentMember), [Base]) Where [Base] is the stored member in Analytic dimension called TimeView dimension, where you load data at

16 Partitioning Execution across multiple partitions is performed serially Performance will be impacted by volume of data being transferred across the partition Query tracking can and should be done against the source model via the partition BSO as a source should be used only with forethought Formulae on the target model should only be written for members where the calculation crosses partitions

17 Partitioning Target tuning Perform all calculations possible on the source(s)

18 Formulas Simplify complex formulas using intermediate formulas Use shortest possible MDX idioms/expressions Progressively eliminate cells needing to be processed for a formula Avoid CURRENTMEMBER references wherever possible Use specific MDX functions/expression patterns that are internally optimized Use built-in calculations (Time Balance, Ranking calculations) ORACLE CONFIDENTIAL

19 MDX Optimization Meta-data vs data Performance difference between the following statements: IIf(Scenario.CurrentMember = Actual,.) Or IIf(Is(Scenario.CurrentMember,Actual),.)

20 Optimized MDX Functions Use the following functions instead of using PARENT, CHILD, LEAD and LAG to navigate a hierarchy OpeningPeriod ClosingPeriod ParallelPeriod LastPeriods PeriodsToDate FirstChild LastChild FirstSibling LastSibling MemberRange RelMemberRange

21 MDX Optimization Goal is to calculate a product s 3 month average Transactions contribution % to [All Merchandise] 3 month average:

22 MDX Optimization Round( ( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions]) ) / (([Time].CurrentMember,Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(2),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) ) ),4)

23 MDX Optimization Round( ( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions]) Remove unnecessary ) / functions. Formatting (([Time].CurrentMember,Ancestor should (Products.CurrentMember, left to the Products.Generations(2)),[Transactions]) reporting tool. + ([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(2),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) ) ),4)

24 MDX Optimization ( ( ([Time].CurrentMember,[Products].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(1),[Products].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(2),[Products].CurrentMember,[Transactions]) ) / (([Time].CurrentMember,Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(2),Ancestor Avoid (Products.CurrentMember, unnecessary references Products.Generations(2)),[Transactions]) ) to ) CurrentMember. It is assumed without explicitly using a function

25 MDX Optimization ( ( ([Time].CurrentMember,[Transactions]) + ([Time].CurrentMember.Lag(1),[Transactions]) + ([Time].CurrentMember.Lag(2), [Transactions]) ) / (([Time].CurrentMember,Ancestor (Products.CurrentMember, Products.Generations(2)),[Transactions]) + ([Time].CurrentMember.Lag(1),Ancestor (Products.CurrentMember, Simplify formulae, no need to Products.Generations(2)),[Transactions]) use 5 functions + where 1 will ([Time].CurrentMember.Lag(2),Ancestor do. 3 CurrentMember (Products.CurrentMember, functions and 2 Lag Products.Generations(2)),[Transactions]) ) ) functions

26 MDX Optimization Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3), (Ancestor (Products.CurrentMember, Products.Generations(2)), Transactions)) Don t use a function where direct referencing can be performed. In this model Gen 2 of the Products dimension will always be the same member

27 MDX Optimization Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3),([All Merchandise], Transactions))

28 MDX Optimization Only perform calculation when data to support the math exists Case When Not IsEmpty(Sum(LastPeriods(3),Transactions)) Then Sum(LastPeriods(3),Transactions)/ Sum(LastPeriods(3), ([All Merchandise],Transactions)) End

29 MDX Optimization Additionally you should break up the complex formula using an additional member: Create a new member [Last 3 Month Transactions] Assign the formula: NonEmpty directive, Sum(LastPeriods(3),Transactions) feature Then the new % Contribution formula becomes: NonEmptyMember [Last 3 Month Transactions] [Last 3 Month Transactions] / ([Last 3 Month Transactions],[All Merchandise])

30 Tuning Calculation Using NON EMPTY Behavior ASO supports association of NON EMPTY behavior properties with formulas in 11.1.x Non empty behavior takes advantage of sparse nature of input data to reduce amount of dynamic calculations Non empty behavior can be defined by adding the following directives before a formula NONEMPTYMEMBER <memberlist> NONEMPTYTUPLE <tuple> ORACLE CONFIDENTIAL

31 Tuning Calculation Using NON EMPTY Behavior The measure [Units] is a loaded value The measure [Avg. Units/Transaction] is calculated using formula [Avg. Units/Transaction] = [Units]/[Transactions] The value of [Avg. Units/Transaction] is empty if value of [Units] is empty ORACLE CONFIDENTIAL

32 Tuning Calculation Using NON EMPTY Behavior The calculation of [Avg. Units/Transaction] can be skipped if value of [Units] is empty We can add non empty behavior for [Avg. Units/Transaction] by rewriting the formula as [Avg. Units/Transaction] = NONEMPTYMEMBER [Units] [Units]/[Transactions] The saving obtained from defining non empty behaviors can be very significant. Savings increase with the complexity of the formula and sparseness of the data. ORACLE CONFIDENTIAL

33 Examples of NON EMPTY behavior [AvgProdTranscation] is empty if [Transactions] for current Product is empty [Measures].[AvgProdTransactions] = NONEMPTYMEMBER [Transactions] AVG(Product.CurrentMember.Children, [Transactions]) [YTDSales] is empty if Sales for entire [Year] is empty [Measures].[YTDSales] = NONEMPTYTUPLE (Year,Sales) Sum(PeriodsToDate(Time.Years, Time.CurrentMember), Sales) ORACLE CONFIDENTIAL

34 Guidelines for Assigning NON EMPTY Behavior If you can identify a member such that a formula is empty if the value of that member is empty use NONEMPTYMEMBER directive with that member The members in NONEMPTYMEMBER should be from the same dimension as the formula member to which you apply this directive. Use multiple members in NONEMPTYMEMBER if one member is not sufficient Use NONEMPTYTUPLE if you need members from multiple dimensions to indicate non empty behavior ORACLE CONFIDENTIAL

35 Use LEAVES and NONEMPTYSUBSET functions For large dimensions (over 100,000 members) use LEAVES function instead of using DESCENDANTS to get leaf level members. LEAVES returns only leaf level members that have non-empty data Leaves(Products) Use NONEMPTYSUBSET to reduce size of your query NonEmptySubset(Crossjoin(Leaves(Customers), Leaves(Products), Sales) ORACLE CONFIDENTIAL

36 Dynamic Calculations Log Messages Application log has some informative messages related to dynamic calculations that can be helpful in tuning The following message indicates an error in a formula. Fix the error message or drop these members from outline if they are not required Error encountered when loading member [<MemberName>]'s calc string [<FormulaExpression>], ignored The following message indicates invalid solve order assignment. Assign correct solve orders for members indicated in the message Invalid formula for member [<A>] (reason: depends on a member [<B>] with higher solveorder ) will be ignored during execution ORACLE CONFIDENTIAL

37 Dynamic Calculations Log Messages The following message indicate that a formula is too complex for Essbase to optimize. Rewrite if possible. Formula for member [% of Total] will be executed in [CELL] mode The following message indicates that a formula is complex and Essbase couldn t detect a non empty behavior for it. The formula for member [MemberName] is Complex. If possible, add a non-empty directive to optimize for sparse data ORACLE CONFIDENTIAL

38 ASO Aggregation Use parallel calculation. Set CALCPARALLEL setting to number of CPU available. Increase ASO cache size higher than the what you have during data load to accommodate parallel calculation. You can use Maxl to accomplish this. Depending on situation a full database aggregation may not be optimal. Optimizing aggregations may require an iterative, fine-tuning process. Query tracking probably the most efficient way to decide what to aggregate, provided queries reflect the users data need properly; However, if you are knowledgeable about user and their query need you can use User-Defined View Selection where you will be able to select top level, bottom level in secondary hierarchy and many others options. Use hierarchy aggregation settings in the outline editor

39 Materialization Query hints On Dimension/Hierarchy

40 Materialization By default Aggregate Storage Option is parallel threaded with two threads, can use up to 8 When loading incremental data, aggregate views are also updated Materialization scripts can be saved and reused to eliminate regeneration of query tracked results The database can be available during materialization without data consistency issues

41 Materialization Query hints Dimensional intersections

42 ASO Query Aggregation and query are related items. Most of the items that we have discussed in previous slides will improve query performance A hint can be used to optimize heavily queried level combinations. Avoid creating too many hints, you will lose effectiveness.

43 Data Load Behavior Within a buffer duplicate records are additive irrelevant of load rule settings When committing the buffer it adheres to data load rule setting related to overwrite, add to, or subtract from Single threaded, can use DLTHREADS cfg settings but thorough testing should be performed Materialized views are updated as part of the data load process during an incremental update

44 Data Load Slice loading Is the primary feature enabling Excel based data submission and trickle feeding functionality Creates subcubes alongside the primary slice of the database Dynamic aggregations are performed across the necessary slices to provide query results Different materialized views might exist within a slice as compared to the primary slice of the database

45 Data Load Optimization Compression dimension should be across the load file Consider removing aggregate views prior to an incremental update of data Data should be loaded as additive values Multiple buffers can be used to parallel load the database. Requires simultaneous Maxl processes to be executed Ignore Zeros and Missing values whenever possible, maxl buffer setting parameter

46 Data Load Optimization Don t load 0 (zero) values if possible Use ASO cache setting. The number is based on the number of cells that would be loaded. While setting ASO cache try not to exceed 25% of the available physical memory to leave more space for the OS file cache For a Database with 1 billion cells or more, the cache size may be set as high as 512 MB or 1 GB if the available memory permits it Avoid (if possible) Text substitutions, field splits/joins and white space truncation in rule file

47 ASO Others If possible allocate space for default and temp table spaces on different physical disk devices where applicable

<Insert Picture Here> Optimizing ASO

<Insert Picture Here> Optimizing ASO Optimizing ASO Steve Liebermensch Consulting Technical Director The following is intended to outline our general product direction. It is intended for information purposes only, and