Scripting OBIEE Is UDML and XML all you need?

Size: px

Start display at page:

Download "Scripting OBIEE Is UDML and XML all you need?"

Edith Berry
5 years ago
Views:

1 Scripting OBIEE Is UDML and XML all you need? Andreas Nobbmann Consultant Business Intelligence Brighton, May 14th, 2009 Basel Baden Bern Lausanne Zürich Düsseldorf Frankfurt/M. Freiburg i. Br. Hamburg München Stuttgart Wien

2 Why scripting OBIEE? Scripting OBIEE Is UDML and XML all you need? 2

3 Agenda What is UDML? How to use UDML? Good examples for UDML usage XML Data is always part of the game. Good examples for XML usage The last word Scripting OBIEE Is UDML and XML all you need? 3

4 Agenda What is UDML? How to use UDML? Good examples for UDML usage XML Data is always part of the game. Good examples for XML usage The last word Scripting OBIEE Is UDML and XML all you need? 4

5 What is UDML? UDML = Universal Database Markup Language Commonly known as: Scripting and definition language to describe databases or perform certain actions against databases In OBIEE: E.g. DDL creation etc. Undocumented feature in BI Server repository to manage mouseless administrative scripting Oracle recommends for example to merge Oracle BI repositories by using UDML Scripting OBIEE Is UDML and XML all you need? 5

6 What to use it for In general for scripting or describing everything stored in your Oracle BI repository file Connection pools Users / groups, LDAP server definitions, Security settings, privileges Initialization blocks, variables physical, aliases, logical tables, views... Primary / secondary keys, Foreign key, complex joins Dimensions / hierarchies External name settings (DISPLAY NAME CN_.. / CD_..)... Scripting OBIEE Is UDML and XML all you need? 6

7 Agenda What is UDML? How to use UDML? Good examples for UDML usage XML Data is always part of the game. Good examples for XML usage The last word Scripting OBIEE Is UDML and XML all you need? 7

8 How to use UDML? Ad Hoc Simply do copy (CTRL+C) on the object you want to script Paste (CTRL+V) it to your favoured editor, edit script and copy back to repository Scripting OBIEE Is UDML and XML all you need? 8

9 How to use UDML? Ad Hoc Attention: Copy is context sensitive, means place your cursor on the father of the object you want to insert the UDML If new Business Model (BM) do not click on any object and paste into the BM window If new table source Go to BM and paste it into And so on Remark: users and groups cannot be copied this way, they can only be merged into repositories Scripting OBIEE Is UDML and XML all you need? 9

10 How to use UDML? UDML generation Use nqudmlgen.exe generates UDML from your repository - N - Q - S does not generate upgrade IDs (needed for versioning) without security objects only security objects (overwrites Q parameter) -N use if you want to create a repository from the scratch, OBIA creates new object ID s c:\oraclebi\server\bin\nqudmlgen.exe U Administrator P Administrator R c:\oraclebi\server\repository\paint.rpd O paint_udml Scripting OBIEE Is UDML and XML all you need? 10

$exe merges UDML into repository and optionally backs it up - I - B - O -8 input script name optional base repository name output repository name is for UTF-8 c:\oraclebi\server\bin\nqudmlexec.$

11 How to use UDML? merging repositories Use nqudmlexec.exe merges UDML into repository and optionally backs it up - I - B - O -8 input script name optional base repository name output repository name is for UTF-8 c:\oraclebi\server\bin\nqudmlexec.exe U Administrator P Administrator I c:\oraclebi\server\repository\changerepo_2_proddb_udml.txt -B c:\oraclebi\server\repository\sales.rpd O c:\oraclebi\server\repository\sales_new.rpd Remark: always make a backup before working with nqudmlexec. Scripting OBIEE Is UDML and XML all you need? 11

12 How to use UDML? update via data source Use nqudmlcli.exe adding UDML to OBIEE repository via Data Source -D -w -P Data Source name as in odbcad32.exe (ODBC Data Source Administrator) optional passphrase for SSL connections if you leave empty it uses the profile of the client you are connecting from c:\oraclebi\server\bin\nqudmlcli.exe U Administrator P Administrator -D AnalyticsWeb I c:\oraclebi\server\repository\changerepo_2_proddb_udml.txt Remark: sometimes this tool is not running because of lacking privileges. Copy it with all DLLs named NQSUDMLxx.dll into another directory and restart. Scripting OBIEE Is UDML and XML all you need? 12

13 Agenda What is UDML? How to use UDML Good examples for UDML usage XML Data is always part of the game. Good examples for XML usage The last word Scripting OBIEE Is UDML and XML all you need? 13

14 Good examples for UDML usage Parsing your repository Development street Automating import of user groups Update row counts Add icons to your repository To be continued...

Good UDML examples Parsing your repository Download UDML parser jar-ball from http://code.google.com/p/udmlparser/. Copy it to a directory you want Use nqudmlgen.

15 Good UDML examples Parsing your repository Download UDML parser jar-ball from Copy it to a directory you want Use nqudmlgen.exe with options N Q -8 for the repository Fire java jar udmlparser0_4.jar udml=yourudmlfile.txt rpdxml=yourudml_xml.xml against YourUDMLFile.txt. You get a well formed XML, which is of course editable and transformable via Style Sheets etc. Thanks to Daniel Galassi for the parser Scripting OBIEE Is UDML and XML all you need? 15

$ora Export the UDML from your DEV repository with nqudmlgen.exe c:\oraclebi\server\bin\nqudmlgen.exe U Administrator P Administrator R c:\oraclebi\server\repository\sales.$

16 UDML examples Development street Assume we have three environments, where DWHDEV, DWHTST and DWHPRD, are the data sources names defined within TNSNAMES.ora Export the UDML from your DEV repository with nqudmlgen.exe c:\oraclebi\server\bin\nqudmlgen.exe U Administrator P Administrator R c:\oraclebi\server\repository\sales.rpd O sales_dev_udml search for the connection pool (DECLARE CONNECTION POOL) Copy the whole block of the connection pool into two text files Scripting OBIEE Is UDML and XML all you need? 16

17 UDML examples Development street cont. In the first file replace the text within the {} with DWHTST In the second file replace it with DWHPRD Adapt your user settings as required Save them as for example DBConnectInfos_UDML_DWHTST and DBConnectInfos_UDML_DWHPRD Generate the repository sales_tst.rpd by using nqudmlexec c:\oraclebi\server\bin\nqudmlexec.exe U Administrator P Administrator I c:\oraclebi\server\repository\dbconnectinfos_udml_ DWHTST -B c:\oraclebi\server\repository\sales.rpd O c:\oraclebi\server\repository\sales_tst.rpd Thanks to Venkat, John and Mark for their blogs around that! Scripting OBIEE Is UDML and XML all you need? 17

18 UDML examples Development street cont. Another way is to copy this into the files Scripting OBIEE Is UDML and XML all you need? 18

19 UDML examples Automating user group import Get your user groups from your table with a simple SQL Select DECLARE SECURITY ROLE usergroup AS usergroup UPGRADE ID description{data Security Group usergroup } PRIVILEGES (READ); from DWH_APP.usergroups Save the result of your SQL in a text file and update your repository with nqudmlexec c:\oraclebi\server\bin\nqudmlexec.exe U Administrator P Administrator I c:\oraclebi\server\repository\usergroups_udml -B c:\oraclebi\server\repository\sales.rpd O c:\oraclebi\server\repository\sales_usergroups.rpd Thanks to Venkat for his blog around that! Scripting OBIEE Is UDML and XML all you need? 19

20 UDML examples update row counts 2 general steps to do so normally Right click on table and do update row count Or use tools / update all row counts WAITING When using hints in Oracle or an event table for Cache you can get the updated row count from there and copy the results into a UDML file DECLARE TABLE "U".."SYSTEM"."S_ETL_DAY" AS "S_ETL_DAY" UPGRADE ID NO INTERSECTION ROW COUNT LAST UPDATED ' :22:43 PRIVILEGES ( READ); Scripting OBIEE Is UDML and XML all you need?

21 UDML examples making the repository intuitive Assign the dollar sign to all fact tables DECLARE TABLE "Paint"..."Fact" AS "Fact" UPGRADE ID 554 ICON INDEX 102 HAVING Assign the stop sign to the database connections when you work on them for a longer period of time DECLARE DATABASE "Forecast" AS "Forecast" UPGRADE ID 162 ICON INDEX 92 TYPE XMLDirect Other examples: Question mark (Index 90), globe (Index 81), work cone (index 81) etc. Scripting OBIEE Is UDML and XML all you need?

22 Agenda What is UDML? How to use UDML Good examples for UDML usage XML Data is always part of the game. Good examples for XML usage The last word Scripting OBIEE Is UDML and XML all you need? 22

23 XML Is used for everything defined in the presentation catalog Requests Prompts, filters Dashboards, dashboard pages User objects, group objects Privileges Descriptions etc. Scripting OBIEE Is UDML and XML all you need? 23

24 XML 3 ways to insert XML into catalog Advanced tab in request Paste XML and Click Set XML In catalog manager Edit and paste Or directly in the file system under <OracleBIData>\web\catalog\<YourCatalog>\path_to_your_report\YourReport Open with editor Paste XML and save Scripting OBIEE Is UDML and XML all you need? 24

25 Agenda What is UDML? How to use UDML Good examples for UDML usage XML Data is always part of the game. Good examples for XML usage The last word Scripting OBIEE Is UDML and XML all you need? 25

26 Good examples for XML usage Automatically create pivot-table label heading style Format all columns with Percent in name Format all columns depending on their data type Editing XML directly in your editor To be continued... Scripting OBIEE Is UDML and XML all you need? 26

27 XML examples create pivot-table Scripting OBIEE Is UDML and XML all you need? 27

28 XML examples create pivot-table Here the XML source: Scripting OBIEE Is UDML and XML all you need? 28

29 XML examples create pivot-table column formulas Now open a good XML editor (for example XMLSpy) and create a XSLT stylesheet Add namespaces, start at / elements, then match all saw:report elements and copy them to target Copy all saw:report = all attributes With this part of the stylesheet you copy all the column formulas. <saw:column formula=""d2 Market"."M01 Market"" columnid="c0"/>... Scripting OBIEE Is UDML and XML all you need? 29

30 XML examples create pivot-table parameters The next part is related to parameters needed for formatting purpose You see the parameters aggrule halign fontstyle aggregation rule within Pivot Allignment Font Style 3 = # of views in compoundview parameters Scripting OBIEE Is UDML and XML all you need? 30

31 XML examples create pivot-table compoundview Next is the standard compoundview Here also the pivottableview1 is added Scripting OBIEE Is UDML and XML all you need? 31

32 XML examples create pivot-table makepivot next part is creating the Pivot table in XML dynamically Parameters are used with { } Parameter {aggrule} Scripting OBIEE Is UDML and XML all you need? 32

33 XML examples format all Percent columns Let s take a look on the pattern functionalities of XML We define the rule for the percent pattern on template saw:column <xsl:if test="contains(@formula, 'Percentage')"> <xsl:attribute name="percent" select="'true'"/> </xsl:if> pattern Pretty easy, isn t it? Scripting OBIEE Is UDML and XML all you need? 33

34 XML examples format on data type Let s derive the data type from the UDML-XML we parsed from the original sales sample Goal: define all columns with data type double like ,00 Scripting OBIEE Is UDML and XML all you need? 34

35 XML examples format on data type Get presentation column depending on your input column on Xpath level PresentationColumnMappingID Get the logical column from your UDML data by searching the presentation column on Xpath level LogicalColumnCalculation Scripting OBIEE Is UDML and XML all you need? 35

36 XML examples format on data type cont. Get the physical column by searching for the logical column on Xpath level PhysicalColumnID and derive the data type PhysicalColumnDatatype from physical column Scripting OBIEE Is UDML and XML all you need? 36

37 XML examples format on data type The test result looks like that of course it can be further extended by formatting per data type Scripting OBIEE Is UDML and XML all you need? 37

) installed on your PC then it probably has the column mode, you can

38 XML examples editing directly in your editor If you have a good editor (like UltraEdit etc.) installed on your PC then it probably has the column mode, you can put your cursor to a column and edit multiple columns at the same time Scripting OBIEE Is UDML and XML all you need? 38

39 Agenda What is UDML? How to use UDML Good examples for UDML usage XML Data is always part of the game. Good examples for XML usage The last word Scripting OBIEE Is UDML and XML all you need? 39

40 The last word UDML and XML are really very useful for scripting Good news: everything can be automated by scripts Cache Seeding, Backup, Synchronizing styles & skins with OC4J, users & user groups, repository, presentation catalog... UDML&XML provide a good approach for creating repositories and reports automatically from the metadata I am working on it Good news: new tool Content Accelerator Framework (CAF) published within : Venkat blogged already about it content-accelerator-framework-caf-synchronizer-a-reviewpreview-xudml/ My resumee: Scripting with OBIEE is fun! Scripting OBIEE Is UDML and XML all you need? 40

41 Thank you very much!? Basel Baden Bern Lausanne Zürich Düsseldorf Frankfurt/M. Freiburg i. Br. Hamburg München Stuttgart Wien

42 If you have any questions or want the scripts Please feel free to drop me a note or give me a call Andreas.Nobbmann@trivadis.com Trivadis AG Basel Consultant Business Intelligence Scripting OBIEE Is UDML and XML all you need? 42

43 Appendix Error Messages UDML parser Data is always part of the game. Scripting OBIEE Is UDML and XML all you need? 43

44 UDML Parser error messages UDML Parser Messages This category contains message numbers 28xxx Near Near String must be in a single line. Cause. The named string must be on a single line. Response. Remove all the new lines within this string Near Near (@2%s): End of file has been reached. Cause. The singly or doubly quoted string does not have a matching quote. Response. Add single or double quote at the end Near near (@2%s): Syntax Cause. A keyword was misspelled or an unexpected identifier was encountered. Response. Verify the syntax and correct any errors before retrying the action Near (@2%s): Duplicate definition. Cause. The same object was defined more than once. Response. Change the name of one of the objects. Scripting OBIEE Is UDML and XML all you need? 44

45 UDML Parser error messages cont Near Multiple matches found in the current repository. Cause. More than one match was found in the repository. The repository may be corrupted. Response. Repair the repository. Restore it from a back-up, or use the Check Consistency command in the Administration Tool to identify compilation errors. Correct any problems before retrying your actions Near (@2%s): Not defined. Cause. The named symbol is not defined either in the script or the repository. Response. Define the named symbol in the script or create an object in the repository. If necessary, contact Siebel Systems technical support for assistance Near (@2%s): This data type is not supported. Cause. The named data type is not supported. Response. Correct the data type and retry the action. If necessary, contact Siebel Systems technical support for assistance Near (@2%s): This database type is not supported. Cause. The named database type is not supported. Response. Correct the database type and retry the action. If necessary, contact Siebel Systems technical support for assistance. Scripting OBIEE Is UDML and XML all you need? 45

46 UDML Parser error messages cont Near This connection type is not supported. Cause. The named connection type is not supported. Response. Correct the connect type and retry the action. If necessary, contact Siebel Systems technical support for assistance Near It cannot be a parent or container of itself. Cause. A circular definition was found. Response. Correct the appropriate definitions to break the circle. If necessary, contact Siebel Systems technical support for assistance Near (@2%s): This role is not defined. Cause. The required role is missing. Each relationship needs to have two roles. This is an internal error. Response. Contact Siebel Systems technical support for assistance Near (@2%s): This object privilege cannot be used for none set. Cause. This object privilege is wrong. This is an internal error. Response. Contact Siebel Systems technical support for assistance Near (@2%s): This object privilege cannot be used for read set. Cause. This object privilege is wrong. This is an internal error. Response. Contact Siebel Systems technical support for assistance. Scripting OBIEE Is UDML and XML all you need? 46

47 UDML Parser error messages cont Near This connection type is not supported. Cause. The named connection type is not supported. Response. Correct the connect type and retry the action. If necessary, contact Siebel Systems technical support for assistance Near It cannot be a parent or container of itself. Cause. A circular definition was found. Response. Correct the appropriate definitions to break the circle. If necessary, contact Siebel Systems technical support for assistance Near (@2%s): This role is not defined. Cause. The required role is missing. Each relationship needs to have two roles. This is an internal error. Response. Contact Siebel Systems technical support for assistance Near (@2%s): This object privilege cannot be used for none set. Cause. This object privilege is wrong. This is an internal error. Response. Contact Siebel Systems technical support for assistance Near (@2%s): This object privilege cannot be used for read set. Cause. This object privilege is wrong. This is an internal error. Response. Contact Siebel Systems technical support for assistance. Scripting OBIEE Is UDML and XML all you need? 47

48 UDML Parser error messages cont Near This object privilege cannot be used for read_write set. Cause. This object privilege is wrong. This is an internal error. Response. Contact Siebel Systems technical support for assistance Near This object privilege cannot be used for erase set. Cause. This object privilege is wrong. Response. Contact Siebel Systems technical support for assistance Near This object privilege cannot be used for grant set. Cause. This object privilege is wrong. Response. Contact Siebel Systems technical support for assistance Near This object privilege cannot be used for all set. Cause. This object privilege is wrong. Response. Contact Siebel Systems technical support for assistance Near In the metadata expression the following error occurred: Cause. The named expression has one or more errors. This can be caused by spelling errors or objects referenced in the expression that are missing. Response. Ignore and then reconstruct the problem expressions. Scripting OBIEE Is UDML and XML all you need? 48

49 UDML Parser error messages cont Near is already in the repository and it's ID is different from the designated ID. Cause. An object of the same name exists in the repository but with a different ID. The repository may be corrupted. Response. Repair the repository. Restore it from a back-up, or use the Check Consistency command in the Administration Tool to identify compilation errors. Correct any problems before retrying your actions Near (@2%s): is an invalid feature. Cause. The system could not find the named feature. Response. Verify the spelling of the named feature Near (@2%s): is an invalid value for the feature (@3%s) Cause. The specified value is invalid for the named Boolean feature. Response. Valid values include words like YES, NO, ON or OFF. Make the appropriate changes before retrying the action Near (@2%s): is an invalid value repository property name. Cause. The property name is wrong. Response. Use 'CustomPresentationLayer' Near (@2%s): This alias appears more than once. Cause. The alias appeared twice in the repository, either as the name or alias of an object of the same type. Response. Find the duplicates in the repository using the Query tool and make the appropriate changes. Scripting OBIEE Is UDML and XML all you need? 49

50 UDML Parser error messages cont Near This alias is the same as an object in the script. Cause. The alias has the same name as another object of the same type in the script. Response. Correct the script so the alias and the object use different names Near This alias is the same as an object in the repository. Cause. The alias has the same name as another object of the same type in the repository. Response. Correct the repository so the alias and the object use different names Near The Administrator security role does not inherit from any other roles. Cause. The Administrators role inherited from another role. Response. The Administrator security role can only be a root level role Near (@2%s): Invalid DateTime format. Cause. The format of the Datetime string is invalid. Response. Correct the format of the Datetime string and retry the action Invalid level relationship found. Please recreate the level(s) relationships. Cause. The level with the required ID does not exist in the repository. The repository may be corrupted. Response. Repair the repository. Restore it from a back-up, or use the Check Consistency command in the Administration Tool to identify compilation errors. Correct any problems before retrying your actions. Scripting OBIEE Is UDML and XML all you need? 50

51 UDML Parser error messages cont Error: The object does not exist. If it is on-line mode, please check in changes from the File menu. Cause. The object with the required ID does not exist in the repository. The object may not be checked in. Response. If the repository is in on-line mode, use the Check-in function from the File menu to make the object available Error: Cannot open the Cause. Cannot open the named UDML script file. The file may not exist, or may be in use. Response. Verify that the file exists and is not in use by another process. If the problem persists, contact Siebel Systems technical support for assistance. Source: (Siebel 2004) Scripting OBIEE Is UDML and XML all you need? 51

$g facts Factxxxx as Fact) equals the string sets does not update directly, creates a new repository c:\oraclebi\server\bin\equalizerpds.$

52 How to use UDML? equalize repositories Use equalizerpds.exe equalizes 2 repositories into a new one -X -Y -O treats name-wise similar facts as one (e.g facts Factxxxx as Fact) equals the string sets does not update directly, creates a new repository c:\oraclebi\server\bin\equalizerpds.exe A Administrator B Administrator -C c:\oraclebi\server\repository\sales_version1.rpd -D Administrator E Administrator -F c:\oraclebi\server\repository\sales_version2.rpd X -Y Scripting OBIEE Is UDML and XML all you need? 52

Copyright 2010, Oracle. All rights reserved.

Copyright 2010, Oracle. All rights reserved. Agenda Interactive Dashboards Master-Detail Linking New Prompt Controls Pivot Table Interactions Enhanced Dashboard Builder Analysis Editor Overview Working with the analysis editor step by step Praxisteil