PPDM Meta Model Practical Applications

Size: px

Start display at page:

Download "PPDM Meta Model Practical Applications"

Karen Bailey
5 years ago
Views:

1 PPDM Meta Model Practical Applications

2 The Situation You have inherited/starting a PPDM database There are data feeds and documentation available You have time and resources to analyze it The vision: It will become the Enterprise Data Repository

3 The Steps Understand what you have Data model structure Data content Review data feeds Rules available; both data & business Repeatable processes Progressive Improvement

4 What is the Meta Model The management of data, and having the tools to manage and execute the various Business Processes in this Business Module offers great opportunity and efficiencies to stakeholders in data management. > 50 core tables Major themes around audit, rdbms, groups, metrics, rules, code versions

5 Data Model Structure PPDM_SYSTEM PPDM_TABLE PPDM_COLUMN PPDM_CONSTRAINT PPDM_INDEX PPDM_GROUP PPDM_GROUP_OBJECT PPDM_SYSTEM_MAP

6 Data Model Structure, continued These tables are the foundation of the meta model If you load the meta model from PPDM, it contains versions: From 3.2 to 3.8 Focus on the implemented model The primary key SYSTEM_ID is used thoroughout the meta model Documentation is available in the database PPDM_TABLE.TABLE_COMMENT PPDM_COLUMN.COLUMN_COMMENT Use these tables for other systems that are being integrated (based upon SYSTEM_ID)

7 Data Model Structure, continued (Oracle) SQL to see what you have: Table Differences (select 'Missing PPDM 3.8 Tables' Table_Review, table_name from ppdm_table where system_id = 'PPDM38 minus select 'Missing PPDM 3.8 Tables',table_name from user_tables) union (select 'Non PPDM 3.8 Tables',table_name from user_tables minus select 'Non PPDM 3.8 Tables',table_name from ppdm_table where system_id = 'PPDM38') Column Differences (select 'Missing PPDM 3.8 Columns' Column_Review,table_name,column_name,data_type from ppdm_column where system_id = 'PPDM38 minus select 'Missing PPDM 3.8 Columns',table_name,column_name,data_type from user_tab_columns) union (select 'Non PPDM 3.8 Columns',table_name,column_name,data_type from user_tab_columns minus select 'Non PPDM 3.8 Columns',table_name,column_name,data_type from ppdm_column where system_id = 'PPDM38')

8 Data Model Structure, continued (Oracle) SQL to see what you have: Index Differences (select 'Missing PPDM 3.8 Indexes' Index_Review,'INDEX' object_type,table_name,index_name,index_category from ppdm_index where system_id = 'PPDM38 minus select 'Missing PPDM 3.8 Indexes','INDEX' object_type,table_name,index_name,index_type from user_indexes) union (select 'Non PPDM 3.8 Indexes','INDEX' object_type,table_name,index_name,index_type from user_indexes minus select 'Non PPDM 3.8 Indexes','INDEX' object_type,table_name,index_name,index_category from ppdm_index where system_id = 'PPDM38') Constraint Differences (select 'Missing PPDM 3.8 Constraints' Constraint_Review,table_name,constraint_name,constraint_type from ppdm_constraint where system_id = 'PPDM38 minus select 'Missing PPDM 3.8 Constraints',table_name,constraint_name,decode(constraint_type,'R','FOREIGN','C','CHECK','P','PRIMARY') from user_constraints) union (select 'Non PPDM 3.8 Constraints',table_name,constraint_name,decode(constraint_type,'R','FOREIGN','C','CHECK','P','PRIMARY') from user_constraints minus select 'Non PPDM 3.8 Constraints',table_name,constraint_name,constraint_type from ppdm_constraint where system_id = 'PPDM38')

9 Data Content What tables are populated? Declare CURSOR get_tables IS SELECT table_name FROM ppdm_table where system_id = 'PPDM38'; gt get_tables%rowtype; rc integer; begin FOR gt IN get_tables LOOP rc := 0; EXECUTE IMMEDIATE 'SELECT count(*) FROM ' gt.table_name INTO rc; IF rc > 0 THEN UPDATE PPDM_TABLE SET row_count = rc, row_count_date = SYSDATE WHERE system_id = 'PPDM38' AND table_name = gt.table_name; ELSE UPDATE PPDM_TABLE SET row_count = 0, row_count_date = NULL WHERE system_id = 'PPDM38' AND table_name = gt.table_name; END IF; COMMIT; END LOOP; END; /

10 Data Content, continued What does this tell me? Tables populated, but is this enough? select COUNT(*) from ppdm_table where system_id = 'PPDM38' AND ROW_COUNT > 0 What else do I need to know? What modules are populated? select DISTINCT(COUNT(group_type ' - ' pgo.group_id)) table_count,group_type ' - ' pgo.group_id PPDM_GROUP from ppdm_group_object pgo,ppdm_table pt,ppdm_group pg where pgo.group_id=pg.group_id and pgo.system_id=pt.system_id and pt.system_id = 'PPDM38' and pgo.table_name=pt.table_name and row_count > 0 group by group_type ' - ' pgo.group_id order by 2 Group by data; TYPE - REFERENCE, SUPPORT, DATA Group by discipline; ID - WELL, LAND, SEISMIC Metrics

11 Rules; Data & Business Used for building rules in the db Primary Key is RULE_ID Links to PPDM_GROUP & PPDM_SYSTEM Allows you to create repeatable processes that track the health of the data in PPDM

12 Example of Rules: BUSINESS_ASSOCIATE Data Rules What is the % of null values in key columns? Ba_type, current_status, ba_category, source select (select count(*) from business_associate) total_rows, 1 - (count(ba_type)/(select count(*) from business_associate)) percent from business_associate where ba_type is not null group by ba_type Business Rules What reference tables have more than 250 values? Two step process Use PPDM_Column to store the distinct values of each column Join PPDM_Column with PPDM_Group_Object where the group_id = REFERENCE

13 Examples of Rules, continued Step 1 declare -- CURSOR get_columns IS SELECT table_name,column_name FROM ppdm_column where system_id = 'PPDM38' and data_type not in ('BLOB'); gc get_columns%rowtype; -- rc integer; begin FOR gc IN get_columns LOOP rc := 0; EXECUTE IMMEDIATE 'SELECT COUNT(DISTINCT(' gc.column_name ')) FROM ' gc.table_name INTO rc; IF rc > 0 THEN UPDATE PPDM_COLUMN SET distinct_value_count = rc, distinct_value_count_date = SYSDATE WHERE system_id = 'PPDM38' AND table_name = gc.table_name AND column_name = gc.column_name; ELSE UPDATE PPDM_COLUMN SET distinct_value_count = 0, distinct_value_count_date = SYSDATE WHERE system_id = 'PPDM38' AND table_name = gc.table_name AND column_name = gc.column_name; END IF; COMMIT; END LOOP; END; /

14 Examples of Rules; continued Step 2 select pc.table_name,pc.column_name,distinct_value_count from ppdm_column pc,ppdm_group_object pgo where pc.table_name=pgo.table_name and pgo.group_id = 'REFERENCE' and distinct_value_count > 250 What does this show you? A list of reference tables that might have too many values in it; What did it show me? These tables met the criteria R_WELL_LOG_CLASS R_WELL_STATUS R_CURVE_TYPE R_FLUID_TYPE R_PDEN_STATUS Next Steps Decide what you want to manage & when Store in the PPDM_Rule tables; both the rules and the outcome Move into PPDM_Metrics and maintain metrics

15 PPDM_EXCEPTION Just started using this table for reporting errors in some utility functions Primary Key is PE_ID (number(10)) Used SYSTEM_ID, TABLE_NAME or CONSTRAINT_NAME Put error message into REMARK

16 What s in it for me? Do you want to: Ensure that deletes are processed correctly Ensure that primary key changes are processed correctly Ensure that your data load worked Ensure that if your data load didn t work, you can capture why Ensure the dynamic display of UI elements Ability to start to optimize your business processes Store your data management knowledge in the database, not in an application

17 PPDM META MODEL: Practical Applications Investigate it Populate it Understand your data & processes Work with it Thank You

PPDM Meta Model Some Functionality

PPDM Meta Model Some Functionality Meta Model Table Groupings 1. PPDM_SYSTEM, PPDM_TABLE, PPDM_COLUMN 2. PPDM_GROUP, PPDM_GROUP_OBJECT 3. PPDM_CONSTRAINT, PPDM_CONS_COLUMN 4. PPDM_SYSTEM_MAP, PPDM_MAP_DETAIL,