Inventory your Data. PPDM Data Management Symposium June 2012

Transcription

1 Inventory your Data PPDM Data Management Symposium June 2012

2 Recap PPDM Meta Model talks Practical Applications - June 2011 Know your Data Oct 2011 Group your Data March 2012 The vision: It will become the Enterprise Data Repository

3 What s next Digging deeper into the content; What is present? UWI s, BA s, Field s, PDEN s, etc. How long has it been there? Has anything disappeared? Manage it like an Inventory

4 Requirements Understand when data comes in UWI s, BA s, Field s, PDen s, etc. Understand what s populated WELL_LICENSE.LICENSEE What s NOT populated WELL_LICENSE.LICENSEE_CONTACT_ID Gap Analysis Process Review

5 Rule Capture Define your data UWI s & BA s Create Rules in PPDM_RULE PPDM_RULE RULE_ID = BA_COUNT or UWI_COUNT RULE_CLASS = Bus Reqmnt RULE_PURPOSE = Completeness USE_CONDITION_TYPE = Weekly

6 Model Analysis Dig into UWI s select PCC.TABLE_NAME,PCC.COLUMN_NAME from PPDM_CONS_COLUMN PCC,PPDM_CONSTRAINT PC where PC.SYSTEM_ID=PCC.SYSTEM_ID and PC.TABLE_NAME=PCC.TABLE_NAME and PC.CONSTRAINT_ID=PCC.CONSTRAINT_ID and PC.REFERENCED_CONSTRAINT_ID = 'W_PK' AND PC.TABLE_NAME!= 'PPDM_RULE_COMPONENT order by PCC.TABLE_NAME;

7 Model Analysis Dig into UWI s select 'COLUMN' TYPE,TABLE_NAME,COLUMN_NAME from PPDM_COLUMN where SYSTEM_ID = 'PPDM38' and COLUMN_NAME like '%UWI%'; More than 125 rows What s missing? WELL WELL_CORE BUT WELL_CORE WELL_CORE_ANALYSIS

8 Designed Consistency Relationships can go down many levels Write once, Store once, Use many times PPDM_GROUP GROUP_ID = UWI COUNT (or BA COUNT) GROUP_TYPE = ANALYTIC RULE_ID = Rule_ID UWI_COUNT RULE_ID = Rule_ID BA_COUNT

9 PPDM_GROUP_OBJECT insert into PPDM_GROUP_OBJECT (GROUP_ID,OBJECT_OBS_NO,SYSTEM_ID,TABLE_NAME, COLUMN_NAME,RULE_ID,SOURCE,ROW_QUALITY) select 'UWI COUNT',OBJECT_OBS_NO,'PPDM38',TABLE_NAME, COLUMN_NAME, 'UWI_COUNT', 'pdc', 'REFERENCE FROM PPDM_COLUMN WHERE SYSTEM_ID = 'PPDM38' AND (TABLE_NAME,COLUMN_NAME) IN (select TABLE_NAME,COLUMN_NAME from PPDM_COLUMN where SYSTEM_ID = 'PPDM38' and COLUMN_NAME like '%UWI%' UNION select PCC.TABLE_NAME,PCC.COLUMN_NAME from PPDM_CONS_COLUMN PCC,PPDM_CONSTRAINT PC where PC.SYSTEM_ID = 'PPDM38' and PC.SYSTEM_ID=PCC.SYSTEM_ID and PC.TABLE_NAME=PCC.TABLE_NAME and PC.CONSTRAINT_ID=PCC.CONSTRAINT_ID and PC.REFERENCED_CONSTRAINT_ID = 'W_PK');

10 RECAP Goal of inventorying PPDM with PPDM Defined business requirements Use PPDM_GROUP & PPDM_RULE Decomposition of the data model Next Steps Repeatable Inventory Process

11 Inventory Process Part I Logic: In-activate any existing rows for UWI_COUNT for every row in the group (UWI COUNT), see if there is any data stored If not, go to the next one If there is, store a new row in PPDM_RULE_COMPONENT Declare cursor GET_DATA is select TABLE_NAME,COLUMN_NAME from PPDM_GROUP_OBJECT where GROUP_ID = 'UWI COUNT' ORDER BY TABLE_NAME; GD GET_DATA%ROWTYPE; --

12 Inventory Process Part II found integer := 0; typ_line varchar2(2000); TYPE cur_type IS REF CURSOR; cur cur_type; QRY varchar2(2000); pkey varchar2(20); begin update PPDM_RULE_COMPONENT set ACTIVE_IND = 'N', EXPIRY_DATE=sysdate where RULE_ID = 'UWI_COUNT' and ACTIVE_IND = 'Y'; COMMIT; for GD in GET_DATA LOOP execute immediate 'select count(*) from ' GD.TABLE_NAME ' where ' gd.column_name ' is not null' into FOUND; if FOUND > 0 then qry := 'select distinct ' gd.column_name ' from ' gd.table_name ' where ' gd.column_name ' is not null'; open cur for qry; LOOP fetch CUR into PKEY; insert into PPDM_RULE_COMPONENT(RULE_ID,COMPONENT_OBS_NO, PPDM_SYSTEM_ID,PPDM_TABLE_NAME,PPDM_COLUMN_NAME,UWI,SOURCE,ROW_QUALITY) values ('UWI_COUNT','OBJECT_OBS_NO,'PPDM38',GD.TABLE_NAME,gd.column_name, PKey,'pDC','REFERENCE'); exit when CUR%notfound; END LOOP; end if; commit; found := 0; END LOOP; end; /

13 What does it tell me? All tables that have a BA (or UWI) field that s populated What does this tell us? WELL_LICENSE. LICENSEE_CONTACT_ID A little bit about the intent

14 What else can it tell me? Reporting on the content of the database Like who licenses, drills and then produces select * from (select PPDM_TABLE_NAME,BA_NAME,SUM(PRC.REMARK) ROW_COUNT from PPDM_RULE_COMPONENT PRC,BUSINESS_ASSOCIATE BA where RULE_ID='BA_COUNT' and PRC.ACTIVE_IND = 'Y' and PRC.BUSINESS_ASSOCIATE=BA.BUSINESS_ASSOCIATE group by PPDM_TABLE_NAME,BA_NAME) PIVOT (SUM(ROW_COUNT) for PPDM_TABLE_NAME in ('PDEN','WELL','WELL_LICENSE')) order by 2,3 desc

15 What can it REALLY tell me? What do you have populated in PPDM? Area tables Linked with UWI Work with data via hierarchy Group tables Linked to TABLE_NAME Work with data via processes

16 What do I do with it? Start a program to continuously measure content You can now start to know what s going on Identify data spikes being + / - Show how to leverage an existing PPDM Add in new module, like Facility

17 PPDM Model Knowledge - Vision Map Structure Time Interval #1 Select RDBMS Architect Schema(s) Run SQL to create RDBMS objects Run PPDM SQL to populate base data Create SQL reports to see what you have Time Interval #2 Time Interval #3 Time Interval #4 Ensure Backups are in place Create SQL to monitor for any type of change Create process for remediation of content changes Run SQL reports to see what you have Decide what you want in your database Test your backups Create process for remediation of structure changes Review SQL reports to see what you have Create SQL reports to see what you have Align old reference data with new Prepare for PPDM 3.9 What s new? Review SQL reports to see what you have What do you need that s new? Create reference data review process PPDM 3.8 & Beyond Validate conversion Create SQL reports to see what data you have Create DQ process & avoid GIGO Create support data review process Content Create SQL reports to see what you have Review PPDM content for applicability Identify system(s) to migrate to PPDM Context Business

18 Summary A great set of tables to help manage PPDM Continous data management that is tuned to business processes Corporate reporting is being enabled; Moving from an OLTP to DSS Validation for more