HPE Project and Portfolio Management Center

Transcription

1 HPE Prject and Prtfli Management Center Sftware Versin: Cntent Pack 1.0 Vertica fr PPM Reprting Custmizatin Guide G t HELP CENTER ONLINE Dcument Release Date: Nvember 2014 Sftware Release Date: Nvember 2014

2 Vertica fr PPM Reprting Custmizatin Guide HPE Prject and Prtfli Management Center Legal Ntices Warranty The nly warranties fr Hewlett Packard Enterprise Develpment LP prducts and services are set frth in the express warranty statements accmpanying such prducts and services. Nthing herein shuld be cnstrued as cnstituting an additinal warranty. HPE shall nt be liable fr technical r editrial errrs r missins cntained herein. The infrmatin cntained herein is subject t change withut ntice. Restricted Rights Legend Cnfidential cmputer sftware. Valid license frm HPE required fr pssessin, use r cpying. Cnsistent with FAR and , Cmmercial Cmputer Sftware, Cmputer Sftware Dcumentatin, and Technical Data fr Cmmercial Items are licensed t the U.S. Gvernment under vendr's standard cmmercial license. Cpyright Ntice Cpyright 2016 Hewlett Packard Enterprise Develpment LP Trademark Ntices Adbe is a trademark f Adbe Systems Incrprated. Micrsft and Windws are U.S. registered trademarks f Micrsft Crpratin. UNIX is a registered trademark f The Open Grup. Oracle and Java are registered trademarks f Oracle and/r its affiliates. This prduct includes an interface f the 'zlib' general purpse cmpressin library, which is Cpyright Jeanlup Gailly and Mark Adler. Dcumentatin Updates The title page f this dcument cntains the fllwing identifying infrmatin: Sftware Versin number, which indicates the sftware versin. Dcument Release Date, which changes each time the dcument is updated. Sftware Release Date, which indicates the release date f this versin f the sftware. T check fr recent updates r t verify that yu are using the mst recent editin f a dcument, g t: This site requires that yu register fr an HPE Passprt and sign in. T register fr an HPE Passprt ID, g t and click Register. Supprt Visit the HPE Sftware Supprt Online web site at: This web site prvides cntact infrmatin and details abut the prducts, services, and supprt that HPE Sftware ffers. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 2 f 56

3 Vertica fr PPM Reprting Custmizatin Guide HPE Prject and Prtfli Management Center HPE Sftware nline supprt prvides custmer self-slve capabilities. It prvides a fast and efficient way t access interactive technical supprt tls needed t manage yur business. As a valued supprt custmer, yu can benefit by using the supprt web site t: Search fr knwledge dcuments f interest Submit and track supprt cases and enhancement requests Dwnlad sftware patches Manage supprt cntracts Lk up HPE supprt cntacts Review infrmatin abut available services Enter int discussins with ther sftware custmers Research and register fr sftware training Mst f the supprt areas require that yu register as an HPE Passprt user and sign in. Many als require a supprt cntract. T register fr an HPE Passprt ID, g t: and click Register. T find mre infrmatin abut access levels, g t: HPE Sftware Slutins & Integratins and Best Practices Visit HPE Sftware Slutins Nw at t explre hw the prducts in the HPE Sftware catalg wrk tgether, exchange infrmatin, and slve business needs. Visit Hewlett Packard Enterprise Self-Slve Knwledge Search at t access a wide variety f best practice dcuments and materials. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 3 f 56

4 Vertica fr PPM Reprting Custmizatin Guide HPE Prject and Prtfli Management Center Cntents HPE Prject and Prtfli Management Center 1 Chapter 1: Intrductin 7 Abut This Guide 7 Related Dcuments 7 HPE PPM Dcuments fr Vertica fr PPM Cntent Pack Accessing PPM Dcumentatin 7 Chapter 2: Vertica fr PPM Reprts Custmizatin 8 Befre Yu Custmize Reprts 8 Obtaining Sample Cntent Packs fr Custmizatin 8 Cntent Pack Structure 9 Running Sample Cntent Packs 10 Custmizing Vertica fr PPM Reprts 13 Extending r Creating Entities 13 Defining Cntent Packs 13 Deplying Cntent Packs 16 Scenaris f Extending r Creating Entities 18 Scenari: Extending Dimensin Entities 18 Defining Cntent Packs 18 Deplying Cntent Packs 25 Scenari: Creating New Dimensin Entities 26 Defining Cntent Packs 26 Deplying Cntent Packs 32 Custmizing ETL Rules 33 What is ETL Rules 33 Why Custmize ETL Rules 34 Hw t Custmize ETL Rules 34 Best Practices 35 Chapter 3: ETL Architecture 37 ETL Architecture Overview 37 ETL Steps Intrductin 38 ETL Step 1: EXT 38 ETL Step 2: SSI 41 ETL Step 3: XREF 42 ETL Step 4: MSI 43 ETL Step 5: XFR 45 HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 4 f 56

5 Vertica fr PPM Reprting Custmizatin Guide HPE Prject and Prtfli Management Center ETL Step 6: KEYLOOKUP 46 ETL Step 7: TARGET 47 ETL Step 8: TSNP 47 ETL Step 9: HIERARCHY 48 ETL Step 10: POSTTARGET 48 Appendix A: Entity Attribute Descriptins 50 Send Us Feedback 56 HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 5 f 56

6 Vertica fr PPM Reprting Custmizatin Guide HPE Prject and Prtfli Management Center HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 6 f 56

7 Vertica fr PPM Reprting Custmizatin Guide Chapter 1: Intrductin Chapter 1: Intrductin "Abut This Guide" belw "Related Dcuments " belw Abut This Guide This guide shws yu hw t custmize Vertica fr PPM reprts step by step. This guide als prvides infrmatin abut the ETL structure and Extract-Transfrm-Lad (ETL) steps, which can help yu better understand hw the ETL prcess wrks. Related Dcuments This sectin lists HPE dcuments that cntain useful infrmatin fr Vertica fr PPM administratrs and users. HPE PPM Dcuments fr Vertica fr PPM Cntent Pack 1.0 Vertica fr PPM Release Ntes fr Cntent Pack 1.0 Vertica fr PPM User's Guide fr Cntent Pack 1.0 Vertica fr PPM Administratr's Guide fr Cntent Pack 1.0 PPM 9.40 Data Mdel Guide Accessing PPM Dcumentatin T btain HP PPM Center dcumentatin g t the HP Sftware Supprt Online website at T access this website, yu must first register fr HP Passprt accunt. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 7 f 56

8 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin Chapter 2: Vertica fr PPM Reprts Custmizatin Read this sectin fr instructins n hw t custmize the Vertica fr PPM reprts. After reading this guide, yu can: Add new fact entity Add new dimensin entity Add attributes fr existing entities Change ETL rules Read the fllwing sectins fr details: "Befre Yu Custmize Reprts" belw "Custmizing Vertica fr PPM Reprts" n page 13 Befre Yu Custmize Reprts Befre yu start t custmize Vertica fr PPM reprts, make sure the fllwing envirnments are available: Vertica database cluster Vertica fr PPM cntent pack Fr instructins n hw t install Vertica fr PPM cntent pack, refer t the Vertica fr PPM Administratr's Guide fr Cntent Pack 1.0. Yu als need t have ETL hands-n experience befre custmizatin. Nte: T test the custmizatin prcess, it is suggested that yu run the sample cntent packs n a test envirnment first. D nt use the prductin envirnment fr testing r cntent develpment because cntent packs cannt be uninstalled. Yu als need t btain the sample cntent packs and understand the cntent pack structures by reading the fllwing sectins: "Obtaining Sample Cntent Packs fr Custmizatin" belw "Cntent Pack Structure " n the next page "Running Sample Cntent Packs " n page 10 Obtaining Sample Cntent Packs fr Custmizatin T btain the sample cntent packs, fllw these steps: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 8 f 56

9 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin 1. G t the HP Supprt Cntract infrmatin page ( 2. Sign in t the HP Passprt page. The HP Supprt Cntract infrmatin page pens. 3. In the Cntract identifier (SAID)* bx, type yur service agreement ID (SAID) and click Add. 4. G t Operatinal Reprts Cntent fr Prject and Prtfli Management - Dwnlads page n (hpln.hp.cm/nde/81/cntentfiles). 5. Select Vertica fr PPM CP Dwnlad the Vertica_Reprting_Custmizatin_Samples.zip file. 7. Extract the entire cntents f Vertica_Reprting_Custmizatin_Samples.zip t yur lcal drive. Cntent Pack Structure Generally speaking, tw cntent pack (.cp) flders are needed under the <VDW_ HOME>/Cntent directry fr a single task: CUSTOMIZATION_PPM.cp: Cntains JSON files that define surce entities, extractin entities, and stream entities. CUSTOMIZATION_TARGET.cp: Cntains JSON files that define target entities. Nte: When deplying cntent packs, yu always need t deply CUSTOMIZATION_TARGET.cp first. Otherwise, yu may nt be able t deply CUSTOMIZATION_PPM.cp and prcess the rest f tasks. The flder structure f the cntent packs is shwn as fllws: CUSTOMIZATION_PPM.cp (rt flder) dwmetadata (flder) entities (flder) Surce entity JSON (file) HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 9 f 56

10 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin streams (flder) Stream entity JSON (file) extmetadata (flder) Extractr entity JSON (file) cp.jsn (file) CUSTOMIZATION_TARGET.cp (rt flder) dwmetadata (flder) entities (flder) Target entity JSON (file) cp.jsn (file) The fllwing table describes the JSON files. JSON file Target entity JSON Surce entity JSON Stream entity JSON Extractr entity JSON CP JSON Descriptin Stres infrmatin related t the target table, such as field names and relatin infrmatin with ther tables. Stres infrmatin related t the surce table such as field names and incremental extractin infrmatin. Stres infrmatin related t the stream, such as the surce entity, target entity, and SSI transfrming SQLs. Stres infrmatin related t data extractin, such as SQLs fr extracting data. Stres infrmatin related t cntent packs, such as target entities and surce entities. This file is the entrance fr the crrespnding cntent pack. Running Sample Cntent Packs Befre yu custmize reprts, it is suggested yu run the fllwing cntent pack examples in yur test envirnment. 1. Run the fllwing cmmand n Linux t ensure that $VDW_HOME is cnfigured crrectly: Ech $VDW_HOME If $VDW_HOME is crrectly cnfigured, the directry that yu have Vertica fr PPM cntent pack installed returns. Fr example, /VDW_HOME HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 10 f 56

11 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin Otherwise, add $VDW_HOME as a system envirnment variable and pint it t the Vertica fr PPM cntent pack directry. 2. Unpack the Vertica_Reprting_Custmizatin_Samples.zip file. Tw cntent pack flders are included: CUSTOMIZATION_PPM.cp and CUSTOMIZATION_ TARGET.cp. Fr instructins n btaining the sample cntent packs, see "Obtaining Sample Cntent Packs fr Custmizatin" n page Place the CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp files in the <VDW_ HOME>/Cntent directry. 4. Run the CntentManager.sh script under the <VDW_HOME>/bin directry t deply the CUSTOMIZATION_TARGET.cp: sh CntentManager.sh --instructin install --cpname CUSTOMIZATION_TARGET; Yu can find the fllwing message frm CntentManager.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: The cntent f package CUSTOMIZATION_TARGET was successfully installed After yu run the cmmand, the fllwing tables are generated: The DIM_ CUSTOMIZATION_CONTACTS table is generated fr extending dimensin entities. The FACT_CUSTOMIZATION_RESOURCE_DEMAND table is generated fr extending fact entities. The DIM_CUSTOMIZATION_TIME_SHEETS table is generated fr creating new dimensin entities. The FACT_ CUSTOMIZATION_TIME_ACTUALS table is generated fr creating new fact entities. 5. Run the CntentManager.sh script under the <VDW_HOME>/bin directry t deply CUSTOMIZATION_PPM.cp: sh CntentManager.sh --instructin install --cpname CUSTOMIZATION_PPM; Yu can find the fllwing message frm CntentManager.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: The cntent f package CUSTOMIZATION_PPM was successfully installed 6. Run the ExtractrEngine.sh script under the <VDW_HOME>/bin directry t extract data frm the PPM database t flat files: sh ExtractrEngine.sh --streamname <Stream_Name> --instancename <PPM_ Instance_Name> Yu can find the fllwing message frm ExtractrEngine.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 11 f 56

12 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin Extractr was successfully executed. The BATCH ID is: <Batch_ID>. Yu als need t make sure that n warning messages are displayed in the cmmand line interface. Replace <Stream_Name> with the fllwing depending n the sample cntent packs yu use: CUSTOMIZATION_CONTACTS_STREAM fr extended dimensin entities CUSTOMIZATION_RESOURCE_DEMAND_STREAM fr extended fact entities CUSTOMIZATION_TIME_SHEETS_STREAM fr new dimensin entities CUSTOMIZATION_TIME_ACTUALS_STREAM fr new fact entities <PPM_Instance_Name> is the PPM instance name yu specified when installing the Vertica fr PPM cntent pack. 7. Run the FlwEngine.sh script under the <VDW_HOME>/bin directry t prcess ETL: sh FlwEngine.sh --batch <Batch_ID> --streamname <Stream_Name> -- instancename <PPM_Instance_Name> Yu can find the fllwing message frm FlwEngine.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: ETL prcess was executed successfully <Batch_ID> used in this cmmand is the batch ID that was generated in Step 6. Replace <Stream_Name> with the fllwing depending n the sample cntent packs yu use: CUSTOMIZATION_CONTACTS_STREAM fr extending dimensin entities CUSTOMIZATION_RESOURCE_DEMAND_STREAM fr extending fact entities CUSTOMIZATION_TIME_SHEETS_STREAM fr new dimensin entities CUSTOMIZATION_TIME_ACTUALS_STREAM fr new fact entities 8. Cnnect t the Vertica database and check whether the data has been laded successfully: select * frm <Target_Schema>.<Table_Name> <Target_Schema> is the name fr the schema that cntains target data and tables fr reprting. Replace <Table_Name> with the fllwing fr different sample cntent packs: DIM_ CUSTOMIZATION_CONTACTS fr extended dimensin entities FACT_CUSTOMIZATION_RESOURCE_DEMAND fr extended fact entities DIM_CUSTOMIZATION_TIME_SHEETS fr new dimensin entities FACT_CUSTOMIZATION_TIME_ACTUALS fr new fact entities Fr mre infrmatin abut these scripts, refer t "Administratin Tasks" f the Vertica fr PPM Administratr's Guide fr Cntent Pack 1.0. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 12 f 56

13 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin Custmizing Vertica fr PPM Reprts This sectin prvides instructins n custmizing Vertica fr PPM reprts: "Extending r Creating Entities" belw "Custmizing ETL Rules" n page 33 Extending r Creating Entities Yu can extend existing entities r create new entities fr bth dimensin and fact tables. This sectin shws yu hw t custmize the entities step by step. Fr examples f hw t extend r create dimensin and fact entities, see "Scenaris f Extending r Creating Entities" n page 18. Nte: It is suggested t run the sample cntent packs in yur test envirnment befre extending entities. See "Running Sample Cntent Packs " n page 10 fr details. Defining Cntent Packs T extend r create entities, yu need t define cntent packs by fllwing these steps: 1. Create tw cntent pack flders: CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp. 2. Create flders under these tw cntent packs. Fr detailed structures, see "Cntent Pack Structure " n page Under CUSTOMIZATION_TARGET.cp, d the fllwing: a. Under the dwmetadata\entities directry, create and define the target entity JSON file accrding t the sample cntent packs at HP Live Netwrk. Fr mre infrmatin fr btaining the sample cntent packs, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. Nte the fllwing when defining the target entity JSON file: Fr details abut the attributes in this JSON file, see "Entity Attribute Descriptins" n page 50. The entity_name attribute shuld be unique because the table name is generated accrding t the entity name. The segmented_by attribute is used fr cluster segmentatin. See HP Vertica Analytics Platfrm Versin 7.0.x Dcumentatin fr details. The schema attribute includes the new attributes: attribute_name: The attribute name. scd (fr dimensin tables nly): Determines whether t update the recrd (scd1) r t insert a new recrd (scd2). HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 13 f 56

14 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin T extend OOTB entities, yu need t define the *_assciated_* attribute: lkup_entity_name: The name f the entity that needs t be extended. rle_entity_name: Includes CUSTOMIZATION_ as the prefix and _EXTEND as the suffix; fr example, CUSTOMIZATION_CONTACTS_EXTEND. *_assciated_* is required fr extended entities while ptinal fr new entities. Refer t the fllwing table fr details. Attribute Descriptin Attribute Type Descriptin Required fr Optinal fr dimensin_ assciated_ dimensin Used t define the reference frm a dimensin table t anther dimensin table (fr example, snwflake mdel). Fr an extended dimensin entity, this attribute is used t specify which entity this table is extended frm. Extended dimensin tables New dimensin tables fact_ assciated_ dimensin fact_ assciated_ fact Used t define the reference frm a fact table t a dimensin table. Used t define the reference frm a fact table t anther fact table. Fr an extended fact entity, this attribute is used t specify which entity this table is extended frm. Extended fact tables; Use nly ne f the attrbutes fr a single entity New fact tables (Nt supprted fr new fact tables) b. Create and define cp.jsn accrding t the samples prvided at HP Live Netwrk. Fr mre infrmatin fr btaining the sample cntent packs, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. Nte the fllwing when defining cp.jsn: The value f cntent_pack_name shuld be the same as defined in cntent_pack f the target entity. cp.jsn in CUSTOMIZATION_TARGET.cp must have target_entities defined. Fr detailed descriptins f the attributes in this JSON file, see "Entity Attribute Descriptins" n page 50 After yu cmplete this step, the target entity JSON file is defined. 4. In CUSTOMIZATION_PPM.cp, d the fllwing: a. Under the dwmetadata\entities directry, create and define the surce entity JSON file accrding t the samples prvided at HP Live Netwrk. Fr mre infrmatin, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. When defining the surce entity JSON file, nte the fllwing: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 14 f 56

15 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin schema: Includes the attribute definitins fr the surce entity. attribute_name: Defines the attribute name. Fr detailed descriptins f the attributes in this JSON file, see "Entity Attribute Descriptins" n page 50. After yu cmplete this step, the surce entity JSON file is defined. b. Under dwmetadata\streams, create and define the stream entity JSON file accrding t the samples prvided at HP Live Netwrk. Fr mre infrmatin, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. When defining the stream entity JSON file, nte the fllwing: cntent_pack: Aligns with cntent_pack defined in the surce entity JSON file. stream_name: Includes CUSTOMIZATION_ as the prefix. surce_entities_includes: Includes the surce entity f the stream; fr example, CUSTOMIZATION_KCRT_CONTACTS. target_entities_includes: Includes the target entity f the stream; fr example, CUSTOMIZATION_CONTACTS. transfrms: see "ETL Step 2: SSI" n page 41fr a definitin. pst_target_transfrms: Optinal. If yu want t prcess ther SQLs after data is laded t the target table, include SQLs in this attribute. Fr mre infrmatin, see "ETL Step 10: POSTTARGET" n page 48. Fr detailed descriptins f the attributes in this JSON file, see "Entity Attribute Descriptins" n page 50. After yu cmplete this step, the stream entity JSON file is defined. c. Under extmetadata, create and define the extractr entity JSON file accrding t the samples prvided at HP Live Netwrk. Fr mre infrmatin, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. When defining the extractr entity JSON file, nte the fllwing: cntent_pack: Aligns with cntent_pack defined in the stream entity and surce entity. entity_name: Includes CUSTOMIZATION_ as the prefix and _EXT as the suffix. surce_entity_name: Defines the surce table name. extractin_view: Selects the attributes defined in the surce entity JSON file. Fr detailed descriptins f the attributes in this JSON file, see "Entity Attribute Descriptins" n page 50. After yu cmplete this step, the extractr entity JSON file is defined. d. Create and define cp.jsn accrding t the samples prvided at HP Live Netwrk. Fr mre infrmatin, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. When defining cp.jsn, nte the fllwing: cntent_pack_name shuld be unique and align with cntent_pack defined in stream entity, surce entity, and extractr entity. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 15 f 56

16 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin cp.jsn in CUSTOMIZATION_PPM.cp must have the streams, surce_entities, and extractin_entities attributes defined. Fr detailed descriptins f the attributes in this JSON file, see "Entity Attribute Descriptins" n page 50. Deplying Cntent Packs T deply cntent packs, fllw these steps: 1. Run the fllwing cmmand n Linux t ensure that $VDW_HOME is cnfigured crrectly: Ech $VDW_HOME If $VDW_HOME is crrectly cnfigured, the directry that yu have Vertica fr PPM cntent pack installed returns. Fr example, /VDW_HOME Otherwise, add $VDW_HOME as a system envirnment variable and pint it t the Vertica fr PPM cntent pack directry. 2. Place CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp under <VDW_ HOME>/Cntent. 3. Run the CntentManager.sh script under the <VDW_HOME>/bin directry t deply CUSTOMIZATION_TARGET.cp: sh CntentManager.sh --instructin install --cpname CUSTOMIZATION_TARGET; Yu can find the fllwing message frm CntentManager.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: The cntent f package CUSTOMIZATION_TARGET was successfully installed The DIM_ CUSTOMIZATION_CONTACTS table is generated in the Vertica database. 4. Run the CntentManager.sh script under the <VDW_HOME>/bin directry t deply CUSTOMIZATION_PPM.cp: sh CntentManager.sh --instructin install --cpname CUSTOMIZATION_PPM; Yu can find the fllwing message frm CntentManager.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: The cntent f package CUSTOMIZATION_PPM was successfully installed 5. Run the ExtractrEngine.sh script under the <VDW_HOME>/bin directry t extract data frm the PPM database t flat files: sh ExtractrEngine.sh --streamname <Stream_Entity_Name> --instancename <PPM_ Instance_Name> Yu can find the fllwing message frm ExtractrEngine.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 16 f 56

17 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin Extractr was successfully executed. The BATCH ID is: <Batch_ID>. <Stream_Entity_Name>: Shuld be the same as defined in stream_name f the stream entity JSON file. <PPM_Instance_Name>: The PPM instance name yu specified when installing the Vertica fr PPM cntent pack. 6. Run the FlwEngine.sh script under the <VDW_HOME>/bin directry t prcess ETL: sh FlwEngine.sh --batch <Batch_ID> --streamname <Stream_Entity_Name> -- instancename <PPM_Instance_Name> Yu can find the fllwing message frm FlwEngine.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: ETL prcess was executed successfully <Batch_ID>: The batch ID that was generated in Step 5. <Stream_Entity_Name>: Shuld be the same as defined in stream_name f the stream entity JSON file. <PPM_Instance_Name>: The PPM instance name yu specified when installing the Vertica fr PPM cntent pack. 7. Cnnect t the Vertica database and check whether the data has been laded successfully: select * frm <Target_Schema>.DIM_<Target_Entity_Name> Or select * frm <Target_Schema>.FACT_<Target_Entity_Name> <Target_Schema>: The name fr the schema that cntains target data and tables fr reprting. <Target_Entity_Name>: Shuld be the same as defined in entity_name f the target entity JSON file. T find the new target tables in views, yu need t manually update the views in the Vertica database with the fllwing SQL queries: CREATE OR REPLACE VIEW <vdwtarget_schema>.<custmized target table>_v AS SELECT <OOTB target table>.*, <custmized target table>.<extended field> as extended_ field FROM <vdwtarget_schema>.<customized target table> right jin <vdwtarget_ schema>.<ootb target table> n <vdwtarget_schema>.<customized target table>.md_enterprise_key = <vdwtarget_ schema>.<ootb target table>.md_enterprise_key; Replace the fllwing variables: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 17 f 56

18 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin <vdwtarget_schema>: The schema that cntains the target tables <custmized target table>: The target table that is created <OOTB target table>: The OOTB target table that is custmized <extended field>: The field that is t be extended Yu can find all views n specified tables in the PPM 9.40 Data Mdel Guide. Nte the fllwing when deplying the cntent packs: Always deply CUSTOMIZATION_TARGET.cp befre deplying CUSTOMIZATION_PPM.cp. If yu change the cntent packs after deplyment, yu need t run CntentManager.sh again t make the changes effective. Yu need t create a Shell t call the Extractr Engine and Flw Engine, and run ETL n a regular basis with crntab. Yu als need t make sure the script can run after the vdwetljb.sh prcess cmpletes. Fr hw t create the Shell script, yu can take vdwcustmizatinetljb.sh that is included in the sample package as an example. Fr instructins n btaining the sample package, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. Fr mre infrmatin abut these scripts, refer t "Administratin Tasks" f the Vertica fr PPM Administratr's Guide fr Cntent Pack 1.0. Scenaris f Extending r Creating Entities Yu can better understand the prcedure f reprt custmizatin by reading the fllwing scenaris: "Scenari: Extending Dimensin Entities" belw "Scenari: Creating New Dimensin Entities" n page 26 By leveraging these scenaris, yu can als d the fllwing with the sample cntent packs that are prvided n HP Live Netwrk: Extending fact entities Creating fact entities Fr mre infrmatin fr btaining the sample cntent packs, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. Scenari: Extending Dimensin Entities If yu want t extend dimensin entities fr the KCRT_CONTACTS table, fllw these steps. The USER_ID and ENABLE_FLAG fields are t be added. Defining Cntent Packs T define cntent packs, fllw these steps: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 18 f 56

19 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin 1. Create tw cntent pack flders: CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp. 2. Create flders under these tw cntent packs. Fr detailed structures, see "Cntent Pack Structure " n page Under CUSTOMIZATION_TARGET.cp, d the fllwing: a. Under the dwmetadata\entities directry, create and define the target entity JSON file CUSTOMIZATION_CONTACTS.jsn by cpying the fllwing t the file: "metadata_layut_versin":"1.0", "versin":"1.0", "cntent_pack":"customization_target", "entity_name":"customization_contacts", "entity_business_name":"customization CONTACTS", "entity_descriptin":"customization CONTACTS", "dimensin": "dimensin_business_name":"customization CONTACTS", "is_cnfrmed":"true", "dimensin_type":"primary", "strage_strategy": "segmented_by":"default", "partitin_by":"na", "schema":[ "attribute":"user_id", "attribute_name":"user_id", "attribute_business_name":"user ID", "attribute_descriptin":"user ID", "attribute_type":"dimensin", "scd":"scd1", "target_data_type":"int", "size":"na", "is_required":"false", "attribute":"enabled_flag", "attribute_name":"enabled_flag", "attribute_business_name":"enabled FLAG", "attribute_descriptin":"enabled FLAG", "attribute_type":"dimensin", "scd":"scd2", "target_data_type":"varchar", "size":"1", "is_required":"false" ], "dimensin_assciated_dimensin":[ HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 19 f 56

20 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin ] "lkup_entity_name":"cntacts", "rle_entity_name":"customization_contacts_extend", "rle_entity_business_name":"cntacts", "descriptin":"cntacts" "dimensin_assciated_dimensin":[ "lkup_entity_name":"cntacts", "rle_entity_name":"extends_cntacts", "rle_entity_business_name":"cntacts", "descriptin":"cntacts" ] CUSTOMIZATION_CONTACTS.jsn defines: Tw fields in the target table: USER_ID and ENABLED_FLAG The target entity that is t be extended: Cntacts After yu cmplete this step, the target entity JSON file is defined. b. Create and define cp.jsn by cpying the fllwing t the file: "metadata_layut_versin": "1.0", "cntent_pack_name": "CUSTOMIZATION_TARGET", "versin": "1.0", "descriptin": "Prject Management & Prtfli Management Cntent Pack, Shared Entities.", "require": "platfrm": ">=1.0.0", "target_entities": [ "name": "CUSTOMIZATION_CONTACTS" ] cp.jsn defines: The cntent pack name: CUSTOMIZATION_TARGET The target entity as defined in CUSTOMIZATION_CONTACTS.jsn: CUSTOMIZATION_ CONTACTS After yu cmplete this step, the cp.jsn file is defined. 4. In CUSTOMIZATION_PPM.cp, d the fllwing: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 20 f 56

21 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin a. Under the dwmetadata\entities directry, create and define the surce entity JSON file CUSTOMIZATION_KCRT_CONTACTS.jsn: "metadata_layut_versin": "1.0", "versin": "1.0", "cntent_pack": "CUSTOMIZATION_PPM", "surce_prduct": "PPM", "entity_name": "CUSTOMIZATION_KCRT_CONTACTS", "entity_business_name": "PPM cntacts table with custmized field", "entity_descriptin": "Cntacts inf with custmized field", "schema": [ "attribute": "CONTACT_ID", "attribute_name": "CONTACT_ID", "attribute_business_name": "Cntact ID", "attribute_descriptin": "Cntact ID", "sql_data_type":"int", "size": "na", "is_bk": "true", "is_cdc": "false", "is_required": "true", "clumn_sequence": "1", "attribute": "USER_ID", "attribute_name": "USER_ID", "attribute_business_name": "User Id", "attribute_descriptin": "User ID f the cntact", "sql_data_type":"int", "size": "na", "is_bk": "false", "is_cdc": "false", "is_required": "false", "clumn_sequence": "2", "attribute": "ENABLED_FLAG", "attribute_name": "ENABLED_FLAG", "attribute_business_name": "ENABLED FLAG", "attribute_descriptin": "ENABLED FLAG", "sql_data_type":"varchar", "size": "1", "is_bk": "false", "is_cdc": "false", "is_required": "false", "clumn_sequence": "3" ] HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 21 f 56

22 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin CUSTOMIZATION_KCRT_CONTACTS.jsn defines: Three fields in the surce table: CONTACT_ID and ENABLED_FLAG fields that are extended; USER_ID that is the primary key f the surce table The surce entity name: CUSTOMIZATION_KCRT_CONTACTS After yu cmplete this step, the surce entity JSON file is defined. b. Under dwmetadata\streams, create and define the stream entity JSON file CUSTOMIZATION_CONTACTS_STREAM.jsn: "metadata_layut_versin":"1.0", "versin":"1.0", "cntent_pack":"customization_ppm", "surce_prduct":"ppm", "stream_name":"customization_contacts_stream", "surce_entities_includes":[ "surce_entity_include":"customization_kcrt_contacts" ], "target_entities_includes":[ "target_entity_include":"customization_contacts" ], "transfrms":[ "sql": "drp table if exists PPM_CUSTOMIZATION_CONTACTS_$MD_SOURCE_ INSTANCE_ID_SSI CASCADE; --==--==-- create table PPM_CUSTOMIZATION_CONTACTS_$MD_SOURCE_INSTANCE_ID_SSI ( MD_BUSINESS_KEY varchar(1000), CUSTOMIZATION_CONTACTS_EXTEND_BUSINESS_KEY varchar (1000), CUSTOMIZATION_CONTACTS_EXTEND_ENTERPRISE_KEY INT, USER_ID INT, ENABLED_FLAG varchar(1), MD_BATCH_ID INT, MD_PROCESS_ID INT, MD_SOURCE_INSTANCE_ID INT, MD_FLAG varchar(10) ); --==--==-- insert int PPM_CUSTOMIZATION_CONTACTS_$MD_SOURCE_INSTANCE_ID_SSI ( MD_BUSINESS_KEY, CUSTOMIZATION_CONTACTS_EXTEND_BUSINESS_KEY, HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 22 f 56

23 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin USER_ID, ENABLED_FLAG, MD_BATCH_ID, MD_PROCESS_ID, MD_SOURCE_INSTANCE_ID, MD_FLAG ) select tab.md_business_key, tab.customization_contacts_extend_business_key, tab.user_id, tab.enabled_flag, $MD_BATCH_ID AS MD_BATCH_ID, $MD_PROCESS_ID AS MD_PROCESS_ID, $MD_SOURCE_INSTANCE_ID AS MD_SOURCE_INSTANCE_ID, tab.md_flag frm ( select t1.md_surce_instance_id ':' t1.md_ business_key as MD_BUSINESS_KEY, t1.md_surce_instance_id ':' t1.contact_id as CUSTOMIZATION_CONTACTS_ EXTEND_BUSINESS_KEY, USER_ID, ENABLED_FLAG, decde(t1.md_ flag,'new','new','del','del','upd') as md_flag, rw_number() ver( partitin by t1.md_ surce_instance_id ':' t1.md_business_key) multi_flag frm CUSTOMIZATION_CONTACTS_STREAM_CUSTOMIZATION_KCRT_ CONTACTS_$MD_SOURCE_INSTANCE_ID_EXT t1 where t1.md_pf_flag = 'D') tab where tab.multi_flag = 1; --==--==-- SELECT ANALYZE_STATISTICS('PPM_CUSTOMIZATION_CONTACTS_$MD_SOURCE_INSTANCE_ ID_SSI'); --==--==-- " ] CUSTOMIZATION_CONTACTS_STREAM.jsn defines: Surce entities that prvide data Target entities that accept data SQLs mainly t transfrm data frm surce entities t target entities After yu cmplete this step, the stream entity JSON file is defined. c. Under extmetadata, create and define the extractr entity JSON file CUSTOMIZATION_ HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 23 f 56

24 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin KCRT_CONTACTS_EXT.jsn: "metadata_layut_versin":"1.0", "versin":"1.0", "surce_prduct":"ppm", "cntent_pack": "CUSTOMIZATION_PPM", "entity_name":"customization_kcrt_contacts_ext", "surce_entity_name":"kcrt_contacts", "extractr" : "OracleDBExtractr", "extractin":[ "extractin_view" : "SELECT cntact_id, user_id, ENABLED_FLAG FROM KCRT_ CONTACTS", "surce_prduct_versin" : "9.30" ] CUSTOMIZATION_KCRT_CONTACTS_EXT.jsn defines: The extractr entity name: CUSTOMIZATION_KCRT_CONTACTS_EXT surce_entity_name as the surce table name extractin_view: SQLs fr extracting data After yu cmplete this step, the extractr entity JSON file is defined. d. Create and define cp.jsn: "metadata_layut_versin": "1.0", "cntent_pack_name": "CUSTOMIZATION_PPM", "surce_prduct": "PPM", "versin": "1.0", "descriptin": "Prject Management & Prtfli Management Cntent Pack, Extend Entities.", "require": "platfrm": ">=1.0.0", "cp": [ "name": "CUSTOMIZATION_TARGET", "versin": ">=1.0.0" ], "streams": [ "name": "CUSTOMIZATION_CONTACTS_STREAM" ], "surce_entities": [ HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 24 f 56

25 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin "name": "CUSTOMIZATION_KCRT_CONTACTS" ], "extractin_entities": [ "name": "CUSTOMIZATION_KCRT_CONTACTS_EXT" ] cp.jsn defines: The cntent pack name: CUSTOMIZATION_PPM Surce entities: Defined in CUSTOMIZATION_KCRT_CONTACTS.jsn Stream entities: Defined in CUSTOMIZATION_CONTACTS_STREAM.jsn Extract entities: Defined in CUSTOMIZATION_KCRT_CONTACTS_EXT.jsn Deplying Cntent Packs T deply cntent packs, fllw these steps: 1. Place CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp under <VDW_ HOME>/Cntent. 2. Run the CntentManager.sh script under the <VDW_HOME>/bin directry t deply CUSTOMIZATION_TARGET.cp: sh CntentManager.sh --instructin install --cpname CUSTOMIZATION_TARGET; Yu can find the fllwing message frm CntentManager.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: The cntent f package CUSTOMIZATION_TARGET was successfully installed The DIM_ CUSTOMIZATION_CONTACTS table is generated in the Vertica database. 3. Run the CntentManager.sh script under the <VDW_HOME>/bin directry t deply CUSTOMIZATION_PPM.cp: sh CntentManager.sh --instructin install --cpname CUSTOMIZATION_PPM; Yu can find the fllwing message frm CntentManager.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: The cntent f package CUSTOMIZATION_PPM was successfully installed 4. Run the ExtractrEngine.sh script under the <VDW_HOME>/bin directry t extract data frm the PPM database t flat files: sh ExtractrEngine.sh --streamname CUSTOMIZATION_CONTACTS_STREAM --instancename <PPM_Instance_Name> HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 25 f 56

26 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin Yu can find the fllwing message frm ExtractrEngine.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: Extractr was successfully executed. The BATCH ID is: <Batch_ID>. <PPM_Instance_Name> is the PPM instance name yu specified when installing the Vertica fr PPM Cntent Pack. 5. Run the FlwEngine.sh script under the <VDW_HOME>/bin directry t prcess ETL: sh FlwEngine.sh --batch <Batch_ID> --streamname CUSTOMIZATION_CONTACTS_STREAM --instancename <PPM_Instance_Name> Yu can find the fllwing message frm FlwEngine.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: ETL prcess was executed successfully <Batch_ID> used in this cmmand is the batch ID that was generated in Step Cnnect t the Vertica database and check whether the data has been laded successfully: select * frm <Target_Schema>.DIM_CUSTOMIZATION_CONTACTS <Target_Schema> is the name fr the schema that cntains target data and tables fr reprting. Scenari: Creating New Dimensin Entities If yu want t create new dimensin entities fr the TM_TIME_SHEETS table, fllw these steps. The TIME_SHEET_ID and DESCRIPTION fields are t be added. Defining Cntent Packs T define cntent packs, fllw these steps: 1. Create tw cntent pack flders: CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp. 2. Create flders under these tw cntent packs. Fr detailed structures, see "Cntent Pack Structure " n page Under CUSTOMIZATION_TARGET.cp, d the fllwing: a. Under the dwmetadata\entities directry, create and define the target entity JSON file CUSTOMIZATION_TIME_SHEETS.jsn by cpying the fllwing t the file: "metadata_layut_versin": "1.0", "versin": "1.0", "cntent_pack": "CUSTOMIZATION_TARGET", "entity_name": "CUSTOMIZATION_TIME_SHEETS", "entity_business_name": "Custmizatin TIME SHEETS", "entity_descriptin": "Custmizatin TIME SHEETS", HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 26 f 56

27 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin "dimensin": "dimensin_business_name":"custmizatin time sheets", "is_cnfrmed":"true", "dimensin_type":"primary", "strage_strategy": "segmented_by":"default", "partitin_by":"na", "schema": [ "attribute": "TIME_SHEET_ID", "attribute_name": "TIME_SHEET_ID", "attribute_business_name": "TIME SHEET ID", "attribute_descriptin": "TIME SHEET ID", "attribute_type": "dimensin", "scd": "scd1", "target_data_type": "INT", "size": "na", "is_required": "false", "attribute": "DESCRIPTION", "attribute_name": "Descriptin", "attribute_business_name": "Custmizatin Descriptin", "attribute_descriptin": "Custmizatin Descriptin", "attribute_type": "dimensin", "scd": "scd1", "target_data_type": "varchar", "size": "650", "is_required": "false" ] CUSTOMIZATION_TIME_SHEETS.jsn defines: Tw fields in the target table: TIME_SHEET_ID and DESCRIPTION. After yu cmplete this step, the target entity JSON file is defined. b. Create and define cp.jsn by cpying the fllwing t the file. The value f cntent_ pack_name shuld be the same value defined in cntent_pack in step 3a: "metadata_layut_versin": "1.0", "cntent_pack_name": "CUSTOMIZATION_TARGET", "versin": "1.0", "descriptin": "Prject Management & Prtfli Management Cntent Pack, Shared Entities.", HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 27 f 56

28 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin "require": "platfrm": ">=1.0.0", "target_entities": [ "name": "CUSTOMIZATION_TIME_SHEETS" ] cp.jsn defines: The target entity CUSTOMIZATION_TIME_SHEETS, as defined in CUSTOMIZATION_TIME_ SHEETS.jsn The cntent pack name: CUSTOMIZATION_TARGET After yu cmplete this step, the cp.jsn file is defined. 4. In CUSTOMIZATION_PPM.cp, d the fllwing: a. Under the dwmetadata\entities directry, create and define the surce entity JSON file CUSTOMIZATION_TM_TIME_SHEETS.jsn: "metadata_layut_versin": "1.0", "versin": "1.0", "cntent_pack": "CUSTOMIZATION_PPM", "surce_prduct": "PPM", "entity_name": "CUSTOMIZATION_TM_TIME_SHEETS", "entity_business_name": "Custmizatin PPM TIME SHEETS table", "entity_descriptin": "Custmizatin TIME SHEETS inf", "schema": [ "attribute": "TIME_SHEET_ID", "attribute_name": "TIME_SHEET_ID", "attribute_business_name": "Time sheet id", "attribute_descriptin": "Time sheet id", "sql_data_type": "INT", "size": "na", "is_bk": "true", "is_cdc": "false", "is_required": "true", "clumn_sequence": "1", "attribute": "DESCRIPTION", "attribute_name": "DESCRIPTION", "attribute_business_name": "Descriptin", "attribute_descriptin": "Descriptin", "sql_data_type": "VARCHAR", "size": "650", HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 28 f 56

29 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin ] "is_bk": "false", "is_cdc": "false", "is_required": "false", "clumn_sequence": "2" CUSTOMIZATION_TM_TIME_SHEETS.jsn defines: Tw fields in the surce table: TIME_SHEET_ID and DESCRIPTION The surce entity name: CUSTOMIZATION_TM_TIME_SHEETS After yu cmplete this step, the surce entity JSON file is defined. b. Under dwmetadata\streams, create and define the stream entity JSON file CUSTOMIZATION_TIME_SHEETS_STREAM.jsn: "metadata_layut_versin":"1.0", "versin":"1.0", "cntent_pack":"customization_ppm", "surce_prduct":"ppm", "stream_name":"customization_time_sheets_stream", "surce_entities_includes":[ "surce_entity_include":"customization_tm_time_sheets" ], "target_entities_includes":[ "target_entity_include":"customization_time_sheets" ], "transfrms":[ "sql": "drp table if exists PPM_CUSTOMIZATION_TIME_SHEETS_$MD_SOURCE_ INSTANCE_ID_SSI CASCADE; --==--==-- create table PPM_CUSTOMIZATION_TIME_SHEETS_$MD_SOURCE_INSTANCE_ID_SSI ( MD_BUSINESS_KEY varchar(1000), CUSTOMIZATION_TIME_SHEETS_EXTEND_BUSINESS_KEY varchar(1000), CUSTOMIZATION_TIME_SHEETS_EXTEND_ENTERPRISE_KEY INT, DESCRIPTION varchar(650), MD_BATCH_ID MD_PROCESS_ID MD_SOURCE_INSTANCE_ID MD_FLAG ) unsegmented all ndes; INT, INT, INT, varchar(10) HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 29 f 56

30 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin --==--==-- insert int PPM_CUSTOMIZATION_TIME_SHEETS_$MD_SOURCE_INSTANCE_ID_SSI ( MD_BUSINESS_KEY, CUSTOMIZATION_TIME_SHEETS_EXTEND_BUSINESS_KEY, DESCRIPTION, MD_BATCH_ID, MD_PROCESS_ID, MD_SOURCE_INSTANCE_ID, MD_FLAG ) select tab.md_business_key, tab.customization_time_sheets_extend_business_key, tab.description, $MD_BATCH_ID AS MD_BATCH_ID, $MD_PROCESS_ID AS MD_PROCESS_ID, $MD_SOURCE_INSTANCE_ID AS MD_SOURCE_INSTANCE_ID, tab.md_flag frm ( select t1.md_surce_instance_id ':' t1.md_ business_key as MD_BUSINESS_KEY, t1.md_surce_instance_id ':' t1.time_sheet_id as CUSTOMIZATION_TIME_ SHEETS_EXTEND_BUSINESS_KEY, DESCRIPTION, decde(t1.md_ flag,'new','new','del','del','upd') as md_flag, rw_number() ver( partitin by t1.md_ surce_instance_id ':' t1.md_business_key ) multi_flag frm CUSTOMIZATION_TIME_SHEETS_STREAM_CUSTOMIZATION_TM_TIME_ SHEETS_$MD_SOURCE_INSTANCE_ID_EXT t1 where t1.md_pf_flag = 'D') tab where tab.multi_flag = 1; --==--==-- SELECT ANALYZE_STATISTICS('PPM_CUSTOMIZATION_TIME_SHEETS_$MD_SOURCE_ INSTANCE_ID_SSI'); --==--==-- " ] CUSTOMIZATION_TIME_SHEETS_STREAM.jsn defines: Surce entities that prvide data Target entities that accept data SQLs mainly t transfrm data frm surce entities t target entities After yu cmplete this step, the stream entity JSON file is defined. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 30 f 56

31 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin c. Under extmetadata, create and define the extractr entity JSON file CUSTOMIZATION_TM_ TIME_SHEETS_EXT.jsn: "metadata_layut_versin": "1.0", "versin": "1.0", "surce_prduct": "PPM", "cntent_pack": "CUSTOMIZATION_PPM", "entity_name": "CUSTOMIZATION_TM_TIME_SHEETS_EXT", "surce_entity_name": "TM_TIME_SHEETS", "extractr": "OracleDBExtractr", "extractin": [ "extractin_view": "SELECT TIME_SHEET_ID, DESCRIPTION FROM TM_ TIME_SHEETS", "surce_prduct_versin": "9.30" ] CUSTOMIZATION_TM_TIME_SHEETS_EXT.jsn defines: The extractr entity name surce_entity_name as the surce table name extractin_view: SQLs fr extracting data After yu cmplete this step, the extractr entity JSON file is defined. d. Create and define cp.jsn: "metadata_layut_versin": "1.0", "cntent_pack_name": "CUSTOMIZATION_PPM", "surce_prduct": "PPM", "versin": "1.0", "descriptin": "Prject Management & Prtfli Management Cntent Pack, Extend Entities.", "require": "platfrm": ">=1.0.0", "cp": [ "name": "CUSTOMIZATION_TARGET", "versin": ">=1.0.0" ], "streams": [ "name": "CUSTOMIZATION_TIME_SHEETS_STREAM" ], HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 31 f 56

32 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin "surce_entities": [ "name": "CUSTOMIZATION_TM_TIME_SHEETS" ], "extractin_entities": [ "name": "CUSTOMIZATION_TM_TIME_SHEETS_EXT" ] cp.jsn defines: The cntent pack name: CUSTOMIZATION_PPM Surce entities: Defined in CUSTOMIZATION_TM_TIME_SHEETS.jsn Stream entities: Defined in CUSTOMIZATION_TIME_SHEETS_STREAM.jsn Extract entities: Defined in CUSTOMIZATION_TM_TIME_SHEETS_EXT.jsn Deplying Cntent Packs T deply cntent packs, fllw these steps: 1. Place CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp under <VDW_ HOME>/Cntent. 2. Run the CntentManager.sh script under the <VDW_HOME>/bin directry t deply CUSTOMIZATION_TARGET.cp: sh CntentManager.sh --instructin install --cpname CUSTOMIZATION_TARGET; Yu can find the fllwing message frm CntentManager.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: The cntent f package CUSTOMIZATION_TARGET was successfully installed The DIM_ CUSTOMIZATION_CONTACTS table is generated in the Vertica database. 3. Run the CntentManager.sh script under the <VDW_HOME>/bin directry t deply CUSTOMIZATION_PPM.cp: sh CntentManager.sh --instructin install --cpname CUSTOMIZATION_PPM; Yu can find the fllwing message frm CntentManager.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: The cntent f package CUSTOMIZATION_PPM was successfully installed 4. Run the ExtractrEngine.sh script under the <VDW_HOME>/bin directry t extract data frm the PPM database t flat files: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 32 f 56

33 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin sh ExtractrEngine.sh --streamname CUSTOMIZATION_TIME_SHEETS_STREAM -- instancename <PPM_Instance_Name> Yu can find the fllwing message frm ExtractrEngine.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: Extractr was successfully executed. The BATCH ID is: <Batch_ID>. <PPM_Instance_Name> is the PPM instance name yu specified when installing the Vertica fr PPM Cntent Pack. 5. Run the FlwEngine.sh script under the <VDW_HOME>/bin directry t prcess ETL: sh FlwEngine.sh --batch <Batch_ID> --streamname CUSTOMIZATION_TIME_SHEETS_ STREAM --instancename <PPM_Instance_Name> Yu can find the fllwing message frm FlwEngine.lg under <VDW_HOME>/lgs if the cntent pack is deplyed successfully: ETL prcess was executed successfully <Batch_ID> used in this cmmand is the batch ID that was generated in Step Cnnect t the Vertica database and check whether the data has been laded successfully: select * frm <Target_Schema>.DIM_CUSTOMIZATION_TIME_SHEETS <Target_Schema> is the name fr the schema that cntains target data and tables fr reprting. Custmizing ETL Rules What is ETL Rules Extract-Transfrm-Lad (ETL) is the prcess f extracting data frm data surce, transfrming data, and lading data t the target data warehuse. ETL, as the cre f Business Intelligence, is a critical step fr deplying data warehuse. ETL cntains the fllwing steps: 1. EXT 2. SSI 3. XREF 4. MSI 5. XFR 6. KEY LOOKUP 7. TARGET 8. TSNP HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 33 f 56

34 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin 9. HIERARCHY 10. POST TARGET Fr details f these steps, see "ETL Steps Intrductin" n page 38. Why Custmize ETL Rules The PPM database includes SQLs that can prcess the ETL steps. At the same time, PPM als supprts ETL prcess custmizatin t meet varius business needs. Because the business envirnments vary, yu can ptimize the ETL perfrmance by custmizing ETL rules. Hw t Custmize ETL Rules Yu can custmize ETL rules by running the fllwing cmmand: sh ArtifactRegister.sh --artifactfile <DATA FILE PATH> --streamname <STREAM NAME> --etlstep <ETL STEP> --register CUSTOMIZATION Supprted Parameters Parameter Mandatry? Sample Value Descriptin streamname Yes PPM_PERSON_STREAM The name f the stream in which the ETL rules need t be custmized; See "Entity Attribute Descriptins" n page 50 fr details. etlstep Yes EXT SSI TSNP XREF MSI XFR KEYLOOKUP TARGET HIERARCHY POSTTARGET ETL step that needs t be custmized artifactfile Yes /temp/xref.sql SQL file that cntains custmized ETL lgic HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 34 f 56

35 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin Supprted Parameters, cntinued Parameter Mandatry? Sample Value Descriptin register N CUSTOMIZATION All nn-hp prvided SQL cmmands shuld be registered as CUSTOMIZATION help N Prints put the help message Fr detailed explanatin, see "Administratin Tasks" in the Vertica fr PPM Administratr's Guide fr Cntent Pack 1.0. Cusmizatin Validatin All the ETL rules are stred in the ETL_GENERATED_ARTIFACT table f the management schema. Yu can query custmized ETL rules by running the fllwing SQLs: SELECT * FROM <MANAGER_SCHEMA>.ETL_GENERATED_ARTIFACT,<METADATA_ SCHEMA>.ETL_METADATA where ETL_GENERATED_ARTIFACT.STREAM_ID = ETL_ METADATA.OBJECT_ID AND ETL_GENERATED_ARTIFACT.REGISTERED_BY='CUSTOMIZATION' AND ETL_METADATA.OBJECT_NAME='<STREAM_NAME>' <MANAGER_SCHEMA> is the name that yu specified fr the schema that cntains management tables when installing the Vertica fr PPM cntent pack. <METADATA_SCHEMA> is the name that yu specified fr the schema that cntains metadata when installing the Vertica fr PPM cntent pack. <STREAM_NAME> is the stream name that is specified when running ArtifactRegister.sh. Best Practices Custmizing ETL rules is a very cmplex prcess. Fllw these steps fr custmizatin: 1. Query the system with the fllwing cmmand t find ut the default rule fr a specified ETL step under a certain stream: SELECT ARTIFACT_CONTENT FROM <MANAGER_SCHEMA>.ETL_GENERATED_ ARTIFACT,<METADATA_SCHEMA>.ETL_METADATA where ETL_GENERATED_ ARTIFACT.STREAM_ID = ETL_METADATA.OBJECT_ID AND ETL_GENERATED_ ARTIFACT.REGISTERED_BY='SYSTEM' AND ETL_METADATA.OBJECT_NAME='<STREAM_ NAME>' AND ETL_GENERATED_ARTIFACT.ETL_STEP='<ETL_RULE_STEP_NAME>' <MANAGER_SCHEMA> is the name that yu specified fr the schema that cntains management tables when installing the Vertica fr PPM cntent pack. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 35 f 56

36 Vertica fr PPM Reprting Custmizatin Guide Chapter 2: Vertica fr PPM Reprts Custmizatin <METADATA_SCHEMA> is the name that yu specified fr the schema that cntains metadata when installing the Vertica fr PPM cntent pack. <STREAM_NAME> is the name f the stream that needs t be custmized. <ETL_RULE_STEP_NAME> is the ETL step that needs t be custmized. Pssible values are SSI, XREF, MSI, XFR, KEYLOOKUP, TARGET, HIERARCHY, TNSP, and POSTTARGET. 2. Cpy the SQLs frm the ARTIFACT_CONTENT field t a TXT file, such as data.txt. Adjust the SQLs and keep variables such as MD_BATCH_ID, MD_PROCESS_ID, and MD_SOURCE_ INSTANCE_ID as is. 3. Run the fllwing cmmand t custmize the ETL rule: Nte: sh ArtifactRegister.sh --artifactfile data.txt --streamname <STREAM_NAME> - -etlstep <ETEL_RULE_STEP_NAME> --register CUSTOMIZATION D nt change the metadata f OOTB entities. D nt add new fields t the existing staging r target tables. During the system upgrade, the system bypasses all custmized ETL artifacts. Smetimes it may cause ETL t break dwn. If that happens, crrect yur ETL SQLs and register again. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 36 f 56

37 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture Chapter 3: ETL Architecture The Vertica fr PPM cntent pack includes a flexible ETL Engine that can generate Vertica SQLs t prcess data based n metadata defined in the JSON frmat in cntent packs. Leveraging the flexible metadata and ETL Engine, yu can easily extended ETL Cntent t extract data frm mre PPM tables and stre them in the Vertica database fr later data cnsumptin. All cntent are packaged t cntent packs. Tw types f cntent packs are available: surce cntent pack and target cntent pack. Surce cntent pack includes surce entity metadata, extractin entity metadata, and stream definitins. Whereas target cntent pack nly cntains target entities. Fr explanatins f surce entity, target entity, extractin entity, and stream, see "Entity Attribute Descriptins" n page 50. After all metadata are develped, administratrs can use the Cntent Manager tl t deply that cntent pack. Fr detailed instructins n hw t deply a cntent pack, see Vertica fr PPM Administratr's Guide. After that, administratrs can schedule ETL jb with crntab t start data extractin. "ETL Architecture Overview" belw "ETL Steps Intrductin" n the next page ETL Architecture Overview Typically an ETL prcess cntains 10 steps. Flw Engine calls Template Engine t generate SQLs based n the ETL templates fr each entity during run time and Flw Engine executes thse 10 steps ne by ne. ETL templates cntains mst f the cmmn ETL patterns, s yu nly need t fcus n the business lgic when develping new cntent. In general, ETL templates supprt the fllwing ETL patterns: Change Data Capture ETL Engine captures nly change data based n the is_cdc attributes defined in the surce metadata. Deleted data can be captured based n the is_bk attributes defined in the surce metadata. Slw Changing Dimensin ETL Engine supprts slw changing dimensin. It autmatically keeps the dimensin histry if yu mark a field as scd2. Late Arriving Dimensin ETL Engine autmatically generates dummy recrd fr late arriving dimensin data, and updates the dummy recrd when data arrives. Surrgate Key Generatin HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 37 f 56

38 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture ETL Engine generates surrgate key autmatically. It generates the enterprise key t identify recrds and the primary key t identify the histry f recrds. Hierarchy Flatten Based n the target metadata definitin, ETL Engine stres hierarchy relatin infrmatin in the BRIDGE_HIEARCH tables. Unlimited hierarchy levels are supprted. Date Timezne Cnversin ETL Engine cnverts date t the data warehuse timezne autmatically. Yu need t specify the PPM database timezne and PPM data warehuse timezne during installatin. ETL Steps Intrductin The ETL prcess includes 10 steps as shwn in the fllwing figure. Fr detailed descriptin f each ETL step and the structure f key staging tables f that step, read the fllwing sectins. ETL Step 1: EXT EXT is the first step f ETL. During EXT, the staging table schema aligns with the surce entity schema. Flat files are input in this step, while *_EXT tables are utput. The EXT step des the fllwing: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 38 f 56

39 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture Lading data frm flat files t staging tables Flat files created by Extractr Engine are cpied frm flat files t staging tables (naming cnventin: *_SRC and *_DEL_SRC). Cleaning data with null/duplicated business key The system checks the data type and data length frm the flat files. The system als checks if the data field is null. It is required that the data is nt null. Rejected data is laded t the VALF (validatin fail) tables (naming cnventin: *_VALF). The VALF tables have the same table structures but all fields data type is VARCHAR (50000), which is t make sure that the data is nt rejected again. The system checks if there is duplicated data based n the is_bk attribute defined in the surce entity. Duplicated data is mved t the VALF tables as well. Generating hash cdes fr recrds The system generates a hash cde fr each recrd and puts the hash cde t the MD_HASH_ CODE field. The system use the hash cde t cmpare table fr data updates. Prefilling data frm snapsht tables t make sure that recrds can be jined in the SSI step. T reduce the lad n the PPM database, Extractr Engine nly extracts changed data. The system puts all data that was laded t snapsht tables (naming cnventin: *_TSNP) in the TSNP step. In the EXT step, the system pulls data back frm the snapsht tables, t make sure that all data can be jined in the SSI step. Checking fr updated, deleted, and inserted data Thrugh cmparing snapsht tables (naming cnventin: *_TSNP), the system tags data with UPD fr update and NEW fr insert. Extractr Engine extracts the business key fields fr all recrds frm the PPM database and cmpare it with the snapsht tables fr deleted recrds. The fllwing table lists the key staging tables. Key Staging Tables Table Name <STREAM_NAME>_<SOURCE_ENTITY_ NAME>_<MD_SOURCE_INSTANCE_ID>_ SRC <STREAM_NAME>_<SOURCE_ENTITY_ NAME>_<MD_SOURCE_INSTANCE_ID>_ SRC <STREAM_NAME>_<SOURCE_ENTITY_ NAME>_<MD_SOURCE_INSTANCE_ID>_ TSNP Table Descriptin Stres data laded frm flat files fr further prcess. Cntains all business key fields, used fr detecting deleted data. Cntains all data that was laded and the histry f recrds. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 39 f 56

40 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture Key Staging Tables, cntinued Table Name <STREAM_NAME>_<SOURCE_ENTITY_ NAME>_<MD_SOURCE_INSTANCE_ID>_ VALF <STREAM_NAME>_<SOURCE_ENTITY_ NAME>_<MD_SOURCE_INSTANCE_ID>_ EXT Table Descriptin Stres rejected data. Cntains data that has been prcessed in the EXT step and is ging t be passed t the SSI step. Mst f the staging tables in this step cntains the fllwing fields. System Reserved Fields in Staging Tables Field Name Field Descriptin MD_ BATCH_ID MD_ PROCESS_ ID MD_ SOURCE_ INSTANCE_ ID MD_FLAG MD_PF_ FLAG MD_ BUSINESS_ KEY MD_HASH_ CODE Data Batch ID. Extractr Engine puts data t flat files and gives a batch ID fr thse data. Administratrs can use this ID t track the data in the staging tables and the target tables, and als check Extractr Engine and Flw Engine lg files during trubleshting. ETL prcess ID. Flw Engine generates prcess ID and stres the ID in tables. Administratrs can use the ID t check the Flw Engine lg and the prcess status in the Derby database. Surce Instance ID. System assigns an ID fr PPM instance when the administratr registers a new PPM instance t the system. Vertica fr PPM users can use this ID t separate data frm different PPM instances. Indicates whether this rw is new, is updated, r needs t be deleted. Indicates whether rw is prefilled. P means the data is prefilled frm the snapsht tables. D means the data is laded frm flat files during the test lad. Primary key that is mapped t ne r mre surce clumns. The system cncatenates MD_SOURCE_INSTANCE_ID as part f MD_BUSINESS_KEY. Stres system-generated hash cdes. The system cmpares this field with the snapsht tables fr data update. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 40 f 56

41 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture ETL Step 2: SSI SSI is the nly step that des nt have any ETL templates. Flw Engine executes SQLs defined in the stream JSON file under the transfrms attribute. *_EXT tables are input in this step, while *_SSI tables are utput. Yu can custmize Vertica fr PPM reprts ttally in yur way in the SSI step. Hwever, yu need t understand the majr functins f this step: Handling business lgic This is the nly step where the entity-specific lgic can be handled. Yu can input SQLs fr handling the business lgic, such as calculatin and aggregatin. Transfrming data mdels frm the surce mdel (defined in surce entities) t the target mdel (defined in target mdels) The utput table f the SSI step (naming cnventin: *_SSI) shuld align with the target mdel in bth the field name and data granularity. Cleaning unnecessary prefilled data T imprve the ETL perfrmance, yu need t remve any unnecessary prefilled data. Yu can refer t the SQLs that cntain where tab1.md_pf_flag= D r tab2.md_flag= D in the sample ETL entities. Fr mre infrmatin abut btaining the sample cntent packs, see "Obtaining Sample Cntent Packs fr Custmizatin" n page 8. Cmbining multiple surce entities int ne target entity If yur target entity needs data frm multiple surce entities, cmbine these surce entities prperly t ppulate data t the *_SSI tables. Yu als need t pay attentin t the fllwing during the SSI develpment: MD_FLAG field MD_FLAG is a very imprtant field that cntrls the data lading behavirs in the fllwing ETL steps. Yu need t mark the recrd as NEW if the recrd is a newly created and UPD if the recrd already exists in the target tables. If yu mark the recrd as DEL, it means that recrd needs t be deleted later. Yu need t cnsider it thrughly when yu cmbine multiple surce entities. Data frm different entities carries different MD_FLAG frm the EXT step. Fields need t be ppulated in the *_SSI tables MD_BUSINESS_KEY: This recrd is used t identify the data granularity. The system generates the enterprise key later based n the value f MD_BUSINESS_KEY. Make sure this value is unique and is nt changed during the test lad. Recrds with different MD_ BUSINESS_KEY fields are cnsidered as different recrds. It is suggested that prefix <MD_ SOURCE_INSTANCE_ID> is included as part f MD_BUSINESS_KEY. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 41 f 56

42 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture MD_BATCH_ID: Yu can ppulate this field with the value frm the *_EXT table r the value frm the <MD_BATCH_ID> runtime variable. MD_SOURCE_INSTANCE_ID: Yu can ppulate this field with the value frm the *_EXT table r the value frm the <MD_SOURCE_INSTANCE_ID> runtime variable. MD_PROCESS_ID: Yu can ppulate this field with the value frm the *_EXT table r the value frm the <MD_PROCESS_ID> runtime variable. Temprary tables that yu create in the SSI step Cnsider the Vertica strage strategy t speed up ETL when yu run ETL n tp f the Vertica database cluster. Yu can use UNSEGMENTED ALL NODES fr small tables and SEGMENTED by hash (md_business_key) all ndes fr large tables in mst cases. Fr details abut Vertica strage impact n perfrmance, refer t HP Vertica Analytics Platfrm Versin 7.0.x Dcumentatin. The fllwing table lists the key staging tables. Key Staging Tables Table Name <SOURCE_PRODUCT_NAME>_ <SOURCE_ENTITY_NAME>_<MD_ SOURCE_INSTANCE_ID> SSI Table Descriptin SSI table is an utput table f the SSI step. It shall align with the target mdel n bth schema and data granularity The SSI table cntains the fllwing system reserved fields. System Reserved Fields in Staging Tables Field Name <ROLE_ ENTITY_ NAME>_ BUSINESS_ KEY Field Descriptin If yu define any entity lkup in the target entity in dimensin_assciated_ dimensin, fact_assciated_dimensin, r fact_assciated_fact attibutes, yu need t create a field in the SSI table, and put relevant business key values int this field. The name f this field shuld cntain <ROLE_ENTITY_NAME> defined in the target entity as the prefix. The value stred in this field shuld be the same as in MD_BUSINESS_KEY f the lkup_entity_name entity. Otherwise ETL Engine is nt able t build reference between entities in the fllwing steps. ETL Step 3: XREF In the XREF step, the system generates MD_ENTERPRISE_KEY fr recrds based n the value f MD_BUSINESS_KEY. Unlike many ther staging tables that are drpped and re-created every time when the ETL Jb runs, *_XREF tables (naming cnventin: *_XREF) are nt cleared. *_XREF tables maintain a mapping relatinship between MD_BUSINESS_KEY and MD_ENTERPRISE_KEY t make sure each MD_ENTERPISE_KEY is uniquely mapping t a MD_BUSINESS_KEY. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 42 f 56

43 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture The system als generates MD_ENTERPRISE_KEY fr ther reference entities if the referred recrd is nt arrived. This is a part f the Late Arriving Dimensin functin. *_SSI tables are input in this step, while *_XREF tables are utput. The fllwing table lists the key staging tables. Key Staging Tables Table Name <TARGET_ENTITY_ NAME>_ XREF Table Descriptin Cntains a mapping relatinship between MD_BUSINESS_KEY and MD_ENTERPRISE_KEY The XREF table cntains the fllwing system reserved fields. System Reserved Fields in Staging Tables Field Name Field Descriptin MD_BATCH_ID BUSINESS_ KEYVALUE ENTERPRISE_ KEY Fr late arriving data, the batch ID is set t -1. The system updates MD_ BATCH_ID when data arrives Stres business key values that are read frm MD_BUSINESS_KEY in the *_ SSI tables Stres generated sequence numbers ETL Step 4: MSI In the MSI step, the system btains the enterprise key generated frm MD_BUSINESS_KEY in the XREF step and stres it in the MSI step result tables (such as *_MSI tables). If the target entity has references t ther entities, it lks up the *_XREF tables f ther entities t btain the enterprise key f the referenced entities and stres it in the <ROLE_ENTITY_NAME>_ENTERPRISE_KEY field. The system als cnverts all date fields frm the PPM timezne t the data warehuse timezne. If the PPM timezne is enabled with daylight saving, daylight saving is als applied t the data warehuse timezne. Vertica fr PPM cntent pack leverages the daylight-saving infrmatin in the Java Runtime Envirnment (JRE). Thus yu need t update the JRE n a regular basis. During EXT, the hash cde fr surce entities has been generated. Because the mdel and data granularity has been changed in the SSI step, in the MSI step, the system regenerates the hash cde fr recrds. T verify if an SCD2 field has been updated, the system generates hash cdes fr all SCD1 fields and all SCD2 fields respectively. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 43 f 56

44 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture The system als checks the *_XREF tables t understand if dummy recrds need t be generated fr late arriving data. If the *_XREF table cntains recrds in which the value f MD_BATCH_ID is - 1, dummy recrds are generated. *_SSI tables and *_XREF tables are input in this step, while *_MSI tables are utput. The fllwing table lists the key staging tables. Key Staging Tables Table Name <SOURCE_PRODUCT_NAME>_ <TARGET_ENTITY_NAME>_<MD_ SOURCE_INSTANCE_ID>_MSI Table Descriptin Output table f the MSI step, including the enterprise key generated in the XREF step, and the dummy recrds fr late arriving data. The MSI table cntains the fllwing system reserved fields. System Reserved Fields in Staging Tables Field Name Field Descriptin MD_ ENTERPRISE_ KEY <ROLE_ ENTITY_ NAME>_ BUSINESS_ KEY <ROLE_ ENTITY_ NAME>_ ENTERPRISE_ KEY MD_HASH_ CODE_SCD1 MD_HASH_ CODE_SCD2 MD_HASH_ CODE Unique identifier fr a recrd. Generated base n MD_BUSINESS_KEY ppulated in the SSI step. Business key f entities that will be referred t. This shuld be exactly the same as MD_BUSINESS_KEY f the referred entity. Enterprise key f the entities that are referred t. The system autmatically ppulates this field thrugh lking up the *_ XREF table f the referred entity based n the value f <ROLE_ENTITY_NAME>_BUSINESS_KEY. Hash cde that is generated by the SCD1 clumns fr the dimensin table. Hash cde that is generated by the SCD2 clumns fr the dimensin table. Hash cde that is generated by all clumns fr the fact table. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 44 f 56

45 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture ETL Step 5: XFR In the XFR step, the system separates incming data t different staging tables based n MD_ FLAG. New recrds are put t the *_XFRN tables and deleted recrds are put t the *_XFRD tables. SCD1-updated recrds are put t the *_XFRU tables. Hwever, fr SCD2-updated recrds, a cpy f SCD2 histry recrds is stred in the *_XFRN tables. Later, the cpy is inserted t the target tables as the latest recrds f SCD2. A cpy is als stred in the *_XFRU2 tables. The system als verifies if a dummy recrd is available in the target tables because f the late arriving data. If s, even if incming data cntains the SCD2 changes, the system stres the recrd t the *_XFRU tables nly t make sure the dummy recrds are updated accrdingly. Fr fact entities, SCD2 is nt supprted, system puts all updated recrds in the *_XFRU tables. *_MSI tables are input in this step, while *_XFRN, *_XFRD, *_XFRU, and *_XFRU2 tables are utput. The fllwing table lists the key staging tables. Key Staging Tables Table Name <SOURCE_PRODUCT_ NAME>_<TARGET_ ENTITY_NAME>_<MD_ SOURCE_INSTANCE_ID>_ DIM/FACT_XFRN <SOURCE_PRODUCT_ NAME>_<TARGET_ ENTITY_NAME>_<MD_ SOURCE_INSTANCE_ID>_ DIM/FACT_XFRD <SOURCE_PRODUCT_ NAME>_<TARGET_ ENTITY_NAME>_<MD_ SOURCE_INSTANCE_ID>_ DIM/FACT_XFRU <SOURCE_PRODUCT_ NAME>_<TARGET_ ENTITY_NAME>_<MD_ SOURCE_INSTANCE_ID>_ DIM/FACT_XFRU2 Table Descriptin Cntains all data that is inserted int the target tables in the TARGET step. Cntains all data that is deleted in the target tables in the TARGET step. The system des nt physically delete the recrds. Instead, it updates MD_ACTIVESTATUSIND t N and MD_DELETEDDATE t reflect the deleted data. Cntains all data that is updated in the target tables in the TARGET step. Cntains all data that is updated in the target tables in the TARGET step. The *_XFU2 tables nly need t update MD_ TRANSLASTIND t N and MD_TRANSENDDATE t sysdate t reflect the last effective date f the SCD2 histry recrds. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 45 f 56

46 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture The XFR table cntains the fllwing system reserved fields. System Reserved Fields in Staging Tables Field Name MD_ CREATEDDATE MD_DELETEDDATE MD_ LASTMODDATE MD_ TRANSLASTIND MD_ TRANSENDDATE MD_ ACTIVESTATUSIND Field Descriptin The date when this recrd was created The date when this recrd was deleted The date when this recrd was updated mst recently Y: This is the latest recrd. N: This is a histry recrd. Applies t SCD2 histry recrds. Fr ther recrds withut histry, the value will be Y The last effective date f this recrd. Applies t the SCD2 histry recrds. Y: The recrd is active N: The recrd is deleted ETL Step 6: KEYLOOKUP In the KEYLOOKUP step, the system generates the primary key fr all new recrds; that is, recrds in the *_XFRN tables are based n the value f MD_ENTERPRISE_KEY. The system als generates the primary key fr late arriving data f referred entities. The mapping between MD_ENTERPRISE_ KEY and the primary key is ne t many because the system assigns a primary key t each histry recrd. *_XFRN tables are input in this step while *_ KEY_LOOKUP tables are utput. The fllwing table lists the key staging tables. Key Staging Tables Table Name <TARGET_ENTITY_NAME>_ DIM/FACT_KEY_LOOKUP Table Descriptin Cntains mapping between MD_ENTERPRISE_KEY and PK_<TARGET_ENTITY_NAME>. The KEY_LOOKUP table cntains the fllwing system reserved fields. System Reserved Fields in Staging Tables Field Name PK_<TARGET_ENTITY_NAME> Field Descriptin Generated primary keys fr new recrds HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 46 f 56

47 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture ETL Step 7: TARGET In the TARGET step, the system lads data t the target tables. It btains the primary key generated in the KEYLOOKUP step, inserts data frm the *_XFRN tables t the target tables, updates the target tables accrding t the data frm the *_XFRU and *_XFRU2 tables, and marks the recrds as deleted thrugh setting MD_ACTIVESTATUSIND t N and updating MD_ DELETEDATE as sysdate. If late arriving data in the previus batch arrives, the system updates MD_BATCH_ID in the *_ XREF tables with the current batch ID. *_XFRN, *_XFRD, *_XFRU, *_XFRU2, and *_KEY_LOOKUP tables are input in this step while DIM_*, FACT_*, r BRIDGE_* tables are utput. The fllwing table lists the key staging tables. Key Staging Tables Table Name DIM_<TARGET_ENTITY_ NAME> FACT_<TARGET_ENTITY_ NAME> BRIDGE_<TARGET_ ENTITY_NAME> Table Descriptin Target dimensin tables that include data fr reprting. Target fact tables that include data fr reprting. Target bridge tables that include data fr reprting f manyt-many relatinships. ETL Step 8: TSNP In the TSNP step, the system maintains snapsht tables. The system cpies all recrds frm *_EXT tables t *_TSNP tables t keep a snapsht. The *_TSNP tables are used as a surce fr prefilled data and als fr Change Data Capture. The system als updates the LAST_EXTRACTION_VALUE field f the DATA_SOURCE_CDC_ STAMP table. Extract Engine extracts data frm the PPM database incrementally based n the value f this field. The *_TSNP tables track the change histry f the PPM data as well. *_EXT tables are input in this step while *_ TSNP tables are utput. The fllwing table lists the key staging tables. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 47 f 56

48 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture Key Staging Tables Table Name <SOURCE_PRODUCT>_<TARGET_ ENTITY_NAME>_<MD_SOURCE_ INSTANCE_ID>_TSNP DATA_SOURCE_CDC_STAMP Table Descriptin Snapsht tables that cntain data extracted frm PPM. It cntains bth the latest data and the histry data. A table in the management schema. Extractr Engine relies n this field t extract data incrementally. ETL Step 9: HIERARCHY If a dimensin entity has hierarchy definitin in the target entity JSON, the system creates a hierarchy table in the HIERARCHY step. If n hierarchy is defined, this step is skipped. The hierarchy table supprts unlimited hierarchy levels. The histry f hierarchy changes is als kept fr future reference. DIM_* tables are input in this step while BRIDGE_HIERARCHY_* tables are utput. The fllwing table lists the key staging table. Key Staging Table Table Name BRIDGE_HIERARCHY_<TARGET_ENTITY_ NAME> Table Descriptin Stres the dimensin hierarchy level infrmatin ETL Step 10: POSTTARGET The bjectives fr the POSTTARGET step are: T create views n tp f the dimensin and fact tables, s that Vertica fr PPM reprting users can understand the data easily. T pen a windw fr cntent engineers t perfrm any analytical functins after all data are laded int the target tables. The system defines templates fr creating views. Vertica fr PPM creates the fllwing types f views based n the table types and metadata defined in the target table: Hierarchy Drill UP View /Hierarchy Drill Dwn View Thse tw views are created fr Vertica fr PPMreprting users, s they d nt have t cmbine BRIDGE_HIEARCHY tables by themselves. If ne dimensin entity has hierarchy defined, the system autmatically creates thse tw views. Users can select Hierarchy Drill UP View if they want t knw all parent recrds and Hierarchy Drill Dwn View if they want t knw all child recrds. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 48 f 56

49 Vertica fr PPM Reprting Custmizatin Guide Chapter 3: ETL Architecture User Data View The system creates User Data View nly if a dynamic_view attribute is defined in the target JSON file. The system leverages the cntext infrmatin defined in PPM t prvide USER_ DATA fields in PPM with a meaningful name in the views. If a dimensin entity has bth bridge and dynamic_view defined, the system generates the cnslidated view autmatically. Cmmn View Fr all ther target tables, the system als creates a view layer. Extended entities can be jined with the OOTB entity thrugh the view layer fr better a user experience. In the stream JSON definitin, yu can als define the pst_target_transfrms attribute. Fr examples, see OOTB PPM_RESOURCE_DEMAND_STREAM.jsn (under <VDW_ HOME>/Cntent/CUSTOMIZATION_PPM.cp/dwmetadata/streams). All SQL-defined pst_target_transfrms attributes are executed in this step as well. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 49 f 56

50 Vertica fr PPM Reprting Custmizatin Guide Appendix A: Entity Attribute Descriptins Appendix A: Entity Attribute Descriptins This sectin lists detailed explanatins fr each attribute f the surce entity, target entity, extractr entity, stream entity, and CP JSON files. Fr detailed file structures, refer t "Cntent Pack Structure " n page 9. The fllwing table shws the attribute descriptins f the surce entity JSON file. Surce Entity JSON File Field metadata_layut_versin versin cntent_pack surce_prduct entity_name entity_business_name entity_descriptin schema attribute attribute_name attribute_business_name attribute_descriptin sql_data_type size is_bk is_cdc Descriptin Versin f the metadata frmat Versin f the entity metadata Cntent pack name. Shuld be the same as cntent_pack_ name defined in cp.jsn Surce prduct Surce entity name Business name f the entity Surce entity descriptin Target entity attribute array Surce table field that needs t be extracted Attribute name. Serves as a field name in the staging tables Attribute business name Surce table field descriptin Field data type Field size The value is true if the attribute serves as a business key The value is ture if data is incrementally extracted. In mst cases, this attribute is used fr LAST_UPDATE_DATE. If it is nt specified as true, Extractr Engine extracts all recrds everytime. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 50 f 56

51 Vertica fr PPM Reprting Custmizatin Guide Appendix A: Entity Attribute Descriptins Surce Entity JSON File, cntinued Field is_required clumn_sequence Descriptin The value is ture if this field is nt null The clumn sequence when creating staging tables The fllwing table shws the attribute descriptins f the target entity JSON file. Target Entity JSON file Field metadata_layut_versin versin cntent_pack entity_name entity_business_name entity_descriptin dimensin dimensin_business_name is_cnfrmed dimensin_type strage_strategy segmented_by partitin_by fact fact_business_name fact_descriptin fact_type Descriptin Versin f the metadata frmat Versin f this entity metadata Cntent pack name. Shuld be the same as cntent_pack_ name defined in cp.jsn Target entity name Business name f the entity Target entity descriptin Included fr a target dimensin entity Dimensin business name Whether this dimensin is cnfrmed; fr future usage Hierarchy r Primary ; use Hierarchy if yu want t define hierarchical structure fr this entity Includes segmented_by and partitin_by Used fr table segmentatin; specify Default if yu want t cpy data f this entity t all Vertica clusters Used fr table partitin Included if this is a target fact entity Business name f the fact Descriptin f the fact entity ACCUMULATED is the nly supprted value HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 51 f 56

52 Vertica fr PPM Reprting Custmizatin Guide Appendix A: Entity Attribute Descriptins Target Entity JSON file, cntinued Field schema attribute attribute_name attribute_business_name attribute_descriptin Descriptin Target entity attribute array Name f this attribute Target entity attribute name Business name f this attribute Descriptin f this attribute attribute_type If this is target fact entity, the value shuld be measure ; therwise the value is dimensin. scd target_data_type size is_required dimensin_assciated_ dimensin fact_assciatin_dimensin fact_assciatin_fact scd1: Update the recrd scd2: Insert a new recrd Target entity attribute type Target entity attribute size The value is ture if this field is nt null Defines the dimensin if this dimensin refers t ther dimensins Defines the fact if this fact refers t ther dimensins Defines the fact table if this fact table refers t ther fact tables Nte: Late arriving fact tables are nt supprted. Make sure the referred fact table is laded prir t this fact table. lkup_entity_name rle_entity_name rle_entity_business_name descriptin Lkup entity name Alias fr the referred entity Business name fr the referred entity Descriptin f this lkup The fllwing table shws the attribute descriptins f the extractr entity JSON file. HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 52 f 56

53 Vertica fr PPM Reprting Custmizatin Guide Appendix A: Entity Attribute Descriptins Extractr Entity JSON file Field metadata_layut_versin versin surce_prduct cntent_pack entity_name surce_entity_name extractr extractin extractin_view surce_prduct_versin Descriptin Versin f the metadata frmat Versin f this entity metadata Surce prduct Cntent pack name. Shuld be the same as cntent_pack_ name defined in cp.jsn Entity name Surce table name The extractr that is called t extract data. Currently OracleDBExtractr is supprted. Cntains the extractin attributes An n-the-fly view frm which extractr extracts data frm. The versin f the surce prduct that the extractr extracts frm The fllwing table shws the attribute descriptins f the stream entity JSON file. Stream Entity JSON file Field Descriptin metadata_ layut_versin versin cntent_pack surce_prduct stream_name surce_ entities_ includes Versin f the metadata frmat Versin f this entity metadata Cntent pack name. Shuld be the same as cntent_pack_name defined in cp.jsn Surce prduct Stream entity name Defines the surce entity f the stream HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 53 f 56

54 Vertica fr PPM Reprting Custmizatin Guide Appendix A: Entity Attribute Descriptins Stream Entity JSON file, cntinued Field Descriptin surce_entity_ include target_entities_ includes target_entity_ include transfrms sql pst_target_ transfrms sql The surce entity name f the stream Defines the target entity f the stream The target entity name f the stream Cntains the attribute related t SQL transfrmatin Queries that transfrm surce entities t target entities Optinal. If yu want t prcess ther SQLs after data is laded t target tables, include SQLs in this attribute. Includes SQL queries that is executed as part f POSTTARGET step The fllwing table shws the attribute descriptins f the CP JSON file. CP JSON File Field metadata_layut_ versin cntent_pack_name versin descriptin require platfrm target_entities name streams name Descriptin Versin f the metadata frmat Cntent pack name Versin f this cntent pack Descriptin f this cntent pack Dependency f this cntent pack Platfrm versin that requires t supprt this cntent pack Target entity sectin. Lists all target entities that yu defined in this cntent pack Target entity name Stream entity sectin; lists all stream entities that yu defined in this cntent pack Stream entity name HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 54 f 56

55 Vertica fr PPM Reprting Custmizatin Guide Appendix A: Entity Attribute Descriptins CP JSON File, cntinued Field surce_entities name extractin_entities name Descriptin Surce entity sectin; lists all surce entities yu defined in this cntent pack Surce entity name Extractr entity sectin; lists all extractin entities yu defined in this cntent pack Extractr entity name HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 55 f 56

56 Send Us Feedback Let us knw hw we can imprve yur experience with the Vertica fr PPM Reprting Custmizatin Guide. Send yur t: HPE Prject and Prtfli Management Center (Cntent Pack 1.0) Page 56 f 56