Getting Information Out of the Informatica Repository. William Flood, ETL Team Lead Charles Schwab

Transcription

1 1

2 Getting Information Out of the Informatica Repository William Flood, ETL Team Lead Charles Schwab 2

3 My Background About Charles Schwab Life at Schwab 3

4 Presentation Agenda Three Ways to Query Informatica Repository Object Queries MetaQuery Repository SQL Mass Update of Sessions Resource Going Forward 4

5 Object Queries 5

6 What is an Object Query A PowerCenter feature to query objects within a repository Repository Manager Mapping Designer Workflow Manager Permission Shared Private Is available without versioning in version 8 6

7 Object Query Options Query Parameters Query Conditions 7

8 Object Query Note Always add the parameter Reusable Status Set the value to Non-Reusable, Reusable By default it only returns Reusable objects 8

9 Object Query Demo Create a Query Set parameters of the Query Run the Query Review results 9

10 Object Query Usages And / Or Logic 10

11 Object Query Suggestions Create as many queries as possible to search for coding standard issues Allow developers to run the coding standard checking queries Makes queries to look for checked out / in objects in non-development folders Use object queries to find objects that may change when a table changes Always verify your results 11

12 Object Query Pros and Cons Pros + Easy to use and share with the team + Interact with the objects directly from the tool + Informatica maintains during upgrades + Results can be exported to a file Cons - Locked in to defined parameters and conditions - Doesn t go down to the transformation level - Query results not always 100% correct 12

13 Object Query Usages Find all objects labeled for a Deployment Group Find deleted objects you want to recover Find all invalidated or impacted objects Find code that isn t meeting company s coding standards Find code checked in the last 24 hours 13

14 MetaQuery 14

15 MetaQuery Metaquery a free GUI tool to query the repository Connects via JDBC to the following database: DB Sybase Oracle SQL Server Output can be exported into excel or text file JRE 1.5 or higher Any version of Windows except Vista PowerCenter version: 8.6 and above 15

16 MetaQuery - Examples Folder Permissions Application Connection DB Connection variable Source File Directory and Source File Name Target File Directory and Target File Name Reject File Directory and Reject File Name Session Instance General Options Maximum Memory allowed Maximum Percentage of total memory allowed Additional Concurrent Pipelines Pre build lookup cache Pushdown Optimization Session Parameter File Teradata Error Table1/Error Table 2 Dynamic Partitioning Session Run Status Session Throughput and load Treat Source rows as Many others 16

17 MetaQuery Logins from windows with JDBC connection 17

18 MetaQuery 18

19 MetaQuery 19

20 MetaQuery Demo 20

21 MetaQuery 21

22 MetaQuery 22

23 MetaQuery Pros + Easy to use and more options then Object Queries + Results can be exported to a file Cons - Limited number of defined queries and conditions - Not made for a versioned repository - Query results not always 100% correct 23

24 MetaQuery Usages Find the Permissions on the Folders See Commands Tasks Values See Lookup and Source SQL Get Repository SQL 24

25 Repository SQL 25

26 MetaQuery Usages MX (Rep) Views Guaranteed to work with upgrades Over 60 views provided by Informatica 26

27 Repository SQL Only run select statements, never updates or deletes Column Decode: Subject_Area = Folder Name Widget_Type_Name = Transformation Name Field_Name = Port_Name Is_visable = 1 really means it the current version If there is no is_visable flag then you must do the max of the version number 27

28 Repository SQL How to Get SQL Look at the SQL in the rep views Monitor the SQL run when using MetaQuery.class files - \Metaquery 3.0\XML Ask Informatica support Always Make Sure SQL: Only pulls the current version of an object. Works with Reusable and Non-Reusable Objects Create views from SQL Create those views in a different database schema 28

29 Repository SQL Pros + Most options, most flexible + Can tie to reporting tools, daily reports Cons - Queries can become invalid if repository changes during upgrades - Most complex to work with - Mx View not always 100% right 29

30 Repository SQL Usages Validate Mapping Standards Validate Parameter Files Settings Validate Logs Paths and Names Validate Targets Have PK Keys Defined 30

31 Pmrep Massupdate 31

32 Pmrep Massupdate Updates session properties: Update all sessions in a folder Update specific sessions based off a list of sessions Update only when a criteria is met Run in test mode (-m) Output from the command is: Folder name Number of sessions that are successfully updated or failed Names of the sessions that are updated Checks in with a blank comment 32

33 Pmrep Massupdate Rules and Guidelines It can update reusable and non-reusable sessions It cannot update sessions that are checked out It cannot update sessions in frozen folders If the Repository Service process node has limited memory, disable repository agent caching before you run MassUpdate or restart the Repository Service after you run MassUpdate 33

34 Pmrep Massupdate 34

35 Pmrep Massupdate pmrep massupdate -t <session_property_type (session_property, session_config_property, transformation_instance_attribute, session_instance_runtime_option)> -n <session_property_name> -v <session_property_value> [-w <transformation_type>] {-i <persistent_input_file> -f <folder_name> } [-o <condition_operator (equal, unequal, less, greater)>] [-l <condition_value>] [-g <update_session_instance_flag>] [-m <test_mode>] [-u <output_log_file_name>] 35

36 Pmrep Massupdate Update specific sessions Create file for massupdate to use create an object query and execute it pmrep executequery -q <Object Query Name>' -u '<Input File Name> pmrep massupdate -t 'session_property' -n 'Session Log File directory' -v '$PMSessionLogDir\' -i '<Input File Name>' m File Format: null,<folder Name>,<Workflow Name>.<Session Name>,session,none,14,non-reusable Update only when a criteria is met pmrep massupdate -t 'session_config_property' -n 'Stop on errors' -v '18' -i <Input File Name>' -o 'unequal' -l '1' Run update with a condition against a folder pmrep massupdate -t 'session_config_property' -n 'Stop on errors' -v 1' -f <Folder Name>' -o 'unequal' -l '1 36

37 Pmrep Massupdate /10/ :12:57 ** Saving... Repository test_ver_mu, Folder cases_auto Validating the flow semantics of Workflow wf_non_reusable_test_ff......flow semantics validation completed with no errors. Validating tasks of Workflow wf_non_reusable_test_ff......workflow wf_non_reusable_test_ff tasks validation completed with no errors. Workflow wf_non_reusable_test_ff updated. Checking-in saved objects...done Massupdate Summary: Number of reusable sessions that are successfully updated: 1. Number of non-reusable sessions that are successfully updated: 1. Number of session instances that are successfully updated: 0. Number of reusable sessions that fail to be updated: 0. Number of non-reusable sessions that fail to be updated: 0. Number of session instances that fail to be updated: 0. 37

38 Pmrep Massupdate 38

39 Pmrep Massupdate Demo 39

40 Resource 40

41 BinaryBolt.com List of Repository SQL Mapping Standard Validation All Ports are Connected Check all Deployment Groups for Overlapping Objects Open for all to: Contribute SQL Comment Find SQL Searchable Currently SQL is all in Oracle Goal: Collect SQL to find all settings in Informatica 41

42 BinaryBolt.com 42

43 BinaryBolt.com 43

44 BinaryBolt.com 44

45 BinaryBolt.com 45

46 BinaryBolt.com 46

47 BinaryBolt.com 47

48 BinaryBolt.com Search Tip: Workflow: Properties Tab: Workflow Log File Name 48

49 Questions? 49