Lab 2: Audit the PARTS table

Transcription

1 Lab 2: Audit the PARTS table In this lab you create a new process. This process will carry out an audit the PARTS table that will be based upon the results of your profiling. Create Reference Data Holding Valid PARTID Patterns During our profiling, we noted that the PARTID was expressed in a number of inconsistent formats: Let s assume that we have discussed this with the relevant business people, and they have told us that all PARTIDs should consist of 5 alpha-numeric characters. The alpha and numeric characters can be in any order, but any pattern that has more or less than 5 characters has an invalid format. Patterns that include non-alpha-numeric characters, such as whitespaces or punctuation, are also invalid. The most foolproof way to perform this kind of check might be by using the Regex Check processor. But for the purposes of training, we will keep things simple (although less foolproof) by using the Pattern Check processor. The Pattern Check processor works by comparing the patterns it finds to a set of valid patterns stored in reference data. So our first job will be to create this reference data. 1. In the Director UI, return to your Profile the PARTS table process, and, in the Canvas, single-click the Patterns profiler. Its results should be displayed in the Results Browser. 2. In the Results Browser, select the PARTID tab. The Results Browser should now display the various patterns that were found in the PARTID attribute. 3. In the Results Browser, select the six exclusively alpha-numeric patterns that have a length of 5 characters (you can select multiple patterns by holding down the Ctrl key on your keyboard whilst clicking). Page 1 of 16

2 4. Right-click any of the highlighted patterns and select Create Reference Data 5. The New Reference Data dialog will open. Click Next >, and select Pattern as the Lookup Column. 6. Click Next >, and then click Next > again without selecting a return column. Page 2 of 16

3 7. Click Next > again without selecting a category, and Name your reference data Valid PARTID Formats. Then click Finish. 8. The Reference Data Editor will open, displaying the six patterns that you selected in step 3. Your Valid PARTID Formats reference data will consist of these six patterns. Click OK. Create Reference Data Holding Valid UMCODEs During our profiling, we noted that the UMCODE attribute had a number of distinct values: Let s assume that we have discussed this with the relevant business people, and they have told us Page 3 of 16

4 that the only valid UMCODEs are PC, KT and PK. All records must have a UMCODE, and all other UMCODES are invalid. We will check the validity of UMCODEs by using the List Check processor. The List Check processor works by comparing the values it finds to a set of values stored in reference data. So our first job will be to create this reference data. 1. Return to your Profile the PARTS table process, and, in the canvas, single-click the Quick Stats profiler. Its results should be displayed in the Results Browser. 2. In the Results Browser, drill-down on the Distinct Values for UMCODE attribute. The Results Browser should now display the various distinct values that were found in the UMCODE attribute. 3. In the Results Browser, select these three values: PC, KT and PK. (you can select multiple patterns by holding down the Ctrl key on your keyboard whilst clicking). 4. Right-click any of the highlighted values and select Create Reference Data 5. The New Reference Data dialog will open. Click Next >, and select UMCODE as the Lookup Column. Page 4 of 16

5 6. Click Next >, and then click Next > again without selecting a return column. 7. Click Next > again without selecting a category, and Name your reference data Valid UM Codes. 8. Click Finish. 9. The Reference Data Editor will open, displaying the three values that you selected in step 3. Your Valid UMCODES reference data will consist of these three values. Click OK. Page 5 of 16

6 Create a New Process Now that you have created your reference data, you can create a process to audit the data from the PARTS table. This process will check for a valid format in the PARTID attribute, and for a valid value in the UMCODE attribute. Records that fail these checks will be tagged with error messages. Since the process will ultimately publish alerts to Case Management you will call it Audit and Publish Errors to Case Management. However, in this lab you will only go as far as identifying the errors. The first part of the process will look like this: 1. In the Director UI, within your Case Management Training project, right-click the Processes node in the Project Browser, and select New Process Page 6 of 16

7 2. The New Process dialog will open. Select your PARTS Table snapshot as the Data Source, and click Next >. 3. Click Next > again without selecting Add Profiling. 4. Name your process Audit and Publish Errors to Case Management, and then click Finish. The new process will open on the canvas, and a Reader will be displayed: Check the Format of the PARTID 1. In the Tool Palette, on the right-side of the screen, click to select the Audit family of processors, and then drag the Pattern Check processor onto the Canvas, dropping it a little below and to the right of the reader. Page 7 of 16

8 2. Wire up the Reader s output port to the Pattern Check processor s input port. 3. The Pattern Check dialog will open. Select PARTID as the Field for Validation, and then navigate to the Options tab. 4. In the Valid Patterns box, click to the right of Reference Data. The Select Resource dialog will open. De-select the Filter by Category checkbox. 5. Select your Valid PARTID Formats reference data and then click OK. 6. In the Categorize unmatched as drop down, select Invalid. 7. Click OK to close the Pattern Check dialog. 8. On the canvas, double click the words Pattern Check, and rename the processor Check for Valid Format in PARTID attribute (Pattern Check). Page 8 of 16

9 Add an Error Message for Invalid PARTID Formats 1. In the Tool Palette, on the right-side of the screen, click to select the Transformation family of processors, and then drag the Add String Attribute processor onto the Canvas, dropping it a little below and to the right of the Pattern Check processor that you renamed Check for Valid Format in PARTID attribute (Pattern Check). 2. Wire up the Pattern Check processor s Invalid port to the Add String Attribute processor s input port. Page 9 of 16

10 3. The Add String Attribute dialog will open. In the Output Attributes box, on the right-side of the dialog, double-click NewString and rename it Error Message:PARTID. Error Message:PARTID will be the name of the new string attribute you are going to add. 4. Navigate to the Options tab and enter a New Attribute Value of Invalid PARTID Format. Invalid PARTID Format is the value that will be inserted into the Error Message:PARTID attribute. 5. Click OK to close the dialog. Page 10 of 16

11 6. On the canvas, double click the words Add String Attribute, and rename the processor Add Error Message for PARTID - Invalid Format (Add String Attribute). Check the Values in the UMCODE Attribute 1. In the Tool Palette, on the right-side of the screen, click to select the Audit family of processors, and then drag the List Check processor onto the Canvas, dropping it a little to the right of the Check for Valid Format in PARTID attribute (Pattern Check) processor. 2. Wire up the Check for Valid Format in PARTID attribute (Pattern Check) processor s All port to the List Check processor s input port. Page 11 of 16

12 3. The List Check dialog will open. Select UMCODE as the Field for Validation, and then navigate to the Options tab. 4. In the Valid values box, click to the right of Reference Data. The Select Resource dialog will open. 5. Select your Valid UM Codes reference data and then click OK. 6. In the Categorize unmatched as drop down, select Invalid. Leave all other options set to their default values. 7. Click OK to close the List Check dialog. 8. On the canvas, double click the words List Check, and rename the processor Check for Valid Value in UMCODE attribute (List Check). Page 12 of 16

13 Add an Error Message for Invalid UMCODEs 1. In the Tool Palette, on the right-side of the screen, click to select the Transformation family of processors, and then drag the Add String Attribute processor onto the Canvas, dropping it a little below and to the right of the List Check processor that you renamed Check for Valid Value in UMCODE attribute (List Check). 2. Wire up the List Check processor s Invalid port to the Add String Attribute processor s input port. Page 13 of 16

14 3. The Add String Attribute dialog will open. In the Output Attributes box, on the right-side of the dialog, double-click NewString and rename it Error Message:UMCODE. Error Message:UMCODE will be the name of the new string attribute you are going to add. Page 14 of 16

15 4. Navigate to the Options tab and enter a New Attribute Value of Invalid Value in UMCODE. Invalid Value in UMCODE is the value that will be inserted into the Error Message:UMCODE attribute. 5. Click OK to close the dialog. 6. On the canvas, double click the words Add String Attribute, and rename the processor Add Error Message for UMCODE - Invalid Value (Add String Attribute). Run Your Process and Examine the Results 1. In the toolbar at the top of the canvas, click to run your new process. 2. In the canvas, single-click each of the processors in turn and examine their results in the Results Browser. You should find that 112 records have an invalid PARTID format and that 10 records have an invalid UMCODE. Single-clicking the Add Error Message Page 15 of 16

16 processors should confirm that the relevant error message has been added to each of the records that failed the two checks. Page 16 of 16