Oracle Enterprise Data Quality Business Rules

Transcription

1 Oracle Enterprise Data Quality Business Rules Release

2 Table of Contents Lab 1: Import and Run a Partially Configured Set of Business Rules... 4 Lab 2: Copy and Modify Existing Rules Lab 3: Add New Rules Lab 4: Frequency Profile the Errors Lab 5: Optional Extension Lab Conditional Rules Lab 6: Optional Extension Lab Incorporate the Business Rules Check Processor in a Wider Process 19 Page 2 of 19

3 Page 3 of 19

4 Lab 1: Import and Run a Partially Configured Set of Business Rules During the labs in this module, you will work with a partially configured set of business rules. Your objective is to examine the existing rules, and then to adapt and extend them. In this lab, you will copy a partially-configured business rules spreadsheet into your EDQ instance s businessrules folder and examine its contents. You will then import a package (a.dxi file) that contains a partially configured Business Rules process. You will run the process and examine its results. Copy a Partially Configured Business Rules Spreadsheet to Your Business Rules Folder Note that the business-rules.xls spreadsheet, which is provided specially to complement this training module, contains a number of working business rules. Business rules spreadsheets must be in the.xls format. They can have any name provided that it ends in.xls, and they must be placed in your EDQ instance s businessrules folder. 1. Copy the business-rules.xls spreadsheet from C:\share\edq_training_assets_12.1.3\Data Files\Business Rules to your EDQ instance s businessrules folder. The EDQ Trn2 virtual machine s businessrules folder is located at /apps/oracle/fmw/user_projects/domains/edq_domain/edq/oedq.local.home/ businessrules. The virtual machine s desktop includes both a shortucut to the share folder and a shortcut to oedq.local.home, so one way to copy the spreadsheet is simply to double click the shortcut to oedq.local.home, open its businessrules folder and then drag the.xls file from one place to the other: Page 4 of 19

5 If you have mapped Windows drives from your host machine, then you could instead copy the files using Windows Explorer on your host machine. For anybody not using the EDQ _Trn2 virtual machine: note that both the businessrules directory is a sub-directory of the oedq.local.home directory. The location of oedq.local.home can vary, depending on your machine s operating system and the type of installation you have carried out. On a Windows 7 machine, the default location of oedq.local.home is: C:\ProgramData\Oracle\Enterprise Data Quality\oedq_local_home. One some Linux virtual machines (though not on the EDQ Trn2 virtual machine) the oedq_local_home directory may be located at /opt/oracle/middleware/edq/config.local. 2. On your host machine, copy the business-rules.xls spreadsheet from C:\share\edq_training_assets_12.1.3\Data Files\Business Rules to the root of the C:\share folder. When you need to make changes to the spreadsheet, you will make them in the copy, and you will still have the original version available should anything go wrong. 3. Open the business-rules.xls spreadsheet from the C:\share folder on your host machine and examine its contents. Answer the following questions: Page 5 of 19

6 1. Navigate to the Rules worksheet. Which attribute does Rule1 apply to, and what type of check does it perform? 2. Rule 19 produces an error message of Title is not valid. What type of check does it perform, and how many rows of reference data does this check use? 3. Rule 5 checks for noise characters. What condition must be met before Rule 5 s check is applied? If its conditions are met, what type of check does rule 5 perform? 4. What four values would cause a record to fail with the error message First Name includes suspect data? 5. What type of check is chdateprehistoric and what will it check for (hint: the error message for the rule that the check is associated with will give you a clue)? Import a Package 1. Open the Director user interface and log in as the dnadmin user (in the EDQ Trn2 virtual machine, the dnadmin user s password is also dnadmin). 2. Right-click at the bottom of the Project Browser and select Open Package File or alternatively follow the menu path File >> Open Package File Page 6 of 19

7 3. Navigate to the file Business Rules.dxi which you will find in C:\share\edq_training_assets_12.1.3\Data Files\Business Rules. Select the file and click Open. A folder named Business Rules.dxi should appear at the bottom of the Project Browser. 4. Drag the Projects node from beneath the Business Rules.dxi folder and drop it on top of the Projects node directly below localhost (localhost is your server name). Page 7 of 19

8 5. A Business Rules project should appear in the Project Browser. Configure the Processor 1. Expand the Business Rules node within the Project Browser, and within the Business Rules project, expand the Processes node. Double click the Check Data process to open it on the Process Canvas. 2. Double click the Business Rules Check processor to open the Business Rules Check dialog. Page 8 of 19

9 3. Navigate to the Options tab and, if necessary, in the XLS File Configuration section, select a File Name of business-rules.xls. 4. Navigate to the Identify tab and, if necessary, select Input Attributes as displayed below: 5. Click OK to close the dialog. Run the Process and Study the Results 1. Click to run the Check Data process. You can see the process running in Director s Tasks window. Page 9 of 19

10 2. Once the Check Data process has completed the Tasks window should become empty. Single Click the Business Rules Check processor in the Process Canvas. You should see its results in the Results Browser. 3. Interrogate the results, drilling-down where necessary to answer the following questions: 1. Why did the record for Mr Carols fail? 2. What errors caused the record for Professor Bill Adams to fail? 3. Which record failed because of suspect data? 4. What error caused the record for Mr Paul Humphries to fail? Page 10 of 19

11 Lab 2: Copy and Modify Existing Rules In this lab you will copy the First Name business rules and modify them to apply to the last name. You will then run the Check Data process and examine its results. For the purposes of training, we are going to assume that we can apply the same set of business rules to first names and last names. Since the checks themselves are not associated with particular attributes, we can to do this by simply copying and modifying existing rules and conditions, renaming them, pointing them at different attributes, and so on. In the real-world, you might want to apply different business rules to first and last names (for example, you might want to list check first names against a list of known valid first names for your locale, but you are unlikely to be able to take this approach with last names). Copy and Modify the First Name Business Rules 1. On your host machine, use Windows Explorer or another file browser to navigate to the copy of the business-rules.xls file that you placed in the machine s shared folder (C:\share, if you followed the instructions in the Overview module). 2. Open the copy of the business-rules.xls spreadsheet in Microsoft Excel or a similar spreadsheet application. If you have any other copy of the same file open, then make sure that you close it. Otherwise you may get confused. 3. Navigate to the Rules worksheet. 4. Copy rules 2 to 9 and paste them into rows 12-19, which are currently empty. 5. In rows 12 to 19: a) Amend the Rule IDs to Rule11-Rule18 respectively. b) Amend each rule label so that references to the first name, where present, are removed, and references to the last name are introduced. c) In the Apply to Attribute column, change the value to atsecondname. d) Where you see the Condition cofirstname_populated change it to cosecondname_populated. e) Where you see the Condition comultiples change it to cosecondnamemulitples. f) Amend the Error Codes so that they are all unique values. g) Amend the Error Messages so that they reference the last name rather than the first name (e.g. change First Name is too short to Last Name is too short.) The Rules worksheet should now look similar to the screenshot below: Page 11 of 19

12 Copy and Modify the First Name Conditions 1. Navigate to the Conditions worksheet. 2. Copy the cofirstname_populated condition and paste it into row 5 of the worksheet, which should currently be empty. 3. In row 5: a) Change the Condition Name to cosecondname_populated. b) Change the Attribute or Check to atsecondname. 4. Copy the Multiples condition and paste it into row 6 of the worksheet, which should currently be empty. 5. In row 6: a) Change the Condition Name to cosecondnamemultiples. b) Change the Attribute or Check to atsecondname. The Conditions worksheet should now look similar to the screenshot below: 6. Save the business-rules.xls spreadsheet. If you are using a recent version of Microsoft Excel, ensure that you save the spreadsheet as an Excel Workbook with a.xls extension. Do not save it with a.xlsx extension, as it will not be recognized by the Business Rules Check processor. Page 12 of 19

13 7. Copy the business-rules.xls spreadsheet from your host s shared folder to the EDQ Trn2 virtual machine s businessrules folder, overwriting the file of the same name that is already there. Remember that the EDQ Trn2 virtual machine s businessrules folder is located at /apps/oracle/fmw/user_projects/domains/edq_domain/edq/oedq.l ocal.home/businessrules. Shortcuts to oedq.local.home and to the share folder are provided on the EDQ Trn2 virtual machine s desktop. Run the Process and Study the Results 1. Navigate back to the Director interface and click to run the Check Data process. You can see the process running in Director s Tasks window. 2. Once the Check Data process has completed the Tasks window should become empty. Single Click the Business Rules Check processor in the Process Canvas. You should see its results open in the Results Browser. 3. Interrogate the results, drilling-down where necessary to answer the following questions: Page 13 of 19

14 1. Look through the error messages, and confirm that the new Last Name Business Rules that you have added appear to be working. 2. How many of the records that fail include errors relating to the last name? Page 14 of 19

15 Lab 3: Add New Rules In this lab you will add two new rules to the business-rules.xls spreadsheet. The first new rule will check whether the Gender attribute is populated with a valid value. During this part of the lab, you will be given step-by-step instructions telling you exactly what to do. The second new rule will check whether the Account Type attribute is populated with a valid value. Since this rule is set up in a very similar fashion to the first, you will not be given step-by-step instructions, but you can refer back to the instructions for setting up the Gender check rule if necessary. After each set of changes, you will run the Check Data process and examine its results. Add a New Rule to Check if the Gender Has a Valid Value 1. Open the copy of the business-rules.xls spreadsheet on your host machine s shared folder (if you followed the instructions in the Overview module, the shared folder will be located at C:\share). 2. Navigate to the Rules worksheet. 3. In the first available empty row create a new rule, copying in the values from the screenshot below: 4. Navigate to the Checks worksheet, and, in the first available empty row, create a new check, copying in the values from the screenshot below: 5. Create a new worksheet called Genders. Populate cells A1 and A2 with the values M and F respectively, as displayed in the screenshot below: 6. Save the business-rules.xls spreadsheet. Page 15 of 19

16 7. Copy the business-rules.xls spreadsheet from your host s shared folder to the EDQ Trn2 virtual machine s businessrules folder, overwriting the file of the same name that is already there. Remember that the EDQ Trn2 virtual machine s businessrules folder is located at /apps/oracle/fmw/user_projects/domains/edq_domain/edq/oedq.l ocal.home/businessrules. Shortcuts to oedq.local.home and to the share folder are provided on the EDQ Trn2 virtual machine s desktop. 8. In the Director interface, Double click the Business Rules Check processor in the Process Canvas. The Business Rules Check dialog opens. 9. Navigate to the Identify tab and map the Gender Input Attribute to the atgender Identifier. 10. Click OK to close the dialog. 11. Run the Check Data process and examine its results, answering the following questions How many records that fail include the error message that the Gender is not valid? (Note: if your answer is zero, then there is an error in your business rule). Add a New Rule to Check if the Account Type attribute has a valid value The process for setting up this business rule is very similar to that for setting up the Gender check business rule. In both cases, you need to use a List Check. Detailed instructions are therefore not provided, but you can refer back to the instructions for adding the Gender check if necessary. 1. Add a business rule and an associated check. Records should pass the business rule if their Account Type matches one of these two values: a) Checking b) Deposit 2. Once you have configured your new business rule, run the Check Data process and examine its results, answering the following questions How many records that fail include the error message that the Account Type is not valid? (Note: if your answer is zero, then there is an error in your business rule). Page 16 of 19

17 Lab 4: Frequency Profile the Errors In this lab you will add a Frequency Profiler to gain an understanding of how often each error occurs. To do this you will frequency profile the Error Message attribute. Since this attribute is a string array that can be populated by several distinct values, you will use the Split Records from Array processor to create a separate row for each combination of customer record and error message (in other words, each error found will appear in its own row). And you will pass the resulting rows of data into the Frequency Profiler. Split Records From Array 1. In Director drag the Split Records from Array processor from the Tool Palette onto the Process Canvas (you will find the Split Records from Array processor in the Transformation family). 2. Wire the input of the Split Records from Array processor to the Business Rules Check processor s Fail port. The Split Records from Array dialog opens. We are using the output from the Business Rules Check processor s Fail port because only records that fail will include error messages. 3. Click to select the three available stringarray attributes (ErrorCodes, ErrorSeverities and ErrorMessages). 4. Click OK to close the dialog. 5. Run the process and then examine its results in the Results Browser. How many input and output records do you have? Drill down on the output records. Examine the First Name and Last Name attributes. Can you confirm that many of your input records have several output records? Frequency Profile the Errors 1. In Director drag the Frequency Profiler from the Tool Palette onto the Process Canvas (you will find the Frequency Profiler in the Profiling family). 2. Wire the input of the Frequency Profiler to the Split Records from Array processor s All port. The Frequency Profiler dialog opens. We are using the output from the Split Record from Array processor s All port because we want to examine the errors generated by all the input records, not just those that generated multiple errors and which were therefore split. 3. Select the ErrorMessages.Split attribute and click OK to close the dialog. 4. Run the process and then examine its results in the Results Browser. Which error occurred most frequently? Page 17 of 19

18 Lab 5: Optional Extension Lab Conditional Rules In this lab you will add three new rules. The first should fail records where the address is populated, but not in a valid format. For example, it should fail addresses like davebarnard@.com or paulfisher.snomail.com. It should pass records that are either unpopulated or which are populated with a value in the correct format. Your rule will require a condition to be set, and you will need to create a check with a check type of RegEx. Your second new rule should fail records where the Account Type is set to Deposit and the balance is negative (i.e. is less than zero). This rule will also require you to create a condition as well as checks. Your third new rule should fail records where the Account Type is set to Checking and the balance is not within the range to (inclusive). Again, a condition will be required as well as some checks. Since this is an extension lab, detailed instructions for how to proceed are not provided. Instead, you should examine the business-rules.xls spreadsheet for examples of similar rules that may help you, and consult Oracle Enterprise Data Quality s online help. If the Address is Populated, Check Whether it is in a Valid Format 1. Construct a business rule to fail records where the address field is populated, but not with a value in a valid format. If you look in the Options tab of EDQ s Check processor, you may be able to find some RegEx that you can reuse in your business rule check. Check Whether Deposit Account Balances Have a Positive Value 1. Construct a business rule to fail records where the Account Type is set to Deposit and the balance is negative (i.e. is less than zero). Check Whether Checking Account Balances are within a Valid Range 1. Construct a business rule to fail records where the Account Type is set to Checking and the balance is not within the range to (inclusive). You can assume that all balances are integers, and that there are no leading zeros. If you examine the existing rules in the business-rules.xls spreadsheet, you may find a rule that performs a similar check on a different field. Page 18 of 19

19 Lab 6: Optional Extension Lab Incorporate the Business Rules Check Processor in a Wider Process In this lab you will create a wider process in order to enable the remediation of errors that are identified by the Business Rules Check processor. Using the same business rules that you have developed in previous labs, your process should: 1. Determine which errors can be automatically fixed within Oracle Enterprise Data Quality, and which will require human intervention to fix. 2. Fix the errors that can be automatically resolved. 3. Write clean records to a set of staged data called Clean Data. 4. Write records that require human intervention to a set of staged data called Human Intervention Required. Since this is an extension lab, detailed instructions for how to proceed are not provided. Page 19 of 19