Date Mining: Applications for Compliance and Internal Auditing. John A. Beattie, CPA, CFE Partner Larry Kanter, CPA, CFF, CFE Partner.

Size: px

Start display at page:

Download "Date Mining: Applications for Compliance and Internal Auditing. John A. Beattie, CPA, CFE Partner Larry Kanter, CPA, CFF, CFE Partner."

Allan Whitehead
5 years ago
Views:

1 Date Mining: Applications for Compliance and Internal Auditing John A. Beattie, CPA, CFE Partner Larry Kanter, CPA, CFF, CFE Partner Syllabus I. Data Mining Defined, Its Benefits, Relationship to the Audit Process and Uses II. III. The Data Mining Process More on Readying Data, Validation and Examples IV. Conclusion 1

2 Part I Data Mining Defined, Its Benefits, Relationship to the Audit Cycle and Uses What is Data Mining? The practice of searching through large amounts of computerized data to find useful patterns or trends Merriam-Webster Online Dictionary The search for hidden information: the locating of previously unknown patterns and relationships within data using a database application Encarta World English Dictionary[North American Edition] 2

3 What is Data Mining? Generally, data mining (sometimes called data or knowledge discovery) is the process of analyzing data from different perspectives and summarizing it into useful information - information that can be used to increase revenue, cuts costs, or both. Data mining software is one of a number of analytical tools for analyzing data. It allows users to analyze data from many different dimensions or angles, categorize it, and summarize the relationships identified. Technically, data mining is the process of finding correlations or patterns among dozens of fields in large relational databases. Jason Frand, PhD, UCLA, Anderson School of Management What is Data Mining? Our working definition and the basis for this presentation. Data Mining should be thought of as a generic term: It can be as simple as extracting one column of data from an Excel Spreadsheet. It can be as complicated as sorting through millions or billions of seemingly unrelated records to develop a pattern, gain an understanding of data attributes or form conclusions based on actual information. 3

4 Data Mining Is Here Data Mining is the future. It is one of the hottest topics in graduate applied statistics and business management programs. Its use as a tool to illuminate operations has arrived. But, interestingly, it is neither new nor a fad. Auditors, Decision Support Specialists and Fraud Examiners have been using matrix analysis and other data mining techniques for years. It is not in itself the keys to the kingdom, but it can be a treasure map when used by skillful navigators. John Beattie, CPA, CFE Presentation to the Healthcare Financial Management Association, Delaware Valley Healthcare Council, June 2005 Data Mining Benefits Automate Testing Test All Transactions Leverage Different Data Sources Enhances Audit Effectiveness Can Integrate Analytics Into New Processes 4

5 Data Mining And The Audit Process Plan (Exploratory, Prior Audit Reports, Focus on Area 1 (Red) of Risk Assessments) Prepare (Identify files, tables and analytics, build scripts, queries, structured queries) Test Execute (Validating results is critical) Review - (Validate again, Save scripts for improved efficiency) Report Use graphs and other illustrative techniques to complement narrative Comparative Reporting of CPT Code Groupings Surgical Pathology Profile % 80.00% 60.00% 40.00% 20.00% 0.00% National State Provider 5

6 Analytics Ad Hoc Created for a specific purpose Repetitive Periodically executed, managed from a secure site. (Task Scheduler in Windows allows scripts to run at a certain point in time.) Continuous Always on Create a separate working folder for each audit or investigation. Application Examples for Internal Audit and Compliance Use the Chart of Accounts to Map Areas and Trial Balance To Identify Unusual Fluctuations, Patterns and Trends Supply Chain and Procurement Payroll and Fringe Benefits (Sick Leave by employee and department with Frequency and Day of Week, Pay Rates Test % Increase and Job Code ) User IS Access Rights Accounts Payable (Master Vendor Files, Rebates, Price Differential on Same Item [Kickback Symptom can be Higher Prices] ) Accounts Receivable Patient Financial Services and Health Information Management Fixed Assets Cash Handling (Ex. Dietary) 6

7 Application Examples for Internal Audit and Compliance Cash Duplicate Payments PFS and HIM Duplicate Medical Record Numbers, Ineligible Dependants Date of Birth Check (Continual Flag), Sanctions Checks (Excluded Providers to Master Employee File), Drug Utilization by Department and Clinic Quality Measurements of Blood Pressure and Pain Assessments, Time series ER Patient Flow, Physician Credentials to Surgeries and Procedures Decision Support - Scheduling and Wait Times in Clinics (Managed Care v. FFS), Bed Turnaround Times, Discharge Times (Submission Time) Payroll Use of Electronic Swipe Cards Swipe Location and Time to Job s Physical Location Date and Time Analysis Example Functions returns 1 if a given date falls between Monday and Friday and 0 if the date falls between Saturday and returns the day of the Converts date to a Julian returns 0 if time if in the AM, 1 if in the PM and -1 for an invalid Calculates the number of days between two returns a number indicating whether a date s value falls within a specified range (1) and (0) if - returns a number indicating whether a time s value falls within a specified range (1) and (0) if not 7

8 Part II The Data Mining Process Data Mining Process Data Mining can be narrowed down to five steps: Locating and Understanding the Data Data Acquisition Verification and Migration Database Design & Development Data Analysis 8

9 Locating and Understanding Data Become involved in the process early; Meet with knowledgeable people to gain an understanding of the data issues; Develop a plan in very specific terms of what to ask for and the form in which it is produced. Ask : Is this the best file? Scripts are used to convert data; Be certain that you request a data dictionary for each set of data (File Layout Understand Structure of the Data); Assist with the preparation of document requests as they relate to data. Data Acquisition The data must be obtained in a form that is easily useable; May require conversion so as not to be in a proprietary software; May require the producing party to restore from back-up; Be intimately involved in the process to insure that the data is received efficiently, accurately and in the proper form; Privacy and protection issues must be addressed adequately; 9

10 Verification and Migration Once it is received, gain a thorough understanding of the data: Does the production contain exactly what we asked for? Is it in the format the we requested? What is in each field? The data is loaded into a database program such as SQL Server; The fields are then analyzed and matched to the data dictionary (in some cases this step is performed prior to loading the data into the database). Database Design and Development Once each field present in each dataset is known, the mapping process begins: Mapping is the first step in linking data from different sources together into one database. Verify record counts against the original data; Insure the integrity of the data. 10

11 Data Analysis Develop queries and reports that transform the data into meaningful information. This is the heart of the process; Verify the queries, verify the queries, verify the queries; Perform a devils advocate analysis of the results; Perform a gut check; Verify the queries again. Part III More on Readying Data, Validation and Examples 11

12 Readying Data The queries are only as good as the source data Often more time is spent building tables and making certain you have the right data than on analysis. Social Security Number can have many different formats: /99/9999 Use a function such as Map(string, format) Map(SSN, ) to determine how many different formats. Need to make SSN look the same. Insert - twice. Insert(string,insert text, location) Insert(Insert(Include(SSN, ), -,4) do same for count 6. (Unconditional expression) = Count left to right Readying Data Programs are flexible enough to accept a variety of data formats, including: XML, Excel, Access Oracle, comma delimited, report image file. Is the record structure consistent? How many lines in a record? Trapping or Anchoring Character a location in the source file that allows the data mining program to find each record. Open Database Connectivity Allows you to connect to a live data source Ask: Do you have Open Database Connectivity to your data? Can you get access? (It is read only access.) 12

13 Splitting Names (ACL) Readying Data John_Doe to Doe or John A Doe Segment is 2 in first example and 3 in second. Username is column header. Example cont d.: Readying Data Split(String, separator, segment) Split(username,,, 1) 13

14 Data Validation Once the data is imported, it should be validated to ensure queries are performed using a complete data set. Types of Validation: Fields contained expected data. Date fields are populated with dates Dates are within the defined period Check for empty fields Fields expected to contain positive numbers are positive Fields expected to contain certain data lengths are populated with that data Length (Include(ssn, )) <>9 (this is an example of a Conditional Expression) Determine if leading zero dropped. Functions can be used to build the data so you can use it. Duplicate entries Gap analysis Data Validation Verify function Check for empty fields Verify date range Command: VERIFY FIELDS Svc_Date ERRORLIMIT 10 IF Svc_Date < ` ` TO SCREEN - This command identifies records where the Service Date is less than April 15, Results are output to the screen at 10/screen. 14

15 Gap analysis Data Validation Command: GAPS ON Svc_Date TO SCREEN This command identifies gaps within the data listed in the Service Date field and outputs the results to the screen. The two most common data problems are blanks and type mismatches. Data Profiling Many data techniques search for specific types of known exceptions in the data. Advanced analysis techniques, such as data profiling, search for unsuspected symptoms. Fields can be analyzed to identify the minimum, maximum, average, highest, and lowest 10 values. Most often recorded amounts (charges) Range of charges Most often used procedures, HCPCS codes, or ICD-9-CM codes. 15

16 Data Mining Example 1 Sort DRG volume in descending order and identify high risk DRGs. Command: CLASSIFY ON DRG TO "DRG 2008.FIL" This command produces a file that sorts, counts, and provides a percent contribution to the total for the different DRGs within the data set. The count column can then be sorted in descending order to identify the high volume DRGs. Payments could have been used instead of volume to identify those DRGs with the highest reimbursement contribution. Data Mining Example 1 The table can then be compared to a list of high risk DRGs. JOIN PKEY DRG IF DRG = 238 ISOLOCALE root This command uses the JOIN command to match the identified DRGs in the current table to a predetermined list of high risk DRGs. In this example, DRG 238 was used for the match. Once identified, the DRG 2008.FIL table allows one to choose all the records with a specific DRG. 16

17 Data Mining Example 1 Charges for each case can be compared to the average and exceptions identified for review. Criteria for exceptions may be claims with charges that are >2 SD above the average. Further mining can be used to identify whether exceptions are related to certain NPIs or associated ICD-9-CM coding patterns. Data Mining Example 2 Vulnerable procedures Certain procedures are expected to be performed on an outpatient basis. Data can be queried to identify a population of one-day stays with procedures expected to be performed on an outpatient basis. Depending on availability of data inpatient one-day stays can be identified based on the admit and discharge dates. Disch_Date - Adm_Date <= 1 This command identifies records where the difference between the discharge and admission dates is 1 or less. 17

18 Data Mining Example 2 The one-day stay population can now be filtered for high risk DRGs or procedure codes. The results of these filters can be subjected to further classification by NPI to identify high volume offenders. A Ratio Analysis of reimbursement to charges can be applied to assess efficiency. Data Mining Example 3 Pharmaceutical agents are developed to address specific diseases or conditions. Manufacturers publish recommended dosages and conditions for these agents in their package inserts. Pharmaceuticals can be cross walked to HCPCS codes, dosages converted to associated units, and diseases/conditions to ICD-9-CM codes. 18

19 Data Mining Example 3 A table is created containing HCPCS codes, maximum units, and associated ICD-9-CM codes. (Benchmark Table) Code is scripted to look for HCPCS codes in the client data. Once identified, each record is compared to the Benchmark Table and exceptions output to a separate table. Cross-Tabulation Distinct values from one field, such as a family of DRGs provide heading for the columns fo the table. (x-axis) Values from a second field, such as NPIs can provide heading for the rows. (y-axis) The result can profile physician DRG assignments. 19

20 GROUP ACL Script EXPORT ASCII to xtab.wsp X_NAMES + comp EXPORT ASCII to xtab.wsp NUM_FIELD + If Xtab_Field = + INCLUDE(%X_axis%), 251, 252,253 ),, _ ),1,3) + EXPORT ASCII to xtab_wsp 0.00 END OPEN Xtab_Temp ACTIVATE Xtab SUMMARIZE ON %y_axis% ACC ALL TO %outfile% PRESORT OPEN Conclusion Confidence Through Clarity John A. Beattie, CPA, CFE Partner, Healthcare Consulting Group Direct: Mobile: John.Beattie@ParenteBeard.com Larry Kanter, CPA, CFF, CFE Partner, Forensics and Litigation Group Direct: Mobile: Larry.Kanter@ParenteBeard.com 20

DO NOT SEND DUPLICATE COPIES OF YOUR LOG AND DO NOT SEND A PRINTED COPY.

DO NOT SEND DUPLICATE COPIES OF YOUR LOG AND DO NOT SEND A PRINTED COPY. AMERICAN BOARD OF UROLOGY 2018 LIFE LONG LEARNING (LLL) LEVEL 2 PEDIATRIC UROLOGY SUBSPECIALTY CERTIFICATION EXAMINATION PROCESS INSTRUCTIONS FOR SUBMISSION OF ELECTRONIC LOGS Please read all instructions