THE STATE OF DATA QUALITY

Transcription

1 THE STATE OF DATA QUALITY

2 HOW TO START A DATA QUALITY PROGRAM Organizations claim to make decisions based on data, but are they really? Ask a few simple questions of the data they are using for those presumed decisions and you ll quickly see confidence wane as doubts about data quality and accuracy emerge. In recent years, so much emphasis (and money) has been placed on the visualization of data, that we ve created a culture that encourages us to brush data quality issues under the rug for the sake of meeting report deadlines. In the rush to show benefit and value from the expensive data warehouses, marts, cubes and business intelligence (BI) solutions we ve implemented, we tend to glance over data quality issues at each stage of the analysis, development, and implementation. There is often no immediate consequence of skirting data quality. BI programs have grown to be so big that the individuals driving innovation and production are not directly responsible for the accuracy of BI outputs, only the vision; and the vision is looking pretty good these days. Our new dashboards and analytic applications are receiving high praise, so high in fact that business users start asking for more and more, faster and faster. We don t have the time to deal with data quality issues in today s pace of business, so we do what we do best. We take in lots of data quickly, store it efficiently, summarize it well, and serve it up effectively for reporting. The business is getting their insights, we re getting thumbs up for timeliness and delivery, but deep down WE KNOW THERE ARE DATA QUALITY ISSUES We know there are data quality issues - values are out of bounds, relationships are missing, sequences of events don t look right. If we stopped to deal with them, we would compromise our BI program delivery dates. Our counterparts who manage the enterprise and line of business applications seem to ignore when we question the data they send to our warehouse, so who are we to be the stewards of perfection? We are in a conscious state of quality ignorance. We know better than to engage the business application owners without the cooperation of the technology owners, so we use our best judgment and make decisions in our analysis. We compromise. We adjust our Extract Transform Load (ETL) code. We map objects, conform values, set defaults, decide if NULL means Yes or No. We clean up data. We find a way to create value, or so WE think. We make compromises so that we can assist business users in publishing new insights across the enterprise, insights that excite! We start to hear the buzz, directly and indirectly, about how good some of these new data discoveries really are but the buzz is usually shortlived. Summaries don t add up to other reports, alignment to categories and hierarchies aren t matching - and where did these categories come from anyway? Overall percentages and distributions don t seem to be reflecting the feel of reality in the business. Confidence in our BI program is eroding quickly. What happened? The rug was pulled out from underneath us thanks to those small data quality issues we tried to hide under it

3 TECHNICALLY, ANARCHY IS A FORM OF GOVERNANCE Information needs to be managed. Whether you are a technical expert or business professional, we all know this and have witnessed the poor results of not managing data and content in a strategic and controlled manner. But who is responsible for telling us which data means what, whether it is accurate, or if there is enough confidence in it to drive decisions? The business tends to think that IT owns all data and is responsible for its accuracy, completeness, timeliness and overall quality. However, IT is often simply the provider of the applications and infrastructure to support the business in managing their own data. Discussions about data and its issues are generally dealt with in functional silos, within functional application teams and their business sponsors and owners. At the edges, where data is required from one silo to another, methods of delivery and transfer are not standardized, not well tested, and capture of changes that are important may or may not be handled properly. There are some data quality aspects that IT manages well enough. There is some management around the application portfolio -- there must be, this is how we budget most funds, after all. There is an infrastructure team that looks after the networks, servers, and everything telecom and internet -- another big chunk of the budget. There is an architecture team that ensures applications meet architecture and design principles and standards -- little budget, little influence, and no real focus on data architecture. Governance is thought to be about change management policy and adapting behaviors to reach desired states, but governance is about responsibilities, accountabilities and decision-making authority. Data governance is a framework that allows us to take appropriate and sanctioned actions on a routine basis to adhere to data quality standards. Consistent governance is especially important in data to ensure issues are being regularly addressed. There is a constant backlog of issues arising out of the insights we ve published for business users - to go along with that spreadsheet we kept about the issues we uncovered when building the application. We ve got issues, real data quality issues, but who is responsible for these? Who is accountable for correcting the data so that our reports provide more accurate insights? Who acts when action is needed? Too often, the answer is no one. There is a sense of anarchy in BI programs that cause an unhealthy tension between IT and business units when questions arise around data quality and how we should handle them - and both parties decline authority. There is some semblance of IT governance, but we don t know how well it works with corporate governance functions. In fact, we rarely hear about this stuff. When we do hear about governance it is usually in terms of new or changing procedures, maybe some new standards or guidelines

4 DEPLOYING A DATA QUALITY WORKING GROUP The only way to overcome anarchy is collaboration and structure. There must be a responsible party. We need some sort of data quality program or a committee to submit issues to for these issues to be resolved. We know this program needs some governance around it to be sustainable, but which comes first? A data governance program or a data quality program? DATA QUALITY OR DATA GOVERNANCE. WHICH COMES FIRST? The answer is that the order doesn t really matter, as long as they are not far apart. It is simply important that you should not do one without the other. However, both data quality and data governance require executive sponsorship and a budget to establish the program and make it sustainable, which takes time, influence, proof, and convincing others it is a high enough priority to get funded. It doesn t mean we shouldn t start those conversations, but what can we do within the business intelligence program to improve the quality and ownership of data now? Manage the issues. Issues related to IT initiatives, regardless of whether related to data or not, surface from within projects or from the business. During analysis and design, we raise issues. During testing, Quality Assurance (QA) raises defects; simply another form of an issue. During User Acceptance Testing (UAT) business users find defects in data or gaps in understanding of rules. All are issues that will eventually require action and therefore need attention no more hiding our concerns under the rug. Managing data quality issues should be approached in the same way that we manage any set of issues in business. There must be a dedicated authority committed to issue management and a cross-functional team to manage processes and control. We call this a Data Quality Working Group. It is important to clarify that this group is not a BI Competency Center (BICC) or a Center of Excellence (CoE), though data quality working groups often report to these authorities. Even if we already have a team within our BI program or Project Management Office (PMO) organization that fields questions related to data accessibility and value, it is best to create a new group specifically tasked with maintenance of data quality issues. The formality of this group is a function of how you manage and operate your BI program. Put enough procedure and process around a working group to keep the issues prioritized and information flowing outward while allowing for flexibility and agility in execution. In general, we have an issue management process that deals with reporting and recording an issue, assessing the issue, and then resolving it. Whatever framework you employ for other issue management, apply it for data quality issues, too. There s no need to reinvent the wheel just because these issues feel new. The makeup and reporting structure of a data quality working group varies from organization to organization, which is why it is so important to develop your own framework for data quality and governance as opposed to trying to implement a boilerplate model. In general, the working group does not need many members, but should include at least one representative who works on the enterprise glossary and metadata, and various data analysts from the BI projects. It may be wise to include a data modeler, a Database Administrator (DBA), and representatives from QA, but again this will vary from organization to organization. Data quality working groups often start as an IT function reporting into a BICC if there is one, then drives outward, growing more diverse as time goes on

5 DATA QUALITY ISSUE MANAGAMENT RECORDING ISSUES Now that we have a data quality working group assembled and ownership of data quality issues contained, we need a place to record issues that arise. Data quality issues originate from multiple sources, all of which should be aggregated and presented collectively to the working group. This allows us to reduce bias and prioritize within the context of all known data quality issues, not from within the silos from which they originated. Most organizations have a system to record issues and capture defects, maybe even a ticketing system. Use whatever current issue management system your company employs to get known data quality issues recorded again, don t reinvent the wheel. If issues are already recorded somewhere, make sure there is an easy method for identifying them as data quality issues. PRIORITIZING ISSUES Prioritization of data quality issues should not be left solely to the interests of the working group, but should align to business priorities as closely as possible. The working group won t naturally prioritize issues in the same order and manner as the business would, so it is best to involve business stakeholders to help prioritize specific issues. Often, the business leaders needed for this type of decision making are already stakeholders associated with BI initiatives through formalized BI Centers of Excellence and Competency Centers. Under the umbrella of a BI program and its charter, the working group can and should take initiative to manage data quality issues across departments, working closely with BICC s and stakeholders for direction and prioritization. Make sure the working group meets regularly to keep the focus on issues at the top of the list, as we all know business priorities can frequently change. Capture standard issue items such as a title, date reported, description, the project, priority, risk, desired resolution date, etc., as well as a few other items unique to a data quality issues: data lineage, primary and secondary applications impacted, and whether the issue seems to be intermittent, recurring or was only observed as one-time occurrence. ASSESSING ISSUES We have a prioritized queue of issues, now it s time to start assessing the problems. There are well-established methods of assessing data quality issues, the key is finding a good and lean process to put it in place within the scope of our BI program. In general, you will find data analysts, QA testers, and developers already do quite a bit of this type of assessment work. We simply need to be a bit more formality about driving their efforts and capturing their findings. Based on initial assessment, we set impact and severity, assign the issue to someone to track and manage it, add notes, and the like. Since these are data quality issues, we add things like data lineage and identify the business data steward or Subject Matter Expert (SME). We attach scripts used in assessing the issue and keep snapshots of data used in our assessment

6 RESOLVING DATA QUALITY ISSUES - AT LEAST AS FAR AS YOU CAN Here comes the challenging part, the part of our BI program that no one wanted to claim ownership of: fixing data quality issues. It s important to look across the quality issues from a more strategic and holistic viewpoint. Although teams will try to solve issues on their own, data quality issues often should be surfaced so more strategic actions can be taken, usually to go along with acceptable tactical solutions that our teams devise. The working group should thoroughly document any recommended actions for the business. Don t just document for due diligence, but make practical recommendations for those who do have the authority or influence over business units to implement change. Much of our corrective data quality actions will be performed within our BI program. And since we ve identified suitable business stewards and SMEs when forming the data quality working group, we should have taken their input into account in the actions we are recommending the BI program perform to correct these data quality issues. Recommended actions are often in line with what IT is accustomed to doing, and generally corrective in nature: new mappings and transforms within our ETL. We can implement some corrective measures from within the BI program, but this will not remediate all data quality issues. There are likely some preventive actions we need to make visible to the business, and hopefully, we have the procedures and forums with which to do so. Without formal data governance controls and information management procedures, it is very difficult to get others outside of a BI program to act. Focusing data quality issues through a working group helps to build the credibility of the data quality concerns and better equips us to recommend corrective actions that the business needs to take. To better encourage action, expose how data quality issues will impact the business through data quality assessments. Producing real business examples of data quality impact will help to gain momentum around remediating business-focused data quality issues. Now that we ve taken the time to look at the issues across our program more strategically, remediated any IT-oriented challenges and provided the business with other recommendations, we can implement the solutions devised, and trust that we have agreement from the business to act as well. DID WE RESOLVE DATA QUALITY? Data quality will never be fully resolved, but it can be controlled. Data quality working groups act as a data authority, identifying issues, prioritizing action and deploying solutions. Our BI programs now have the start of a data quality program, but to really control these issues, we need a long-term quality solution that reduces the frequency and allows us to act sooner. Just as the size and makeup of data quality working groups will vary from company to company, no two data quality programs should look alike. There is no cookie-cutter solution to data quality. You need to develop your own custom data quality framework to effectively manage data across the enterprise. If your organization is ready to reduce data quality issues and implement a solution, contact CCG at (813) to speak with a Consultant about your custom framework