Data quality and data governance are both indispensable for organizations that want to become Data Intelligent enterprises. Data quality and data governance are certainly related, but they are two separate disciplines. Often, organizations purchase a data quality tool hoping that it will solve their issues with data accuracy and trust. However, organizations need data governance first to create the foundation for enterprise-scale data quality. In this post, I’ll discuss the differences between the two and why organizations need data governance to improve their data quality.
The difference between data quality and data governance
Data quality describes the accuracy, completeness, and consistency of data. Organizations need high quality data that they can trust so they can make important decisions. Without high quality data, an organization cannot become data driven because they cannot trust their data. This lack of trust means the organization cannot use their data to make impactful business decisions, which can lead to inefficiency, missed opportunities, and ultimately, financial loss.
Data governance is a collection of practices and processes to standardize and automate the management and use of data within an organization; it’s a system of authority and control over the management of data assets. Data governance provides a framework for collaboration through a shared language;. teammates within and across departments can communicate using the same terminology and analyze the same data. Additionally, clarifying roles and responsibilities eliminates confusion and makes data processes and collaboration easy to follow.
How do these strategies overlap?
To put it simply, you can’t have data quality without good data governance. Organizations need proper data governance before they should even consider a separate enterprise-scale data quality tool.
Organizations use data governance for a number of reasons. Data governance impacts security, privacy, accuracy, compliance, roles and responsibilities, management, integration, and more. Organizations use data governance to
- Increase transparency around data, its use, its availability, and its management
- Standardize data systems, policies, and procedures
- Resolve data issues
- Ensure regulatory and organizational compliance
All of these pursuits are required tasks for improving and monitoring data quality.
Successfully incorporating data quality and data governance
So where should a data team start if it wants to improve data quality and data governance? Organizations have thousands and thousands of different data elements. Which ones should they focus on? Which ones could be left out of scope? Which ones are making the greatest impact on the business and should be managed first?
Data teams can address these questions from two different angles:
- Critical data elements: identify what is critical for the business; this could be a regulatory report, a cube, a KPI, etc
- Data value: estimate the shelf-life of poor data quality or, in other words, the risk associated with bad quality; focus first on those areas with the highest risk
In both cases, once organizations detect and prioritize the areas of focus, they can use data governance to create a collaborative framework for managing and defining policies, business rules, and assets to provide the necessary level of data quality control. It’s essential to have IT and the business on the same page and collaborating to define and improve data quality and data usage.
For example, the data owners can define key systems and processes involved, while the business can state what standards the data should adhere to when it moves through the systems. This is where policies, requirements, and business rules are created and agreed on.
Once you know how the data flows through the organization and you know what the standards are, asking the data quality team to translate these standards into data quality rules and run them on the data in those systems is more streamlined.
Enabling data quality with data governance capabilities
Once the organization determines where they should focus their data quality efforts (rather than trying to profile everything), business analysts can use this high quality data to make impactful business decisions. But first, they need to trust that data. Having a simple and direct way to identify data errors and see them quickly resolved is essential to maintain (and/or restore) that confidence.
A data helpdesk comes in handy for resolving data issues. This is a key component of data service management, a high-level maturity state in which all employees of an organization have access to one central place where all data is documented. A data helpdesk enables organizations to handle all data related issues in an efficient way, using the data governance organization, roles, and responsibilities. These data governance features help ensure trust in an organization’s data so business analysts can be sure they are using the best data available to make their data-driven decisions.
Although data quality and data governance are two separate disciplines, they work together in parallel. In fact, having a strong data governance foundation is essential to successful data quality. Without data governance, organizations cannot trust their data, and therefore, cannot guarantee the quality of their data. Data governance serves as the first step to ensuring access to the highest quality data available by providing necessary context and understanding around your data.
Data quality then comes in to compare datasets across an enterprise. When data quality is implemented on top of a data governance foundation, data quality provides much more than a simple view of quality. It provides a view of quality in the context of how the business uses data, enabling data stewards to identify problems that need to be addressed, enabling what-if analysis, and ultimately providing business analysts and data scientists with more accurate data to perform their analysis. With data governance as the foundation and data quality alongside, organizations can be sure that their data is accurate, trustworthy and secure, thus enabling digital transformation across the entire enterprise.
Data quality and data governance with Collibra
Collibra delivers a platform for data governance that serves as the necessary foundation to a data quality tool and, ultimately, drives Data Intelligence. With Data Stewardship, Intuitive Workflows, Data Helpdesk, and more, Collibra enables organizations to give context to their data sets at the metadata level and maintain control over this data.
Many Collibra customers also integrate a data quality tool into their environment to enhance data usability. When the two tools work together, Collibra offers the rules, procedures and standards to not only clean the raw data, but also to illustrate data errors, peculiarities and issues, in order to help compile the best standards and monitor the data quality over time. Then the data quality tool translates the business aspects into technical aspects to test the data and provide ongoing evaluation. With up to date and well governed data, business users can better trust and analyze data to drive valuable insights.