IT’S YOUR DATA, AND THIS IS YOUR BLOG

Welcome to the Collibra Blog, where CDOs, data stewards, and data citizens go to learn about true data governance.

subscribe

Metrics Matter

Share: Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone

Metrics Matter

In 1999, a $125 million satellite, designed to observe weather patterns on Mars, was about to begin its orbit of the red planet. Instead, the Mars Climate Orbiter plunged into the atmosphere and was incinerated. Why? Engineers had calculated the thrust needed to nudge the satellite into orbit in standard pounds while systems aboard the MCO relied on metric newtons to measure impulse.  As my granny might have put it: but for a nail the kingdom was lost.

Today, the story is a textbook example of how assumptions can derail even our most carefully planned projects. For the Mars Climate Orbiter, metrics, quite literally, mattered.

For businesses, metrics of a different kind matter. Metrics—whether about customer profitability, sales growth, or P/L—drive insight. But without real transparency into how those metrics are calculated, businesses can wind up flying too near the atmosphere, courting disaster with no real understanding of the imminent risk.

Interestingly, the problem is quite similar to the disconnect that plagued the Mars Climate Orbiter. The data those engineers and scientists had wasn’t “bad” data. It wasn’t inaccurate or outdated. It simply wasn’t fit for purpose—and so the metrics that got calculated with that data were incorrect. Without transparency into those metrics—and the data that defined them– assumptions, rather than fact, governed decision making.

I see business users unwittingly making the same mistake at almost every organization I visit. In business, metrics rule. And some very smart people are working hard to deliver accurate metrics to drive their businesses forward. But what are the assumptions behind these metrics? What attention is being paid to the data grounding them?

In many companies I’ve worked with, “data quality” is often identified as an issue to be resolved. But often, the issue isn’t as much about the data as it is about the metrics built from that data. Take, for example, the question that data users must be asked at least once a quarter: “How many customers do we have?” It should be an easy one to answer. But it’s not unusual for different parts of the organization to have different definitions of what a customer is—digital might only count customers who have logged onto the website or a company app, marketing probably includes prospects, whereas finance is only interested in those who’ve created an actual account. Each is valid, but each needs to be understood in its own context. Without transparency into these different contexts, a data user can pull perfectly good data and still come up with the wrong answer.

Very smart people are paid very good wages to build metrics and dashboards that answer questions like these. They grab the data that’s available to them—having no reason to distrust it—put it into their Excel spreadsheets, and work diligently to craft that data into intelligible information. But without appropriate context—without transparency—those metrics that the business depends on, the metrics that business users are working so hard to build and curate, are simply missing the mark. Even worse, inaccurate reports and dashboards eventually become the authoritative source for information—you’ve cemented those assumptions into your business practice.

So what’s the answer? I believe we have to start thinking about “data governance” more broadly and begin to put processes in place that also govern our metrics and reporting assets—i.e. a system of information governance.   (That’s one of the reasons I recommend organizations—whether or not they are banks—align with BCBS 239 principles since they clearly differentiate between data risk and reporting risk.)

Information governance can deliver the kind of transparency your business users need to build better metrics. It gives business users the tools they need to find and understand their data. It helps them see who created the data in the first place—and why. It provides clear—and shared—data definitions that can be understood in business terms. And it provides transparency about how information (metrics) are defined so they are used in the right context.

When viewed in this way, “data governance” ceases to be a technical problem and instead becomes a business solution—helping you deliver trustworthy metrics that can improve the quality of your decisions, accelerate product development, and optimize the customer experience. And those are the metrics that will really matter.

 

Simon is the Collibra Global Financial Services Market Lead. During his 20+ years of management consulting services experience at leading consultancies EY and PwC, Simon helped financial services institutions design and implement information and data governance processes to support internal, external, and regulatory reporting including BCBS 239 and CCAR.
  • Irina Williams, PerfexLab

    “…Data quality is often identified as an issue to be resolved. But often, the issue isn’t as much about the data as it is about the metrics built from that data.” – Simon, you hit the nail here.
    What’s the point in an accurate data if it can be then misinterpreted in reports?
    The main purpose of all data initiatives is to deliver trusted and accurate data to support effective decision-making, right? The only thing is that businesses consume data specifically in the form of metrics, which they receive from reports.
    And here are two questions: 1. How to build a business glossary of metrics that we can govern? 2. Because most companies already have thousands of reports in use, how to make sure that metrics used in the reports are accurate as defined in the business glossary?
    In our company we have come to the conclusion that the only way to kill two birds with one stone is to make an inventory of reports with all metrics that they contain. Once done, we have a full catalogue of all metrics used in the company with understanding which of them contradict or duplicate another (e.g. the same metric used with different names in different reports or otherwise a metric with the same name calculated differently). Having this information we can start governing it: unify metrics and update reports where they are used.