Metadata Knowledge Graph: The Brain Powering Data Intelligence
There’s an old story about a group of people who have to guess what an object is just by touch. Each one makes a different guess, which leads to disagreement and distrust. It turns out everyone is wrong—the object is an elephant, but one was holding the trunk, one had a tusk, and so on. Nobody got it because no one had the full story.
That’s how it is inside data-heavy organizations. There’s plenty of data to guide every business decision, but the reality is so short of the ideal. And remember, there’s more to come. At Collibra, we advocate the notion of Data Intelligence, where all data—regardless of source or format—is easily accessible to a broad ecosystem of business users empowered to trust, adapt, enhance and share it. We believe this helps organizations turn data into a strategic asset, reinvent business processes, gain a competitive advantage and boost digital transformation.
But intelligence is also a fundamentally human characteristic, and we need more than volume. Content without context leads to chaos rather than clarity, so we need data to be supplemented with transparency, lineage and history. But let’s go one level deeper. Even if we can see the whole picture, each of us brings a different perspective to it—and that’s a good thing. Diversity of opinions, experience and expertise can all contribute to the greater good. However, without the right technologies and processes, it can add to the confusion.
So what does it take to achieve the right state of knowledge? To achieve any kind of intelligence, we need a brain—a pulsating, dynamic entity with many nodes sending information. And that, in a data-sensitive setting, is the knowledge graph.
To see how this works, think of how different constituencies come together in a business environment.
- There’s John the data steward, whose job it is to make sure everyone is speaking the same language. Sounds simple, but terminology is a major challenge; five salespeople will have five different definitions of the word “customer.” Developing clear, articulated and validated business terms—and defining code values for reference data—is critical. That helps resolve issues around data access, which is even more important.
- Dora is in data protection, and it’s a big job. She identifies, documents and analyzes compliance, conducts regular audits and leads training on customer and employee privacy (a huge issue in this environment). Every time there’s news of a breach, or a fine for non-compliance, it sends a shudder down the corporate spine. She’s dedicated to seeing the data doesn’t get into the wrong hands—but still reaches the right people.
- David the data engineer builds pipelines to collect data, ensures integration with scripts and standards and develops frameworks to serve the data. He needs to work very closely with Dora—and at times, it can seem like they are the best of enemies.
- Luke the data scientist needs to understand the data better than anyone else, keep it up to date and document the operating model. It’s his primary task to see that users come to trust the data, or they won’t use it.
There are many more competing and collaborating interests, but you get the idea—and it could lead to major discord.
Now imagine how this plays out in the Collibra platform. It’s a “brain” that features not only information from multiple brain cells, nodes and other assets, but the collective wisdom that comes from multiple edges, connections and relationships between those sources and data nuggets. That’s what transforms information into knowledge.
As consumers and producers—the human elements creating Data Intelligence—we draw knowledge through the lens of each application tailored to our particular needs, but we still have the visibility provided by the entire graph. This is how we get a clear, defined and powerful view of the big picture because everybody’s looking at the same version of the interconnected truth.
There are good reasons why some data and IT executives are uneasy about adding more data into the mix, particularly when it’s from more sources with more perspectives and in more formats. The concerns over compliance and security are certainly justified—and the goal of turning data into a strategic asset through greater access and to drive collaboration can seem like a fantasy.
This is what makes us so different: The Collibra platform provides full visibility into the entire data landscape, building on a rich trove of expertise and perspective and serving our particular needs, even as it ensures full compliance with the endless stream of privacy mandates. This is how we offer the right foundation to unite people and processes and balance rights and responsibilities. And it’s how we practice Data Intelligence, drive innovation and generate greater company value.
VP of Engineering