There are three types of metadata in Collibra:
- Technical Metadata: used by IT practitioners to keep track of the details of how the data is processed. Collibra can capture technical metadata, or acquire it from another tool specific to the IT organization.
- Business Metadata: enables business people to understand what the information means, and whether they can use it in their own context. This type of metadata is never static, as using data in new ways gives it new meaning and new relationships. Examples of business metadata include the relationship with reference data or the identity of a master entity and its relationship to the current data. (e.g. product images and product masters).
- Operational Metadata: used to reflect the state of the data. For example, operational metadata could be a record of who and how many people have accessed the data, or it could be links to open issues that exist with that data.
- In addition, some metadata is not about data at all, but about analytical models and their subcomponents (such as map/reduce jobs), visualizations, and assumptions. All types of metadata require different management approaches, and Collibra supports them all with the same flexibility, ease of use, and process orientation that makes Collibra the leader in data governance.
Collibra Enriches Metadata
All of this metadata is necessary to provide the complete business context and lineage for the data. This variety of metadata demands a system that is flexible and provides the tools needed to source and enrich metadata. Flexibility allows organizations to represent all the different types of metadata in a way that is consistent with their meaning and purpose. Sourcing and enriching metadata is an ongoing task. the Collibra platform uses a combination of three approaches for sourcing and enrichment:
- Collibra Connect and its purpose built connectors access the metadata stored in a large number of databases, applications, and other sources. This approach provides the ability to harvest the metadata as needed, and to recognize changes when they occur.
- Collibra allows you to “crowdsource” enrichment information from the stakeholders, stewards, and subject matter experts. This approach leverages the knowledge and experience of the entire organization, and can be easily scaled.
- Automated machine learning approaches are being added to facilitate automatic enrichment. As the scope and scale of our big data increases, this will become increasingly important.
Integration for Ongoing Modification of Metadata
Given the huge number and types of metadata, sources, and targets, it is nearly impossible to create customized integration for each one. Collibra solves this problem through Collibra Connect, an Enterprise Service Bus (ESB) for metadata. Collibra Connect provides connectors to all types of data sources, traditional and big data, as well as applications and data management tools such as ETL. It delivers connectors to retrieve information from data quality scanners, data models, profiling tools, and other sources of metadata. Collibra Connect also provides process connections to allow data processes to interact with other processes, such as IT service management. Collibra Connect brings all your metadata together, with templates that script the interactions between these sources and the other Collibra solutions.
"Collibra is a valuable addition for organizations that want to extract more value from their current data investments. As pioneers in data authority, they are truly redefining the way the industry looks at data governance"