From dev to data: embracing observability into DataOps
Data engineers have a clear mandate: make high-quality, relevant, and timely data accessible to data consumers. But as the volume of data and data-driven activities multiply rapidly, there are a lot of questions that arise, such as: Can the data quality program scale with data? Can it deliver trusted data in near-real-time? Does it detect problems in advance? The observability into data health from ingestion to consumption across pipelines, applications and infrastructure provides huge opportunities to improve data quality.