Build custom data quality checks without getting lost in translation

Poor data quality is a silent killer, costing companies an average of $12.9 million annually according to Gartner. A primary cause is the painful gap between business and IT, where simple business logic gets lost in translation. This forces technical teams into a frustrating cycle of interpreting vague requests, leading to endless email chains and weeks of delay just to write a single, precise SQL rule. What if you could eliminate that friction and empower your business experts to define robust checks themselves using the same natural language they use every day? Collibra makes this possible, empowering your team to turn business knowledge directly into data quality action—no coding required.

What’s new: Custom rules

Collibra is thrilled to announce the General Availability of custom rules with Text2SQL for Data Quality & Observability (Cloud). This release provides an intuitive user interface to write, validate and apply custom SQL rules to your data quality jobs. A standout capability is the ability for non-technical users to generate these rules using AI (Text2SQL), democratizing data quality for everyone involved. This flexible and efficient rule-writing experience is set to drive adoption and automate the governance of critical data like never before.

How custom rules help

Your cloud data is a goldmine, but only if you can trust it. While adaptive rules catch basic errors like NULLs, empties and uniqueness, they miss the crucial context that drives your business. What happens when a product code is sold in the wrong country? These are the business-specific rules that are notoriously difficult to define and translate into complex SQL. Most data tools buckle under this pressure, unable to manage different syntaxes or scale custom monitoring. This failure creates a cascade of problems: inconsistent data, untrustworthy reports, and ultimately, poor business decisions.

Custom rules with Text2SQL directly addresses these pain points by offering:

• Simplified rule definition for unique organizational needs
• AI-assisted rule creation (Text2SQL) for non-technical users
• Efficient validation to prevent syntax errors before execution
• Flexible filtering and rule tolerances to apply rules to specific data subsets and minimize false positives
• Immediate data quality scores in the data catalog

How custom rules works

This new capability is centered around a robust Rule Workbench, providing a comprehensive environment for crafting and managing your data quality rules. Within the workbench, you define a rule name and select a primary column that the rule focuses on. A powerful rule filter allows you to apply the rule to a specific subset of rows in your dataset, without needing a separate check for that.

The core of the workbench is the SQL Editor, designed to streamline rule definition. It features syntax highlighting for easy comprehension and an option for auto-formatting to ensure readability. Before running, you can validate your query to catch syntax errors, ensuring accuracy.

Beyond technicalities, the Rule Workbench integrates seamlessly with broader governance by offering rule settings. Here, you can assign Data Quality dimensions for effective business reporting and set a tolerance percentage for acceptable data breaks. You can also configure alerts for exception, breaking or passing statuses, ensuring timely notifications.

For non-technical users, an AI rule creation feature (Text2SQL) allows generating SQL queries from natural language prompts, making rule-writing accessible to all.

Once configured, you can preview the rule's potential impact by viewing sample breaking records before the job runs. These rules are applied to data quality jobs within the Collibra Platform, connecting directly to your overall data and analytics governance strategy.

Why you should be excited

Custom rules help increase efficiency across the entire organization from data quality admins or data quality managers. It helps these personas:

For Data Quality Admins:

• Streamline customization: Easily write, validate and apply custom SQL rules to tailor data quality monitoring to your organization's unique needs, driving platform adoption
• Enhance control: Define rule parameters like dimension assignment, tolerance thresholds and alert configurations to ensure rules align with organizational resources and operational policies
• Reduce technical barrier: Empower non-technical team members to create rules using the AI (Text2SQL) feature, freeing up technical resources and accelerating data quality strategy implementation

For Data Quality Managers:

• Proactive issue identification: Configure monitor-specific alerts for rules in exception, breaking or passing status, ensuring you're the first to know about critical data quality issues and can support rapid resolution
• Contextual understanding: Assign data quality dimensions to rules for effective business reporting and clear explanations of rule intentions
• Efficient management: Add new rules directly from the monitoring overview or job details page, integrating seamlessly with your existing data quality workflows and leveraging profiling and adaptive rule insights

Use cases

Here are some real-world scenarios where custom rules with Text2SQL can make a significant impact:

• Financial compliance validation: Automatically validate that customer financial records adhere to specific regulatory standards, such as ensuring all transaction IDs must follow a specific format (digits and alphanumeric, for example). This automates the governance of critical compliance data
• Healthcare data consistency: Implement rules to verify the consistency of patient demographic data by checking for matching patient IDs, consistent date formats and valid insurance codes. The AI (Text2SQL) feature can help medical record specialists define these complex consistency checks without deep SQL knowledge
• E-commerce product data accuracy: Create rules to monitor product catalog entries for accurate pricing, correct inventory levels (e.g., StockQuantity > 0), and complete product descriptions. Custom filters can be applied to focus on specific product categories or high-value items, ensuring that only relevant breaks are flagged for immediate action

Key takeaways

The launch of custom rules with Text2SQL perfectly embodies our "AI-powered governance, everywhere" theme by bringing advanced automation and intelligent assistance to the heart of data quality. This feature not only streamlines the creation of essential data quality rules but also makes data governance more accessible and efficient for everyone, driving broader adoption of our new platform. It ensures that your critical data is continuously monitored and governed with precision, allowing you to trust your data at every turn.

Join Collibra’s Product Premiere to learn about:

• AI democratizes rule creation: Non-technical users can now write complex SQL rules with AI assistance (Text2SQL)
• Custom rules are flexible and efficient: Tailor data quality monitoring to your exact organizational needs with a powerful, intuitive workbench
• Automated governance is amplified: Rules automate the monitoring and enforcement of quality for critical business and governance assets

• 

  Kate Wendell

  Kate Wendell

  Director, Product Management

  Collibra

• 

  Marc Campabadal

  Marc Campabadal

  Customer Advocate

  Collibra

• 

  Nataliya Shurupova

  Nataliya Shurupova

  Senior Manager, Software Engineerin

  Collibra