What is data quality monitoring?

Data quality monitoring is the ongoing practice of checking data against defined rules, like not-null, uniqueness, range, and freshness checks, every time new data lands, not just once at build time. It catches breakage early: a null spike, a schema drift, a stale table, before it reaches a dashboard or a report.

How it works generically

Most data quality monitoring is built from expectations: explicit, testable rules like "this column is never null" or "this value falls between 0 and 100." A validation engine runs those expectations against new data on a schedule or on every pipeline run, and failures get flagged, logged, or used to block a downstream step before bad data spreads further.

Why it matters

Bad data is expensive precisely because it's quiet. A pipeline that runs successfully but writes null revenue or duplicate customer IDs doesn't throw an error, it just produces a dashboard that's wrong. Monitoring turns a silent failure into a visible one, ideally before a stakeholder makes a decision off the bad number.

OptimaFlo's current default

OptimaFlo has real Great Expectations integration built for this: a working suite builder, context manager, and validator that can generate and run expectations against Bronze, Clean, and Ready tables. That code path is functional, not a stub. It is off by default, though. Automatic expectation generation and the enforcement gate that blocks a pipeline on a failed check both sit behind a single environment flag, OPTIMAFLO_EXPECTATION_FIRST, and that flag defaults to off. With the flag unset, generation makes zero LLM calls and the enforcement check returns no expectations, a structural no-op. A workspace or organization has to explicitly turn the flag on to get automatic, ongoing data quality monitoring today.

Related terms

• Data lineage: Tells you where a bad value came from; data quality monitoring tells you a value is bad in the first place.
• Data governance: Data quality monitoring is one of the controls a governance program typically requires.