What is data profiling?

Data profiling is the process of examining a table's actual data to understand its shape: how many values are null, how many distinct values a column has, what the minimum and maximum look like, and what patterns show up in the data. It answers "what does this table actually contain" before you write a transform or a quality check against it.

What a profile typically includes

A profile usually reports, per column: null rate, count of distinct values (cardinality), a sample of the most common values, minimum and maximum for numeric or date columns, and an inferred data type based on the values seen rather than just the declared schema. Some profilers also detect patterns, flagging that a text column looks like it holds emails or phone numbers, which is a hint toward what kind of data it actually is.

Why it matters

A profile catches problems before they reach a dashboard: a column that's supposed to be required but is 40% null, a "boolean" column that actually has five distinct values, an ID column with duplicates where you expected uniqueness. It also informs what to build next: cardinality and null rates are exactly the inputs that shape which quality expectations are worth writing for a table.

How OptimaFlo profiles tables

OptimaFlo runs a DuckDB-powered profiler service that computes these statistics directly against a table: cardinality, null rates, value distributions, min and max, inferred data type, and pattern detection, with a configurable sample size so a profile can run against a subset of a large table instead of scanning it in full. It can profile a table at any layer, Raw, Clean, or Ready, not just at connect time when a source's schema is first inferred.

Related terms

• Data quality monitoring: The ongoing checks a profile's findings often feed into, turning a one-time statistic into a standing expectation.
• Data ingestion: The step that lands data in a table before profiling can examine it.