Data quality in production
Quality is enforced by tests that run every time data moves, or it decays. There is no third state.
Data drifts. A source renames a column, a job half-fails on a Tuesday, a duplicate slips in through a retry. Without checks in the pipeline, the first person to notice is a manager reading a wrong number in a quarterly report, three weeks later, with no idea which of forty tables to blame.
The fix is boring, which is why it works. Row counts, types, ranges, uniqueness, freshness, checked on every run. A failed critical check stops the pipeline before bad data lands. A soft check raises an alert and lets the run finish. The art is in deciding which is which.
In production the same checks keep running, each with a threshold and a named owner. When something drifts, the alert points at the exact table and the exact check. Fixes stay small because problems get caught while they’re still one bad run old.