Data Quality Recruiting

Data quality is the discipline that keeps analytics trustworthy: the tests, monitors, and validation gates that catch broken rows before a decision consumes them. Its practitioners define what correct means as executable assertions and prove it continuously. Demand for the seat has hardened because business reporting and AI training both fail visibly when the inputs go bad. dbt's documentation ships four built-in data tests out of the box, unique, not_null, accepted_values, and relationships, and treats each test as SQL that hunts failing rows [1][1] Add data tests to your DAG — dbt Developer Hub (accessed 2026-09-28). Databricks describes the monitoring side: anomaly detection analyzes historical commit patterns to build a per-table model of when the next update should arrive, marking a table stale when a commit is unusually late [2][2] Data quality monitoring — Databricks Documentation (accessed 2026-09-28). Hiring for this craft means finding people who have owned the thresholds, not just run the checks.

Challenges in Data Quality Recruiting

Automated data testing moved from a pipeline nicety to a governance requirement

In dbt, a data test is a SQL select that returns rows disproving an assertion: the not_null test selects nulls, the unique test selects duplicates, and relationships checks referential integrity against a parent table [1][1] Add data tests to your DAG — dbt Developer Hub (accessed 2026-09-28). Singular tests are free-form SQL saved in the project; generic tests are parameterized query blocks applied across models and columns [1][1] Add data tests to your DAG — dbt Developer Hub (accessed 2026-09-28). Great Expectations generalizes the same idea outside the warehouse: an Expectation is a verifiable assertion about data, an Expectation Suite is a collection of them, and a Checkpoint runs suites against data and fires Actions such as alerts on the results [3][3] GX Core overview — Great Expectations Documentation (accessed 2026-09-28). The distinction that matters for hiring sits between the test and the regime. Almost every data practitioner has written a test at some point. Far fewer have built the arrangement where hundreds of tests run on a schedule, failures page the right owner, and every test has a known reason to exist. That arrangement also has an economics: a test that pages nobody costs compute and attention, and a suite full of them trains the team to ignore the channel entirely.

Anomaly detection in data pipelines arrives on commit history and row counts

Manual rules fail at estate scale. Databricks' data quality monitoring builds a per-table model from commit history to predict the next update; a commit that arrives unusually late marks the table stale, and completeness works the same way, comparing the last 24 hours of rows against a predicted range [2][2] Data quality monitoring — Databricks Documentation (accessed 2026-09-28). Snowflake implements the identical pattern as an option on data metric functions: the platform trains an algorithm on historical values and flags results outside the predicted range, currently for row count and freshness [4][4] Detecting anomalies in data quality — Snowflake Documentation (accessed 2026-09-28). Neither platform asks the operator to write the detection logic, which is the point. Scale outruns rules: a thousand tables cannot each carry a hand-tuned expectation, so the platform learns each table's rhythm and flags deviation. Two consequences follow. Anomaly detection in data pipelines is now a platform feature, so tool familiarity is nearly worthless as a differentiator. The scarce skill moved to the decisions around the algorithm: which tables earn monitoring, which metrics deserve anomaly detection, and what a detected anomaly triggers downstream.

Data observability turns alerts into triage and root cause

Snowflake's account-wide quality dashboard makes the operational goal explicit: healthy tables, monitoring coverage as a percentage of the estate, and open incidents grouped by type, volume anomalies beside freshness anomalies [5][5] Using the data quality monitoring dashboard — Snowflake Documentation (accessed 2026-09-28). That grouping is the heart of data observability: an incident view plus the lineage needed to trace a stale table back to the job that stopped feeding it. Teams that skip this step end up with monitors firing into channels nobody reads. Observability also changes the failure economics: an unowned incident decays into alert fatigue, and muted channels are how real breaks go unnoticed for weeks. Practitioners who have run observability at scale can describe their triage path: how an alert moves from page to runbook, which upstream jobs get correlated, and how they cut alert noise without cutting coverage. Candidates who configured a monitoring product cannot; their experience stops at the alert.

Data integrity monitoring splits rule ownership from metric ownership

Snowflake's system data metric functions enumerate the metric side of the craft: blank counts, null counts, freshness, schema change counts, and distribution statistics from quantiles to standard deviation, each attachable to tables and views on a schedule [6][6] System data metric functions — Snowflake Documentation (accessed 2026-09-28). Beside them sit expectation-style checks, declarative assertions about acceptable values. The two mechanisms answer different questions, and the split shows up as a real career boundary. Metric owners watch distributions drift and page when the median moves; rule owners encode invariants such as "status is one of five values" and treat any violation as a defect. A candidate who has only ever written thresholds has no experience with referential invariants. One who has only written dbt-style tests has never designed a monitoring regime for a thousand tables. The strong hire has done both and knows which failure each mechanism is built to catch, because the two failures look similar in production and cost very different amounts of time to find.

Data validation at the gate is a different seat from validation at the table

Placement is the decision CVs never show. Great Expectations is explicit about the options: a Checkpoint integrated with an orchestrator enables scheduled, automated testing on an essential table, while interactive validation of a dataframe serves exploration [3][3] GX Core overview — Great Expectations Documentation (accessed 2026-09-28). dbt puts the same choice in the release path, where tests run against model changes before merge [1][1] Add data tests to your DAG — dbt Developer Hub (accessed 2026-09-28). Validate at ingestion and you catch producer bugs early but own a queue with a high failure rate. Validate in CI at transformation time and you stop bad merges, but nothing watches the table after deploy. Validate continuously and you catch the silent upstream break at the cost of ongoing compute. Senior practitioners place per asset and can defend the choice. The interview question is simple: for their last estate, which assets were gated where, and which incident made them change a placement. The answer reveals whether the candidate optimized for their own queue or for the consumer's trust.

Owned failure thresholds separate automated data testing practice from tool badges

Quality CVs read identically: dbt tests, GX, monitoring, expectations. Verification therefore asks for the decisions behind the tools. Which threshold did the candidate set for row volume, and what historical pattern justified it? Which expectations fired in production, who got paged, and how did the resolution change the suite? What happened the time a freshness alert was a false alarm, and how did they retune it without hiding a real stall? A practitioner who owned a regime can narrate a specific failing record through its remediation; someone who operated tooling describes features. The cost of getting it wrong lands as analyst hours burned on bad numbers, model runs poisoned by silent schema drift, and a quality program that exists as a dashboard while trust keeps eroding.