
Problem
Data problems in a payments business are expensive and quiet: a payment posted twice, a fee engine missing its cap, a ledger out by 12.50, a settlement feed that stopped two days ago. Rule-based monitors catch them early but produce alerts nobody reads, and "AI anomaly detection" on its own can't be audited.
Approach
I built the monitor in n8n with one design rule: SQL decides what failed; Claude only explains why; code checks Claude.
- Rules are data. 16 SQL rules across 7 dimensions (completeness, validity, uniqueness, integrity, consistency, timeliness, volume) live as rows in PostgreSQL, each with an owner and a severity. A database function runs them all, stores the failing records, and opens, updates or resolves one incident per rule.
- Claude Opus 5.5 explains only new failures, in one call per run with structured outputs: likely cause, business impact, suggested fix, confidence, evidence and related rules. Emails are masked before anything leaves the database.
- A guardrail in code accepts only evidence ids Claude was shown and links to rules that really failed; anything else becomes a plain fallback note.
- Jira: a Bug per new incident with the explanation, the evidence and the SQL to reproduce it; comments when it is still failing or fixed. An HTML report page shows every rule, the explained failures and the run history.

Result
Evaluated end to end against 16 planted defects on fictional Contoso Pay data (2026-11-16, Claude Opus 5.5, mock Jira): 10 of 10 quality gates passed.
- 16 of 16 defects caught with exactly the right record ids, and 0 false alarms on clean data.
- 16 of 16 explanations passed the code check and named a plausible root cause, for example "the fee engine applies a flat 1.5% without the 0.25 minimum or the 15.00 cap, and truncates instead of rounding".
- About $0.15 per run with new failures; $0 when nothing is new, because ongoing incidents only get a Jira comment.
Testing found three real problems, each fixed and covered by a test:
- Claude found a bug in one of my rules: the volume check compared today's partial day, which would raise a false alarm every morning. Fixed, with edge-case tests for a partial day and a day with zero transactions.
- My guardrail was too strict: it rejected two good explanations that cited real ids from the detail text. I replayed that real response offline, fixed the check, and kept it as a regression test, alongside an invented id, a refusal and an API error.
- A DNS failure broke one Claude call. The monitor still detected everything and filed tickets with fallback notes, and the Claude and Jira calls now retry.
What this shows
Data quality engineering and AI assurance together: deterministic, auditable checks; an AI that adds judgement but can't change a result; a guardrail tested against real model output; and an evaluation that measures detection, false alarms, explanations and cost.