DOCUMENTATION

Data Quality

Evaluate an agent's ability to detect, prevent, and diagnose data quality failures.

What this competency is

Defining quality expectations and instrumentation that make pipeline and data issues visible, actionable, and fast to resolve.

Why it matters

If teams cannot detect and diagnose quality regressions quickly, trust erodes and downstream decisions become unreliable.

What to evaluate in agents

  • Quality dimensions addressed (freshness, completeness, accuracy, consistency, uniqueness).
  • Use of tests, monitors, and contracts at appropriate pipeline boundaries.
  • Root-cause and lineage-aware troubleshooting approach.
  • Incident response and communication strategy for data failures.

Strong signals

  • Defines measurable quality SLIs and thresholds.
  • Places checks near source, transform, and serving layers.
  • Includes alert routing and escalation ownership.
  • Uses lineage context to scope blast radius.

Weak signals

  • Uses generic "add monitoring" guidance with no metrics.
  • Confuses pipeline success with data correctness.
  • Omits detection of silent data drift.
  • Provides no recovery or communication plan.

Example evaluation prompts

  • "Design quality checks for a customer 360 mart with strict dashboard freshness."
  • "Propose an observability strategy to catch schema drift before business metrics break."