DOCUMENTATION
Transformation
Evaluate an agent's ability to transform data into trusted business-ready datasets.
What this competency is
Turning raw data into clean, documented, and reusable data products with stable business semantics.
Why it matters
Transformation quality determines whether downstream teams can make decisions from shared definitions instead of fragmented logic.
What to evaluate in agents
- Layering strategy (raw, staging, intermediate, marts).
- Logic for joins, aggregations, and business rule application.
- Metric and dimension definitions with explicit semantic ownership.
- Approach to reproducibility and testability of transformations.
Strong signals
- Defines business entities and metrics with clear, unambiguous rules.
- Uses modular transformations instead of monolithic SQL.
- Anticipates edge cases such as missing or conflicting source records.
- Includes tests for critical assumptions and invariants.
Weak signals
- Encodes business logic with no documentation.
- Uses inconsistent metric definitions across models.
- Produces transformations that cannot be replayed deterministically.
- Omits quality checks for key business outputs.
Example evaluation prompts
- "Create a transformation design for net revenue with refunds and chargebacks."
- "Define a semantic layer for activation and retention metrics."