DOCUMENTATION

Scalability

Evaluate an agent's ability to scale data systems and optimize performance with clear trade-offs.

What this competency is

Designing systems that sustain increasing load while meeting latency, throughput, and cost targets.

Why it matters

A design that works at current scale can fail quickly under growth if performance bottlenecks and scaling boundaries are not addressed.

What to evaluate in agents

  • Load characterization (data volume, concurrency, growth, skew).
  • Bottleneck analysis across compute, storage, network, and coordination.
  • Scaling strategy (vertical, horizontal, caching, precomputation).
  • Performance measurement and capacity planning approach.

Strong signals

  • Defines measurable performance goals and test methodology.
  • Identifies likely hotspots and mitigation tactics.
  • Connects scaling choices to cost implications.
  • Uses staged optimization based on observed bottlenecks.

Weak signals

  • Suggests "scale out" with no bottleneck model.
  • Optimizes prematurely without workload evidence.
  • Ignores skew and hot partition behavior.
  • Omits capacity planning or saturation signals.

Example evaluation prompts

  • "Scale a dashboard serving layer from 100 to 10,000 concurrent users."
  • "Diagnose and improve end-to-end latency in a streaming enrichment pipeline."