Eval-Driven Development

Treat improving an LLM app as an explicit, repeatable flywheel, not a one-time build: build an instrumented agent → Error Analysis on real traces → write evals only where the analysis justifies them → improve, prove which change moved the metric, wire evals into CI/CD, monitor production for drift → new failures re-enter as permanent regression tests. Evals are the engine ("evals are the new unit tests") because non-deterministic, subjective outputs make vibes-based iteration unreliable — a plausibly "better" change can silently regress.

The key inversion vs. naïve testing: error-analysis-first. You don't write a generic eval suite up front; you write evaluators reactively for the failures you discover, so coverage tracks real problems instead of imagined ones. Code-based evals catch deterministic failures; LLM-as-Judge handles subjective ones.

Claims

Don't trust vibes; measure. principle — durable: non-deterministic/subjective outputs mean an unmeasured change can silently regress; gate on evidence.
Every production failure becomes a permanent regression test. principle — durable: the system's memory of what can go wrong only grows (P3 "accrete, never drop" in llmops-continual-improvement).
Write evals reactively for discovered failures, not proactively for imagined ones. best practice — context: given that error analysis is actually being done first; premature comprehensive eval suites waste effort on the wrong failure modes.
Wire evals into CI/CD and guard against overfitting the eval set. best practice — context: once a product is maturing and prompt/model/tool changes ship regularly.
Count experiments run, not features shipped. best practice — context: roadmap framing for an LLM product; time-box exploration with clear decision points.
Eval infrastructure is debugging and fine-tuning infrastructure — you get them for free. principle — durable: the trace database, the assertions, the search/label tooling, and the synthetic-data generator you build for evals are the same assets you need to root-cause a complaint and to curate a fine-tuning set; there is enormous overlap.
Don't rely on generic eval frameworks to measure your AI — build eval specific to your problem. principle — durable: quality is domain-specific, so a bought generic scorer measures the wrong thing; start with what you already have (spreadsheets, your analytics system) before buying tools.
Make the quality gate enforceable and local, graded against domain-specific rubrics, not taste. best practice — context: agent-generated output where "looks fine" is unreliable; a second source (Buildable) runs review as a real gate over surface-specific rubrics (design-token discipline, local-first seam, accessibility, state coverage) with --strict failing rather than warning on drift — "design quality as an enforceable local gate," the part it says a hosted builder can't give you. Corroborates domain-specific evals as a shipped gate, and the CI build-verification of its Golden Templates is the same "measure, don't vibe" applied to the exemplar itself.
Fine-tuning is best for syntax, style, and rules; RAG supplies context and up-to-date facts. best practice — context: choosing between fine-tuning and retrieval to fix a failure mode; they address different classes of problem and aren't substitutes.
A score is the precondition for optimizing anything — where no eval exists, automated improvement of prompts, skills, or agents is not merely unwise but impossible. principle — durable, and a sharper statement of "don't trust vibes." SkillOpt's text-space optimizer accepts a skill edit only against a held-out score (Validation-Gated Update); with no score there is no accept rule and the loop degenerates into unbounded rewriting. An eval made blocking and automatic is a gate — the practice, promoted to infrastructure.
Evals are an economic requirement, not only an epistemic one: without a mechanical checker, spend past a low ceiling buys answers that exist and are never found. principle — durable, and the first argument on this page that doesn't route through non-determinism. A second, independent source (Nate B Jones) derives "you need evals" from a repeated-sampling result rather than asserting it: where an automatic grader existed, coverage converted straight into results; where the model had to pick its own best answer, majority voting and reward models all stalled around 100 attempts. The money spent past that line is the source's "shaded area… that's money you're not getting back." This gives the vault a cost case for evals to sit beside the correctness case, which is the case that persuades an operator who was never worried about silent regressions. Figures are the source's claims — see that distillate's Key claims. See Repeated-Sampling Scaling.

Error Analysis — the input that decides which evals to write.
Validation-Gated Update — an eval turned into a hard accept/reject rule on artifact changes; the enforcement end of this page's discipline.
LLM-as-Judge — the evaluator for subjective gates.
Levels of Evaluation — the cost-ordered menu of evals the flywheel draws from.
Synthetic Data Generation — the flywheel's data-generation and curation engine.
Agentic Workflow Patterns — the evaluator-optimizer pattern is this flywheel collapsed into an inference-time loop; an independent corroboration of generate→evaluate→refine.
Spec-Driven Development — SDD makes this structural: tests are the executable, verifiable half of the spec, and "tests validate behavior" is "measure, don't vibe" turned into an artifact.
Repeated-Sampling Scaling — the economic case for this page's discipline: a checker is what turns attempts into results, so an eval is the precondition for scaling spend, not just for trusting output.
Agent-Shape Triage — "is checking much cheaper than producing?" promoted to one of four questions that decide whether a task is worth agents at all.
Distillate: AI Evals — Self-Study Plan (Hamel & Shreya).
Distillate: Your AI Product Needs Evals.
Distillate: Learnings from a No-Code Library: Keeping the Spec-Driven Development Triangle in Sync — corroborates tests-as-source-of-verifiable-truth from the spec-driven-development angle.
Distillate: Buildable — A Local, Build-Verified App-Builder Brain for Coding Agents — review as a rubric-graded, --strict local gate, and CI build-verification as the eval that makes a template "golden."
Distillate: Building Effective Agents — corroborates "measure and iterate; add complexity only when it demonstrably improves outcomes."
Distillate: microsoft/SkillOpt — training skills like weights, without touching weights — corroborates "measure, don't vibe" by making a held-out score the load-bearing precondition for training a skill at all.
Distillate: 1.6M agents registered for OpenClaw and did NOTHING. — derives "you need evals" from economics rather than asserting it: without a mechanical checker, coverage never converts and marginal spend buys unfindable answers.
Corroborates the flywheel in llmops-continual-improvement.

Eval-Driven Development

Claims

Related

Linked from