AI Assisted Software Engineering

Part II · The Life Cycle, Audited

13 Testing, Evaluation, and Tests as Steering

RecalibrateLab: Do your tests mean anything?Feeds M5

Load-bearing assumption

Tests are written with intent independent of the defect

Verdict: Recalibrate

Testing assumes a test's expected values come from intent independent of the defect; when the agent that wrote the code also writes the tests, that independence must be enforced by structure, and components that call a model need evaluations over repeated trials instead of single runs.

Learning objectives

After this chapter you should be able to:

  1. Distinguish fault, error and failure, and apply the test levels, the pyramid, coverage and mutation score.
  2. Explain why coverage measures execution rather than adequacy, and compute a mutation score.
  3. State testing's load-bearing assumption, show how an agent-written test can encode the defect it should catch, and explain why different authorship alone does not establish independence.
  4. Design structural independence: acceptance tests derived from the specification, written first, and read-only to the implementing agent.
  5. Compute pass@k and pass^k from trial counts, choose between them for a given component, and set a release threshold with a stated confidence.
  6. State what evidence would overturn the verdict.

Opening case

A composite case, not a real engagement.

The dispatch dashboard from Chapter 6 reaches review with a strong-looking report: 214 tests, all passing, 93% line coverage, a green CI badge. This is how one defect got past all of that.

The chapter continues from here.

What the chapter covers

  1. 13.1 The classical method
    • 13.1.1 Fault, error, failure
    • 13.1.2 Levels, the pyramid, and oracles
    • 13.1.3 Coverage is not adequacy
    • 13.1.4 Mutation testing
    • 13.1.5 What the classical method got right
  2. 13.2 The assumption it depends on
  3. 13.3 What agents change
    • 13.3.1 One author, one blind spot
    • 13.3.2 Tests that change until they pass
    • 13.3.3 Tests as steering
    • 13.3.4 Nondeterministic components and evals
    • 13.3.5 LLM-as-judge
    • 13.3.6 Property-based testing
  4. 13.4 The verdict
  5. 13.5 What replaces it: structural independence
    • 13.5.1 Four rules
    • 13.5.2 Tests as steering: red, green, and a human
    • 13.5.3 Designing an evaluation for a component that calls a model
    • 13.5.4 Dispatch, twice
    • 13.5.5 What the replacement keeps
  6. 13.6 Lab: Do your tests mean anything?
  7. 13.7 What would make this verdict false

Lab and prediction

Lab: Do your tests mean anything?

Feeds milestone M5. How the labs fit the project

This chapter's own prediction: Agent-written suites have lower mutation scores than spec-derived acceptance suites, and at least one agent-written test fails once a specification-contradicting behavior is corrected.

A result that contradicts a prediction, honestly reported, is a result. You are graded on accurate records and reasoned interpretation, not on agreeing with the book.

Summary

  • Coverage measures execution, not adequacy; mutation score measures whether a suite has teeth.
  • Load-bearing assumption: tests are written with intent independent of the defect. When one agent writes both, a test can encode the bug and a pipeline can stay green while meaning nothing. Different authorship alone does not establish independence (Knight and Leveson 1986).
  • Evidence: LLM-generated oracles mainly capture actual behavior, models have tampered with graders they could reach, and output varies even at temperature 0.
  • Verdict: recalibrate. The structure holds; independence must be enforced, and "pass" becomes a rate for model-calling components.
  • Replacement: acceptance tests derived from the specification, written first, read-only to the implementer, checked by mutation testing, with a held-out subset, used to steer the agent through red and green.
  • Evals: pass@k for exploration, pass^k for must-work paths, inputs from the operational profile, and trials sized to the confidence you need.

Key terms

Fault, error, failure
a flaw in code or design; the incorrect internal state it causes; the visible departure from required behavior.Also used in 4, 8, 15, 17
Test oracle
the mechanism that decides whether observed behavior is correct.
Coverage
the share of code a suite executes; not a measure of adequacy.Also used in 1, 2, 4, 5, 12, 14, 15
Mutant, equivalent mutant
a copy of the program with one small change; one that behaves identically and cannot be killed.
Mutation score
mutants killed divided by mutants generated minus equivalent mutants.Also used in 2, 18
Test independence
the expected value in a test does not derive from the code under test. Different authorship alone does not establish it.Also used in 12
Read-only acceptance test
a spec-derived test the implementing agent cannot modify, enforced by permissions and CI.
Defect-encoding test
a test whose expected value contradicts the specification.
Eval
a test suite for a nondeterministic component: inputs, a scoring rule, repeated trials, a rate.Also used in 14
pass@k, pass^k
the probability that at least one, or all, of k trials succeed; pass^k equals pᵏ when trials are independent (Yao et al. 2024; Chapter 15).
LLM-as-judge
using a model to grade another model's output; subject to position, verbosity and self-preference biases.

Connections

Where this chapter and another point to each other most often.

See it on the map

Further reading

  • Inozemtseva and Holmes (2014). The largest study of coverage and effectiveness; read its threats to validity.
  • DeMillo, Lipton, and Sayward (1978). The origin of mutation testing, in a few pages.
  • Chen et al. (2021), Section 2.1. Where pass@k and its unbiased estimator are defined.
  • Yao et al. (2024). The definition and use of pass^k; read how fast reliability falls as k grows.
  • Barr et al. (2015). A survey of the oracle problem, this chapter's theme in general form.