Chapter 2. The Assumption Audit: A Method That Outlasts the Tools

Learning objectives

After this chapter you should be able to:

Explain why a method can give confident, wrong answers when the conditions it was built for change.
Identify the load-bearing assumption of a method using the negation test.
Carry out the six steps of an assumption audit on a named mechanism or activity (the unit of analysis).
Assign one of four verdicts with the ordered decision rule, and defend it against someone who disagrees.
Grade evidence by design, relevance, measurement quality, uncertainty and replication.
Turn a verdict into a prediction that your own project data could show to be wrong.

Opening case

A composite case, not a real engagement.

A product team of six engineers adopts coding agents in the spring. By summer, the engineering manager has a planning problem. The roadmap was estimated as always: story points per ticket, a velocity of about forty points per two-week sprint, a release date read off the burndown. The burndown rule was informal: a ticket's points counted when the ticket closed, and a ticket closed when its pull request opened. Six weeks into the quarter, the burndown looks wonderful. Tickets close at nearly twice the old rate. Then the integration environment fills with failures, two senior engineers spend a week doing nothing but reviewing agent pull requests, and the release slips by a month anyway.

The chapter continues from here.

What the chapter covers

2.1 Why a method instead of a list of answers
2.2 Where the idea comes from
2.3 Load-bearing assumptions
2.4 The six steps
2.5 The four verdicts
2.6 Grading the evidence
2.7 A worked example: earned value
2.8 A contrasting example: when the method holds
2.9 The audit at a glance
2.10 Auditing the audit

Lab and prediction

Lab: Audit a practice from your own work

Feeds milestone M0. How the labs fit the project

A result that contradicts a prediction, honestly reported, is a result. You are graded on accurate records and reasoned interpretation, not on agreeing with the book.

Summary

Methods whose assumptions fail keep producing confident, wrong answers. The negation test finds the load-bearing assumption.
Every audit names its unit of analysis; verdicts are comparable only for like units, and one practice can earn different verdicts at different units.
The decision rule is ordered: Q1 reversed scarcity (activities), Q2 the assumption still holds, Q3 a fix by values or definitions alone, Q4 otherwise breaks. A new input artifact to an unchanged procedure is Q3.
Audit the method at its strongest, then ask what teams do instead and what agents change. Earned value requires objective completion criteria; crediting at ticket close is a shortcut agents make much costlier.
Grade evidence by design and by relevance, measurement quality, uncertainty and replication. Write the chain observation, inference, mechanism, test.
Verdicts are dated judgments. The tally summarizes them and is not itself a result.

Key terms

Assumption audit: a six-step check of whether a method still works after its design conditions have changed.Also used in 1, 18
Load-bearing assumption: an assumption about a method's setting whose failure would make the method's output unreliable (term from Dewar 2002).Also used in 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14
Negation test: negating a candidate assumption to see whether the method's output becomes unreliable.Also used in 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
Unit of analysis: the specific mechanism or activity an audit examines.Also used in 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
Decision rule: the ordered questions Q1 to Q4 that assign a verdict (Section 2.5).Also used in 3, 4, 6, 7, 11, 12, 13, 14, 17
Holds, recalibrate, breaks, inverts: the assumption survives; new values or definitions restore the method; the method needs new variables, records or steps; an activity's scarce and abundant resources have traded places.Also used in 1, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18
Signpost: an observable event that would warn that a vulnerable assumption is failing (Dewar 2002).Also used in 1, 4, 18
Evidence chain: observation, inference, proposed mechanism, test.
Statement of practice: a description of what practitioners do and why; without documented cases or measurements, not validation.Also used in 4, 6, 8, 12, 18

Connections

Where this chapter and another point to each other most often.

Chapter 1: The Coordination Problem, 1968 to Now
From Chapter 1: In Chapter 2 he shows that a task that partitions but needs communication pays training costs, which grow linearly with headcount, and intercommunication costs, which grow as n(n − 1)/2: three workers have three pairs, ten have 45, twenty have 190.
Chapter 3: Life-Cycle Models When Iteration Is Cheap
From Chapter 3: The load-bearing assumption is: > Iteration is expensive, so phase boundaries and loop length are set by the cost of rework. It is an assumption about the process (Chapter 2).
Chapter 4: Requirements as the Primary Artifact
That is an inversion (Chapter 4, which also reports the verdict for the requirements document itself). ### Split verdicts Some areas get two verdicts.
Chapter 5: Size Estimation: What Function Points Still Measure
Size estimation (Chapter 5) breaks for lines of code as a stand-in for effort and survives for function points, which never measured code.
Chapter 6: Cost and Schedule Estimation After COCOMO
Take COCOMO II, the cost model in Chapter 6.
Chapter 7: Project Management and Where Value Is Earned
Hassan's book has no citations at all, a reason to check its claims (see the Claim audit), not to dismiss them. ## 2.7 A worked example: earned value Here is a complete audit of one method; Chapter 7 develops it.
Chapter 8: Coordination at Scale: Shared Context vs. Shared Messages
From Chapter 8: Apply the negation test from Section 2.3.
Chapter 9: Risk Management: Project Risk and Adversarial Risk
Risk management (Chapter 9) recalibrates for project risks, which still have estimable probabilities, and breaks for the simple unconditional probability-times-loss formulation applied to adaptive threats like prompt injection.
Chapter 10: Design for Change, and for Agents
Agents have a harder version of the limit: a finite context window and, unless the harness persists something, no memory between sessions (Chapter 10).
Chapter 11: Construction When Code Is Cheap
Now code is abundant and understanding is scarce (Chapter 11, which also gives the verdict for construction practices themselves). ### The decision rule Boundaries invite argument, so the rule is ordered.
Chapter 12: Reviews: Inspecting Evidence, Not Diffs
A method can keep its structure while the thing it operates on changes: inspection that examines an evidence package instead of a raw diff is still inspection, with the same roles and the same logic (Chapter 12), so its verdict is recalibrate.
Chapter 13: Testing, Evaluation, and Tests as Steering
From Chapter 13: All four judge assertion quality; none asks where the expected value came from, so a test that encodes the bug can satisfy all four. ## 13.4 The verdict Recalibrate. Chapter 2's decision rule (Section 2.5) first asks whether the assumption still describes agent-assisted work.
Chapter 14: Configuration Management When Agents Commit
From Chapter 14: Table 14.2 runs the negation test (Section 2.3).
Chapter 15: Reliability of Nondeterministic Systems
Reliability growth models (Chapter 15) assume the program is fixed between fixes, that execution is deterministic, and that failures are independent.
Chapter 17: Coordination Without Communication: The Bound and the Leak Test
This is a bound on a stylized game; Chapter 17 states its assumptions and limits.
Chapter 18: Running the Whole Life Cycle: The Engineer as Manager of an Agent Team
From Chapter 18: Appendix G lists them on one page. - M0: Baseline agent-session log. A log of your first agent session, building a small, specified utility, in a repository with a convention file and pre-commit hooks (Chapters 1 and 2).

See it on the map

02 The Assumption Audit: A Method That Outlasts the Tools