16 Roles and the Four Conditions for Assurance

Learning objectives

After this chapter you should be able to:

1. State the four conditions for an agent role to add assurance, and explain why a role that fails any one adds only the appearance of assurance.
2. Test a role with a seeded-flaw comparison, a claim-to-artifact audit, a channel inventory and an ownership check.
3. Name the nine roles, the characteristic failure of each, and the human who owns its decisions.
4. Distinguish implementation separation, diversity and measured independence, and design the check that measures what a same-model reviewer adds.
5. Write a three-scenario qualification check and an escalation matrix, and say when recertification is required.
6. Design a small agent team in which every role is checked against the four conditions and holds least privilege.

Opening case

A composite case, not a real engagement.

A five-person group is building an invoicing service for a distributor. Its engineering lead has set up a team of agents: an orchestrator, three implementers, a project-management agent, and a "red team" agent that attacks each feature before merge. All run on the same model. She designed the team so she would not have to read status reports.

The chapter continues from here.

What the chapter covers

1. 16.1 From agents to agent teams
   • 16.1.1 Where roles help, and where they harm
2. 16.2 The four conditions
   • 16.2.1 Incremental detection
   • 16.2.2 Evidence, not self-report
   • 16.2.3 Explicit channels
   • 16.2.4 Human ownership
3. 16.3 The nine roles
4. 16.4 Qualifying and recertifying agent roles
5. 16.5 Accountability as an engineering requirement
6. 16.6 Designing an agent team: the Dispatch example
7. 16.7 What would make these conditions unnecessary?

Lab and prediction

This chapter's own prediction: Across the class, the first-run pass rate on scenario 2 is lower than on scenario 1.

A result that contradicts a prediction, honestly reported, is a result. You are graded on accurate records and reasoned interpretation, not on agreeing with the book.

Summary

• A role adds assurance when it catches what the generator's checks missed and shows the evidence. The four conditions are necessary, not sufficient, and unvalidated.
• Incremental detection: measure what a checking role catches beyond the generator's checks, and the joint miss rate (P10). Separation and diversity may help; only a measurement establishes independence, and a correlated layer still adds detection.
• Evidence, not self-report: compliance by declaration accepts a claim without confirming it independently. Test with a claim-to-artifact audit and a tamper test.
• Explicit channels: inventory every route between roles; anything unlogged is a candidate hidden channel.
• Human ownership: a named person owns each role and decision, can reconstruct decisions from logs and can suspend the role. That nobody typed a change never negates accountability; ownership has a capacity limit.
• Qualify each role with three scenarios, recertify after any model change (Chapter 14), and let roles refuse. Dispatch shows the design: diversity by method, deterministic gates, an escalation matrix, least privilege.

Key terms

Incremental detection

the share of the generator's missed defects that a checking role catches.Also used in 1, 5, 7, 9, 11, 12, 15, 18

Implementation separation, diversity, independence

running layers as separate sessions or prompts; differing in model, method or information; a measured joint miss rate equal to the product of the miss rates.Also used in 12, 15

Compliance by declaration

accepting that a check happened because the actor says so (Hassan 2026, §9.9.6, p. 277).Also used in 7, 12, 13

Role owner / decision owner

who answers for a role's configuration, and who answers for the decision it informs.

Role card, escalation matrix

a repository file giving a role's owner, model, permissions and qualification record; a table of decision type, owner, consult path and required evidence.Also used in 8, 18

Connections

Where this chapter and another point to each other most often.

• Chapter 1: The Coordination Problem, 1968 to Now
  From Chapter 1: Chapter 16 develops and tests them; earlier chapters cite them by name. > 1.
• Chapter 3: Life-Cycle Models When Iteration Is Cheap
  Those are the four conditions this chapter states, tests and designs for. ## 16.1 From agents to agent teams Chapters 3 through 15 treated an agent mostly as one worker whose output a person checks.
• Chapter 4: Requirements as the Primary Artifact
  It needs an executable attack: replay the request as a different user and expect a refusal. - The four specification ambiguities. No agent role settles these; a person does (Chapter 4).
• Chapter 5: Size Estimation: What Function Points Still Measure
  Fed other agents' completion reports, it credits claims. - Estimation relies on calibration data (Chapters 5 and 6).
• Chapter 6: Cost and Schedule Estimation After COCOMO
  Fed other agents' completion reports, it credits claims. - Estimation relies on calibration data (Chapters 5 and 6).
• Chapter 7: Project Management and Where Value Is Earned
  They harm where a management agent would inherit an assumption that Part II found shaky. - Project management relies on "progress is measured by work completed" (Chapter 7).
• Chapter 8: Coordination at Scale: Shared Context vs. Shared Messages
  Chapter 8 compares architectures; this chapter asks a prior question: when does a role add assurance, and when only a confident report?
• Chapter 9: Risk Management: Project Risk and Adversarial Risk
  Calibrated on agents' own estimates, which tend to follow the happy path (Hassan 2026, §3.5, p. 116), it turns optimism into data. - Risk relies on "risks have estimable probabilities" (Chapter 9).
• Chapter 11: Construction When Code Is Cheap
  From Chapter 11: Ownership survives as accountability, not comprehension (Chapter 16, condition 4), and pairing survives as a second reader who checks (Chapter 12).
• Chapter 12: Reviews: Inspecting Evidence, Not Diffs
  With only one model family, diversity has to come from method and from the human, which is why the engineers still read the evidence package (Chapter 12).
• Chapter 13: Testing, Evaluation, and Tests as Steering
  For adaptive threats the simple unconditional probability misleads, and a model will still return one. - Testing relies on tests written "with intent independent of the defect" (Chapter 13).
• Chapter 14: Configuration Management When Agents Commit
  A testing agent sharing the generator's model and context may not have it. - Review and provenance rely on a history that records how each change was produced (Chapter 14).
• Chapter 15: Reliability of Nondeterministic Systems
  A harness that commits under a person's identity does not supply it, though that person still answers for the merge. - Layered checking multiplies miss rates, which needs independent failures (Chapter 15).
• Chapter 17: Coordination Without Communication: The Bound and the Leak Test
  For pairs of roles that make converge-or-split decisions, run the leak test of Chapter 17.
• Chapter 18: Running the Whole Life Cycle: The Engineer as Manager of an Agent Team
  From Chapter 18: Under the fourth condition from Chapter 16, you are the named human owner of every role on your team, and accountability does not transfer to an agent (Tantithamthavorn 2026, §6.8).

See it on the map

Further reading

• Knight and Leveson (1986). The classic test of the independence assumption.
• Kim et al. (2025). How often different models make the same mistakes, and why.
• Hassan (2026), §8.4 (pp. 235–247) and §9.9 (pp. 275–279). Qualification, escalation, delegation, compliance by declaration.
• Nissenbaum (1996). Why accountability erodes when many hands contribute.