Chapter 9. Risk Management: Project Risk and Adversarial Risk

Learning objectives

After this chapter you should be able to:

Apply Boehm's six-step risk process, compute risk exposure and risk reduction leverage, and keep a risk register with mitigation and contingency plans.
State the load-bearing assumption of risk management and use the negation test to show where it fails in two different ways.
Describe the new project-risk classes agents introduce, separating what has been measured from what is practitioner experience.
Explain prompt injection, the confused deputy and the lethal trifecta, and say why a simple unconditional probability times loss fits an attacker-chosen event poorly and what a conditional threat model asks instead.
Build a two-part risk register: Part A with reversibility, blast radius and a basis for every probability, Part B with a structural check for every agent path.
Record an unreviewed merge as a risk decision, and state what evidence would overturn the verdict.

Opening case

A composite case, not a real engagement.

Six weeks into the dispatch dashboard build, the consultant's risk register has twelve rows. Each has a probability, a loss in hours, and an exposure that is their product. The top rows are familiar: misread reassignment rules, requirement churn, a date the client will not move. Near the bottom sits "Prompt injection: P = 0.05, L = 20 h, RE = 1.0 h. Action: monitor." Nobody could say where 0.05 came from. She had picked a small number because nothing had happened yet.

The chapter continues from here.

What the chapter covers

9.1 The classical method
- 9.1.1 Where it came from
- 9.1.2 The six steps
- 9.1.3 Risk exposure
- 9.1.4 The top-ten list and top-ten tracking
- 9.1.5 Risk reduction leverage
- 9.1.6 The register, and NASA's conditional likelihood
- 9.1.7 A worked register for Dispatch
- 9.1.8 What it got right
9.2 The assumption it depends on
9.3 What agents change
- 9.3.1 Project risks with no base rates
- 9.3.2 Adversarial risk: events someone chooses
9.4 The verdict
9.5 What replaces it: the two-part register
- 9.5.1 Part A: project risks, with new columns
- 9.5.2 Unreviewed merges as a recorded decision
- 9.5.3 Part B: structural adversarial exposure
- 9.5.4 The same example: Dispatch, both parts
9.6 Lab: Two-part risk register
9.7 What would make this verdict false

Lab and prediction

Lab: Two-part risk register

Feeds milestone M4. How the labs fit the project

The lab tests this prediction from the working paper the book grows from (Smith 2026a):

P7 Adversarial risks scored by simple unconditional probability × loss show no relationship to incidents; structural flags do

This chapter's own prediction: Part A rows with a measured basis (E1, E2) track observed frequency more closely than judged rows (E3).

A result that contradicts a prediction, honestly reported, is a result. You are graded on accurate records and reasoned interpretation, not on agreeing with the book.

Summary

Boehm's method (1991) ranks risks by exposure, RE = P(UO) × L(UO), chooses remedies by risk reduction leverage, and keeps a register with a named owner. For intentional threats NASA uses a conditional likelihood given that the hostile agent is present (NASA 2022).
Load-bearing assumption: risks have estimable probabilities: P can be grounded, and the event is not chosen by an adaptive adversary.
Project risk: recalibrate. Agents add risk classes with few or no base rates (one study measured 19.7% of recommended packages as hallucinated, with a wide spread by model; Spracklen et al. 2025) and make reversibility and blast radius matter. The fix is new rows, a stated basis for every P, and an irreversibility rule.
Adversarial risk: breaks, in the narrow sense that the simple unconditional P × L cannot be grounded for attacker-chosen events. Conditional-likelihood practice survives in structural form.
Replacement: a two-part register. Part A is probabilistic, with reversibility, blast radius and basis columns. Part B is a structural check per agent path: remove a leg first. Merging unreviewed code is a recorded decision with a named owner.
Prediction P7: unconditional P × L scores show no relationship to incidents, while structural flags do.

Key terms

Risk exposure (RE): probability of an unsatisfactory outcome times the loss if it occurs.Also used in 1, 3, 4, 5, 6, 7, 8, 11, 12, 14, 15, 17, 18
Risk reduction leverage (RRL): reduction in exposure per unit of cost.
Conditional likelihood: the likelihood that an intentional threat succeeds given that the hostile agent is present (NASA 2022).
Reversibility / blast radius: whether an action's effects can be undone at tolerable cost (R1 to R3) / what one failure can affect before a person can intervene (B1 to B3).Also used in 10, 14
Hallucinated dependency: a package or interface a model recommends that does not exist.
Prompt injection: an attack in which instructions planted in content a model processes are followed as if legitimate; indirect when planted in content the agent later reads.Also used in 2
Confused deputy: a program that uses its own authority for a caller's purpose (Hardy 1988).
Lethal trifecta: access to private data, exposure to untrusted content and the ability to communicate externally, in one agent (Willison 2025).
Structural exposure: risk that follows from an agent path's wiring, assessed without an unconditional probability.Also used in 16
Two-part register: Part A, probabilistic project risks; Part B, structural adversarial exposure.

Connections

Where this chapter and another point to each other most often.

Chapter 2: The Assumption Audit: A Method That Outlasts the Tools
Apply the negation test (Section 2.3) to each.
Chapter 6: Cost and Schedule Estimation After COCOMO
That differs from Boehm's unconditional P, and Section 9.3.2 returns to it. ### 9.1.7 A worked register for Dispatch Return to the dispatch dashboard from Chapters 5 and 6.
Chapter 12: Reviews: Inspecting Evidence, Not Diffs
Tantithamthavorn (2026, §10.4) lists seven patterns of AI-induced debt; what makes them distinctive, he says, is plausibility: they look like competent code from someone who does not know your project, so they pass review (Chapter 12).
Chapter 14: Configuration Management When Agents Commit
The rate is specific to model and prompt and will move with every release (Chapter 14), so treat it as a prior and replace it with a count from your own log.
Chapter 16: Roles and the Four Conditions for Assurance
It is owned, not scored: NASA's accountable individual applied to a merge, and condition 4 of Chapter 16. ### 9.5.3 Part B: structural adversarial exposure A path is an agent or agent role together with everything it reads, every tool it can call and every system it can touch.

See it on the map

09 Risk Management: Project Risk and Adversarial Risk