The engineer as manager of an agent team

G.1 Labs by chapter

Table G.1 lists each lab by the heading it carries in its chapter (after "Lab:"), the milestone it feeds and the prediction it tests. "Chapter's own" means a prediction stated in that chapter without a P-number (Section G.4). A lab that "bears on" a prediction produces data relevant to it without testing it. A result that contradicts a prediction is a result: report it as it is.

Table G.1. Labs, milestones and predictions.

The Chapter 17 lab has two paths. The core path is for every student: implement the small game, confirm the bound by enumeration, and analyze supplied pilot-style logs, with no model calls. The extended path audits your own agent stack under four conditions at scale, about 8,000 model calls for four conditions at 1,000 rounds, and suits volunteers or a project group. Fill in the cost and time worksheet of Chapter 17 (Table 17.7) before choosing. Both paths keep pre-registration and honest statements of uncertainty, and the core path is enough for M3.

G.2 Rules that apply to every lab

1. Pre-register before you collect data. Write the question, the prediction and the quantity you will measure, date it and commit it (Appendix E, Section E.8, shows the form).
2. Keep the work log. Most labs use it as their dataset (Appendix B).
3. Report against the prediction. Say whether the prediction held, and if not, whether the verdict is wrong or the prediction was badly chosen. You are graded on accurate records and reasoned interpretation, not on agreeing with the book.
4. End with the closing question. What would have to change about agents for your conclusion to stop being true? (Appendix A, Section A.8.)

G.3 Milestones

Table G.2. The project milestones.

The work log runs from the first week through the final and is turned in with each milestone. From M3 on it also records escalations and refusals, with minutes waiting (Appendix B, Section B.2). The handoff happens after M4 (Appendix C). Chapter 18 describes the project in full; read its Sections 18.1 to 18.3 in week one.

G.4 Predictions

The ten predictions come from the working paper this book grows from (Smith 2026a, Table 7). Each can fail on your project's data. The Chapters column names the labs that test each one.

Table G.3. The ten predictions.

Chapters 13 and 16 bear on P10 without testing it. Chapter 18's lab tests P1, P2, P4, P5 and P8 again on your own whole-project data; at class size the pooled results are exploratory.

Eight chapters also state a prediction of their own, with no P-number.

• Chapter 5: specification hours track function points more closely than lines, and review hours track lines presented more closely than function points.
• Chapter 8: most coordination defects trace to undeclared dependencies, not to failures of declared channels.
• Chapter 9: Part A rows with a measured basis (E1, E2) track observed frequency more closely than judged rows (E3).
• Chapter 11: comprehension plus review hours exceed prompting hours; agent-native rework leaves fewer defects than a hand edit on a semantic conflict, at a higher review time.
• Chapter 13: agent-written suites have lower mutation scores than spec-derived acceptance suites, and at least one agent-written test fails once a specification-contradicting behavior is corrected.
• Chapter 14: changing only the model version changes measured behavior on a project's eval suite.
• Chapter 15: a growth-model forecast fitted on one baseline undershoots the defect count observed after a regeneration by more than its Poisson limit allows.
• Chapter 16: across the class, the first-run pass rate on scenario 2 is lower than on scenario 1.

Chapter 18 adds one (Section 18.4.5): of the questions the package should have answered, most ask for a reason or a state of affairs, not a rule.

G.5 Post-mortem and oral-defense outline

The final deliverable has two parts, both in Chapter 18 (Section 18.5). Draw every claim from your work log, not from memory.

The blameless post-mortem. Choose the event from the log: the one that cost the most hours against your estimate, or surprised you most. The subject is the system, not a person; "the agent did it" is not a root cause, because you configured the system the agent ran in and you own the merge. The same holds for the author of the project you inherited: the subject is the package.

Table G.4. Post-mortem sections (Chapter 18, Section 18.5.1).

Take the post-mortem through three stages.

1. Draft. Due shortly before M6.
2. Peer review. A classmate answers three questions in writing, reading your log and not only your prose. Can each claim in the timeline be traced to a log entry? Would removing the stated root cause have prevented the impact? Could each action item be checked?
3. Revision. Revise, and attach a note saying what changed in response and what you declined to change, with reasons.

The oral defense. The defense is individual and short, about 6 minutes. It has three parts.

1. Your own design. Defend your coordination design, say whether your M3 prediction of where coordination would fail held, and answer the one thing your inheritor's handoff audit says your handoff missed (Appendix C, Section C.6).
2. The inherited project. Demonstrate it from a clean checkout.
3. One hard question. The question comes from the readings or from a tension between the companion books. Answer it with your own data. Examples: is a handoff package a message or compressed shared context, answered from your question log? Your reviewer and your generator ran on the same model: what does your log say about whether they failed together? Pick one verdict in Table A.4 and say what your project's data say about it.

An answer that cites a log entry earns more than one that cites a book.

G.6 A suggested 14-week schedule

This is one way to fit the book into a semester. It is a suggestion, not a requirement; adapt it to your calendar. No week assigns more than two chapters, and the first four weeks assign five, not eight, because those are the weeks in which you also set up the log, write a specification and pre-register estimates.

Table G.5. Chapters by week.

Chapter 18 is the guide to the project, not a chapter to learn in one sitting, so the table lists it in three readings: Sections 18.1 to 18.3 in week 1 (the project, its milestones and the work log), Sections 18.4 to 18.8 at the handoff, and the whole again in week 13. Weeks 2 to 4 carry one chapter each, because you are also writing a specification and counting function points; the work log runs from day one.

Five notes on the schedule.

1. The four conditions arrive early, in brief. Chapter 1 gives the four conditions for assurance in a box (condition 1 is incremental detection), and the Part II chapters from Chapter 6 on cite them by name. That box carries you until Chapter 16 in week 7, where they are developed and tested. Chapter 16 sits before M3 because M3 needs it, and so before Chapters 12, 13 and 15, whose roles it discusses. Treat the role descriptions in Sections 16.3 and 16.4 as a preview, and return to them in weeks 12 and 13 when reviews, testing and reliability have been taught; rerun your role's qualification suite then (Chapter 16, lab step 4).
2. The leak test has a core path. Every student does the core path in the coordination week (week 6): implement the small game, confirm the bound and analyze supplied logs. The extended path, about 8,000 model calls for four conditions at 1,000 rounds, is for volunteers or a group, and can run from week 6 alongside later chapters. Give students the cost and time worksheet before they choose (Chapter 17, Table 17.7).
3. The handoff splits the semester. Weeks 1 to 9 are greenfield, on your own project. Weeks 10 to 14 are brownfield, on the project you inherit. Chapters 11, 12, 13 and 15 are all taught against inherited code. Chapter 14 and the handoff share week 9, so start the package in week 8.
4. The two reveals come late. Estimate vs. actual (M6) and the handoff data (P8) need the full log, so they land in weeks 13 and 14. Pool the class's held-out estimates in week 14 (Appendix E, Section E.9), and treat the pooled results as exploratory.
5. Fewer weeks. In a 10-week term, the two-chapter cap will not hold. Assign Chapters 3, 5, 13 and 15 as reading with shortened labs, run only the core leak-test path, and combine weeks 2 and 3, 8 and 9, and 10 to 12. Say in the first class which chapters are reading only.