Appendix E. Estimation Worksheet: Four Models

E.1 Rules

Estimate the total human hours for the same project with four models, before construction starts. Units are human hours throughout, so the models can be compared. Chapter 6 explains the models and the reasoning behind the agent-aware one. Model 1 is the naive baseline: COCOMO II at nominal settings, with no reuse adjustment and no local calibration. Model 2 changes TOOL alone. Section E.6 adds COCOMO II at its strongest, which is the comparison that P4 names.

1. Fill in every field. Write the basis for every number.
2. Date the worksheet and commit it. Do not change it afterward (Section E.8).
3. Keep a fifth line for the recalibrated agent-aware estimate you make after the first milestone. Never overwrite the original four.
4. Quote the agent-aware result as a range with the base case marked, never as one number.

E.2 Model 1: function points to lines to COCOMO II

Fields:

1. Unadjusted function points (FP): from Chapter 5. ______
2. Lines of code per FP: language ratio, with its source or the word "illustrative." ______
3. Size (KSLOC): FP × ratio ÷ 1,000, adjusted for reuse and requirements volatility. ______
4. Scale factors: PREC, FLEX, RESL, TEAM, PMAT. Look up the values for your ratings in the model manual. ______
5. Effort multipliers: the seventeen. Leave nominal (1.00) unless you can write the reason for another rating. List each exception with its reason. ______
6. Results: PM, hours, TDEV, average staff. ______

The equations (Chapter 6, Section 6.1.2):

PM = A × Size^E × ∏ EMᵢ

E = B + 0.01 × Σ SFⱼ

TDEV = C × PM^F, with F = D + 0.2 × (E − B)

with A = 2.94, B = 0.91, C = 3.67 and D = 0.28 (the 2000 calibration). Size is in KSLOC, PM in person-months, TDEV in calendar months, and one person-month is 152 hours. Hours = PM × 152. Average staff = PM ÷ TDEV.

At nominal settings the five scale factors are PREC 3.72, FLEX 3.04, RESL 4.24, TEAM 3.29 and PMAT 4.68, which sum to 18.97 and give E = 1.0997.

E.3 Model 2: COCOMO II with TOOL very high

Repeat Model 1 with the tool-use multiplier TOOL set to very high. Its calibrated range runs from 1.17 (very low) to 0.78 (very high) (Chapter 6, Section 6.1.2). Everything else stays as in Model 1.

• TOOL rating and value: 0.78. ______
• PM, hours, TDEV: ______

This is the change from the nominal settings through TOOL alone. It is not the most COCOMO II can say about generated code; Section E.6 uses the rest of the model.

E.4 Model 3: story points times velocity

Fields:

1. Total story points (SP): from your backlog. ______
2. Velocity: story points completed per sprint. State where the number came from, and whether it was measured before you used agents. ______
3. Human hours per sprint: hours you expect to spend on the project in one sprint. ______
4. Sprint length (weeks). ______
5. Results:

Sprints = SP ÷ velocity; Hours = Sprints × hours per sprint; Weeks = Sprints × sprint length

Velocity counts what a team's definition of done counts. With agents, a ticket can close at code-complete, long before the work is verified (Chapter 2, opening case; Chapter 7). Say in the basis which meaning of "closed" your velocity used.

E.5 Model 4: the agent-aware model

Fields (the inputs of Chapter 6, Section 6.5.1):

• FP: function points. ______
• s: specification hours per function point. ______
• t_s, n: steering hours per agent task, and the number of agent tasks (about 200 delivered lines each). ______
• L: delivered lines, forecast from your function point count (Chapter 6, Section 6.5.4). ______
• γ: generation ratio, first-pass lines presented for review per delivered line (γ ≥ 1). Each presented line counts once, the first time a reviewer reads it; alternatives and discarded attempts count, re-reading does not. ______
• r: review rate, lines per hour. ______
• P: acceptance properties or checks, forecast the same way. ______
• v: hours per property to write, run and investigate. ______
• ρ: rework fraction: re-review, re-verification and re-prompting after defects, as a fraction of first-pass review plus verification. ______
• L_existing, r_c: lines of existing code to understand, and the rate at which you read unfamiliar code. Zero for greenfield work. ______
• Product multiplier on review and verification (RELY, CPLX), if above 1.00. ______
• w, T, p: loaded hourly rate, tokens consumed, price per token. ______
• h_week, agent critical path: human hours per week, and the longest chain of agent runs that cannot be parallelized. ______

The equations (Chapter 6, Sections 6.5.1 and 6.5.2):

H = H_spec + H_steer + (1 + ρ) × (H_review + H_verify) + H_comp

H_spec = s × FP; H_steer = t_s × n; H_review = γL / r; H_verify = v × P; H_comp = L_existing / r_c

Cost = H × w + T × p

TDEV ≥ max(agent critical path, H / h_week)

Symbols as defined in Chapter 6. Every logged hour maps to one term, and the other category is not modeled (Appendix B, Table B.2). Write the other three scenarios of Chapter 6, Section 6.5.3, in the same commit: optimistic, conservative (the verified careful-review rate) and stress. Each changes several parameters together from your base case; s and v stay at base. The scenarios show how far apart plausible workflows lie; they are not a probability interval.

To calibrate after the first milestone, compute s, t_s, γ, r, v and ρ from your log (Appendix B, Section B.7), re-estimate the remaining work, and record the result as a fifth line.

E.6 COCOMO II at its strongest

A replacement should beat the best version of what it replaces (Chapter 6, Section 6.5.6). Write the settings below in the pre-registration commit, with the basis for each, before you compute anything and before you see an actual. Take them from the model manual's definitions, not from the answer you want.

• Reuse. Treat the generated lines as adapted code (ASLOC) and convert them to equivalent new lines.
• Automatic translation. If you argue for it, set AT, the percent of code translated by tool, and ATPROD, its productivity.
• Local calibration. Fit A to projects other than yours.

The equations (Chapter 6, Section 6.1.6):

ESLOC = ASLOC × (1 − AT/100) × AAM

AAM = [AA + AAF × (1 + 0.02 × SU × UNFM)] / 100 when AAF ≤ 50; [AA + AAF + SU × UNFM] / 100 otherwise

AAF = 0.4 × DM + 0.3 × CM + 0.3 × IM

PM_auto = ASLOC × (AT/100) / ATPROD

DM and CM are the percent of design and code modified, IM the integration and test effort as a percent of normal, AA assessment and assimilation (0 to 8), SU software understanding (10 to 50) and UNFM unfamiliarity (0 to 1). ESLOC replaces Size in the effort equation. Fields:

• ASLOC: ______ DM, CM, IM: ______ AA, SU, UNFM: ______ AT, ATPROD: ______
• Basis for each setting: ______
• Local A: computed at the reveal, from classmates' actuals only (Section E.9). ______

Chapter 6 fixed its central and favorable settings before computing, as an example. Central: DM 10, CM 20, IM 100, AA 8, SU 30, UNFM 1.0. Favorable: DM 5, CM 10, IM 50, AA 4, SU 20, UNFM 0.6. Its black-box row (DM 0, CM 0, IM 10, AA 2) shows what the mechanism can do for a component that arrives already tested, and is not a forecast.

E.7 The Dispatch column

The Dispatch running example (Chapter 6, Sections 6.1.4, 6.5.3 and 6.5.6), with illustrative parameters: 120 function points, about 50 lines of Java per function point, 6 KSLOC delivered, s = 0.25 h/FP, t_s = 1/3 h for each of n = 30 agent tasks, γ = 2.0, r = 400 lines/hour, P = 40, v = 0.5 h/property, ρ = 0.3. Nothing here has been tested against an outcome.

Table E.1. The Dispatch estimates, all in human hours.

The last column is the value of A (or ATPROD) at which that COCOMO II row would equal 105 hours. The central reuse setting needs an A 21 times smaller than the global value. A local A is how COCOMO II is meant to adapt, but then A is doing the work of the missing terms; only prediction on projects it was not fitted to can show that it can.

The agent-aware working, term by term: H_spec = 0.25 × 120 = 30 hours; H_steer = (1/3) × 30 = 10 hours; H_review = (2.0 × 6,000) ÷ 400 = 30 hours; H_verify = 0.5 × 40 = 20 hours; rework = 0.3 × (30 + 20) = 15 hours; H_comp = 0 (greenfield); H = 30 + 10 + 1.3 × 50 = 105 hours. Review, verification and rework are 65 of the 105.

Table E.2. Dispatch scenarios for Model 4. All parameters illustrative.

Quote Dispatch as 90 to 200 hours with 105 as the base case. The base case uses the fast review rate, so a fixed-price bid should carry the conservative figure or more. The example does not fix a story-point count, so Model 3 has no canonical value. Use your own project's backlog.

E.8 Pre-registration statement

Put this in the repository with the four estimates and the Section E.6 settings, in the same commit. The commit hash is the proof of date.

PRE-REGISTRATION
Project: ______   Author: ______   Date: ______
Four estimates of total human hours, before construction:
  1. COCOMO II, nominal (naive baseline)   ____ h
  2. COCOMO II, TOOL very high              ____ h
  3. Story points x velocity                ____ h
  4. Agent-aware: base ___ optimistic ___ conservative ___ stress ___ h
COCOMO II at its strongest: DM __ CM __ IM __ AA __ SU __ UNFM __
  AT __ ATPROD __ ; basis written before any computation.
Every parameter and its basis is in this commit. None will change.
A recalibrated estimate after the first milestone is line 5, separate.
Prediction (P4): model 4 has the lower held-out error.
Which model wins, and why: ______      Commit: ______

E.9 Accuracy and held-out validation

When the project ends, fix the comparable actual (Chapter 18, Section 18.6): your hours before the handoff plus the inheritor's hours on the item you deferred, excluding comprehension and the instructor's change request. Report a second figure that includes comprehension, which is what the handoff cost. Then complete Table E.3.

Table E.3. Accuracy worksheet. Rows 6 to 8 are held out: computed without your own actual.

MRE = |actual − estimate| / actual

Also record the signed error, (estimate − actual) ÷ actual, so you can see whether a model errs high or low. Check the split of hours by category as well as the total: a model with the right total and the wrong split is right for the wrong reason.

Fitting is not predicting. Parameters computed from your own log reproduce its hours by construction (Appendix B, Section B.7), so a close fit says nothing about prediction. An actual that falls inside a range produced by refitting parameters, or inside your scenario range, is compatible with the model; it does not show the model is right (Chapter 18, Section 18.6). Report that compatibility under its own name.

The test. With N completed projects, estimate project i from the other N − 1, for every project in turn (leave-one-project-out).

• For COCOMO II, set A₋ᵢ = exp(mean over j ≠ i of ln(Hⱼ / Ĥⱼ)), where Hⱼ is project j's comparable actual and Ĥⱼ its estimate computed with A = 1 (the manual's method, Chapter 6, Section 6.5.6). Use the Section E.6 settings, fixed in advance.
• For the agent-aware model, set each parameter to its pooled value in the other projects' logs.
• Both models see the same function point count and the same forecasts of L and P, and are scored on the same comparable actuals.

PRED(25) is the fraction of estimates with MRE ≤ 0.25. It is meaningful only across many projects: pool your class's held-out results and report the median MRE and PRED(25) for each model, with fitted and held-out figures side by side and labeled. Use them to compare models on the same projects, not to judge one estimate. COCOMO II has one fitted constant here and the agent-aware model six, which helps it in-sample and is why only held-out results count. A class of 20 or 30 projects is a pilot.

E.10 After the actuals

For your own project, separate two kinds of error.

• Bad parameters: the model's form was right, but a value was wrong. Name the parameter, its true value from the log, and the estimate you would have made. This is a diagnosis after the fact, not a validation.
• Bad model: no parameter values could have produced the right answer, because the model has no term for where the hours went.

Then write one paragraph on what you would change in the agent-aware model. Prediction P4 is that the agent-aware model has lower held-out error than COCOMO II at its strongest. If your numbers say otherwise, say so; a result that contradicts the prediction, honestly reported, earns the same credit as one that confirms it.