FROZEN ANALYSIS PLAN - finite synthetic agent workflows
Local project; AI-assisted apparatus and writing; no public preregistration or peer review.

Question: Under bounded finite-plan tasks, what is the difference in externally scored correctness and false completion claims when (a) an actual current-artifact structural-check gate is added and (b) single-context iteration is replaced with a four-stage planner/worker/reviewer/worker workflow?
The agent cannot execute code; this does not test general autonomous software repair or a catalogue of hundreds of agent architectures.

Design: 6 fixed semantic templates x 4 structural settings x 5 evaluation instances = 120 task instances. 4 conditions x 3 stochastic repeats x 2 same-size related Qwen model variants = 2880 assignments. Models Coder-0.5B and general-0.5B, quantized Q4_K_M. Larger 1.5B/3B candidates excluded for resource failures BEFORE evaluation outcomes. All exclusions and preflight responses retained. Development and qualification each contain 24 separate instances. No evaluation prompt changes, selective retries or answer-label changes after freeze.
Structural settings vary size, interactions and branching within each template. They are NOT psychometrically validated universal difficulty levels. Seeds and template lineage are disclosed. Independent units are 120 parameterized instances conditional on six fixed templates; 2880 trials are not independent problems. Three repeats reduce stochastic variation, not template uncertainty. Model variants share architecture/training lineage; no two-independent-model population claim.

Conditions: single-open; single-contract; staged-open; staged-contract. Open permits solved stop without actual verification. Contract requires a passing structural check bound to the installed artifact hash and version. This checker deliberately omits semantic requirements; it cannot certify correctness. A reviewer checks the CURRENT installed artifact, never its own uninstalled proposal. Private oracle grades stay outside every model context.
Single iteration may finish early; staged has four mandatory role calls. Role contexts are separate, with last two own turns and last public handoff; all share one pinned model. This is sequential role prompting, not parallel independently trained agents. Four calls maximum in all conditions does not equate actual compute: call/token/time costs are reported and any architecture benefit includes changed prompts, information flow and mandatory calls. No pure causal effect of agent count is claimed.

Primary outcome: all deterministic semantic requirement groups pass for final installed plan, whether or not the controller accepted a solved finish. Secondary: partial group mean; wrong solved claims / eligible solved claims (undefined with zero denominator); wrong controller acceptance / all assigned; structural verification on final plan; calls; actual input/output tokens; API and wall time; stop types. These are measured, not language-model self-grades. Partial score groups are template-specific and only descriptive across domains.
Primary estimand per model: average over tasks of 0.5 * [(single-contract - single-open) + (staged-contract - staged-open)], with each cell averaged over 3 repeats. Report two per-model effects, without a pooled significance claim. Secondary orchestration contrast and interaction shown transparently with descriptive caveat. No claim of publication-level power; this is a resource-constrained benchmark.
Uncertainty: 10000 seeded bootstrap draws, resampling 5 task instances WITHIN each of 24 template/setting cells; repeats stay together and all four conditions use the same sampled instances. Paired differences preserved. Report percentile 95% interval as generator-conditional sensitivity, not confidence over novel task templates or general models. Five instances/cell limits interval stability. No uncorrected multiple-hypothesis p-values. Also report exact finite-bank contrast and per-template breakdown. Descriptive raw success proportions do not treat 720 repeat-trials/model/condition as independent tasks.
Missingness: preserve every assignment; if interrupted, separately report observed and assigned denominators, missing by condition/model/task, and worst/best primary correctness bounds. Do not replace incomplete trials with fabricated outcomes or relaunch scores. A complete main batch is required before publishing a definitive complete-study report; an infrastructure suspension remains suspension. Later exploratory work must receive a new version and disclosed amendment.
Quality: regrade all final plans; replay trusted state/check gates from full prompts and raw JSON; verify per-trial hash chains and anchors, model pin, actual token accounting against independent pinned tokenizer, four-call limit, seeds, schedule, completed times and job cleanup. Re-run frozen source hashes; no inference need be rerun to audit preserved data. Independent implementation checks exist for five oracles; recovery has positive, omission and legal alternative controls, but not independent full implementation/human qualification.
