Hiro development journal

Completing Hiro's first governed arithmetic promotion

2026.08.21 at 21:52:39 America/Los_Angeles Promotion completed, platform hardened, queue requalified and running Machine-readable JSON

Executive summary

Hiro completed a genuine autonomous code promotion. Candidate f0c7363 repaired a reproduced arithmetic-response failure, cleared contemporaneous public and held-out evaluation without score or category regression, passed security and four valid moderate-risk canary checkpoints, passed the governor's complete 695-test gate, and fast-forwarded the active branch.

The terminal path exposed two additional worktree-infrastructure defects rather than candidate defects: Stage 5 worktrees lacked the ignored pinned runtime, and Windows line-ending conversion invalidated byte-frozen benchmark assets. After repairing only that infrastructure, the same candidate revision passed the exact affected tests and the governor's unchanged complete suite.

The platform was then hardened so every Stage 5 integration worktree provisions the pinned runtime, byte-frozen benchmark directories are excluded from line-ending conversion, and live canary probation starts when asynchronous construction finishes instead of when its scheduler tick began.

Post-promotion validation passed 698 repository tests and five exact-revision end-to-end qualification cycles. Hiro restarted successfully; task and benchmark endpoints return 200; the promoted incident is implemented; the queue is qualified and has already resumed its next candidate.

Work completed

Repair the final approval path without weakening it

Completed

An earlier fully canaried candidate was rejected because a platform test required the repaired arithmetic defect to remain reproducible. The test was decoupled from live product behavior by injecting its failed replay deterministically.
FullSuiteGateError now retains the complete final-test receipt. A bounded harness-revision retry is available only to fully canaried automatic incidents, so an evaluator failure cannot be relabeled as candidate merit.
Three exact platform revisions passed complete repository suites and five-cycle qualifications while builder guidance was narrowed and retry lineage was preserved.

Construct and evaluate the retained arithmetic candidate

Completed

Qwen's first rebuilt patch rewrote every recognized arithmetic response. Stage 4 correctly rejected it after public and held-out scores each fell by 0.025 and reasoning fell by 0.0667 in both suites, despite passing security.
Builder guidance was changed from formatting to recovery, a production guard requires valid multi-sentence arithmetic explanations to remain byte-for-byte unchanged, and construction diagnostics now carry the exact failed assertion into the next bounded revision.
Candidate sandbox-20260822065925-910656d2-01 changed only core/response_envelope.py and its autonomous regression test. Across 192 contemporaneous executions, public baseline and candidate scores were both 0.9291667; held-out scores were both 0.9041667; category regressions were empty; security passed; latency ratios 1.003 and 1.047 remained below 1.10.

Run an honest moderate-risk canary

Completed

The scheduler initially used a timestamp captured before the 20-minute asynchronous build. The live start was corrected forward to actual Stage 5 completion at 2026-08-22T07:19:26Z, and one raced premature receipt was invalidated in an append-only event.
Valid checkpoints 0, 5, 15, and 60 all passed. Every checkpoint used a fresh originating execution, captured-boundary replay, and independent platform tests; candidate-authored tests were excluded from independent canary evidence.
The timing path is now permanently fixed: deterministic qualification runs retain supplied test time, while live runs record canary start and deadline from actual candidate readiness.

Diagnose and repair the terminal worktree failure

Completed

After checkpoint 60, the first governor suite ran 695 tests: 685 passed and ten failed with Windows child-process launch and frozen-asset integrity errors. The failures clustered in benchmark, launcher, and preflight tests and did not involve the arithmetic change.
Direct reproduction proved two causes. The integration worktree lacked its ignored .runtime junction, so tests could not launch the pinned interpreter. Git's global core.autocrlf setting also converted byte-frozen benchmark and harness files in the external checkout.
After provisioning the runtime and restoring exact Git-blob bytes, all 17 affected tests passed. The same candidate revision, with the same four canaries and unchanged governor policy, then passed all 695 tests in 391.04 seconds and fast-forwarded the active branch.

Harden, requalify, and restart the live system

Completed

Platform revision 162b0d3 provisions the host pinned runtime in Stage 5 worktrees, protects frozen benchmark trees from line-ending filtering, and fixes post-build canary timing. New behavioral tests cover each contract.
A fresh external worktree passed all 17 benchmark-integrity, launcher, and preflight tests. Its cleanup exposed a Windows junction-removal hazard that removed the shared virtual environment; the environment was rebuilt from checked requirements and revalidated before restart.
The final promoted-and-hardened head passed 698 tests in 402.78 seconds and five qualification cycles. Hiro restarted with its checked launcher; local task and benchmark endpoints return 200; Qwen 3.8 remains running.

Decisions and reasoning

Do not weaken score, category, security, canary, or full-suite gates to manufacture a promotion.
Classify reproducible runtime, checkout, and evaluator failures as infrastructure only after direct candidate-worktree evidence separates them from candidate behavior.
Reuse the exact fully evaluated candidate after repairing its worktree infrastructure, rather than rebuilding it or discarding earned canary evidence.
Make canary time fail-conservative: asynchronous build time never counts as probation time.
Require fresh external-worktree verification, a complete repository suite, and five-cycle exact-revision qualification before resuming the live queue.
Keep prompt-injection, hacking, scope, and invariant checks intact while removing unrelated text-style rejection behavior from candidate merit decisions.

Validation and evidence

Check	Status	Result
Candidate Stage 4 comparison	passed	192 paired executions; public score 0.9291667 versus 0.9291667; held-out score 0.9041667 versus 0.9041667; no category regression; security passed; both latency ratios below 1.10.
Corrected moderate-risk canary	passed	Valid checkpoints 0, 5, 15, and 60 passed after probation was reset to actual candidate readiness. Each used fresh originating, captured replay, and independent-test evidence.
Governor promotion gate	passed	The unchanged governor ran 695 tests in the repaired candidate worktree; all passed in 391.04 seconds. Main fast-forwarded from 7e5fb04 to candidate f0c7363.
Post-promotion platform tests	passed	73 focused tests passed, a new external worktree passed 17 integrity/launcher tests, the rebuilt pinned environment passed 19 restoration checks, and the complete repository passed 698 tests in 402.78 seconds.
Exact revision pipeline qualification	passed	Five of five end-to-end cycles passed on 162b0d33. Packet qualification-20260822T090024Z.json; SHA-256 d685f456d2858e5e8c450973aa36b72d844eed9110e44240533ce14ca7a13e45.
Live restart and queue	passed	Task and benchmark endpoints returned 200; qualification is current; incident-d2074a2645707df25a06f74e is implemented; implemented count is four; the scheduler resumed with another candidate.

Current state

Candidate f0c736390ef73994e0c9ab73591ac1a9078d2d48 is promoted and recorded as implemented.
Hiro is running on qualified platform revision 162b0d33b189f3347b5af25bada872a7cdabdd1f with Qwen 3.8 available.
The active queue reports 251 actionable items, four implemented incidents, and a subsequent candidate already in progress.
No score, category, security, canary, or governor threshold was waived for this promotion.

Next steps

Observe the next automatically selected candidate through the same construction, paired-evaluation, canary, and governor path.
Add an explicit junction-safe worktree cleanup helper and its real Windows test before automating removal of old integration worktrees.
Register the hiro_contract pytest mark to remove the known warning without changing evaluation behavior.
Use the completed promotion as the first production calibration point for future evaluator variance and causal-attribution analysis.