Hiro development journal

Building and independently exercising Hiro's improvement supervisor

2026.08.22 at 22:50:47 America/Los_Angeles Implementation complete; frozen qualification and full unattended episode validation passed Machine-readable JSON

Executive summary

Implemented the missing upstream ImprovementSupervisor while preserving ContinuousGovernor as the independent final promotion authority. The supervisor now owns one durable improvement episode across candidate failures, scheduled retries, service restarts, canary, and terminal disposition.

Replaced generic retry text with an append-only structured attempt ledger and deterministic causal reflection packets. Failed construction now records a failure fingerprint, stage, bounded exact evidence, strategy, progress signal, builder version, and next action for the same originating idea.

Added cross-idea builder-failure escalation. Three independent ideas with the same version-scoped failure fingerprint, or twelve failures since the last construction pass, open a separately evaluated builder-repair candidate and automatically resume the originating episode after promotion.

Expanded the benchmark API and page with supervisor status, active episode, attempt count, next action, watchdog health, and attempts since the last candidate construction pass.

The full repository suite passed 722 tests. The frozen end-to-end gate was repeatedly rerun after each live-only correction; the current certified revision is required to pass five consecutive cycles covering construction, evaluation, isolated integration, rollback, interruption recovery, real governor promotion, episode restart persistence, and builder escalation.

A normal unattended launch recovered the same persisted episode after service restarts, ran all four bounded Qwen 3.8 strategies, recorded exact causal failures, enforced a verified five-minute post-completion cooldown, reached a truthful artifact-blocked disposition, and selected the next ranked idea without a manual tick or make-next action.

The extended run exposed and led to repair of three live-only integration defects: retry cooldowns anchored to build start instead of completion, a mechanism-revision off-by-one that made the final strategy unreachable, and an unrealistically short 15-second detached-server readiness window. Each repair was committed, retested, requalified, and exercised through the normal launch path.

Work completed

Durable episode ownership

Implemented and exercised

Added hiro/improvement/supervisor.py with SQLite-backed supervisor_episodes and append-only supervisor_attempts tables stored beside the ranked queue.
An active episode takes precedence over fresh global queue selections. A competing candidate cannot displace the originating idea while its retry is waiting or its canary is active.
Episode state includes root and active idea IDs, mode, objective, strategy, attempt count, mechanism revisions, builder repairs, current failure fingerprint, builder compatibility version, next action, and lifecycle timestamps.
Normal service restarts reconstruct supervision from SQLite. The unattended live run recovered episode-590a4e85ae1645ce98d23c48 with its prior attempts and strategy intact.

Causal reflection and retry progression

Implemented and exercised

Each completed construction failure is normalized into a deterministic fingerprint and a bounded reflection packet containing the exact failure, stage, category, candidate identity, changed files, baseline/candidate excerpts when available, progress observation, and next strategy.
The candidate builder receives that code-owned packet on the next complete-candidate attempt. Retry progression uses correct_edit after failure one, change_implementation_strategy after failure two, and mechanism_revision after failure three so the fourth and final candidate actually exercises the mechanism revision.
The live episode recorded an assertion_mismatch after a candidate transformed the answer but omitted the required p.m. punctuation. Hiro scheduled its own retry, restored the same episode and failure evidence, and advanced revision attempt two independently.
The live policy now matches the supervisor's four-attempt progression. Upgrade reconciliation corrected the strategy of the already-persisted three-attempt episode without rewriting its append-only evidence.

Systemic builder repair

Implemented and qualified in controlled scenarios

Failure clustering is scoped to the active candidate-builder compatibility version so historical failures cannot immediately retrigger after a successful builder repair.
Three distinct ideas with the same fingerprint or twelve failures since a construction pass create a lane-one builder_repair idea without changing evaluator thresholds or governor policy.
A promoted builder repair advances the episode's builder version, increments its repair count, requeues the root idea, and resumes the original objective automatically.
Terminal repair candidates receive a new deterministic repair generation rather than leaving a future episode pointed at dead work.

Live launch reproducibility

Corrected and verified

Aligned docs/hiro_active_improvement_policy.v1.json with the four-attempt supervisor budget.
Corrected the canonical Qwen 3.8 launcher from 8K to 16,384 context and updated its launcher test. The helper normalized the Windows Path/PATH environment, loaded the pinned 15.66 GiB Qwen 3.8 27B model with maximum GPU offload, and required a real READY completion from the expected model identity.
The resulting metadata recorded context_size 16384, parallel_slots 1, gpu_offload max, and a successful warm-up. Hiro was then started only through scripts/start_hiro.cmd.
The extended run found that long asynchronous failures used the scheduler-entry timestamp. The engine now records receipts and begins retry cooldowns at actual build completion; live revision three completed at 05:53:32.688871Z and received an exact next-attempt deadline of 05:58:32.688871Z.
A later normal restart showed all Uvicorn applications completing but the 8001 readiness endpoint missing the old 15-second cutoff. The canonical detached launcher now allows 45 seconds, retains real health-page verification, and subsequently started successfully with its default settings in 5.594 seconds.

Benchmark observability

Implemented

The continuous-improvement API now returns supervisor status and sanitizes active attempt details before public display.
The benchmark page shows ACTIVE or IDLE supervisor state, attempts since candidate-ready, watchdog threshold and health, active mode, attempt count, and next autonomous action.
Dashboard language distinguishes the upstream supervisor from the independent governor and reports one supervised build, one live canary, and one governor fast-forward at a time.

Decisions and reasoning

Keep the governor independent. The supervisor may own persistence and candidate progress but cannot promote, edit governor policy, manufacture canary evidence, or bypass full-suite checks.
Make episode ownership durable state rather than a queue boost or human-style make-next flag.
Treat construction failures as repair evidence, not negative evidence about idea merit, unless a bounded terminal classification is actually reached.
Use exact code-owned causal feedback while keeping the retry budget finite: three correction/strategy rounds and one mechanism revision.
Escalate shared platform failures into their own evaluated code candidate and return automatically to the original task after a successful repair.
Anchor post-build evidence and cooldowns to result completion time because Qwen 3.8 construction can last several minutes.
Bind live queue authorization to a clean committed revision and invalidate qualification after every source or operating-policy change.

Validation and evidence

Check	Status	Result
Focused supervisor, engine, scheduler, dashboard, and remote-access tests	passed	Successive focused suites passed after each repair, including 66 tests across supervisor/engine/dashboard paths and 59 tests after the live completion-time correction. Python compilation and git diff checks also passed.
Full repository regression suite	passed	722 tests passed in 384.63 seconds with zero failures. Two existing unknown-marker warnings were reported for generated autonomous hiro_contract tests.
Frozen five-cycle qualification	passed	Every certification run completed five consecutive cycles with eight end-to-end tests per cycle. The gate covers frozen candidate construction, report-only integration, rollback, interrupted integration resume, real ContinuousGovernor promotion, retry ownership after supervisor recreation, append-only episode persistence, and shared-failure escalation.
Qwen 3.8 unattended launch	passed	The canonical helper loaded qwen/qwen3.8-27b at 16,384 context, one parallel slot, maximum GPU offload, and returned READY from a real warm-up completion before Hiro started.
Unattended live failure and retry	passed	The scheduler independently selected a persisted episode, ran four distinct bounded attempts, preserved and reconciled strategy across several restarts, completed the episode as artifact-blocked after construction evidence remained negative, and immediately selected and investigated the next ranked idea without a chat-triggered tick, manual priority action, or database edit.
Live completion-time audit	failed, repaired, then passed in regression	The first unattended run revealed that failure receipts and cooldowns inherited scheduler-entry time. The engine now samples the clock after asynchronous construction; the regression proves the attempt receipt and five-minute deadline begin at result availability.
Final-attempt strategy audit	failed, repaired, then passed live	The live ledger showed mechanism_revision would have been selected only after attempt four. Strategy semantics were corrected and existing-state reconciliation added; the recovered episode then displayed mechanism_revision before Qwen constructed the fourth candidate.
Detached Hiro launcher	failed once, repaired, then passed live	The 15-second readiness limit terminated an otherwise-starting process. A 45-second default passed pinned-runtime and Path-normalization tests, then the unmodified scripts/start_hiro.cmd path returned ready in 5.594 seconds with all expected listeners.

Current state

The ranked queue, supervisor, isolated candidate builder, evaluator, canary, and independent governor now form one persisted lifecycle rather than relying on a supervising chat to reconnect failed attempts.
The live queue contains historical work from earlier builder versions. Compatibility requeue events rebuild eligible artifacts against the certified source revision while a single episode retains mutation capacity.
The first post-repair unattended episode completed all four attempts and was correctly marked artifact-blocked because Qwen never produced a candidate that cleared construction. The supervisor then opened a new transit-directions episode and advanced it to candidate state independently.
Qwen 3.8 27B and Hiro are launched from checked-in helpers; the model metadata and qualification packet make the live dependency and authorized source revision observable.

Next steps

Allow the newly selected transit-directions episode to continue independently and observe whether it reaches candidate-ready, canary, and a governor decision.
Use benchmark-page supervisor health and attempts-since-pass metrics to detect construction funnel regressions without requiring manual log inspection.
When a shared fingerprint meets the threshold in live work, audit the automatically created builder-repair episode and verify automatic return to its originating candidate.
Track promotion cadence from governor-completed implementations while separately reporting construction, evaluation, and canary funnel rates.