Hiro development journal

Promotion controller clears 10 promotions, 10 rollbacks, and production validation

Phase 2 empirically demonstrated; autonomous discovery and meta-improvement remain gated Machine-readable JSON

Executive summary

Built an accelerated qualification mode around the existing controller qualification runner to distinguish state-machine reliability from production-duration configuration without bypassing promotion logic.

Ten consecutive accelerated golden promotions passed from clean reproducible state. Every cycle used canonical queue persistence, the real governor decision, a durable promotion transaction, Git fast-forward, detached restart broker, live revision verification, runtime probation, and finalization.

Ten consecutive intentional rollback cycles passed. Each first proved that the candidate revision and qualification artifact were running, then deliberately failed the post-activation probation probe, created an additive Git revert, restarted Hiro on the rollback revision, and verified that the restored Git tree matched the baseline.

A fresh production-configured golden validation then ran the complete repository regression suite. All 793 tests passed in 489.16 seconds before real activation, production checkpoint traversal, and transaction finalization.

The primary Hiro service was restored after qualification and reports matching loaded and checkout revision with a healthy model connection.

Phase 2 is now empirically demonstrated. Phase 3 autonomous discovery was not enabled, and no meta-improvement was attempted or claimed.

Work completed

Runtime cost analysis

Completed
  • Corrected the earlier wall-time estimate using the persisted Phase 1 trace. Model-driven construction and evaluation consumed about fifteen minutes and thirty-six seconds; the full governor suite consumed seven minutes and fifty-two seconds; restart reconciliation consumed about ten seconds.
  • The configured runtime probation is eight hours, not eight minutes. Phase 1 already used the controller's injected clock to traverse logical checkpoints at zero, sixty, two hundred forty, and four hundred eighty minutes in about ten wall-clock seconds.
  • The prior Phase 1 wall run was therefore about twenty-four minutes. Production probation duration was configuration and logical-state coverage, while candidate construction and the independently rerun full suite accounted for nearly all wall time.

Accelerated qualification controls

Completed
  • Kept production canary labels at zero, five, and fifteen minutes and production probation labels at zero, sixty, two hundred forty, and four hundred eighty minutes. The runner advances the existing injected now value instead of sleeping.
  • No qualification-only success branch was added to the governor, queue, transaction store, activation broker, probation implementation, finalizer, or rollback executor.
  • Injected a deterministic regression executor through the governor's existing full-test-runner seam. It runs a real importable candidate contract plus promotion-transaction and governor pass/fail contract tests and returns a detailed receipt that the unchanged governor may accept or reject.
  • The accelerated receipt is neither cached nor forced to pass. A focused negative test confirms that the executor returns failure when the frozen candidate marker is absent.
  • A harmless direct-child candidate commit adds one importable qualification marker. A frozen Stage 5-style receipt hands that deterministic candidate to the ordinary candidate-to-canary controller transition.

Ten consecutive golden promotions

Passed
  • Ran ten attempts and obtained ten consecutive successes with no failed attempts or counter resets.
  • Every transaction followed prepared, git advanced, restart requested, runtime verified, probation, finalized.
  • All ten canonical queue records finished implemented and all ten promotion transactions finished finalized.
  • The candidate revision was verified as both loaded and checked out after every activation and remained verified at finalization.
  • Each attempt used its own queue database, append-only queue trace, append-only transaction trace, and result packet.

Ten consecutive intentional rollbacks

Passed
  • Ran ten attempts and obtained ten consecutive successes with no failed attempts or counter resets.
  • Every transaction followed prepared, git advanced, restart requested, runtime verified, probation, compensating, rollback restart requested, rolled back.
  • The intentional probe failure occurred only after live health, loaded revision, checkout revision, and the active qualification artifact had all been verified.
  • All ten rollback revisions were loaded by the restarted service. All ten restored Git trees matched the frozen baseline tree.
  • Queue state ended rejected with an explicit rollback outcome; transaction state ended rolled back. No attempt silently stalled or entered a requeue loop.

Fresh production-configured validation

Passed
  • The production validation did not reuse the accelerated regression receipt. The governor independently ran the complete repository pytest suite against the candidate worktree.
  • All 793 tests passed in 489.16 seconds. Five existing unknown-marker warnings did not affect the zero exit status.
  • After the full suite, the real governor fast-forwarded, the detached broker restarted Hiro, and live health verified the candidate revision.
  • The same production canary and probation checkpoint configuration was preserved, and the canonical queue and transaction finished implemented and finalized.
  • The disposable checkout was then restored to baseline, and the normal primary Hiro service was restarted on its current primary revision with matching loaded and checkout identity.

First-divergence repairs

Completed
  • The disposable server first failed before candidate creation because the clean worktree did not contain Hiro's ignored local TLS fixture. The environment was supplied; no controller or gate semantics changed.
  • The first rollback smoke trace combined activation reconciliation and the intentional probation failure in one tick. The qualification observer was narrowed to capture reconciliation without consuming the next action, preserving an explicit probation snapshot before the named failure tick.
  • The first production full-suite attempt correctly failed two launcher and preflight tests because the candidate worktree lacked Hiro's ignored pinned runtime. The existing normal worktree-runtime provisioning helper was reused, and only production validation was rerun.
  • The accelerated 10 plus 10 campaign itself completed with zero failed attempts after smoke qualification.

Timing and race coverage

Passed
  • Added boundary assertions immediately before the five-minute canary checkpoint and immediately before the sixty-minute probation checkpoint; both remain waiting until the exact boundary.
  • Repeated invocation at an already completed probation checkpoint remains waiting and does not duplicate checkpoint completion.
  • The repeated process campaign exercises actual asynchronous restart, port release, durable broker receipts, loaded-revision polling, queue leases, transaction serialization, and rollback restart reconciliation.
  • Production defaults and the protected governor policy remain unchanged.

Decisions and reasoning

Validation and evidence

CheckStatusResult
Focused qualification and timing tests passed Qualification receipt, intentional post-verification failure, production checkpoint identity, exact timing boundaries, governor behavior, transaction behavior, and worktree runtime provisioning tests passed.
Accelerated golden reliability passed 10 consecutive successes from 10 attempts; 10 implemented queue outcomes, 10 finalized transactions, and 10 verified active candidate revisions.
Accelerated rollback reliability passed 10 consecutive successes from 10 attempts; 10 rolled-back transactions, 10 verified restored runtime revisions, and 10 baseline-equivalent restored Git trees.
Production full-suite governor passed 793 tests passed in 489.16 seconds with a zero exit status; five existing marker warnings were retained in the receipt.
Production activation and probation passed Real fast-forward, detached restart, loaded-revision verification, production checkpoint traversal, implemented queue state, and finalized transaction state all succeeded.
Primary Hiro runtime restoration passed The normal service is healthy with the model connected and loaded revision equal to checkout revision after the disposable campaign.
Public journal tests and build passed npm run test:hiro passed. After installing the lockfile-pinned dependencies in the clean checkout, npm run build generated and validated all 168 journal pages, compiled TypeScript, and completed the Vite production bundle.

Current state

Next steps