Hiro development journal

Finite diversity campaign for Hiro Step 3

2026.07.23 at 07:46:06 America/Los_Angeles Published Machine-readable JSON

Executive summary

Designed a finite execution-grade roadmap Step 3 campaign that a lighter supervising model can initiate, monitor, resume, and close through deterministic controller commands and a fixed evidence decision table.

The plan directly addresses the July 21 repetition failure. That session recorded 44,592 observations but only 2,820 structural signatures, a 93.68% structural-reuse rate. Seven of eight families had only four to eight effective topologies.

The replacement campaign uses ten capability families, eight finite phases, a maximum of 360 planned cases including opening and closing canonical checks, global historical novelty rejection, family and topology caps, mastery retirement, sealed blind validation, and mandatory worker closeout.

Work completed

Repetition diagnosis

Completed

Inspected the existing development-cycle runbook, v1 runner, generated catalogs, adaptive Daylab rotation, and stateful adversarial generator.
Confirmed that the v1 runner extends beyond its nominal cycle count until the timebox and that signatures are checked only within each generated batch rather than against historical catalogs.
Recomputed the continuous-session corpus: 929 cycles, 44,592 observations, 2,820 unique signatures, and 93.68% structural reuse.

Finite capability portfolio

Designed

Defined ten families spanning temporal calendar reasoning, task replanning, email grounding, cross-tool coordination, approval lifecycle, concurrency, recovery, memory continuity, source judgment, and productive autonomy.
Defined a finite sequence from calibration through breadth, evolving state, cross-family composition, adaptive micro-waves, blind frontier, and closeout.
Set the campaign target at 320-420 valid observations, with a manifest maximum of 348 noncanonical cases plus twelve opening and closing canonical workflow checks.

Enforceable novelty and mastery

Designed

Structural signatures exclude superficial names, prose, dates, and labels and instead encode constraint graphs, event ordering, tool classes, approval transitions, resource versions, fault sequences, temporal relations, oracle class, and family composition.
The plan requires exact and near-duplicate rejection against repository suites and historical development and adversarial catalogs, at least 85% noncanary novelty, a fifteen-percent family cap, and a two-percent topology cap.
Mastered family-band combinations retire after two fresh statistically supported waves and return only as small rotating canaries.

Weaker-model supervision contract

Designed

The supervisor follows machine-readable checkpoints and allowed-next-actions instead of reconstructing state from conversation history.
A fixed table determines whether to increase difficulty, hold a band, create a bounded diagnostic wave, permit one evidence-based proposal, reject a batch, or close the campaign.
Terminal handling covers verified duplicate workers, wrapper timeouts, endpoint failures, incomplete reports, exhausted generators, failed candidates, blind leakage, and hard-stop closeout.

Decisions and reasoning

Do not use the v1 continuous development runner or repeated Daylab suites for roadmap Step 3.
Make the deterministic controller responsible for variety, budgets, checkpointing, and stopping; the lighter model supervises rather than improvises the loop.
Use a finite manifest and begin closeout thirty minutes before the deadline even when an earlier phase is incomplete.
Permit at most one qualifying low- or medium-risk candidate per adaptive micro-wave and never alter evaluators, oracles, assertions, or failed cases to gain credit.
If the required v2 controller cannot be implemented and validated within its bounded sixty-minute prerequisite, stop as step3_controller_not_ready rather than reverting to repetition.

Validation and evidence

Check	Status	Result
Historical structural-reuse audit	completed	The July 21 continuous session contained 44,592 cataloged cases and 2,820 unique structural signatures, yielding 93.68% reuse. Seven families had only four to eight signatures; long-context attention supplied 2,778.
Plan structure validation	passed	The runbook contains all ten required families, all eight phases, the finite manifest, novelty/family/topology limits, mastery retirement, supervisor decision table, closeout contract, evidence packet, and lighter-model handoff prompt. Planned maximum is 360 cases including canonical open and close checks.
Step 3 controller availability	not-ready	The specified hiro.benchmarks.development_cycle.step3 interface does not yet exist. The runbook therefore makes controller implementation and validation a bounded prerequisite and prohibits fallback to v1.
Capability evaluation	not-run-by-design	No Step 3 evaluation campaign, model run, candidate change, production connector, service, or schedule was started during this planning session.

Current state

The Step 3 execution plan is available in docs/HIRO_STEP3_DIVERSITY_CAMPAIGN.md.
The existing development-cycle guide now points Step 3 operators to the finite campaign and explicitly disallows the v1 runner for this purpose.
The plan is ready for controller implementation; it is not yet safe to start the long campaign with the old runner.

Next steps

Implement the finite Step 3 controller, expanded generator catalog, global novelty index, mastery state, and deterministic checkpoint interface.
Test prepare, validate-plan, run/resume, status, and idempotent close commands, including duplicate-worker and deadline behavior.
Run a short dry validation that generates and scores manifests without beginning the full campaign.
After the readiness gate passes, hand the exact runbook prompt to the lighter supervising model for the finite campaign.