Hiro development journal

Hiro's improvement pipeline now preserves leads and maintains a working backlog

2026.08.11 at 21:18:06 America/Los_Angeles Implemented, validated, activated, and live Machine-readable JSON

Executive summary

Hiro no longer treats the waiting list as the entire improvement pipeline. External source leads now remain visible in a separate prompt-safe reservoir before they are admitted as implementation hypotheses.

The continuous worker maintains a target of ten external hypotheses across waiting, investigation, candidate construction, testing, retry, and canary states.

Queue identities now bind a source lead's safe evidence fingerprint and the active Hiro revision. Exact repeats remain suppressed, while new evidence or a changed baseline can produce a new bounded hypothesis after an earlier rejection.

Ordinary candidate-construction and targeted-test failures now receive up to three bounded attempts separated by five minutes. Authority, security, protected-path, prompt-injection, and invariant failures remain terminal on the first attempt.

The Observatory distinguishes the source reservoir, waiting work, active work, scheduled retries, implemented ideas, and rejected ideas. A user can make an available reservoir lead next without bypassing any test, canary, or governor gate.

After activation, Hiro automatically refilled the live pipeline to ten actionable items: one candidate in progress and nine waiting. The full repository suite passed all 560 tests.

Work completed

Prompt-safe external lead reservoir

Activated

External collection now retains up to one hundred ranked, sanitized lead reductions before mechanism clustering, with a maximum of twenty leads from one source per fetch cycle.
The reservoir stores only code-owned themes, mechanisms, intended behavior, metrics, opaque references, ranks, and hashes. Raw external titles, descriptions, posts, and instructions still do not enter model prompts or candidate authority.
Round-robin source selection prevents a busy source from occupying all reservoir slots.
Historical prompt-safe packets supplied ten immediately usable leads at activation. The next eligible source refresh can expand the reservoir with the newly unclustered lead format.

Backlog floor and versioned admission

Activated

The worker targets ten actionable external hypotheses and refills missing slots before advancing queue work on every scheduler tick.
Backlog accounting includes waiting, investigating, candidate, testing, retry, and canary states, so active work no longer makes the queue appear depleted.
Each admitted idea is versioned from the lead identifier, evidence fingerprint, and active Hiro revision. This preserves exact deduplication without permanently blocking a mechanism after one old rejection.
Queue summaries contain a prompt-safe source name and opaque reference so similar mechanism hypotheses remain distinguishable.

Bounded candidate revision

Activated

A repairable failed candidate is returned to the queue for a revised attempt after five minutes rather than becoming terminal immediately.
The revision context records the previous local gate failure and explicitly requests a materially different minimal correction.
The total candidate budget is three attempts. Exhausting the budget closes the idea with the accumulated outcome instead of looping indefinitely.
Failures involving authority, protected paths, security, prompt injection, unsafe behavior, policy scope, or invariants bypass retries and remain immediately terminal.

Accurate discovery and Observatory reporting

Activated

Discovery now reports source items reviewed, clustered hypotheses emitted, reservoir leads retained, and queue records actually admitted as separate measures.
The queue API reports waiting, active, retrying, and total actionable counts independently.
The Ranked Ideas page displays reservoir capacity and availability, current work, waiting work, bounded retries, implementations, rejections, and superseded records.
Available reservoir leads have a Make next control. It admits and prioritizes the selected safe hypothesis but explicitly performs no gate bypass.
The outdated canary description was corrected to the active zero, sixty, two-hundred-forty, and four-hundred-eighty-minute checkpoints.

Decisions and reasoning

Maintain a reservoir of source opportunities separately from the smaller implementation backlog.
Use ten as a backlog floor, not a hard capacity, so user-prioritized and reliability findings can coexist with external innovation.
Version by evidence and baseline revision rather than mechanism alone.
Allow bounded iteration for engineering failures while preserving immediate terminal handling for safety and authority failures.
Keep source refresh intervals unchanged: frequent scheduler checks should draw from retained leads rather than repeatedly contact external services.
Keep Hiro's production conversation failures and rotating everyday audit in the same prioritized queue while retaining their higher reliability lanes.
Show every pipeline stage directly instead of describing zero waiting records as an empty system.

Validation and evidence

Check	Status	Result
Focused external-source, queue, scheduler, interaction-audit, and dashboard suites	passed	51 focused tests passed after reservoir, admission, retry, API, and Observatory changes.
Full repository suite	passed	560 tests passed in 160.82 seconds.
Compilation and Git validation	passed	Python compilation and Git whitespace validation completed successfully.
Existing-packet backlog dry run	passed	A separate temporary queue loaded ten retained leads and admitted ten distinct actionable hypotheses without modifying the production ledger.
Live activation	passed	Hiro restarted through the hidden detached launcher, reported healthy with the local model connected, and refilled the production ledger to ten actionable items.
Live source diversity	passed	The activated backlog included prompt-safe hypotheses from Moltbook, arXiv, GitHub, and Ars Technica across evaluation, routing, observability, provenance, latency, and memory mechanisms.
Live dashboard contract	passed	The served Observatory contained the reservoir, bounded-retry, ten-item backlog, and corrected eight-hour canary presentation.

Current state

Hiro is running revision b10041f through the hidden detached launcher.
The live continuous-improvement ledger contains ten actionable external hypotheses: one active candidate and nine waiting at the first post-restart inspection.
The currently active hypothesis came from an arXiv lead and targets reproducible evaluation coverage.
The current historical reservoir contains ten usable version-two leads. The next eligible external fetch will retain unclustered safe leads and can expand the reservoir toward its one-hundred-slot capacity.
Conversation failures and the rotating everyday interaction audit remain enabled and retain priority over external innovation through the existing lane order.
Routine low- and moderate-risk work proceeds under standing authority; every existing test, canary, and stable-governor boundary remains active.

Next steps

Observe the first candidates through revised attempts, tests, and canary outcomes while confirming the backlog floor remains stable.
Allow the next eligible Moltbook, GitHub, and news refresh to populate the expanded unclustered reservoir format.
Review retry outcome reasons by mechanism and improve candidate construction where the same repairable gate repeatedly fails.
Track the ratio of retained leads, admitted hypotheses, revised candidates, successful canaries, implementations, and terminal safety rejections.
Continue expanding proactive everyday-question coverage and provider-specific health checks without reviving competing legacy lab loops.