Hiro development journal

Diagnosing the continuous queue's rejection bottleneck

2026.08.15 at 21:28:44 America/Los_Angeles Diagnosis complete; corrective implementation not yet started Machine-readable JSON

Executive summary

Hiro's queue currently contains 141 ideas: 120 rejected, 18 superseded, three marked implemented, and no actionable records.

The rejection count does not mean that 120 hypotheses were independently disproven. Most failures occur in candidate construction, and construction-tool failures are currently recorded as idea rejection.

Across 301 generated candidate packets, 227 failed construction and only 74 became candidate-ready. All 74 candidate-ready packets were then rejected by evaluation.

The evidence points to two primary system bottlenecks: brittle model-authored text patches during repair, and promotion statistics that require global score separation from a small high-baseline suite even for narrow targeted fixes.

Work completed

Queue outcome audit

Completed

Counted 120 rejected records, three implemented records, 18 superseded records, and zero actionable records from the live continuous-improvement endpoint.
Ninety-three rejected ideas consumed all three queue attempts; twenty consumed one attempt, one consumed two, and six legacy records show zero attempts.
The rejected population includes 56 broad-interaction records and 21 tool-routing records; interaction audit supplied 56 records and the older Moltbook feed supplied 43.
Two of the three implemented records are reconciled bootstrap incidents rather than ordinary successes through the current generative candidate funnel. The remaining implementation is a deterministic acceptance seed.

Candidate construction funnel

Completed

Inspected all 301 frozen candidate packets associated with the autonomous sandbox history.
Two hundred twenty-seven packets, or 75.4 percent, ended candidate_failed before independent evaluation; 74 became candidate_ready.
Across 681 internal builder attempts, 446 failed because planned patches were rejected, 211 failed candidate validation, and 24 failed because every required targeted test was not added or changed.
The most frequent patch-tool failures were 190 stale or ambiguous modify anchors, 132 attempts to create a test file that a prior repair step had already created, and 69 empty patch plans. Sixteen attempts were correctly rejected by safety-pattern checks.

Independent evaluation funnel

Completed

All 74 candidate-ready packets were rejected by the evaluator; none proceeded through the ordinary generative path to promotion.
Every candidate-ready packet had overlapping public and held-out confidence intervals. Seventy-two missed the public global score-delta threshold and 73 missed the held-out threshold.
Fifty-four candidate-ready packets also exceeded the configured latency ratio, 28 regressed the epistemics category, and five regressed instruction following.
A representative candidate improved public score by 0.0104 and passed 40 targeted and regression tests, but failed the required 0.0200 global delta and confidence-separation rules.

Decisions and reasoning

Do not interpret builder protocol failures as evidence against an idea.
Do not weaken invariant, held-out, authority, scope, or safety gates in response to the low promotion rate.
Separate targeted improvement evidence from global non-regression evidence; a narrow fix should prove its intended repair on paired relevant cases without needing to move a small global suite by two percentage points.
Treat single-run latency as noisy evidence and require repeated warmed measurements before rejecting an otherwise valid candidate for performance.
Preserve the 120 historical outcomes append-only. Any recovery should create explicit second-generation trials rather than rewriting rejection history.

Validation and evidence

Check	Status	Result
Live queue status	passed	The live endpoint reported 141 total ideas, 120 rejected, 18 superseded, three implemented, zero actionable, and a closed circuit breaker.
Frozen candidate packet census	passed	Read 301 candidate packets: 227 candidate_failed and 74 candidate_ready. The packet count matches the queue's accumulated candidate-attempt volume.
Builder failure classification	passed	Classified 681 builder attempts and counted the dominant validation and patch-application failure modes from their frozen evidence.
Evaluator rejection classification	passed	All 74 ready candidates were evaluation-rejected; confidence overlap affected all 74, global score thresholds affected nearly all, and latency affected 54.

Current state

The queue has no actionable records because the current admission batch has been exhausted through rejection or supersession.
The circuit breaker is closed; this is a quality and measurement bottleneck, not a currently open infrastructure breaker.
Historical evidence is intact in the append-only queue ledger, frozen candidate packets, and sandbox run packets.
No corrective code or queue-state mutation was performed during this diagnosis.

Next steps

Add a first-class funnel and failure taxonomy so construction, evaluation, safety, infrastructure, and disproven-hypothesis outcomes are separately visible.
Make repair attempts deterministic and fresh-workspace based, preserve target-test context, and surface exact builder failure details to the queue.
Replace the global-improvement gate with targeted paired improvement plus global non-regression, while retaining held-out and invariant checks.
Use repeated warmed latency measurements and uncertainty-aware comparison.
After validating the corrected funnel, selectively retrial deduplicated high-value construction failures as new append-only lineages.