Hiro development journal

Reviewing Contrastive Language Models as a local Hiro scorer

2026.09.27 at 13:41:15 America/Los_Angeles Source review and deferred record; no integration or experiment Machine-readable JSON

Executive summary

Reviewed CLM as a possible local ranking/decision component alongside the deferred Jev idea. It is a candidate scorer, not a replacement for Hiro's generative reasoning model or external discovery.

The source documentation supports an exploratory evaluation, but does not establish GGUF equivalence, Hiro-specific calibration, or a production advantage.

Work completed

Primary-source and existing-state review

Completed

Read https://github.com/Contrastive-LM/CLM, src/clm/embedder.py, src/clm/engine.py and https://huggingface.co/Contrastive-LM/CLM-v0.1-8B after following the supplied Reddit discussion.
Reference heads use a frozen Qwen3-8B encoder with last-token pooling; the small head download does not include the encoder.
Current serving documentation uses vLLM. The embedding client accepts an OpenAI-compatible endpoint, but that does not prove quantized llama.cpp embeddings preserve trained-head rankings or calibration.
Inspected existing Hiro opportunity priority, continuous idea scoring and bounded external-lead selection. A learned scorer could be compared at these existing boundaries, without creating another research queue.
Checked open handoffs; the existing ranked-autonomy handoff was not executed.

Deferred catalog follow-up

Completed

At the user's request, added deferred-clm-local-decision-scoring to the existing Research Map.
Retained the supplied Reddit URL, upstream repository/model card/code links, review details, limitations, related Jev identity and reconsideration conditions.
Updated the deferred-ideas navigation guide. No runtime or authority changes.

Decisions and reasoning

Treat this as a potential local scoring companion, not evidence that Jev is obsolete.
API-level similarities do not establish equal accuracy, calibration or generalization.
Candidate-relative softmax scores are not calibrated probabilities that an action is safe or correct; all options may be poor.
Retain existing independent evaluation, admission, promotion and authority checks. A ranking signal cannot authorize actions.
A small future offline comparison should use frozen real Hiro decision records and independent labels, compare current rules, raw embedding similarity and CLM, and measure ranking quality, bad-option rejection, latency and memory. First test the required encoder/pooling behavior if using GGUF. No experiment is authorized by this review.

Validation and evidence

Check	Status	Result
Documentation and code review	completed	Read upstream implementation/model card and selected Hiro source functions; no runtime reproduction.
Published benchmarks	not_reproduced	Reported agentic verifier scores rely on fine-tuned heads and small held-out task sets; they are not zero-shot Hiro results. Published latency hardware differs from the local RTX 5090.
GGUF compatibility and local resources	not_run	No model downloaded, runtime installed, embedding parity tested, or coexistence memory measured.
Deferred record	passed	Source schemas, unique identities, source references and existing Markdown export inclusion validated.

Current state

CLM is a plausible research candidate. Existing Jev deferral and discovery configuration remain unchanged.
Reference serving truncates at 2048 tokens unless configured otherwise. Longer-state behavior and quantization need evaluation.
CLM ranks supplied options; it does not expand an absent discovery search space or independently produce novel options.

Next steps

If selected for further work, evaluate one existing low-authority ranking decision offline before any integration.