{ "schemaVersion": 1, "date": "2026.07.15", "title": "Reliable direct retrieval for Hiro journal updates", "publicationStatus": "Detailed development update", "executiveSummary": [ "The public Hiro journal was audited after ChatGPT Voice could not retrieve a dated update that remained accessible in a normal browser and regular ChatGPT. Direct production HTTP tests showed that the existing HTML and JSON URLs already returned clean 200 responses, correct content types, no cookies, and identical bodies for browser, curl, generic-bot, GPTBot, and ChatGPT-style user agents.", "OpenAI's retrieval tool nevertheless rejected the dotted dated URL before issuing a normal page request, while search discovery returned no result because the journal is intentionally noindex. The site now offers hyphenated date aliases, stable latest-update endpoints, an llms.txt discovery document, explicit robots access, and a public diagnostics report so AI systems do not need to depend on dotted paths or search indexing.", "A new automated validator exercises the deployed HTML, JSON, discovery, robots, and diagnostics endpoints across representative user agents. It checks status codes, redirects, cookies, MIME types, response equality, body completeness, and the full versioned JSON schema after successful deployments.", "After direct ChatGPT Voice retrieval remained inconsistent, the journal indexing policy was changed from unlisted/noindex to public and indexable so Voice can use web search as a redundant retrieval path.", "A Gemma 4 12B reliability review compared the local configuration with the official Google QAT model card, current llama.cpp server controls, current upstream issue reports, and observed GPU state. The model should fit a 12 GB RTX 5070 at the 4K context; the leading risks are competing VRAM allocation, an unpinned runtime executable, and missing explicit flash-attention, KV-cache, memory-fit, and observability settings." ], "workstreams": [ { "title": "Production retrieval audit", "status": "Completed", "details": [ "Both bballstatistics.com and www.bballstatistics.com were tested for the dated HTML page and full JSON endpoint.", "Browser, curl, GPTBot-compatible, generic-bot, and ChatGPT-User request headers all received HTTP 200 with the same response type and no Set-Cookie header.", "The HTML response was static and complete without client-side JavaScript. The JSON response contained the full journal entry rather than metadata alone.", "No server-side user-agent discrimination, challenge page, authentication requirement, or redirect difference was observed." ] }, { "title": "Stable AI retrieval aliases and discovery", "status": "Implemented", "details": [ "Each dotted journal date now receives equivalent hyphenated HTML and JSON aliases, avoiding URL parsers that treat dotted path segments conservatively.", "Stable /hiro/latest and /hiro/latest.json endpoints always expose the newest complete update.", "The machine-readable journal index now includes schemaVersion 1 plus stableUrl and stableJsonUrl fields for every entry.", "A root llms.txt document identifies the journal index, latest update, full JSON, and retrieval diagnostics URLs.", "An explicit robots.txt allows /hiro/ and /llms.txt while the journal pages retain noindex metadata. Direct retrieval is supported without asking search engines to index the journal." ] }, { "title": "Automated post-deployment validation", "status": "Implemented", "details": [ "The validator discovers the latest date from the journal index and checks index, latest, hyphenated, legacy dotted, robots, discovery, and diagnostics endpoints.", "For browser, curl, generic-bot, and ChatGPT user agents it requires HTTP 200, no redirect, no cookie, correct MIME type, a nontrivial body, and byte-identical content.", "The latest JSON must include schemaVersion, date, title, executive summary, workstreams, decisions, validation, current state, and next steps with non-empty full-content arrays.", "A deployment-status GitHub Actions workflow runs the validator against both apex and www production hostnames after successful deployment and can also be launched manually." ] }, { "title": "Voice-compatible search discovery", "status": "Implemented", "details": [ "Removed noindex metadata from generated journal HTML.", "Added an explicit sitemap covering the journal index, latest alias, and every stable dated page.", "Kept robots access explicit and preserved full HTML and JSON direct retrieval endpoints.", "The journal remains absent from the NBA site navigation, but search engines are now permitted to discover it." ] }, { "title": "Gemma 4 12B on 12 GB research and local diagnosis", "status": "Completed", "details": [ "The official QAT Q4_0 GGUF is approximately 6.5 GiB, leaving theoretical headroom on a 12 GB GPU for a modest context and runtime buffers.", "Observed GPU allocation was approximately 10.7 GiB of 12.2 GiB while LM Studio was active, leaving insufficient clean headroom for a separately launched fully offloaded server.", "The launcher invokes llama-server by bare command name, but that command was not resolvable in the inspected shell; an NVIDIA CUDA backend exists under LM Studio and should be pinned explicitly or replaced with a separately versioned official runtime.", "Current llama.cpp supports automatic device fitting, flash attention, quantized KV caches, explicit micro-batching, prompt-cache controls, and per-slot metrics. The launcher does not currently set most of these.", "Recent upstream Gemma 4 reports include CUDA and experimental MTP instability, so the initial reliable profile should avoid multimodal projection and speculative MTP until a text-only baseline is proven." ] } ], "decisions": [ "Preserve noindex because the journal is public but intentionally unlisted. Retrieval reliability should come from stable direct URLs and machine-readable discovery, not search ranking.", "Keep the legacy dotted URLs working while making hyphenated dates and latest aliases the canonical retrieval paths. This improves compatibility without breaking existing references.", "Treat an AI tool's pre-request URL rejection separately from website availability. A clean 200 response cannot override a client-side safety policy, but simpler aliases and explicit discovery remove avoidable triggers.", "Validate deployed behavior across hostnames and user agents rather than assuming a successful local browser render proves AI accessibility.", "Use www.bballstatistics.com for all AI-facing canonical and discovery URLs because it serves query-string requests directly, while the apex hostname can trigger a Vercel managed challenge.", "Prioritize reliable ChatGPT Voice access over the previous unlisted-journal policy. Search discovery is now an intentional fallback when a Voice environment cannot directly fetch the supplied URL.", "Treat the present failure as an operational runtime and resource-isolation problem before changing response behavior or evaluator expectations." ], "validation": [ { "check": "Pre-change production HTTP matrix", "status": "passed", "result": "Both hostnames returned 200 with text/html or application/json as appropriate for browser, curl, generic-bot, GPTBot, and ChatGPT-style user agents; no cookie or differential response was observed." }, { "check": "OpenAI retrieval-tool reproduction", "status": "passed", "result": "The direct dotted URL was rejected as unsafe by the retrieval layer even though independent HTTP requests succeeded, reproducing the user-visible mismatch between retrieval surfaces." }, { "check": "Frontend production build", "status": "passed", "result": "Journal generation, TypeScript compilation, and the Vite production build completed successfully after the retrieval changes." }, { "check": "Local production-preview retrieval matrix", "status": "passed", "result": "The final validator completed 50 requests with zero failures across HTML, full JSON, robots, llms discovery, diagnostics, and cache-busting query-string endpoints. All representative user agents received byte-identical content." }, { "check": "Post-deployment production retrieval matrix", "status": "passed", "result": "Both apex and www production hostnames passed 42 of 42 requests each with zero failures. Browser, curl, generic-bot, and ChatGPT-style agents received 200 responses, correct MIME types, no redirects or cookies, identical bodies, and complete JSON." }, { "check": "Apex query-string compatibility", "status": "failed-external", "result": "Vercel returned a JavaScript security checkpoint and HTTP 403 for apex journal URLs with query parameters across all tested user agents. Equivalent www URLs returned clean 200 responses. No application user-agent rule caused the difference." }, { "check": "Indexability and discovery build", "status": "passed", "result": "The generator produced indexable HTML and a valid sitemap, and the complete TypeScript/Vite production build passed." }, { "check": "Gemma runtime research and environment inspection", "status": "passed", "result": "Confirmed the local 6.98 GB model file, RTX 5070 12.2 GB capacity, approximately 10.7 GB current VRAM allocation, missing llama-server command resolution, installed LM Studio CUDA backend, and current upstream runtime controls and known Gemma 4 issue reports." } ], "currentState": [ "The journal remains static, public, unlisted, and noindex.", "Legacy dotted URLs remain available alongside hyphenated and latest aliases.", "Full versioned JSON is available without authentication, cookies, or JavaScript.", "The remaining external limitation is that an AI client may independently refuse an unindexed or previously unseen domain before making an HTTP request.", "After deployment, OpenAI's retrieval tool still rejected the apex latest, JSON, llms, and hyphenated URLs as unsafe before issuing an HTTP request. This external URL-trust decision is not explained by the verified website response behavior.", "All AI-facing canonical, index, llms, and diagnostics URLs now use www.bballstatistics.com. The apex hostname remains available for plain requests but is not advertised for AI retrieval because Vercel may challenge query-string requests.", "The journal is now intended to be public and indexable; search-engine discovery may take time after deployment.", "No Gemma launcher or response behavior was changed during this research pass; the result is a staged implementation and measurement plan." ], "nextSteps": [ "Confirm ChatGPT Voice can retrieve /hiro/latest or the hyphenated dated URL after deployment.", "Monitor deployment validation failures for CDN, MIME type, redirect, schema, or user-agent drift.", "Keep stable aliases and full JSON generation in the standard journal publication workflow.", "Escalate the bballstatistics.com domain-level safe-open rejection to OpenAI support with the successful production HTTP matrix as evidence.", "If apex query-string support is required, explicitly approve a narrowly scoped Vercel managed-firewall bypass for /hiro, /llms.txt, and /robots.txt after reviewing the security tradeoff.", "Confirm search engines index the sitemap and test ChatGPT Voice using a title-based web search after discovery propagates.", "Create an isolated Gemma runtime profile, unload competing GPU models, pin a current CUDA llama.cpp binary, benchmark explicit memory-safe configurations, and accept a profile only after repeated Hiro-shaped latency and cancellation tests." ], "disclosureNote": "This public diagnostics entry contains no credentials, tokens, private personal data, or actionable security weakness details. It distinguishes website HTTP behavior from external AI retrieval policy." }