{ "schemaVersion": 1, "date": "2026.07.17", "title": "Dashboard recovery and verified workday startup", "publicationStatus": "published", "executiveSummary": [ "Hiro's local dashboard was unreachable because no process was listening on its web ports. The browser's connection-refused message was accurate and was unrelated to the dashboard API key.", "The dashboard/API process was restarted independently, preserving the intentional shutdown of the Gemma model server and Telegram notifications.", "The first hidden-process restart exposed a Windows console-encoding issue: the Unicode startup banner could not be written through the inherited cp1252 output encoding, so the process exited before binding a port.", "A second restart forced UTF-8 for the child process and succeeded without changing application source.", "The HTTPS root dashboard and its authenticated evaluation data feed were then verified. No evaluation, behavior, resolver, model, scoring, policy, or production-service change was made.", "At the user request, the approved Gemma launcher subsequently started the exact Gemma 4 12B QAT model with maximum GPU offload, a 4096-token context, and one inference slot. Startup took 20.599 seconds and the launcher warmup took 0.260 seconds.", "The first Telegram start exposed the same cp1252 background-output issue. Telegram was restarted with a UTF-8 child environment and reached its polling state successfully.", "A final local-only model request returned READY in 0.217 seconds. The dashboard, model, and Telegram bot were all running, while the active evaluator count remained zero.", "The phone web app was prompting for an API key because its private dashboard and chat requests correctly received HTTP 401 after browser-session storage was cleared.", "Access logs confirmed that the phone was connecting directly through the private Tailscale network, while unrelated public internet scanners were also reaching the protected Funnel endpoint. Disabling authentication globally would therefore have exposed private and control routes.", "Hiro now trusts only direct loopback and Tailscale address ranges. Those clients no longer need the API key, while public Funnel and other internet clients continue to require it.", "The focused operational-boundary suite passed 15 tests and the complete repository suite passed 89 tests. Live checks returned 200 for no-key loopback and tailnet requests and 401 for a no-key public Funnel-style request.", "The local benchmark dashboard was separately checked after the phone-authentication change. Its page and every read-only evaluation API it uses now load without an API key from trusted loopback access.", "A follow-up search-indexing audit found no current technical block: the dated HTML, full JSON, robots file, and sitemap all returned HTTP 200 with correct content types; robots allowed the journal, the sitemap listed the canonical date URL, and the page declared index, follow.", "Repository history showed that the July 14 page was initially published with noindex and became eligible for indexing only on July 15 at 4:24 PM Pacific. The search engine had therefore had roughly two days, not the full time since original publication, to reconsider it.", "Public search returned no results for the entire bballstatistics.com domain, not merely the Hiro page. The journal is also absent from site navigation, leaving the sitemap and machine-readable discovery document as its primary crawl signals.", "Official Google guidance says recrawling can take from a few days to a few weeks and does not guarantee inclusion. No indexing directive or website behavior was changed during this diagnostic pass.", "The bballstatistics.com domain property was verified in Google Search Console and the canonical July 14 Hiro URL was inspected directly.", "Google initially reported the URL as unknown, with no crawl, referring page, or referring sitemap. A priority indexing request was submitted and Googlebot Smartphone fetched the page successfully at 2:19 PM Pacific.", "After the fetch, Google changed the state to Crawled - currently not indexed. A live URL test then reported that the URL is available to Google and that the page can be indexed.", "The Hiro journal index was also added to the priority crawl queue to provide a referring page. The sitemap submission was accepted, but Search Console currently reports that the sitemap could not be read; that unresolved Google-side fetch state remains a follow-up.", "Public retrieval and discovery were strengthened without adding Hiro to the NBA site's navigation. The generator now publishes a standards-based Atom feed containing the full update text and advertises that feed from every journal HTML page and from llms.txt.", "The sitemap now carries an accurate date-only last-modified value for the journal index, latest alias, and every dated canonical page. A build-time validation gate checks the complete JSON schema, sitemap dates, Atom coverage and content, and IndexNow ownership file after every journal generation.", "The deployed production validator completed 66 retrievals with zero failures across browser, curl, generic-bot, and ChatGPT-style user agents. The HTML, JSON, robots file, discovery document, sitemap, Atom feed, diagnostics, and ownership file all returned HTTP 200 with the expected content types and no user-agent-dependent bodies.", "IndexNow received the journal index, latest alias, and all four canonical dated URLs. The first response was HTTP 202 while ownership-key validation was pending; a retry after the key was live returned HTTP 200, confirming receipt. This notification does not guarantee crawl selection or indexing.", "At the user's explicit request, the earlier unlisted-homepage choice was reversed. The public NBA statistics homepage now contains a clear, visible, crawlable HTML link to the Hiro journal." ], "workstreams": [ { "title": "Connection-refused diagnosis", "status": "Completed", "details": [ "Confirmed that the standard dashboard and API ports had no active listeners.", "Confirmed that the dashboard is served by Hiro's FastAPI process rather than by the model server.", "Inspected the prior dashboard runtime record and logs, which showed that the previous process had served evaluation overview requests successfully before stopping." ] }, { "title": "Dashboard-only recovery", "status": "Completed", "details": [ "Started only Hiro's API/dashboard process in a hidden background session.", "Left the Gemma endpoint on port 8080 stopped.", "Left Telegram notifications disabled and the Telegram bot stopped.", "Preserved the intentional-shutdown marker used by Hiro's lifecycle monitoring." ] }, { "title": "Windows background encoding diagnosis", "status": "Completed", "details": [ "The first restart attempt failed before application startup because the Unicode startup banner was encoded through a cp1252 output stream.", "The retry explicitly selected UTF-8 for Python input/output and startup execution.", "The UTF-8 retry reached application startup and brought up both the HTTPS dashboard endpoint and the companion HTTP API endpoint." ] }, { "title": "Dashboard and evaluation-feed validation", "status": "Completed", "details": [ "Verified that the canonical local dashboard root returned a successful HTML response over HTTPS.", "Verified the evaluation overview through the normal API-key-protected path without exposing the key.", "The evaluation overview reported 100 completed runs and retained proposal-only promotion mode." ] }, { "title": "Workday Gemma startup", "status": "Completed", "details": [ "Ran the approved lifecycle launcher after the user explicitly requested that Hiro be available through Telegram during the workday.", "Verified the exact served model identifier, 4096-token context, one inference slot, and maximum GPU offload from startup telemetry.", "The launcher measured 20.599-second startup latency and 0.260-second warmup latency with a READY response.", "A separate final readiness request returned READY in 0.217 seconds." ] }, { "title": "Telegram startup recovery", "status": "Completed operationally", "details": [ "The lifecycle marker was initially enabled while the bot process had exited because its Unicode startup banner inherited cp1252 output encoding.", "Restarted the Telegram lifecycle with UTF-8 forced in the child environment.", "Verified that Telegram notifications were enabled, the managed bot process was alive, and the Telegram application had entered its started polling state." ] }, { "title": "Evaluation isolation", "status": "Completed", "details": [ "Inspected running Python command lines after startup.", "Confirmed that no evaluation, regression-lite, emergence, or ordered-binding process was active.", "No new evaluation run or promotion decision was created." ] }, { "title": "Phone web app authentication boundary", "status": "Completed and live", "details": [ "Diagnosed repeated no-key 401 responses from the phone chat, lab-status, and dashboard data requests.", "Confirmed that the phone was a direct authenticated Tailscale client rather than a public Funnel client.", "Added a narrow trusted-client classifier for loopback, Tailscale IPv4 CGNAT, and Tailscale IPv6 address ranges.", "Kept API-key enforcement enabled for public and ordinary LAN addresses; the existing browser prompt remains available for untrusted clients.", "Added an explicit configuration toggle that can disable tailnet trust and restore key requirements everywhere.", "Restarted only the web API/dashboard; the model and Telegram services stayed online." ] }, { "title": "Benchmark dashboard trusted-access verification", "status": "Completed; no additional code change required", "details": [ "Confirmed that the benchmark page itself returned HTTP 200 over the local HTTP listener.", "Confirmed that overview, run detail, comparison, and lineage requests each returned HTTP 200 without an API key from loopback.", "Inspected the benchmark client logic and confirmed it prompts only after an HTTP 401; the remaining API-key button is for explicitly authenticating an untrusted public client.", "Determined that the observed prompt came from a browser page session loaded before the trusted-client fix took effect." ] }, { "title": "Search indexing delay diagnosis", "status": "Completed; no active block found", "details": [ "Verified clean HTTP 200 responses and correct MIME types for the canonical HTML, full JSON, robots file, and sitemap.", "Verified an index, follow page directive, an exact canonical URL, explicit robots permission, and inclusion of the dated URL in the sitemap.", "Confirmed that exact-title, exact-URL, and site-restricted public searches still returned no result.", "Reviewed publication history and identified the original noindex launch followed by the indexable-policy change on July 15.", "Identified weak discovery signals: the domain has no visible indexed footprint and the unlisted journal has no normal site-navigation link.", "Reviewed official Google, Bing Webmaster Tools, and IndexNow guidance for crawl requests, sitemap submission, URL inspection, and push notification." ] }, { "title": "Google Search Console inspection and crawl requests", "status": "Priority crawl submitted; indexing pending", "details": [ "Completed domain-property ownership verification without publishing account details.", "Inspected the exact canonical July 14 URL and captured Google initial state: unknown URL, no last crawl, no referring sitemap, and no referring page.", "Requested indexing once; Search Console confirmed that the URL was added to a priority crawl queue.", "Confirmed a subsequent successful Googlebot Smartphone fetch with crawling allowed.", "Ran the live URL test; Google reported that the URL is available and the page can be indexed.", "Requested indexing once for the Hiro journal index so its dated-entry links can become a referring-page signal.", "Submitted the full sitemap. Google acknowledged the submission, then displayed an unresolved could-not-fetch/could-not-read status with zero discovered pages." ] }, { "title": "Public retrieval and discovery hardening", "status": "Deployed and validated", "details": [ "Added a public Atom feed at /hiro/feed.xml with one entry per journal date and full plain-text content covering summary, workstreams, decisions, validation, current state, and next steps.", "Advertised the Atom feed through an HTML alternate link, llms.txt, and the public retrieval diagnostics document.", "Added date-only last-modified elements to every sitemap URL, using the newest journal date for the index and latest alias and each entry date for canonical dated pages.", "Added a root-hosted IndexNow ownership file and a reusable submission command that reads the canonical URL set from the generated sitemap.", "Added a deterministic generated-artifact validator to the normal journal generation and production build pipeline.", "Extended the live retrieval validator to cover the feed, sitemap dates, ownership file, and full Atom content under four common user agents." ] }, { "title": "Homepage discovery link", "status": "Deployed and validated", "details": [ "Added a semantic navigation block directly to the static homepage document.", "Added the plain anchor text Hiro development updates with href /hiro outside the React application root.", "Applied minimal visible styling while keeping the link functional with JavaScript disabled.", "Published the change through the existing Vercel deployment without changing the Hiro page URLs, schema, or retrieval behavior." ] } ], "decisions": [ "Recover the dashboard independently instead of restarting the full Hiro lifecycle, because the user needed read-only visibility while model and Telegram shutdown state should remain intact.", "Treat the first restart failure as an operational environment issue, not as evidence of a dashboard application defect.", "Use an explicit UTF-8 child-process environment for this recovery rather than altering source during an operational incident.", "Direct users to the root dashboard URL; the evaluation data lives behind authenticated API routes and is loaded by the root user interface.", "Make no evaluation or behavioral-improvement claims from this session because no evaluation run was performed.", "Start the full model and Telegram lifecycle only after the user explicitly requested workday access.", "Treat the Telegram marker/process mismatch as a failed startup until the managed process and its application-start log are both verified.", "Use a local-only READY request as the final model check; do not run evaluations merely to prove conversational availability.", "Do not remove API authentication globally because the public Funnel endpoint receives unsolicited internet traffic.", "Use the direct socket client address as the trust signal; do not accept an arbitrary browser-supplied identity header.", "Trust only loopback and documented Tailscale ranges, not generic private LAN ranges.", "Leave the front-end prompt logic unchanged so untrusted clients can still authenticate normally after a 401 response.", "Do not create a benchmark-specific authentication exception because the shared trusted-client boundary already covers every benchmark API and preserves one consistent security policy.", "Do not misclassify the current delay as a robots, status-code, content-type, canonical, or page-level indexing defect.", "Measure indexing eligibility from the July 15 removal of noindex, not from the original July 14 publication time.", "Treat search indexing and OpenAI safe-open retrieval as separate systems; indexing may help discovery but cannot guarantee that a Voice retrieval environment will accept the domain.", "Do not make another speculative metadata change without evidence from Google Search Console or Bing URL Inspection.", "Use Search Console evidence instead of inferring crawl state from public search results.", "Submit each priority URL only once because Google states that repeat submissions do not change queue position or priority.", "Do not change page metadata after the live test confirmed current index eligibility.", "Treat sitemap acceptance and sitemap processing as separate states; do not claim the sitemap passed while Google reports that it could not be read.", "Keep Hiro out of the site's ordinary navigation because the journal remains intentionally unlisted; strengthen machine discovery through standards-based public artifacts instead.", "Use date-only sitemap last-modified values because the journal source has reliable publication dates but not durable per-update timestamps.", "Treat an IndexNow HTTP 200 as confirmation that the URL set was received, not as evidence that any participating engine crawled or indexed it.", "Do not alter scoring, model behavior, Hiro runtime services, credentials, or security policy as part of website retrieval work.", "Treat the user's explicit homepage-link request as authorization to supersede the earlier unlisted-navigation decision.", "Place the link in the server-delivered index.html rather than only in React so crawlers and retrieval clients can discover it without executing JavaScript.", "Use one direct relative URL to the stable /hiro index and avoid tracking parameters, redirects, or client-side click handlers." ], "validation": [ { "check": "Initial listener inspection", "status": "passed", "result": "No dashboard listener was present, matching the browser's ERR_CONNECTION_REFUSED result." }, { "check": "First hidden restart", "status": "failed", "result": "The process exited before binding because the Unicode startup banner could not be encoded with cp1252." }, { "check": "UTF-8 restart", "status": "passed", "result": "Both FastAPI application instances completed startup; HTTPS port 8000 and HTTP port 8001 were served." }, { "check": "Dashboard root", "status": "passed", "result": "GET https://127.0.0.1:8000/ returned HTTP 200 with text/html content." }, { "check": "Authentication boundary", "status": "passed", "result": "The protected evaluation route rejected an unauthenticated request with HTTP 401." }, { "check": "Authenticated evaluation overview", "status": "passed", "result": "The read-only overview returned HTTP 200, reported 100 completed runs, and reported proposal_only promotion mode." }, { "check": "Model isolation", "status": "passed", "result": "The local model endpoint remained unavailable after dashboard recovery; Gemma was not restarted." }, { "check": "Telegram isolation", "status": "passed", "result": "Telegram notifications remained disabled and the Telegram bot remained stopped." }, { "check": "Source and evaluation changes", "status": "passed", "result": "No Hiro source, evaluator, scoring, held-out data, policy, grounding, credential, certificate, or environment configuration was changed." }, { "check": "Gemma launcher", "status": "passed", "result": "The exact gemma-4-12b-it-qat-q4_0 model started with 20.599-second startup latency and a 0.260-second READY warmup." }, { "check": "Initial Telegram start", "status": "failed", "result": "The managed bot exited during its Unicode startup banner because the hidden child inherited cp1252 output encoding." }, { "check": "UTF-8 Telegram recovery", "status": "passed", "result": "The managed Telegram bot stayed alive, notifications were enabled, and the application log reported that polling had started." }, { "check": "Final local model readiness", "status": "passed", "result": "A local-only chat-completions request returned READY from gemma-4-12b-it-qat-q4_0 in 0.217 seconds." }, { "check": "Evaluator process audit", "status": "passed", "result": "Zero evaluator processes were active after the workday startup." }, { "check": "Phone request-path diagnosis", "status": "passed", "result": "Local access logs showed the phone requests arriving directly from the authenticated tailnet and receiving 401 before the change." }, { "check": "Focused operational-boundary suite", "status": "passed", "result": "15 tests passed, covering trusted ranges, rejected public and ordinary LAN ranges, no-key tailnet access, key enforcement, and the disable switch." }, { "check": "Complete repository suite", "status": "passed", "result": "89 tests passed in 9.46 seconds." }, { "check": "Live loopback no-key access", "status": "passed", "result": "The protected lab-status endpoint returned HTTP 200 without an API key from loopback." }, { "check": "Live tailnet no-key access", "status": "passed", "result": "The protected lab-status endpoint returned HTTP 200 without an API key over the direct Tailscale address." }, { "check": "Live public boundary", "status": "passed", "result": "A no-key Funnel-style request carrying a public client address returned HTTP 401." }, { "check": "Service continuity", "status": "passed", "result": "The Gemma model endpoint remained HTTP 200 and the Telegram bot remained running after the web API restart." }, { "check": "Benchmark page no-key load", "status": "passed", "result": "GET http://127.0.0.1:8001/benchmarks returned HTTP 200." }, { "check": "Benchmark overview no-key load", "status": "passed", "result": "The evaluation overview endpoint returned HTTP 200 from loopback without an API key." }, { "check": "Benchmark detail no-key load", "status": "passed", "result": "A real evaluation run detail endpoint returned HTTP 200 from loopback without an API key." }, { "check": "Benchmark comparison no-key load", "status": "passed", "result": "A real candidate-versus-baseline comparison returned HTTP 200 from loopback without an API key." }, { "check": "Benchmark lineage no-key load", "status": "passed", "result": "The evaluation lineage endpoint returned HTTP 200 from loopback without an API key." }, { "check": "Canonical HTML indexing audit", "status": "passed", "result": "The July 14 HTML returned HTTP 200, text/html, index and follow metadata, and the exact www canonical URL." }, { "check": "Machine-readable endpoint audit", "status": "passed", "result": "The full JSON endpoint returned HTTP 200 with application/json content." }, { "check": "Robots audit", "status": "passed", "result": "robots.txt returned HTTP 200, allowed the Hiro path, and referenced the sitemap." }, { "check": "Sitemap audit", "status": "passed", "result": "sitemap.xml returned HTTP 200 with application/xml content and included the canonical July 14 URL." }, { "check": "Public search discovery", "status": "failed", "result": "Exact-title, exact-URL, and site-restricted searches returned no result; no page from the domain was visibly indexed." }, { "check": "Indexing-policy history", "status": "diagnostic", "result": "The page launched with noindex and changed to index, follow on July 15 at 4:24 PM Pacific." }, { "check": "Search Console ownership", "status": "passed", "result": "The bballstatistics.com domain property was ownership-verified." }, { "check": "Initial canonical URL inspection", "status": "diagnostic", "result": "Google reported the July 14 URL as unknown, with no crawl, sitemap, or referring page." }, { "check": "July 14 priority crawl request", "status": "passed", "result": "Search Console confirmed that the canonical dated URL was added to the priority crawl queue." }, { "check": "Googlebot fetch", "status": "passed", "result": "Googlebot Smartphone crawled the page at approximately 2:19 PM Pacific; crawl was allowed and page fetch was successful." }, { "check": "Post-crawl index state", "status": "pending", "result": "Google changed the URL state to Crawled - currently not indexed." }, { "check": "Live URL eligibility test", "status": "passed", "result": "Search Console reported URL is available to Google and Page can be indexed." }, { "check": "Journal index priority request", "status": "passed", "result": "The canonical /hiro journal index was added to the priority crawl queue." }, { "check": "Sitemap submission", "status": "partial", "result": "Google accepted the sitemap submission but currently reports Sitemap could not be read and zero discovered pages." }, { "check": "Generated retrieval artifact gate", "status": "passed", "result": "The build-time validator passed for four journal entries, full latest JSON content, every sitemap last-modified value, full Atom feed coverage, and the public IndexNow ownership file." }, { "check": "Full website production build", "status": "passed", "result": "The TypeScript and Vite production build completed after the new generation and validation steps." }, { "check": "Live multi-user-agent retrieval suite", "status": "passed", "result": "66 deployed requests completed with zero failures across browser, curl, generic-bot, and ChatGPT-style user agents; bodies were identical per endpoint and all expected status and content-type checks passed." }, { "check": "Live Atom and IndexNow ownership endpoints", "status": "passed", "result": "The deployed Atom feed returned HTTP 200 with application/xml, and the root ownership file returned HTTP 200 with text/plain." }, { "check": "IndexNow canonical URL notification", "status": "passed", "result": "After an initial HTTP 202 pending key validation, the retry returned HTTP 200 for the journal index, latest alias, and four canonical dated pages." }, { "check": "Homepage-link production build", "status": "passed", "result": "The complete TypeScript/Vite production build and generated Hiro artifact gate passed with the new static homepage navigation link." }, { "check": "Live raw homepage HTML", "status": "passed", "result": "The deployed homepage returned HTTP 200 text/html and contained the exact anchor Hiro development updates outside the React root." }, { "check": "JavaScript-independent discovery", "status": "passed", "result": "The Hiro link is present in the initial HTML response and does not require hydration, cookies, authentication, or client-side JavaScript." } ], "currentState": [ "The local dashboard is available at https://127.0.0.1:8000/.", "The dashboard process is running independently and its authenticated evaluation feed is healthy.", "The dashboard requires the configured Hiro API key for private evaluation data; the root interface stores the supplied key only for the browser session.", "Gemma 4 12B QAT is loaded locally and ready on port 8080.", "Telegram notifications are enabled and the managed Telegram bot is running for workday access.", "No evaluator is running.", "Evaluation history remains read-only through the dashboard and promotion mode remains proposal-only.", "The dashboard and Telegram hidden Windows startup paths remain sensitive to output encoding unless UTF-8 is explicitly selected.", "Direct Tailscale and loopback web-app clients no longer receive the API-key prompt.", "Public Funnel and other untrusted clients still require the configured API key.", "The tailnet trust behavior is configurable and can be disabled without removing authentication.", "The local benchmark dashboard and all of its read-only evaluation APIs no longer require an API key from loopback or direct Tailscale access.", "The journal has no current technical indexing prohibition, but public search still shows no indexed result for the domain.", "The strongest next evidence must come from Google Search Console URL Inspection and Bing Webmaster Tools rather than repeated unauthenticated search queries.", "Google now knows and has successfully crawled the canonical July 14 page, but has not yet selected it for the index.", "Both the dated page and the Hiro journal index are in Google priority crawl queues.", "The live page passes Google current index-eligibility test.", "Google sitemap processing remains unresolved and should be rechecked after the new property finishes initial processing.", "A full-content Atom feed is live at https://www.bballstatistics.com/hiro/feed.xml and is advertised from journal HTML and llms.txt.", "The live sitemap includes last-modified dates for the index, latest alias, and each canonical dated page.", "Future deployments can rerun deterministic generated-artifact checks and notify IndexNow with one repository command.", "IndexNow has confirmed receipt of the current canonical journal URL set, while Google indexing remains separately pending in Search Console.", "The public www.bballstatistics.com homepage now links directly to /hiro with the visible label Hiro development updates.", "The homepage supplies an ordinary internal referring link for crawlers in addition to the sitemap, Atom feed, llms.txt, and IndexNow notifications." ], "nextSteps": [ "Recheck the canonical July 14 URL in Search Console after Google processes the priority crawl; do not resubmit it repeatedly.", "Recheck the sitemap status after property initialization. If it still cannot be read, inspect Google reported fetch details before changing the valid public XML.", "Monitor public search only after Search Console changes the page from Crawled - currently not indexed to an indexed state.", "Close and reopen any benchmark tab that was loaded before the trusted-client change so its stale prompt state is discarded.", "Have the phone reopen or refresh the Hiro web app while Tailscale is connected and confirm the session no longer presents the API-key prompt.", "Persist UTF-8 mode in the managed Telegram launcher and add a focused regression test so future lifecycle starts cannot recreate the marker/process mismatch.", "Add or standardize a dashboard-only launcher that always selects UTF-8, records its process identity, and verifies the root and authenticated overview endpoints.", "Make dashboard lifecycle status distinct from model and Telegram lifecycle status so an intentional model shutdown cannot obscure dashboard availability.", "Keep model and Telegram startup under explicit user control; do not infer that dashboard recovery authorizes inference or notification services.", "Continue to require proposal-only evaluation gates before any behavioral promotion.", "Add the domain to Bing Webmaster Tools when an authenticated owner session is available, then submit and inspect the sitemap there.", "Recheck Google Search Console sitemap processing and the July 14 URL after the new sitemap deployment; do not repeat the existing priority request.", "Run the production retrieval validator and IndexNow notification command after future journal updates.", "Recheck Search Console discovery after Google recrawls the homepage and sitemap; do not repeat the existing priority indexing request." ], "disclosureNote": "This public entry omits credentials, API values, private data, and sensitive operational details. It records only safe, high-level evidence needed to explain the dashboard recovery." }