Reference implementation. This portfolio project is not an official service of any transit agency.

Nightly evaluation snapshot

This page mirrors the most recent completed nightly evaluation run on main, published automatically whether that run passed its gates or not — see why. It is not a promoted release: this repository's cryptographically pinned promotion-evidence pipeline (documented here) remains blocked on a live deployment matching a passing run, and publishes a separate "Verified" page only when that is true. This page's job is narrower: never let the public hub go silent about what the corpus and prompts at HEAD actually score.

Canonical evaluation evidence hub · Live rider assistant

Nightly status

Nightly gate: FAILED

At least one gate the nightly run enforces failed against this scoreboard (see the suite table and the linked case-level report for which cases). This page still publishes the real numbers rather than hiding them.

Evaluated 2026-10-23T09:31:58Z (full, live provider — every model call served from cache), corpus 10deac978967, from this CI run. This page's publisher checks for a newer nightly run several times a day whether or not one has actually appeared, and last did so at 2026-10-23T09:33:01Z — if that instant is recent and the run above is not, the checker is working and the nightly itself is what stopped.

That status was settled when this page was built, and nothing rebuilds it as it ages. It expires 2026-10-24T21:31:58Z — past that instant the nightly workflow has gone quiet for longer than one missed night can explain, whatever the heading still says. Your browser compares that instant to now and replaces this sentence with what it found; with scripting off it cannot, so make the comparison against today yourself.

Last night's score

318/385 (82.6%) across every case in the suite, published exactly as scored — including the cases that failed.

Scores by evaluation suite
Suite	Passed	Pass rate
conversation	9/10	90.0%
cross_agency	16/21	76.2%
edge_cases	102/124	82.3%
freshness	22/30	73.3%
groundedness	59/70	84.3%
multilingual	34/40	85.0%
refusal	41/45	91.1%
sensitivity	22/30	73.3%
stretch_tagalog	13/15	86.7%

Review the case-level report (same file this run wrote to docs/eval-report.html, republished unedited).