Skip to content

Python: benchmark API

Note

Like Python: benchmark harness, this documents python/oracles/api.py — internal to this repository's own benchmark tooling, not the stable public API described in Python: core API.

oracles.api is the only runtime bridge between a benchmark run's results.json and the web/ React UI: the React app fetches /api/v1/report at load time instead of inlining a build-time fixture, and the response models are the frozen oracles.bench_contract.BenchReport contract, so the OpenAPI schema this app publishes is the type source for web/src/contract.ts.

oracles.api

FastAPI app serving the frozen BenchReport contract.

The benchmark engine emits results.json (the camelCase BenchReport payload) into a run-centric tree under .spectrafit_reports/benchmark/. This app is the only runtime bridge to the web/ React UI: the React app fetches /api/v1/report at load time instead of inlining a build-time fixture, so there is one data flow:

benchmark run → results.json → FastAPI → React

The response models ARE the frozen contract (:class:oracles.bench_contract.BenchReport), so the OpenAPI schema this app publishes at /openapi.json is the type source for web/src/contract.ts (generated via openapi-typescript). Pydantic, not a hand-kept JSON Schema, is the contract.

Routing — Rosetta bridge: The endpoints live on a single :class:fastapi.APIRouter mounted twice:

- ``/api/v1/*`` — canonical prefix going forward. No deprecation headers.
- ``/api/*``   — legacy alias kept through 0.1.0, removed no earlier than
  the RFC 8594 Sunset date below (2026-12-06). Every response carries
  ``Deprecation: true`` and ``Sunset: <date>`` so callers see the migration
  window in HTTP itself (RFC 8594 / draft-ietf-httpapi-deprecation-header).

Run: 

uv run --extra benchmark python -m uvicorn oracles.api:app --port 8000

Security

CORS is scoped to the local Vite dev origin: the report payload carries git provenance (see _capture_git_provenance() in oracles/engine.py), so a wildcard origin becomes a real exposure the moment this app ever binds to a non-localhost interface. Add the production origin explicitly when one exists — do not revert to "*".

router = APIRouter() module-attribute

All endpoints live on a single router; the FastAPI app mounts it twice below so there is one source of truth for handlers:

  • _CANONICAL_PREFIX (/api/v1)
  • _LEGACY_PREFIX (/api)

root()

Redirect GET / to /docs (the interactive API explorer).

Prevents the bare {"detail":"Not Found"} 404 that confuses developers who open http://localhost:8000/ expecting something useful. The UI lives on :5173; /docs is the best landing for a bare API hit on :8000.

list_runs()

Return the run ids under <REPORTS_ROOT>/benchmark/ (newest first).

get_latest_report()

Return the newest run's validated :class:BenchReport.

get_report(run_id)

Return the validated :class:BenchReport for the given run_id.

get_trust()

Latest run's verification ledger slice (trust block + inference).

The downloadable 'verification ledger' the UI links to for reviewers. The full V&V detail (wires, NIST datasets, nested-adequacy, \(\sigma\)-calibration, speed-significance) lives here rather than as a dedicated UI page; the landing carries only a compact, externally-anchored trust signal. camelCase wire form, like /report.

The /api/v1 vs /api dual mount

Every route above is registered once, on a single APIRouter, then mounted twice — the module docstring calls this the "Rosetta bridge." /api/v1/* is the canonical prefix going forward and carries no deprecation headers. /api/* is a legacy alias kept for one cycle: every response on that prefix is stamped with Deprecation: true and a Sunset date per RFC 8594 / the draft-ietf-httpapi-deprecation-header convention, so callers see the migration window in the HTTP response itself rather than a changelog.

oracles.api._LEGACY_PREFIX = '/api' module-attribute

Info

Legacy /api/* alias kept through 0.1.0, per RFC 8594 § 3 — see the CHANGELOG [Unreleased] -> Deprecated entry. Not removed on the first tagged 0.1.0 release; removal lands no earlier than the Sunset date in a later 0.1.x.

Warning

Sunset is six months from the Rosetta-bridge landing date (2026-06-06 → 2026-12-06) — this alias is scheduled for removal.

The middleware that stamps those headers only tags responses whose request path actually resolved under the legacy prefix — a bare /apicruft 404 does not inherit them:

oracles.api._DeprecatedAliasMiddleware

Bases: BaseHTTPMiddleware

Stamp Deprecation/Sunset headers on legacy /api/<endpoint> responses.

Only requests whose path resolved against a route registered under the legacy prefix (i.e. /api/runs, /api/report, /api/report/{run_id}) are stamped. Unrelated 404s such as /api/v1abc or /apicruft never inherit the deprecation headers, and the canonical /api/v1/* surface stays clean.

dispatch(request, call_next) async

Stamp deprecation headers on legacy-prefixed responses, then return them.

Parameters:

Name Type Description Default
request Request

The incoming request; both its url.path and the presence of request.scope["route"] are inspected (see Note below for why route.path itself is not used) — an untagged 404 under /api/ (no route matched) is never stamped.

 required
call_next RequestResponseEndpoint

The next handler in the middleware chain; awaited once to obtain the response before it is (maybe) stamped.

 required

Returns:

Type Description
Response

The downstream response, with Deprecation, Sunset, and
Response

Link headers added when the request path fell under the
Response

legacy /api/* prefix (and not the canonical /api/v1/*
Response

one); unmodified otherwise.

Note

Discrimination uses the REQUEST path, not route.path: as of FastAPI 0.140.0, app.include_router(router, prefix=...) no longer produces prefix-combined APIRoute objects reachable via request.scope['route'] — that key resolves to the router-relative route (path == "/runs") for BOTH the /api/* and /api/v1/* mounts, so a route.path-based check would silently never match either prefix. request.url.path is the actual incoming path and is unaffected by that internal representation.

CORS is scoped deliberately, not by default

The module docstring's Security note is an operational constraint, not incidental configuration: /api/v1/trust and /api/v1/report embed the run's git provenance in the response body, so a wildcard CORS origin would turn that provenance into a cross-origin exposure the moment this app binds to anything beyond the local Vite dev server. allow_origins is pinned to http://localhost:5173 and http://127.0.0.1:5173 — add a production origin explicitly if one is ever needed; do not widen it to "*".

See also