PRD 0001: Pruned public search index#

Source of truth for this epic. Tracker: epic issue. The epic links here; this file is not embedded in the issue body.

Problem#

The docs site gates the operations and project sections behind GitHub OAuth, but Sphinx writes one whole-site search index (searchindex.js) and one cross-reference inventory (objects.inv) at the site root, where nginx serves them publicly. An anonymous visitor can read the gated content in outline — 670 section headings verbatim in a measured build — and confirm whether any specific string (hostname, key ID, username, path) appears in the gated pages, attributable per page. The gated runbooks are therefore readable in outline by anyone, even after the raw-path leak (#22) was closed. Reported as #24.

Solution#

Anonymous visitors get a search index that covers only public pages — the public index, served at the root searchindex.js so the default artifact is safe even if serving configuration regresses. Logged-in users get the full index, served at the same URL, so search behaves identically for both audiences and gated runbooks stay searchable for operators. objects.inv requires login. Decided in ADR 0001.

Implementation decisions#

  • The Sphinx configuration owns the canonical list of gated prefixes (operations section, project section, the /private/ convention).

  • A Sphinx build-finished hook prunes all gated docnames from the search index using Sphinx’s own IndexBuilder machinery (not hand-rolled JSON rewriting), emitting the public index at the output root and the full index at a second, gated path.

  • nginx serves both variants at one URL: an exact-match location for /searchindex.js runs auth_request; authenticated requests receive the full index, anonymous requests fall back to the public index. Exact-match locations are required because the static-asset cache regex location outranks any prefix gate for *.js.

  • objects.inv is gated outright with an exact-match location; intersphinx resolves at build time against local inventories, so no external consumer needs the deployed file.

  • The full-index path is never directly reachable without authentication.

Testing decisions#

  • Static configuration tests in the style of the existing gating test suite (AST-parse the Sphinx config, regex-check the nginx config): the gated prefix list exists and covers the three gated areas; nginx carries the two exact-match locations; nginx’s gated locations and the Sphinx gated-prefix list agree, so drift between the two enforcement points is a test failure.

  • A build-artifact test that parses the built public index and asserts zero gated docnames, headings, or gated-only terms survive; skips when no build output exists so the static suite stays runnable without a Sphinx build.

  • Final verification is human-in-the-loop on staging with the pmo submodule checked out: anonymous vs authenticated fetches of searchindex.js differ as specified, and objects.inv requires login.

Out of scope#

  • Wiring pytest into CI — tracked separately as #25.

  • Splitting the site into separate public and gated Sphinx builds (rejected in ADR 0001).

  • Search UX changes; search.html itself stays public.

  • Auditing the gated pages’ content — this epic controls exposure, not what the runbooks say.