0001 — Prune gated pages from the public search index#

Context#

The site is one Sphinx build; access control is applied at serve time by nginx (auth_request → oauth2-proxy), per the documentation strategy decision. Sphinx writes whole-site artifacts — searchindex.js and objects.inv — at the output root, outside every gated prefix, so nginx serves them publicly. The search index contains the gated operations and project pages’ section headings verbatim and maps every indexed term back to its page, so an anonymous visitor can read the gated content in outline and confirm the presence of any specific string in it (issue #24: 56 gated documents, 670 headings, 1803 gated-only terms in a real build).

Path-prefix gating cannot cover these files: nginx’s static-asset cache block is a regex location, and regex locations win over prefix locations for *.js. Only exact-match (location =) blocks can intercept them.

Decision#

  • Keep the single Sphinx build. After the build, prune all gated docnames from the search index, producing two artifacts: the public index (ungated pages only) and the full index (every page).

  • The file at the site root, searchindex.js, is the public index. This is the fail-safe default: if nginx configuration ever regresses, the publicly served file leaks nothing.

  • nginx serves both variants at the same URL: an exact-match location for /searchindex.js runs auth_request; authenticated requests receive the full index, anonymous requests fall back to the public index. Sphinx’s search JavaScript works unchanged for both audiences.

  • objects.inv is gated outright. Intersphinx resolves at build time against local inventories inside this repo, and no external project builds against the deployed site’s inventory.

  • The list of gated prefixes is owned by docs/conf.py; the static test suite asserts that docs/nginx.conf gates exactly the same prefixes, so drift between the two enforcement points is a test failure.

  • Pruning uses Sphinx’s own IndexBuilder machinery in a build-finished hook rather than hand-rolled JSON rewriting, to avoid coupling to the index file format across Sphinx versions.

Consequences / Trade-offs#

  • Anonymous search keeps working, restricted to public content; logged-in operators keep full-site search — including the runbooks, where search matters most.

  • The gated-prefix list now drives two enforcement points (Sphinx post-build pruning and nginx locations), bridged only by a test — the test suite is load-bearing for security, and it does not yet run in CI (issue #25).

  • The post-build hook depends on Sphinx internals (IndexBuilder); a Sphinx major upgrade may require touching it.

  • nginx must special-case two exact-match locations ahead of the cache regex block; the same-URL variant serving is nonstandard nginx (auth fallback via error_page), so it needs a comment and a staging check.

Alternatives considered#

  • Split builds — build public and gated content as separate Sphinx invocations so the public index never sees gated docs. Cleanest isolation, but breaks the single-build aggregator architecture, doubles build time, and leaves the gated section without search or with a second search setup.

  • Gate the index files outright — one nginx line, but Sphinx’s search fetches one global index, so anonymous search dies entirely.

  • :nosearch: metadata injection — inject Sphinx’s nosearch file metadata into gated pages via a source-read hook. Removes gated pages from the only index, so logged-in users lose runbook search; injection is also format-dependent (RST field vs MyST frontmatter) across submodules.

Status#

Accepted.