Operations-Focused Routers#

Routers primarily serving observatory automation with critical operations buffering.

Executed Obs Units Router#

Path: /executed_obs_units

Purpose: Record actual telescope observations

Key endpoints:

  • POST /start - Start observation (buffered)

  • PUT /{id}/finish - Finish observation (buffered)

  • GET /{obs_unit_id} - Query observations (smart query)

Example from code:

Characteristics:

  • @critical_operation decorator

  • Pre-generated UUIDs

  • Smart queries merge buffer + database

  • High reliability requirement

Raw Data Files Router#

Path: /raw_data_files

Purpose: Register data files produced by instruments

Key endpoints:

  • POST / - Register file (buffered)

  • POST /bulk - Register multiple files (buffered)

  • GET /{id} - Get file metadata

Example:

@router.post("/bulk")
@critical_operation
async def register_files_bulk(
    files: List[RawDataFileCreate],
    _transaction_builder = Depends(get_transaction_builder)
):
    # Bulk create step
    files_step = _transaction_builder.bulk_create(
        model_class=models.RawDataFile,
        data_list=[f.dict() for f in files],
        step_id="bulk_create_files"
    )

    return {"count": len(files), "status": "buffered"}

Raw Data Package Router#

Path: /raw_data_package

Purpose: Group related data files

Key endpoints:

  • POST / - Create package (buffered)

  • GET /{id} - Get package with files

  • PUT /{id}/finalize - Mark complete

Characteristics:

  • Groups files by observation

  • Size limits (50GB max recommended)

  • Status tracking (building → complete)

Staging Router#

Path: /staging

Purpose: Data staging for transfer

Key endpoints:

  • POST /stage - Request staging (buffered)

  • GET /status/{id} - Check status

  • DELETE /{id} - Cancel staging

Operation Chain: Diagnosis, Reset, Reconcile#

Once observatory automation has filed observations and the data-transfer pipeline runs, the resulting state is the uniform Operation chain — one row per stage (OperationKind: packaging / bundling / transfer / unpack / archive). The model is defined in ops-db (see the ops-db Operation model reference and the data-transfer two-axes page); the routers below only traverse it.

Read-side traversal (admin diagnose, /api/admin):

  • GET /api/admin/diagnose/{operation_type}/{operation_id} - All DB breadcrumbs for one operation, keyed by (operation_type, operation_id)

  • GET /api/admin/diagnose/by-package/{name} - Fan out to every operation involving a package

These traverse the chain by FK (no artifact→operation ORM relationship) and key on the operation’s OperationKind.value plus its row id — the same attribution the overview uses (attribute_failed_package charges the failing op latest-step-first).

Reset and reconcile (admin, on the /api/transfer router):

# Reset any operation kind uniformly: status -> PENDING, retry_count++,
# OperationFailureEvent history preserved (ADR-0001). {kind} is an OperationKind value.
POST /api/transfer/operation/{kind}/{id}/reset
POST /api/transfer/operation/{kind}/reset-all-failed

# Reconcile a stale state=FAILED package from its op chain (ADR-0004):
# all stages complete -> ARCHIVED, mid-pipeline -> TRANSFERRING, none started -> WAITING.
POST /api/transfer/raw-data-package/{id}/reconcile
POST /api/transfer/raw-data-package/reconcile-inconsistent

The legacy per-id reset paths (POST /api/transfer/data-transfer/{id}/reset, .../long-term-archive/{id}/reset) are retained as #95-safe shims: they resolve the Operation row(s) behind the legacy row by FK and reset those, never reading the soon-dropped status/counter columns.

Next Steps#