========
Overview
========

System Context
--------------

The calibration engine sits at the heart of the CCAT data reduction chain,
between raw data ingestion and science-ready reduction:

.. mermaid::

   graph LR
     FITS[FITS files<br/>telescope] -->|zarr-fits| L0[L0 Zarr<br/>raw counts]
     L0 -->|calibrate| L1[L1 Zarr<br/>T_A* spectra]
     L1 -->|reduction_pipeline| L2[L2 Zarr<br/>gridded maps]

     style L0 fill:#e3f2fd
     style L1 fill:#e8f5e9
     style L2 fill:#fff3e0

The engine accepts L0 Zarr stores produced by the FITS-to-Zarr converter
and writes calibrated L1 Zarr stores consumed by the downstream reduction
pipeline (baseline subtraction, flagging, gridding, mapping).

What It Does
~~~~~~~~~~~~

1. **Ingest** — reads raw 5D count arrays and metadata from L0 Zarr stores
   (optionally converting FITS folders on the fly)
2. **Calibrate** — applies the heterodyne calibration equation using HOT/COLD
   loads, atmospheric model, and receiver parameters to produce antenna
   temperature :math:`T_A^*`
3. **Write** — stores calibrated spectra, system temperature, flags, and
   quality metrics in L1 Zarr stores

Workspace
---------

Seven-crate Cargo workspace:

.. list-table::
   :header-rows: 1
   :widths: 20 50 15 15

   * - Crate
     - Purpose
     - C deps
     - Feature gate
   * - **cal-schema**
     - Shared L0/L1 schema constants (zero dependencies)
     - none
     -
   * - **cal-core**
     - Pure calibration math — RJ correction, gamma, Tsys, PWV fitting, modes
     - none
     -
   * - **cal-io**
     - Zarr I/O, pipeline orchestration, optional FITS ingest
     - cfitsio (optional)
     - ``fits-ingest``
   * - **cal-cli**
     - ``calibrate`` binary (clap CLI)
     - cfitsio (optional)
     - ``fits-ingest``
   * - **cal-py**
     - PyO3 Python bindings
     - cfitsio (optional)
     - ``fits-ingest``
   * - **zarr-fits-core**
     - FITS → Zarr v3 conversion library
     - cfitsio
     -
   * - **zarr-fits-cli**
     - Standalone ``zarr-fits`` conversion binary
     - cfitsio
     -

Build Profiles
~~~~~~~~~~~~~~

.. list-table::
   :header-rows: 1
   :widths: 20 30 20 30

   * - Profile
     - Binaries
     - C deps
     - Use case
   * - **Pure Zarr** (default)
     - ``calibrate``
     - none
     - Production calibration from pre-converted Zarr stores
   * - **Full** (``fits-ingest``)
     - ``calibrate``, ``zarr-fits``
     - ``libcfitsio-dev``
     - Development, ingestion, end-to-end processing

Supported Telescopes
--------------------

Telescope support is driven by YAML profiles (see :doc:`developer/telescope-profiles`):

- **CCAT/FYST** — 200–1200 GHz, Atacama site (5612 m)
- **SOFIA/upGREAT** — 1000–5000 GHz, airborne (13 km)
- **Generic** — fallback for any FITS data with standard keywords

Parity Validation
-----------------

The engine is validated against the legacy kalibrate (C++/Fortran) pipeline
across 7 datasets covering all production modes:

.. list-table::
   :header-rows: 1
   :widths: 30 20 20 15 15

   * - Dataset
     - Mode
     - Pixels
     - Max Dev
     - Status
   * - HFAV_OTF
     - OTF TP
     - 7 HFA
     - 0.004%
     - PASS
   * - HFAV_OI_BLANK_TEST
     - OTF TP
     - 7 HFA
     - 0.034%
     - PASS
   * - OFF_EXCLUDE_TEST
     - OTF TP
     - 1 LFA
     - 0.001%
     - PASS
   * - LFAV_HONEYCOMB
     - OTF-OFF
     - 1 LFA
     - 0.0004%
     - PASS
   * - HFAV_4G_MISSING_DATA
     - multi-band
     - 11
     - 0.635%
     - PASS
   * - HFA_4G_LOAD_TABLE
     - multi-band + load_temp
     - 11
     - 0.642%
     - PASS
   * - SIMULATED_DATA_HFA_4G
     - multi-band (Tier 3)
     - 5
     - 0.034%
     - PASS

All deviations are below the 1% acceptance threshold and arise from
controlled differences (f64 vs f32 truncation in the legacy code).