Crate Map

The workspace is organised into seven crates with clear dependency boundaries. The key design principle is that cal-core has no I/O dependencies — it operates purely on in-memory arrays.

Dependency Graph

        graph BT
  schema[cal-schema<br/>L0/L1 constants]
  core[cal-core<br/>pure math]
  io[cal-io<br/>Zarr I/O + pipeline]
  cli[cal-cli<br/>calibrate binary]
  py[cal-py<br/>Python bindings]
  zfc[zarr-fits-core<br/>FITS→Zarr library]
  zfcli[zarr-fits-cli<br/>zarr-fits binary]

  core --> schema
  io --> core
  io --> schema
  io -.->|fits-ingest| zfc
  cli --> io
  cli --> core
  cli -.->|fits-ingest| zfc
  py --> io
  py --> core
  zfc --> schema
  zfcli --> zfc

  style schema fill:#e8eaf6
  style core fill:#e3f2fd
  style io fill:#e8f5e9
  style zfc fill:#fff3e0
    

Dashed arrows indicate optional dependencies gated by the fits-ingest feature flag.

Crate Details

cal-schema

Zero-dependency crate containing string constants for all L0 and L1 array names, group names, and schema versions. Both the writer (zarr-fits-core) and reader (cal-io) depend on this single source of truth, preventing schema drift.

Key file: crates/cal-schema/src/lib.rs

cal-core

Pure calibration math with no filesystem access. Dependencies are limited to ndarray, rayon, serde, nalgebra, and levenberg-marquardt.

Modules:

• math/ — Rayleigh-Jeans, gamma, temperature, transmission, dump averaging

• atm/ — AtmosphericModel trait + TableAtm interpolation

• pwv/ — PWV fitting (grid, LM, external)

• modes/ — observation mode preparation (TP, OTF)

• calibrate.rs — vectorized calibration loop

• atmosphere.rs — atmosphere determination orchestration

• scan.rs — core data types (ScanData, CalibrationLoad, etc.)

• config.rs — CalibrationConfig, ObsMode, PhysicsConstants

• flags.rs — u16 bitmask constants and utilities

• qa.rs — quality metrics

cal-io

Zarr I/O via the zarrs crate (sync API only, no async runtime) and pipeline orchestration.

Modules:

• raw.rs — L0 Zarr reader (5D data, metadata, coordinates)

• calibrated.rs — L1 Zarr writer

• resolve.rs — bridges raw L0 into CalibrationLoad(Full)

• pipeline.rs — store-level orchestration with prefetch

• atm_table.rs — text ATM table parser

• atm_table_bin.rs — binary ATM table I/O

• ingest.rs — FITS ingest (behind fits-ingest feature)

cal-cli

Thin binary wrapping cal-io’s pipeline with clap argument parsing, indicatif progress bar, and manifest mode. Contains zero calibration logic.

Binary name: calibrate

cal-py

Thin PyO3 surface exposing calibrate_store(), calibrate_scan_py(), list_scans(), and optionally convert_fits_folder() to Python. Contains zero logic — all work is dispatched to cal-core and cal-io.

zarr-fits-core

FITS→Zarr v3 conversion library. Reads individual FITS files via cfitsio, groups by scan, writes Zarr v3 stores with streaming per-subscan memory.

zarr-fits-cli

Standalone zarr-fits binary for FITS conversion without calibration.

Feature Flags

Feature	Crate	Effect
fits-ingest	cal-io	Adds pub mod ingest wrapping zarr-fits-core
fits-ingest	cal-cli	Enables -f/--fits-folder and --zarr-cache flags
fits-ingest	cal-py	Exposes convert_fits_folder() to Python

Without fits-ingest, the workspace has zero C library dependencies — everything compiles with a standard Rust toolchain.