Data Structures#

Data flows through the pipeline as a sequence of typed structures. This page documents each type and its array shapes.

        graph LR
  SD[ScanData<br/>raw 5D counts] --> CL[CalibrationLoadFull<br/>gamma, Trec per C,R,A]
  SD --> PD[PreparedData<br/>ref_data, ON indices]
  CL --> AR[AtmosphereResult<br/>tr_s, tr_i, PWV]
  CL --> CR[CalibrationResult]
  AR --> CR
  PD --> CR[CalibrationResult<br/>spectra, Tsys, flags]

ScanData#

Raw observation data for one scan, loaded from L0 Zarr.

  • data: Vec<Array4<f64>> — per-subscan [C, D, R, A] arrays (stored as Vec instead of a single 5D array to avoid contiguous allocation of >1 GB)

  • sobsmode: Vec<String> — mode per subscan (ON, OFF, OTF-ON, etc.)

  • mjd, exptime, elevation, azimuth, pamb, tamb — per-subscan metadata

  • signal_freq, image_freq, freq_res, freq_off, ref_channel — frequency info

  • otf_lon, otf_lat: Option<Array2<f64>> — OTF coordinates [D, S] (if available)

  • pixel_offset_lon, pixel_offset_lat: Option<Array3<f64>> — per-pixel focal-plane offset [R, A, S], per-subscan (issue #48); legacy [R, A] stores read as [R, A, 1]

  • scan_number, source, instmode, line, rest_freq_hz, velocity_source_kms

Source: cal-core/src/scan/scan_data.rs

CalibrationSnapshot#

Raw HOT/COLD calibration data from the nearest calibration scan.

  • data: Vec<Array4<f64>> — per-subscan [C, D, R, A]

  • sobsmode: Vec<String> — HOT, COLD, COL, SKY, ZERO

  • thot, tcold: Array1<f32> — physical load temperatures (K)

  • load_temp_sig, load_temp_img: Option<...> – per-channel load RJ temperatures (if LOAD_TEMP_ARRAY available)

  • find_hot_cold() method – locates HOT and COLD subscan indices

Source: cal-core/src/scan/cal_snapshot.rs

CalibrationLoad (Single Pixel)#

Resolved calibration data for one pixel (r, a):

  • hot_counts, cold_counts: Array1<f64> — [C]

  • signal_freqs, image_freqs: Array1<f64> — [C] Hz

  • gamma, t_rec_prime, t_rec_ssb: Array1<f64> — [C]

  • bad_channels: Array1<bool> — [C]

  • ta_prime_hot, ta_prime_cold: Array1<f64> — [C] primed temps

  • a_signal, a_image: Array1<f64> — [C] sky coupling coefficients

  • gsxs, gixi: Array1<f64> — [C] sideband gain products

  • famb, airmass, t_hot, t_cold, t_amb, p_amb: f64 — scalars

Constructors (new, new_with_load_temps) take &CalibrationParams rather than separate foeff/gain_image/ physics arguments.

Source: cal-core/src/scan/cal_load.rs

CalibrationLoadFull (All Pixels)#

Same quantities as CalibrationLoad but with [C, R, A] arrays, enabling vectorized calibration without per-pixel loops. Provides extract_pixel(r, a) -> CalibrationLoad for backward compatibility.

Source: cal-core/src/scan/cal_load.rs

PreparedData#

Mode-specific ON/OFF pairing result:

  • ref_data: Array4<f64> – [C, R, A, S_on] reference spectra (broadcast across dumps for standard modes)

  • ref_data_5d: Option<Array5<f64>> – [C, D, R, A, S_on] per-dump OFF reference (used by OTF-chopped for per-dump subtraction)

  • on_subscan_indices: Vec<usize> – indices into ScanData.data for ON subscans (zero-copy access)

  • off_index_per_on: Vec<usize> – OFF subscan index paired with each ON (for per-subscan atmosphere fitting)

  • beam_pairings: Option<Vec<BeamPairing>> – beam A/B metadata for DBS adhesive coupling

  • t_int: Array1<f64> – [S_on] integration times

Source: cal-core/src/modes/prepare.rs

CalibrationResult#

Final calibration output for one scan:

  • spectra: Array5<f64> — [C, D, R, A, S] $T_A^*$ (K)

  • t_sys: Array4<f64> — [C, R, A, S]

  • t_int: Array1<f64> — [S]

  • t_rec_ssb, gamma: Array3<f64> — [C, R, A]

  • tau_signal, tau_image: Array1<f64> — [C]

  • t_sky: Array3<f64> — [C, R, A]

  • flags: Array5<u16> — [C, D, R, A, S]

  • signal_freqs, image_freqs: Array1<f64> — [C]

  • otf_lon, otf_lat, otf_airmass: Option<Array2<f64>> — [D, S]

  • pixel_offset_lon, pixel_offset_lat: Option<Array3<f64>> — [R, A, S], per-subscan focal-plane offset, pass-through from L0 (issue #48)

  • pwv: f64 – fitted PWV (mm)

  • Plus metadata: scan_number, source, arrays, pixels, strategies

Source: cal-core/src/scan/cal_result.rs