Flag System#

Flags are stored as a u16 bitmask per spectral element [C, D, R, A, S], shared across the calibration and reduction stages.

Bit Layout#

Bit

Name

Source

Description

0

BAD_CHANNEL

Calibration

Hot/cold signal absent, T_rec unphysical, or clip threshold

1

FILL_VALUE

Data format

Padded/missing data (NaN in source)

2

SPIKE

Reduction pipeline

Spectral spike detected

3

RFI

Reduction pipeline

Radio frequency interference

4

TSYS_OUTLIER

Reduction pipeline

Tsys deviates from median

5

ELEVATION_LOW

Reduction pipeline

Elevation below threshold

6

ML_FLAGGED

ML pipeline

Machine-learning anomaly detection

7

MANUAL

User

Manual flagging

8–15

(reserved)

—

Future use

Bits 0–1 are set by the calibration engine. Bits 2–7 are set by the downstream reduction pipeline. Bits 8–15 are reserved.

Flag Operations#

Implementation: cal-core/src/flags.rs

// Check if a specific bit is set
fn is_flagged(flag: u16, bit: u8) -> bool

// Set a bit
fn set_flag(flag: &mut u16, bit: u8)

Array-Level Utilities#

  • flag_channel(flags, ch, bit) — set bit for one channel across all D, R, A, S dimensions

  • flag_spectrum(flags, d, r, a, s, bit) — set bit for one spectrum (all channels)

  • flag_scan(flags, bit) — set bit for entire scan

  • count_flagged(flags, bit) — count elements with a specific bit set

Adding a New Flag#

  1. Define the constant in cal-core/src/flags.rs

  2. Set the flag at the appropriate point in the pipeline

  3. Document the bit in the table above

  4. Update downstream consumers that check flags != 0

Downstream Contract#

Consumers should check flags before using spectral data:

# Python: mask flagged data
import numpy as np
good = flags == 0  # no flags at all
spectra_clean = np.where(good, spectra, np.nan)

# Check specific flag
bad_channel = (flags & (1 << 0)) != 0