Utilities#

✓
Documentation Verified Last checked: 2025-01-21 Reviewer: Christof Buchbender

This page documents utilities for database initialization and data loading. These are used for setting up a fresh database instance.

Database Initialization#

The opsdb_init command-line tool initializes a fresh ops-db instance with observatory infrastructure, instruments, and observation units.

ccat_ops_db.opsdb_init.opsdb_initialization(verbose=False, drop=False, local_url=None, docker_compose=False, database=None, data_archive_mode='development')[source]#

This function provisions the ccat_obs_db for testing purposes

Command-Line Usage:

# Initialize with default settings
opsdb_init

# Initialize and drop existing tables
opsdb_init --drop

# Initialize with local data-center repository
opsdb_init --local_url /path/to/data-center

# Initialize for production mode
opsdb_init --data_archive_mode production

# Verbose output
opsdb_init -v --drop

Observation Unit Initialization#

ccat_ops_db.init_obs_units.init_obs_units(installation_path, input_yaml_file, session)[source]#
ccat_ops_db.init_obs_units.read_obs_units(obsunits_list, obs_unit_type='chai_obs_unit', inpar_parameters_folder=None, tiling_parameters_folder=None, table_azimuth_file=None, table_mapping_params_file=None, session=None)[source]#

reads csv file with first line as header and returns a list of dictionaries

ccat_ops_db.init_obs_units.read_sources(source_file, file_format, source_type, session)[source]#

Read a source file and add sources to the database session

Parameters:

  • source_file (str) – Input file name with source namd and coordinates

  • file_format (str) – kosma (kosma_software source file format) or csv. If csv, the coordinate system is assumed to be icrs.

  • source_type (str) – None is fixed target with coordinates. Other names should be compatible with the object in models.

  • session (database session)

ccat_ops_db.init_obs_units.read_pre_scheduled_slots(pre_scheduled_file, session)[source]#

Read pre-scheduled slots and add them to the database session

Parameters:

  • pre_scheduled_file (str) – Input file name with pre-scheduled obsunits and time

  • session (database session)

These functions parse YAML/CSV files from the data-center repository and populate the database with sources, observation units, and scheduling information.

Utility Functions#

These functions provide helpers for adding or updating database entities.

Sources#

ccat_ops_db.utils.add_or_update_source(session, values, source_type=None)[source]#

Example:

from ccat_ops_db.utils import add_or_update_source

values = {
    "name": "Orion-KL",
    "slam": "05:35:14.5",
    "sbet": "-05:22:30.0",
    "frame": "icrs",
    "vlsr": 9.0
}
add_or_update_source(session, values, source_type="fixed_source")

Observation Units#

ccat_ops_db.utils.add_or_update_obs_unit(session, values, obs_unit_type='chai_obs_unit', inpar_parameters_list=None, tiling_parameters=None, instrument_module_configurations=None, azimuth_range=None, mapping_parameters=None)[source]#
ccat_ops_db.utils.add_inpar_and_tiling(session, observation_configuration, inpar_parameters_list=None, tiling_parameters=None)[source]#

Add inpar and tiling parameters to the observation_configuration

Parameters:

  • session (database session)

  • observation_configuration (ccat.opsdb.observation_configuration) – Observation configuration with inpar and tiling to be associated

  • inpar_parameters_list (list) – List of dictionary of inpar_parameters

  • tiling_parameters (list) – List of dictionary of each line in tiling

Returns:

observation_configuration – Updated observation configuration

Return type:

ccat.opsdb.observation_configuration

Pre-Scheduled Slots#

ccat_ops_db.utils.add_pre_scheduled_slot(session, values)[source]#

Data Center Repository Structure#

The initialization process expects data files in the data-center repository with the following structure:

data-center/
├── observatory/
│   └── *.toml (observatory definitions)
├── telescope/
│   └── *.toml (telescope definitions)
├── instruments/
│   └── *.toml (instrument and module definitions)
├── roles/
│   └── *.toml (user role definitions)
├── users/
│   └── *.toml (user definitions)
├── observing_programs/
│   └── *.toml (observing program definitions)
├── lines/
│   └── *.toml (spectral line definitions)
├── obs_modes/
│   └── *.toml (observation mode definitions)
├── sites/
│   ├── sites.development.toml
│   ├── sites.staging.toml
│   └── sites.production.toml (site and location definitions)
└── obs_units/
    ├── input_file_obs_units_to_db.yaml (master input file)
    ├── sources/
    ├── obsunits/
    ├── inpar_parameters/
    └── tiling_parameters/

The --data_archive_mode flag selects which sites configuration to load (development, staging, or production).