Models API Reference#
This page provides the complete Python API reference for all SQLAlchemy models in ops-db.
For conceptual explanations of these models, see:
Complete Model Documentation#
- class ccat_ops_db.models.LocationType(value)[source]#
Bases:
EnumLocation type enum
The location type determines the role of the location in the data transfer system.
SOURCE: Telescope instrument computers BUFFER: Input/transit buffers LONG_TERM_ARCHIVE: Permanent storage PROCESSING: Temporary processing areas
- SOURCE = 'source'#
- BUFFER = 'buffer'#
- LONG_TERM_ARCHIVE = 'long_term_archive'#
- PROCESSING = 'processing'#
- class ccat_ops_db.models.StorageType(value)[source]#
Bases:
EnumStorage type enum
The storage type determines the physical storage medium of the location.
DISK: Traditional disk storage S3: Object storage (AWS S3 or compatible) TAPE: Tape-based archival storage
- DISK = 'disk'#
- S3 = 's3'#
- TAPE = 'tape'#
- class ccat_ops_db.models.RouteType(value)[source]#
Bases:
EnumRoute type enum
The route type determines the route of the data transfer.
DIRECT: RELAY: Route through intermediate site CUSTOM: Custom location-to-location override
- DIRECT = 'direct'#
- RELAY = 'relay'#
- CUSTOM = 'custom'#
- class ccat_ops_db.models.Status(value)[source]#
Bases:
EnumStatus enum
These Status labels are used to make the Status checks uniform accross the entire data transfer system.
These are used accros many concepts such as RawDataPackage, DataTransferPackage, DataTransfer, etc.
PENDING: Is used to mark a concept as waiting for execution e.g. a RawDataPackage in PENDING will be scheduled to be build. SCHEDULED: Marks a concept as scheduled for execution e.g. a RawDataPackage in IN_PROGRESS: Marks that a celery job is working on this concept. COMPLETED: Succesfully completed the concept. FAILED: Failed to execute the concept.
- PENDING = 'pending'#
- SCHEDULED = 'scheduled'#
- IN_PROGRESS = 'in_progress'#
- COMPLETED = 'completed'#
- FAILED = 'failed'#
- class ccat_ops_db.models.PackageState(value)[source]#
Bases:
EnumPackage state enum
The package state determines the state of the package.
WAITING: Waiting for transfer TRANSFERRING: Transferring ARCHIVED: Archived FAILED: Failed
- WAITING = 'waiting'#
- TRANSFERRING = 'transferring'#
- ARCHIVED = 'archived'#
- FAILED = 'failed'#
- class ccat_ops_db.models.PhysicalCopyStatus(value)[source]#
Bases:
EnumAn enumeration.
- PRESENT = 'present'#
- STAGED = 'staged'#
- DELETION_POSSIBLE = 'deletion_possible'#
- DELETION_PENDING = 'deletion_pending'#
- DELETION_SCHEDULED = 'deletion_scheduled'#
- DELETION_IN_PROGRESS = 'deletion_in_progress'#
- DELETION_FAILED = 'deletion_failed'#
- DELETED = 'deleted'#
- LOST = 'lost'#
- class ccat_ops_db.models.TriggerType(value)[source]#
Bases:
EnumAn enumeration.
- CONTINUOUS = 'continuous'#
- CRON = 'cron'#
- MANUAL = 'manual'#
- class ccat_ops_db.models.RunStatus(value)[source]#
Bases:
EnumAn enumeration.
- PENDING = 'pending'#
- STAGING_DATA = 'staging_data'#
- SUBMITTED = 'submitted'#
- RUNNING = 'running'#
- COLLECTING_RESULTS = 'collecting_results'#
- COMPLETED = 'completed'#
- FAILED = 'failed'#
- CANCELLED = 'cancelled'#
- class ccat_ops_db.models.DataProductType(value)[source]#
Bases:
EnumAn enumeration.
- SCIENCE = 'science'#
- QA_METRIC = 'qa_metric'#
- PLOT = 'plot'#
- LOG = 'log'#
- STATISTICS = 'statistics'#
- INTERMEDIATE = 'intermediate'#
- class ccat_ops_db.models.OperationKind(value)[source]#
Bases:
str,EnumIdentity axis of a pipeline operation and the polymorphic discriminator for the uniform Operation model (ops-db #94, ADR-0003).
The values are BYTE-IDENTICAL to the frozen breadcrumb strings already stored in
OperationFailureEvent.operation_typeand used by recovery, the circuit breaker, and routing. Keeping them equal makes this a code refactor rather than a data migration, and makes the operation-kind drift class (recovery’sarchivevs the task’slong_term_archive) unrepresentable: this enum is the single source of truth for identity.strsubclassing lets a member compare and serialize as its raw string, so it can be stored directly in the discriminator column and matched against the legacy breadcrumb strings without translation.Note the deliberate name/value asymmetry on two members: PACKAGING ==
raw_data_packageand ARCHIVE ==long_term_archive– the names follow the stage, the values follow the frozen strings.- PACKAGING = 'raw_data_package'#
- BUNDLING = 'data_transfer_package'#
- TRANSFER = 'transfer'#
- UNPACK = 'unpack'#
- ARCHIVE = 'long_term_archive'#
- STAGING = 'staging'#
- class ccat_ops_db.models.Site(**kwargs)[source]#
Bases:
BaseRepresents a physical or logical site where data can be stored or processed.
- id#
- name#
- short_name#
- site_location#
- long_term_archive_transfers#
- locations#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DataLocation(**kwargs)[source]#
Bases:
BaseBase class for all data storage locations with polymorphic storage types.
- id#
- name#
- location_type#
- site_id#
- site#
- active#
- priority#
- decommissioned_at#
- decommission_reason#
- storage_type#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DiskDataLocation(**kwargs)[source]#
Bases:
DataLocationDisk-based storage location.
- id#
- path#
- host#
- user#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- active#
- decommission_reason#
- decommissioned_at#
- location_type#
- name#
- priority#
- site#
- site_id#
- storage_type#
- class ccat_ops_db.models.S3DataLocation(**kwargs)[source]#
Bases:
DataLocationS3-compatible object storage location.
- id#
- bucket_name#
- region#
- endpoint_url#
- access_method#
- path_style#
- coscine_project#
- coscine_resource#
- get_s3_credentials(site_name: str) tuple[str, str][source]#
Resolve S3 credentials from the environment for this location.
Credentials are looked up (via dynaconf) under the per-location name:
{Site.short_name}_{DataLocation.name}_S3_ACCESS_KEY_ID {Site.short_name}_{DataLocation.name}_S3_SECRET_ACCESS_KEY
Because data-transfer’s dynaconf prefix is
CCAT_DATA_TRANSFER, the actual environment variables carry that prefix. For the Cologne site (short_name="cologne") with along_term_archivelocation:CCAT_DATA_TRANSFER_COLOGNE_LONG_TERM_ARCHIVE_S3_ACCESS_KEY_ID=... CCAT_DATA_TRANSFER_COLOGNE_LONG_TERM_ARCHIVE_S3_SECRET_ACCESS_KEY=...
Do NOT embed the site name in the location name: the convention already prefixes
site.short_name, so a location literally namedcologne_long_term_archiveyields the doubled, easy-to-miss..._COLOGNE_COLOGNE_LONG_TERM_ARCHIVE_.... Name the location justlong_term_archive.If the per-location variable is unset, this falls back to the global
S3_ACCESS_KEY_ID/S3_SECRET_ACCESS_KEY(default placeholder"MUST_SET_VIA_ENV"). A misspelled/doubled name therefore surfaces at the S3 layer asInvalidAccessKeyIdrather than a config error — the data-transfer worker logs the resolved env-var name and a masked key fingerprint (utils.get_s3_client) to make this diagnosable.- Parameters:
site_name (str) – The site’s
short_name(passed by the pipeline).- Returns:
(access_key_id, secret_access_key)
- Return type:
tuple[str, str]
- get_coscine_api_token(site_name: str) str[source]#
Get the Coscine API token for this location (secret, env-resolved).
Mirrors
get_s3_credentials(): looks for a per-location token{Site.name}_{DataLocation.name}_COSCINE_API_TOKENand falls back to the deployment-wideCOSCINE_API_TOKEN. The non-secret project/resource live on the location (coscine_project/coscine_resource).
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- active#
- decommission_reason#
- decommissioned_at#
- location_type#
- name#
- priority#
- site#
- site_id#
- storage_type#
- class ccat_ops_db.models.TapeDataLocation(**kwargs)[source]#
Bases:
DataLocationTape-based storage location.
- id#
- library_name#
- mount_path#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- active#
- decommission_reason#
- decommissioned_at#
- location_type#
- name#
- priority#
- site#
- site_id#
- storage_type#
- class ccat_ops_db.models.ObservingProgram(**kwargs)[source]#
Bases:
Base- id#
- name#
- short_name#
short name without spaces to be used in selection
- description#
- lead_id#
- lead#
- sub_observing_programs#
- instruments#
- obs_units#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.SubObservingProgram(**kwargs)[source]#
Bases:
Base- id#
- name#
- short_name#
short name without spaces to be used in selection
- description#
- observing_program_id#
- observing_program#
- obs_units#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.Source(**kwargs)[source]#
Bases:
BaseA source is a celestial object
This class serves as a base class for various types of sources. It is a polymorphic class in SQLAlchemy, not instantiated directly, but used to provide common attributes. Subclasses, implemented as separate database tables, inherit from the source class and can have additional specific attributes.
See classes that are based on this class for more information on the implemented types of sources.
- id#
- name#
The name of the source
- version#
Version number of the current parameters
- history#
Change history
- type#
The polymorphic identity
- obs_units#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.FixedSource(**kwargs)[source]#
Bases:
SourceA source that has a fixed coordinates
This class is a subclass of the Source class and inherits all attributes from the Source class.
The class implements the skycoord property that returns a SkyCoord object from the ra_deg and dec_deg columns.
- id#
- ra_deg#
The right ascension in degrees (ICRS)
- dec_deg#
The declination in degrees (ICRS)
- slam#
Longitude string of the original input
- sbet#
Latitude string of the original input
- vlsr#
The local standard of rest velocity in km/s
- frame#
The frame of the coordinates of the original input
- property skycoord#
Return a SkyCoord object from the ra_deg and dec_deg columns
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- history#
Change history
- name#
The name of the source
- obs_units#
- type#
The polymorphic identity
- version#
Version number of the current parameters
- class ccat_ops_db.models.SolarSystemObject(**kwargs)[source]#
Bases:
SourceA source that is a solar system object
This class is a subclass of the Source class and inherits all attributes from the Source class.
- id#
- eph_name#
Standard ephemeris name of the source
- naif_id#
The NAIF ID of the source
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- history#
Change history
- name#
The name of the source
- obs_units#
- type#
The polymorphic identity
- version#
Version number of the current parameters
- class ccat_ops_db.models.ConstantElevationSource(**kwargs)[source]#
Bases:
SourceA source that is observed at a constant elevation
This class is a subclass of the Source class and inherits all attributes from the Source class.
- id#
- ra_deg_min#
The minimum right ascension of the area in degrees
- ra_deg_max#
The maximum right ascension of the area in degrees
- dec_deg_min#
The minimum declination of the area in degrees
- dec_deg_max#
The maximum declination of the area in degrees
- slam_min#
The minimum longitude string of the original input
- slam_max#
The maximum longitude string of the original input
- sbet_min#
The minimum latitude string of the original input
- sbet_max#
The maximum latitude string of the original input
- vlsr#
The local standard of rest velocity in km/s
- frame#
The frame of the coordinates of the original input
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- history#
Change history
- name#
The name of the source
- obs_units#
- type#
The polymorphic identity
- version#
Version number of the current parameters
- class ccat_ops_db.models.Line(**kwargs)[source]#
Bases:
BaseA spectral line that is observed by an instrument
- id#
- name#
The name of the spectral line
- rest_frequency#
The rest frequency of the line
- side_band#
The side band of the line
- available#
Whether this line is ready to be scheduled
- comment#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.Observatory(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- telescopes#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.Telescope(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- lon_deg#
- lat_deg#
- alt_m#
- instruments#
- observatory_id#
- observatory#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.Instrument(**kwargs)[source]#
Bases:
Base- id#
- name#
- instrument_type#
- description#
- telescope_id#
- telescope#
- modules#
- observing_programs#
- available#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.InstrumentModule(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- instrument_id#
- instrument#
- instrument_module_configurations#
- available#
- raw_data_packages#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.InstrumentModuleConfiguration(**kwargs)[source]#
Bases:
Base- id#
- type#
- instrument_module_id#
- instrument_module#
- raw_data_files#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- obs_units#
- primary_obs_units#
- class ccat_ops_db.models.ChaiModuleConfiguration(**kwargs)[source]#
Bases:
InstrumentModuleConfiguration- id#
- line_id#
- line#
- if_ghz#
Intermediate frequency (IF) in GHz at the center of the signal sideband
- config_parameters#
List of instrument configuration parameters
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- instrument_module#
- instrument_module_id#
- obs_units#
- primary_obs_units#
- raw_data_files#
- type#
- class ccat_ops_db.models.PrimeCamModuleConfiguration(**kwargs)[source]#
Bases:
InstrumentModuleConfiguration- id#
- config_parameters#
List of instrument configuration parameters
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- instrument_module#
- instrument_module_id#
- obs_units#
- primary_obs_units#
- raw_data_files#
- type#
- class ccat_ops_db.models.ObservationConfiguration(**kwargs)[source]#
Bases:
Base- id#
- type#
- obs_units#
- azimuth_range#
Azimuth range lookup table for constant elevation scans
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ChaiObservationConfiguration(**kwargs)[source]#
Bases:
ObservationConfiguration- id#
- chai_tilings#
- ntilelines#
Number of tile lines to be grouped (for socring in scheduler)
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- azimuth_range#
Azimuth range lookup table for constant elevation scans
- obs_units#
- type#
- class ccat_ops_db.models.PrimeCamObservationConfiguration(**kwargs)[source]#
Bases:
ObservationConfiguration- id#
- version#
Version number of the current parameters
- history#
Change history
- mapping_parameters#
List of mapping parameters
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- azimuth_range#
Azimuth range lookup table for constant elevation scans
- obs_units#
- type#
- class ccat_ops_db.models.ObsMode(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- obs_units#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ObsUnit(**kwargs)[source]#
Bases:
Base- id#
- name#
- version#
Version number of the current parameters
- history#
Change history
- phase#
Common name of campaigns that share the observing requests.
- group#
List of other ObsUnits that belong to the same group
- group_type#
Reason of grouping. ‘equal’ indicates balanced scheduling, ‘either’ indicates resetting cadence once one of them are observed.
- equal_tolerance#
Tolerance of inbalanced schedule when group_type is equal
- min_alt#
Minimum allowed altitude (elevation) [deg]
- max_alt#
Maximum allowed altitude (elevation) [deg]
- min_rotang#
Minimum allowed rotation angle [deg]
- max_rotang#
Maximum allowed rotation angle [deg]
- nominal_alt#
Either fixed altitude in [deg] or ‘var’, which allows a flexible choice
- min_lsa#
Minimum LSA [deg]
- max_lsa#
Maximum LSA [deg]
- lsa_margin#
Allowed deviation from min_lsa and max_lsa [deg]
- cadence#
Requested cadence for scheduling [day]
- requested_time_h#
Requested total observing hours. Mandatory for PrimeCam. For CHAI it is calculated by contents in ChaiTiling
- unit_duration_h#
Observing duration [h]
- trans_ref#
Reference transmission
- priorities#
Science priority
- available#
Whether this ObsUnit is ready to be scheduled
- pre_scheduled_basis#
Whether this Obsunit is executed on pre-scheduled basis, meaning that it has a very low priority outside of the pre-scheduled slots.
- additional_parameters#
Any special parameters or constraints to be stored
- source_id#
- source#
- observing_program_id#
- observing_program#
- sub_observing_program_id#
- sub_observing_program#
- obs_mode_id#
- obs_mode#
- primary_instrument_module_configuration_id#
- primary_instrument_module_configuration#
- instrument_module_configurations#
- observation_configuration_id#
- observation_configuration#
- executed_obs_units#
- pre_scheduled_slots#
- raw_data_packages#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.PreScheduledSlot(**kwargs)[source]#
Bases:
Base- id#
- start_time#
- end_time#
- obs_unit_id#
- obs_unit#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ExecutedObsUnit(**kwargs)[source]#
Bases:
Base- id#
- start_time#
- end_time#
- status#
Status of this observation. ‘running’ indicates currently running, ‘success’ or ‘completed’ will be counted as a successfully completed observation.
- mean_pwv#
Mean pwv [mm] during observation
- mean_elevation#
Mean elevation [deg] during observation
- achievement_factor#
Correction factor for the achieved time compared to the reference transmission
- quality#
If discard, not considered in counting achievement
- obs_unit_id#
- obs_unit#
- obs_info#
Static ancillary infomation (e.g. ObsUnit version, tiling ID for CHAI)
- obs_progress#
Field for progress tracking
- raw_data_packages#
- raw_data_files#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ChaiTiling(**kwargs)[source]#
Bases:
Base- id#
- version#
Version number of the current parameters
- history#
Change history
- priority_in_tiling#
- tile_id#
CHAI internal tile ID
- tile_offset_x#
tile position along x-axis
- tile_offset_y#
tile position along y-axis
- x_or_y#
Scan direction
- tile_unit_scaling_x#
Scaling factor for the script-defined tiling unit for x-direction
- tile_unit_scaling_y#
Scaling factor for the script-defined tiling unit for y-direction
- edge#
Edge name if half of the tile should be observed
- goal_ncycle#
Goal number of cycle
- chai_observation_configuration_id#
- chai_observation_configuration#
- chai_inpar_parameter_id#
- chai_inpar_parameter#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ChaiInparParameter(**kwargs)[source]#
Bases:
Base- id#
- name#
- version#
Version number of the current parameters
- history#
Change history
- chai_tilings#
- lam#
Map center offset in longitude [arcsec]
- bet#
Map center offset in latitude [arcsec]
- cormap#
Map cood. system
- line_range#
v,-10:10:v”)
- Type:
Range of velocities [km/s] affected by lines (e.g. “-70
- Type:
-30
- goal_resolution#
Spectral resolution [km/s] with which the scientific analysis will be done
- refname#
Reference name when using the absolute positions. Set NAN to use relative
- refoffl#
Reference position relative to on in longitude [arcsec]
- refoffb#
Reference position relative to on in latitude [arcesc]
- corref#
Reference cood. system
- mode#
Observing mode; otfl or otfb
- otfpattern#
OTF pattern file name specified in kosma_software
- ton#
On integration time [sec]
- toff#
Forced off time [s] (-1 to use the default calculation = default)
- repetition#
Repetition number
- offononoff#
Sequence of OFF and ON (1 = OFF-ON-ON-OFF (default), 0 = OFF-ON)
- stepl#
Step size in longitude [arcsec]
- stepb#
Step size in latitude [arcsec]
- mapsizel#
Size of map in longitude [arcsec]
- mapsizeb#
Size of map in latitude [arcsec]
- nmapl#
number of raster positions along longitude
- nmapb#
number of raster positions along latitude
- mapangle#
Position angle of the map (counter-clock) [degree]
- crosssizel#
Size of cross in longitude [arcsec]
- crosssizeb#
Size of cross in latitude [arcsec]
- reverseflg#
0 = (+x,+y), 1 = (-x,-y), 2 = (-x,+x,-y,+y)
- Type:
OTF scan direction in cross
- scan_dir#
OTF scan direction (1 is +x or +y, -1 is -x or -y)
- scan_order#
Order of OTF lines (1 is +x or +y, -1 is -x or -y)
- evendump#
If 1 allow even number of dump positions without hitting the center. Default = 0
- novertical#
If 1 skip the second (vertical) scan for cross observations. Default = 0
- pointingflg#
Flag to indicate that it is a pointing session. Default = 0
- offperload#
Define how many off for one load measurment
- onperload#
Define how many on positions for one load measurement
- repperload#
Define how many repetition for one load measurment
- lineperoff#
Define how many otf scan lines for one off measurement
- onperoff#
Define how many on positions for one off measurement
- offperpattern#
Define how many off measurement per pattern
- refpoint#
Reference point (as is used in setpoint)
- act_pixflg#
Flag to use the actual pixel position of refpoint. Default = 0
- nextflg#
If Y, enable telescope moving when writing data. Default = N
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.Role(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- github_team_mappings#
GitHub teams that map to this role
- permissions#
List of permissions granted to this role
- users#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.User(**kwargs)[source]#
Bases:
Base- id#
- email#
- username#
- first_name#
- last_name#
- title#
- affiliation#
- password#
- github_id#
GitHub user ID
- github_username#
GitHub username
- preferences#
User preferences and settings
- roles#
- last_login_at#
- current_login_at#
- last_login_ip#
- current_login_ip#
- login_count#
- active#
- confirmed_at#
- observing_programs#
- api_tokens#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ApiToken(**kwargs)[source]#
Bases:
BaseAPI tokens for programmatic access to the API
- id#
- user_id#
- user#
- name#
Human-readable name for the token
- token_hash#
Hashed token value
- token_prefix#
First few characters for identification
- scopes#
List of permission scopes for this token
- created_at#
- expires_at#
Token expiration time
- last_used_at#
Last time token was used
- active#
Whether token is active
- usage_count#
Number of times token was used
- last_used_ip#
IP address of last usage
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DataTransferRoute(**kwargs)[source]#
Bases:
BaseDefines routes for data transfer between sites and locations.
The database implements a flexible routing system that supports: 1. Direct routes between specific locations 2. Relay routes through intermediate sites 3. Custom location-to-location overrides
- id#
- name#
- origin_site_id#
- origin_site#
- destination_site_id#
- destination_site#
- route_type#
- transfer_method#
- origin_location_id#
- origin_location#
- destination_location_id#
- destination_location#
- relay_site_id#
- relay_site#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.RawDataFile(**kwargs)[source]#
Bases:
BaseRepresents a raw data file from an instrument.
- id#
- name#
- relative_path#
- created_at#
- instrument_module_configuration_id#
- instrument_module_configuration#
- file_type#
- size#
- checksum#
- description#
- source_location_id#
- source_location#
- raw_data_package_id#
- raw_data_package#
- data_transfer_package_id#
- data_transfer_package#
- executed_obs_unit_id#
- executed_obs_unit#
- state#
- physical_copies#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.RawDataPackageMetadata(**kwargs)[source]#
Bases:
BaseModel for storing additional metadata for raw data packages.
This table stores metadata that is not part of the core database models but is needed for IVOA-compatible metadata generation. It includes: - Instrument-specific parameters - Additional quality metrics - Extended provenance information - Custom metadata fields
- id#
- raw_data_package_id#
- created_at#
- updated_at#
- instrument_specific#
- quality_metrics#
- provenance#
- custom_metadata#
- raw_data_package#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.RawDataPackage(**kwargs)[source]#
Bases:
BaseA raw data package is a bundle of raw data files that were observed in an observation unit. But they should never be larger than 50GB in size.
- id#
- name#
- relative_path#
This is the relative path to the raw data package. The absolute path of the location of this file for each archive is stored in the DataLocation table. The path is relative to the raw_data_path of the DataLocation.
- size#
- executed_obs_unit_id#
- executed_obs_unit#
- instrument_module_id#
- instrument_module#
- obs_unit_id#
- obs_unit#
- created_at#
- checksum#
- raw_data_files#
- data_transfer_package_id#
- data_transfer_package#
- long_term_archive_transfers#
- analyze_status#
- physical_copies#
- package_metadata#
- state#
- staging_jobs#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DataTransferPackage(**kwargs)[source]#
Bases:
Base- id#
- hash_id#
- file_name#
- size#
- checksum#
- relative_path#
- origin_location_id#
- origin_location#
- raw_data_files#
- raw_data_packages#
- data_transfers#
- physical_copies#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DataTransfer(**kwargs)[source]#
Bases:
BaseTracks data transfers between locations.
- id#
- process_id#
- start_time#
- end_time#
- data_transfer_method#
- transfer_program_log#
- origin_location_id#
- origin_location#
- destination_location_id#
- destination_location#
- data_transfer_package_id#
- data_transfer_package#
- unpack_start_time#
- unpack_end_time#
- unpack_log#
- logs#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.SystemLog(**kwargs)[source]#
Bases:
Base- id#
- type#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DataTransferLog(**kwargs)[source]#
Bases:
SystemLogLog entries for data transfers with references to log files.
This model implements a lightweight approach where: - Basic status info and log file path are stored in the database - Full command outputs are stored in files - Detailed metrics are stored in InfluxDB
- id#
- data_transfer_id#
- data_transfer#
- timestamp#
- status#
Status of this transfer attempt (success/failure)
- log_path#
Path to file containing full command output
- property content: str | None#
Read and return the full log content if file exists.
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- type#
- class ccat_ops_db.models.OperationFailureEvent(**kwargs)[source]#
Bases:
SystemLogAppend-only failure-history record for any pipeline operation (#85).
One row per failure occurrence (retryable and permanent), never erased by a reset/retry — the durable trail behind the denormalized per-row
error_contextcache.Generic across operation types, so it keys on
(operation_type, operation_id)— the pair the system already correlates on (Celery task state, recovery handlers, circuit breaker, routing) — and carries NO foreign key to the operation row (it can’t reference one of several tables). Precedent for the bare integer id:PhysicalCopy.deletion_task_id.- id#
- operation_type#
- operation_id#
- error_type#
- error_context#
- created_at#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- type#
- class ccat_ops_db.models.DataArchive(**kwargs)[source]#
Bases:
Base- id#
- name#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.PhysicalCopy(**kwargs)[source]#
Bases:
BaseBase class for tracking physical copies of files across different storage locations.
This class implements a polymorphic pattern to track physical copies of different types of files (RawDataFile, RawDataPackage, DataTransferPackage) across different storage locations and types. Each physical copy represents an actual file on disk or other storage medium.
- id#
- type#
- data_location_id#
- data_location#
- created_at#
- verified_at#
- status#
- checksum#
- deletion_task_id#
- deleted_at#
- consumed_by_operations#
- produced_by_operations#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.RawDataFilePhysicalCopy(**kwargs)[source]#
Bases:
PhysicalCopyTracks physical copies of individual raw data files.
- id#
- raw_data_file_id#
- raw_data_file#
- property full_path#
Get the full path/S3 key for this physical copy.
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- checksum#
- consumed_by_operations#
- created_at#
- data_location#
- data_location_id#
- deleted_at#
- deletion_task_id#
- produced_by_operations#
- status#
- type#
- verified_at#
- class ccat_ops_db.models.RawDataPackagePhysicalCopy(**kwargs)[source]#
Bases:
PhysicalCopyTracks physical copies of raw data packages.
- id#
- raw_data_package_id#
- raw_data_package#
- property full_path#
Get the full path/S3 key for this physical copy.
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- checksum#
- consumed_by_operations#
- created_at#
- data_location#
- data_location_id#
- deleted_at#
- deletion_task_id#
- produced_by_operations#
- status#
- type#
- verified_at#
- class ccat_ops_db.models.DataTransferPackagePhysicalCopy(**kwargs)[source]#
Bases:
PhysicalCopyTracks physical copies of data transfer packages.
- id#
- data_transfer_package_id#
- data_transfer_package#
- property full_path#
Get the full path/S3 key for this physical copy.
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- checksum#
- consumed_by_operations#
- created_at#
- data_location#
- data_location_id#
- deleted_at#
- deletion_task_id#
- produced_by_operations#
- status#
- type#
- verified_at#
- class ccat_ops_db.models.LongTermArchiveTransfer(**kwargs)[source]#
Bases:
Base- id#
- site_id#
- site#
- origin_data_location_id#
- origin_data_location#
- destination_data_location_id#
- destination_data_location#
- raw_data_package_id#
- raw_data_package#
- logs#
- start_time#
- end_time#
- last_attempt_time#
- error_message#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.LongTermArchiveTransferLog(**kwargs)[source]#
Bases:
SystemLog- id#
- long_term_archive_transfer_id#
- long_term_archive_transfer#
- log#
- timestamp#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- type#
- class ccat_ops_db.models.StagingJob(**kwargs)[source]#
Bases:
Base- id#
- status#
- failure_error_message#
- start_time#
- active#
- end_time#
- retry_count#
- raw_data_packages#
- logs#
- origin_data_location_id#
- origin_data_location#
- destination_data_location_id#
- destination_data_location#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.StagingJobLog(**kwargs)[source]#
Bases:
SystemLog- id#
- staging_job_id#
- staging_job#
- log#
- timestamp#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- type#
- class ccat_ops_db.models.OperationGroup(**kwargs)[source]#
Bases:
BasePolymorphic parent that bundles N co-created operations (ADR-0003).
Its status is DERIVED from its child operations and is never set independently – modelled as a read-only Python property, not a mapped column, so the derivation rule is the single source of truth and the group cannot drift from its children. Built minimal/claim-free here; the retention claim lives on the StagingOperationGroup specialization.
- id#
- type#
- created_at#
- operations#
- property status: Status#
Derive the group status from the child operations.
Rule (in precedence order): - any child FAILED -> FAILED - all children COMPLETED -> COMPLETED - any child past PENDING -> IN_PROGRESS (work is in flight) - otherwise (empty, or all PENDING/SCHEDULED) -> PENDING
An empty group has nothing to derive from and reads as PENDING (not-yet-started), which is also the safe default before children are attached. SCHEDULED is folded into the pre-start bucket: a group with only SCHEDULED children has dispatched nothing yet, so it is not in flight; the moment any child reaches IN_PROGRESS (or beyond, while others lag) the group reads IN_PROGRESS.
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.StagingOperationGroup(**kwargs)[source]#
Bases:
OperationGroupStaging specialization of OperationGroup carrying the retention claim.
This is the uniform-model successor to the legacy
StagingJob(which is retained, untouched, during the additive transition).activeis the retention claim that holds the staged packages until released – the deletion manager treats an active staging group as a hold on its copies. The job’s status is still derived from its child staging operations on the OperationGroup base; only the claim is added here.- id#
- active#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- created_at#
- operations#
- type#
- class ccat_ops_db.models.Operation(**kwargs)[source]#
Bases:
BasePolymorphic base for every pipeline operation (joined-table inheritance).
Uniform fields shared by all stages live here; stage-specific fields live on the subclasses.
operation_kindis the polymorphic discriminator and holds anOperationKindvalue (== the frozen breadcrumb string).- id#
- operation_kind#
- status#
- retry_count#
- failure_error_message#
- error_context#
- start_time#
- end_time#
- operation_group_id#
- operation_group#
- consumed_copies#
- produced_copies#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.PackagingOperation(**kwargs)[source]#
Bases:
OperationPackaging stage: assembles raw data files into a RawDataPackage.
- id#
- raw_data_package_id#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- consumed_copies#
- end_time#
- error_context#
- failure_error_message#
- operation_group#
- operation_group_id#
- operation_kind#
- produced_copies#
- retry_count#
- start_time#
- status#
- class ccat_ops_db.models.BundlingOperation(**kwargs)[source]#
Bases:
OperationBundling stage: assembles packages into a DataTransferPackage.
- id#
- data_transfer_package_id#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- consumed_copies#
- end_time#
- error_context#
- failure_error_message#
- operation_group#
- operation_group_id#
- operation_kind#
- produced_copies#
- retry_count#
- start_time#
- status#
- class ccat_ops_db.models.TransferOperation(**kwargs)[source]#
Bases:
OperationTransfer stage: moves a DataTransferPackage between locations.
Sibling of UnpackOperation under the same DataTransferPackage (ADR-0003) – the transfer/unpack unicorn record splits into two independent rows.
- id#
- data_transfer_package_id#
- origin_location_id#
- destination_location_id#
- transfer_method#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- consumed_copies#
- end_time#
- error_context#
- failure_error_message#
- operation_group#
- operation_group_id#
- operation_kind#
- produced_copies#
- retry_count#
- start_time#
- status#
- class ccat_ops_db.models.UnpackOperation(**kwargs)[source]#
Bases:
OperationUnpack stage: verifies and unpacks a transferred DataTransferPackage.
Sibling of TransferOperation under the same DataTransferPackage (ADR-0003). Anchored per (data_transfer_package_id, destination_location_id): a package transferred to several destinations is unpacked/verified independently at each, so there is one UnpackOperation per destination, mirroring the sibling TransferOperation’s destination_location_id. This makes the transfer+unpack pair 1:1 per destination. destination_location_id is nullable for the additive transition (no backfill).
- id#
- data_transfer_package_id#
- destination_location_id#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- consumed_copies#
- end_time#
- error_context#
- failure_error_message#
- operation_group#
- operation_group_id#
- operation_kind#
- produced_copies#
- retry_count#
- start_time#
- status#
- class ccat_ops_db.models.ArchiveOperation(**kwargs)[source]#
Bases:
OperationArchive stage: copies a RawDataPackage to long-term storage.
- id#
- raw_data_package_id#
- origin_location_id#
- destination_location_id#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- consumed_copies#
- end_time#
- error_context#
- failure_error_message#
- operation_group#
- operation_group_id#
- operation_kind#
- produced_copies#
- retry_count#
- start_time#
- status#
- class ccat_ops_db.models.StagingOperation(**kwargs)[source]#
Bases:
OperationStaging stage: downloads + unpacks a package into a processing area.
Typically grouped under a StagingOperationGroup (the retention-claim parent).
- id#
- raw_data_package_id#
- origin_location_id#
- destination_location_id#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- consumed_copies#
- end_time#
- error_context#
- failure_error_message#
- operation_group#
- operation_group_id#
- operation_kind#
- produced_copies#
- retry_count#
- start_time#
- status#
- class ccat_ops_db.models.ReductionSoftware(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- ghcr_image_url#
- github_repo_url#
- created_at#
- active#
- versions#
- reduction_steps#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ReductionSoftwareVersion(**kwargs)[source]#
Bases:
Base- id#
- reduction_software_id#
- version_tag#
- image_digest#
- build_date#
- changelog#
- created_at#
- is_latest#
- reduction_software#
- executed_steps#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ReductionStepConfig(**kwargs)[source]#
Bases:
Base- id#
- name#
- version#
- description#
- config_parameters#
- environment_variables#
- command_template#
- resource_requirements#
- created_at#
- reduction_steps#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DataGrouping(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- instrument_module_id#
- filter_rules#
- created_at#
- active#
- instrument_module#
- pipelines#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.Pipeline(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- data_grouping_id#
- processing_location_id#
- trigger_type#
- trigger_config#
- enabled#
- priority#
- created_by_user_id#
- created_at#
- updated_at#
- data_grouping#
- processing_location#
- reduction_steps#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.ReductionStep(**kwargs)[source]#
Bases:
Base- id#
- name#
- description#
- pipeline_id#
- step_order#
- reduction_software_id#
- pinned_version_id#
- reduction_step_config_id#
- group_by#
- cooldown_seconds#
- schedule#
- max_concurrent_runs#
- version_policy#
- max_retries#
- created_at#
- pipeline#
- reduction_software#
- pinned_version#
- reduction_step_config#
- executed_steps#
- upstream_dependencies#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- downstream_dependencies#
- class ccat_ops_db.models.ExecutedReductionStep(**kwargs)[source]#
Bases:
Base- id#
- reduction_step_id#
- reduction_software_version_id#
- status#
- sub_group_key#
- sub_group_metadata#
- execution_command#
- hpc_job_id#
- hpc_queue#
- staging_job_id#
- start_time#
- end_time#
- processing_time_s#
- peak_memory_gb#
- cpu_hours#
- logs#
- failure_error_message#
- retry_count#
- created_at#
- trigger_reason#
- reduction_step#
- reduction_software_version#
- staging_job#
- input_packages#
- input_products#
- data_products#
- step_logs#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DataProduct(**kwargs)[source]#
Bases:
Base- id#
- name#
- relative_path#
- size#
- checksum#
- file_type#
- product_type#
- executed_reduction_step_id#
- data_grouping_id#
- quality_metrics#
- metadata_#
- created_at#
- executed_reduction_step#
- data_grouping#
- raw_data_lineage#
- physical_copies#
- consumed_by_steps#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.DataProductPhysicalCopy(**kwargs)[source]#
Bases:
PhysicalCopy- id#
- data_product_id#
- data_product#
- property full_path#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- checksum#
- consumed_by_operations#
- created_at#
- data_location#
- data_location_id#
- deleted_at#
- deletion_task_id#
- produced_by_operations#
- status#
- type#
- verified_at#
- class ccat_ops_db.models.ExecutedReductionStepLog(**kwargs)[source]#
Bases:
SystemLog- id#
- executed_reduction_step_id#
- executed_reduction_step#
- log#
- timestamp#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- type#
- class ccat_ops_db.models.SystemSettings(**kwargs)[source]#
Bases:
BaseKey-value store for operational configuration settings.
Used by data-transfer SettingsManager to read runtime-tunable parameters (poll intervals, buffer thresholds, transfer limits, etc.).
- id#
- key#
- value#
- description#
- updated_by#
- updated_at#
- created_at#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- class ccat_ops_db.models.OperationalConfigLog(**kwargs)[source]#
Bases:
SystemLogAudit log for operational configuration changes.
- id#
- setting_key#
- old_value#
- new_value#
- action#
- changed_by#
- changed_at#
- __init__(**kwargs)#
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in
kwargs.Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.
- type#