Managers

The three manager modules that drive the pipeline orchestration loop.

trigger_manager

Trigger Manager - evaluates pipeline triggers and creates ExecutedReductionStep records.

Polls enabled pipelines, resolves data groupings via FilterEngine, evaluates per-step trigger conditions with gap evaluation and cooldown, and creates PENDING runs.

ccat_workflow_manager.managers.trigger_manager.process_triggers()

Main trigger evaluation loop - called by the CLI poll loop.

workflow_manager

Workflow Manager - processes PENDING ExecutedReductionSteps through staging and HPC submission.

Picks up PENDING runs, stages input data, builds execution commands, writes manifests, and submits to HPC backend.

ccat_workflow_manager.managers.workflow_manager.process_pending_runs()

Main workflow processing loop - called by the CLI poll loop.

result_manager

Result Manager - monitors HPC jobs and collects outputs into DataProducts.

Polls ExecutedReductionSteps in SUBMITTED/RUNNING state, checks HPC job status, discovers outputs by convention (directory structure), and creates DataProduct records.

ccat_workflow_manager.managers.result_manager.process_running_steps()

Main result collection loop - called by the CLI poll loop.