API Reference

Complete API documentation for the CCAT Data Transfer System. This section provides detailed documentation for developers working with the system.

API Documentation:

• Managers
  • Raw Data Package Manager
  • Data Transfer Package Manager
  • Transfer Manager
  • Archive Manager
  • Deletion Manager
  • Staging Manager
  • Overview
  • Manager/Worker Pattern
• Services
  • Data Integrity Manager
  • Buffer Manager
  • Disk Monitor
  • Health Check
  • Task Monitor Service
  • Task State Manager
  • Recovery Service Runner
  • Overview
  • Service Integration
• Core Utilities
  • Database
  • Configuration
  • Exceptions
  • Utilities
  • Decorators
  • Logging Utils
  • Metrics
  • Notification Service
  • Operation Types
  • Queue Discovery
  • Settings Manager
  • Overview
  • Core Architecture
• Celery Tasks
  • Setup Celery App
  • Overview
  • Distributed Processing
  • Task Integration

Overview

The API is organized into functional areas:

Managers (Managers)

Core business logic classes that orchestrate data transfer operations. These follow the Manager/Worker pattern described in Philosophy & Design Principles.

Services (Services)

Supporting services and utilities that provide monitoring, health checks, and operational support.

Core (Core Utilities)

Fundamental components including database connections, configuration, exceptions, and utility functions.

Tasks (Celery Tasks)

Celery task definitions and distributed processing components.

Architecture Notes

All managers follow the same pattern:

1. Manager Process: Scans database, prepares operations, submits tasks

2. Worker Process: Executes actual work, updates database with results

See Philosophy & Design Principles for detailed explanation of the Manager/Worker duality.

For higher-level operational guidance, see the main documentation sections:

• Core Concepts - Core concepts and data model

• Pipeline Architecture - Data processing pipeline

• Monitoring & Failure Recovery - System monitoring and observability

• Data Lifecycle Management - Data lifecycle management