Queue Discovery#

Dynamic queue discovery system for data transfer operations.

class ccat_data_transfer.queue_discovery.QueueDiscoveryService(session: Session)[source]#

Bases: object

Service for discovering and managing Celery queues based on data locations.

__init__(session: Session)[source]#
get_queue_name_for_location(data_location: DataLocation, operation: str) → str[source]#

Generate standardized queue name from data location and operation.

discover_all_queues() → List[str][source]#

Discover all required queues from database.

get_queues_for_location(data_location: DataLocation) → List[str][source]#

Get all queues for a specific location.

get_primary_buffer_for_site(site: Site) → DataLocation | None[source]#

Get the primary (highest priority) active buffer for a site.

get_all_active_buffers_for_site(site: Site) → List[DataLocation][source]#

Get all active buffers for a site ordered by priority.

get_sites_with_location_type(location_type: LocationType) → List[Site][source]#

Get all sites that have a specific location type.

get_sites_with_buffers() → List[Site][source]#

Get all sites that have buffer locations.

ccat_data_transfer.queue_discovery.route_task_by_location(operation_type: OperationType, data_location: DataLocation) → str[source]#

Route task to appropriate queue based on operation and location.

ccat_data_transfer.queue_discovery.get_transfer_task(origin_location: DataLocation, dest_location: DataLocation) → str[source]#

Select appropriate transfer task based on storage types.

Overview#

Celery queue discovery and management for the data transfer system.

Key Components#

  • Queue discovery mechanisms

  • Queue management

  • Queue routing and configuration