# CCAT System Integration Documentation

## Overview

This documentation covers the CCAT System Integration, including:

-   Docker Compose configuration and deployment
-   Secrets management via Ansible Vault and .env provisioning
-   Configuration management with Dynaconf
-   GitHub PAT (Personal Access Token) rotation procedures
-   Backup and restore procedures
-   Monitoring with Grafana, Prometheus, and Loki
-   Service orchestration across multiple nodes

## Key Components

**Infrastructure Services:**

-   PostgreSQL database with replication
-   Redis cache with TLS
-   InfluxDB for time-series data
-   MinIO object storage
-   Grafana dashboards
-   Prometheus monitoring
-   Loki log aggregation

**Deployment Configurations:**

-   Local development (docker-compose.local.yml)
-   Staging environment (docker-compose.staging.yml)
-   Production nodes:
    -   Database node (docker-compose.production.db.yml)
    -   Data processing node (docker-compose.production.data.yml)
    -   Input node (docker-compose.production.input.yml)
    -   Reuna VM (docker-compose.production.reuna-vm.yml)

## Quick Links

-   `Operations Database </ops-db/docs/index>`{.interpreted-text
    role="doc"}
-   `Data Transfer Documentation </data-transfer/docs/index>`{.interpreted-text
    role="doc"}

## Documentation Contents

::: {.toctree maxdepth="2" caption="Contents:"}
source/getting-started source/configuration source/deployment
source/secrets-management source/monitoring source/backup-restore
source/pat-rotation source/matrix source/proposal-update-workflow-full
source/proposal-update-workflow-small-team source/background/index
:::

## Getting Started

For setup instructions, see the project README and the Getting Started
guide.

## Related Documentation

-   [Main
    Documentation](https://ccatobs.github.io/data-center-documentation/)
-   [Operations Database](https://ccatobs.github.io/ops-db/)
-   [Data Transfer System](https://ccatobs.github.io/data-transfer/)

# Indices and tables

-   `genindex`{.interpreted-text role="ref"}

\* `search`{.interpreted-text role="ref"} ======= System Integration &
Deployment ================================

The system-integration repository contains Docker Compose
configurations, Kubernetes manifests, and supporting infrastructure for
deploying the complete CCAT data center stack across different
environments.

## Deployment Environments

**Local Development**

:   -   `docker-compose.localdev.yml` - Infrastructure containers
        (postgres, redis, grafana, etc.)
    -   `process-compose.localdev.yml` - Application processes with
        hot-reload via process-compose
    -   `docker-compose.dev.yml` - Alternative: full stack in Docker
        with volume-mounted code

**Staging**

:   -   `docker-compose.staging.yml` - Staging environment
    -   `docker-compose.staging.batleth.yml` - Batleth staging server
    -   `cologne-staging-k8s/` - Kubernetes configuration for Cologne
        staging

**Production**

:   -   `docker-compose.production.*.yml` - Production deployment
        configurations
    -   Separated into database, data transfer, and input components

## Infrastructure Components

**Core Services**

-   **PostgreSQL**: Operations database with replication support
-   **Redis**: Caching and session management
-   **MinIO**: S3-compatible object storage
-   **InfluxDB**: Time-series metrics storage

**Monitoring & Observability**

-   **Prometheus**: Metrics collection and alerting
-   **Grafana**: Dashboards and visualization
-   **Loki**: Log aggregation
-   **Promtail**: Log collection agent

**Supporting Services**

-   **Nginx**: Reverse proxy and load balancing
-   **Celery**: Distributed task queue for background jobs

## Directory Structure

`postgres/`

:   PostgreSQL initialization scripts and configuration

`grafana/`

:   Grafana dashboards and provisioning configuration

`prometheus/`

:   Prometheus monitoring configuration

`loki-config/` & `promtail-config/`

:   Log aggregation configuration

`jenkins_pipelines/`

:   CI/CD pipeline definitions

`tools/`

:   Deployment and management scripts

## Deployment Guides

### Kubernetes Deployments

The `cologne-staging-k8s/` directory contains Kubernetes manifests for
deploying on the Cologne cluster:

-   Deployment configurations
-   Service definitions
-   Ingress routing
-   Persistent volume claims

See `cologne-staging-k8s/README.md` for detailed deployment
instructions.

### Docker Compose Deployments

Docker Compose files are organized by environment and component:

1.  **Choose your environment**: local, staging, or production
2.  **Select components**: database, data transfer, input processing
3.  **Run**: `docker-compose -f <config-file> up -d`

## Initialization Scripts

`init_ops_db_entrypoint.sh`

:   Initializes the operations database schema and loads seed data

`init-influxdb.sh`

:   Sets up InfluxDB databases and retention policies

`init-minio.sh`

:   Configures MinIO buckets and access policies

## Monitoring

The monitoring stack provides comprehensive observability:

-   **System Metrics**: CPU, memory, disk, network via Prometheus
-   **Application Metrics**: Custom metrics from Python services
-   **Logs**: Centralized logging with Loki
-   **Dashboards**: Pre-configured Grafana dashboards for all services

Access Grafana at the configured endpoint to view dashboards and alerts.