Running Locally with Docker Compose#

The fastest way to run the complete ops-db-api stack locally is using Docker Compose, which sets up the API, PostgreSQL, and Redis with a single command.

Prerequisites#

  • Docker (version 20.10 or higher)

  • Docker Compose (version 2.0 or higher)

Verify you have Docker installed:

docker --version
docker compose version

Quick Start (Docker Compose)#

If a docker-compose.yml file exists in the repository:

# Clone the repository
git clone https://github.com/ccatobs/ops-db-api.git
cd ops-db-api

# Start all services
docker compose up -d

# View logs
docker compose logs -f

# Stop all services
docker compose down

This will start:

  • API server on port 8000

  • PostgreSQL on port 5432

  • Redis on port 6379

Access the API at http://localhost:8000/docs

Manual Docker Setup#

If Docker Compose configuration doesn’t exist, you can run services manually:

Step 1: Create a Docker Network#

docker network create ccat-network

Step 2: Start PostgreSQL#

docker run -d \
  --name ccat-postgres \
  --network ccat-network \
  -e POSTGRES_USER=ccat_ops_user \
  -e POSTGRES_PASSWORD=ccat_ops_password \
  -e POSTGRES_DB=ccat_ops_db \
  -p 5432:5432 \
  -v ccat-postgres-data:/var/lib/postgresql/data \
  postgres:14

Step 3: Start Redis#

docker run -d \
  --name ccat-redis \
  --network ccat-network \
  -p 6379:6379 \
  -v ccat-redis-data:/data \
  redis:7

Step 4: Build and Run the API#

# Build the Docker image
docker build -t ccat-ops-db-api .

# Run the API container
docker run -d \
  --name ccat-api \
  --network ccat-network \
  -p 8000:8000 \
  -e MAIN_DB_HOST=ccat-postgres \
  -e LOCAL_DB_HOST=ccat-postgres \
  -e REDIS_HOST=ccat-redis \
  -e SECRET_KEY=your-secret-key-change-in-production \
  ccat-ops-db-api

Connecting to Local vs Remote Database#

The API supports connecting to different databases for main (write) and local (read) operations:

Development Configuration (Single Local DB)#

For local development, use the same database for both:

# .env file
SITE_TYPE=main
MAIN_DB_HOST=localhost
MAIN_DB_PORT=5432
LOCAL_DB_HOST=localhost
LOCAL_DB_PORT=5432

This configuration:

  • Writes directly to the database (no buffering)

  • Reads from the same database

  • Simplest setup for development

Observatory Simulation (Buffered Writes)#

To test transaction buffering locally:

# .env file
SITE_TYPE=secondary
MAIN_DB_HOST=remote-main-db.example.com  # Simulated remote
MAIN_DB_PORT=5432
LOCAL_DB_HOST=localhost  # Local replica
LOCAL_DB_PORT=5432

This configuration:

  • Buffers write operations in Redis

  • Reads from local database

  • Merges buffered data with database reads

  • Simulates observatory environment

Testing Both Configurations#

You can switch between configurations by changing the SITE_TYPE environment variable:

# Test as main site (direct writes)
export SITE_TYPE=main
uvicorn ccat_ops_db_api.main:app --reload --port 8000

# Test as secondary site (buffered writes)
export SITE_TYPE=secondary
uvicorn ccat_ops_db_api.main:app --reload --port 8001

Setting Site Type for Testing#

The SITE_TYPE environment variable controls the API’s behavior:

SITE_TYPE=main#

Behavior:

  • All write operations execute directly on the database

  • No transaction buffering

  • No LSN tracking

  • Simplest configuration

Use when:

  • Developing new endpoints

  • Testing database queries

  • Running at the main site (Cologne)

SITE_TYPE=secondary#

Behavior:

  • Critical operations buffered in Redis

  • Background processor executes buffered transactions

  • LSN tracking monitors replication

  • Smart queries merge database + buffered data

Use when:

  • Testing transaction buffering

  • Simulating observatory environment

  • Testing replication lag scenarios

Example .env for Development#

# Development configuration (main site)
SITE_NAME=development
SITE_TYPE=main
MAIN_DB_HOST=localhost
LOCAL_DB_HOST=localhost
REDIS_HOST=localhost

Example .env for Observatory Simulation#

# Observatory simulation (secondary site)
SITE_NAME=observatory
SITE_TYPE=secondary
MAIN_DB_HOST=main-db-host  # Points to main DB
LOCAL_DB_HOST=localhost     # Local replica
REDIS_HOST=localhost
CRITICAL_OPERATIONS_BUFFER=true
LSN_TRACKING_ENABLED=true

Accessing Logs and Debugging#

Docker Compose Logs#

View logs from all services:

# All services
docker compose logs -f

# Specific service
docker compose logs -f api

# Last 100 lines
docker compose logs --tail=100 api

Individual Container Logs#

# API logs
docker logs -f ccat-api

# PostgreSQL logs
docker logs -f ccat-postgres

# Redis logs
docker logs -f ccat-redis

Accessing the Database#

Connect to PostgreSQL using psql:

# With Docker Compose
docker compose exec postgres psql -U ccat_ops_user -d ccat_ops_db

# With individual container
docker exec -it ccat-postgres psql -U ccat_ops_user -d ccat_ops_db

Accessing Redis#

Connect to Redis using redis-cli:

# With Docker Compose
docker compose exec redis redis-cli

# With individual container
docker exec -it ccat-redis redis-cli

Example commands:

# Check connection
PING

# List all keys
KEYS *

# View transaction buffer
LRANGE site:development:transaction_buffer 0 -1

# Check buffer size
LLEN site:development:transaction_buffer

See Redis Inspection for more Redis debugging commands.

API Logs in Development Mode#

When running with uvicorn directly (not Docker):

# Start with INFO level logging
uvicorn ccat_ops_db_api.main:app --reload --log-level info

# Debug level for detailed logs
uvicorn ccat_ops_db_api.main:app --reload --log-level debug

Logs will show:

  • Request/response for each API call

  • Database queries executed

  • Transaction buffering operations

  • Background processor activity

  • LSN tracking progress

Hot Reload for Development#

When running locally (not in Docker), uvicorn’s --reload flag enables hot reload:

uvicorn ccat_ops_db_api.main:app --reload --port 8000

This will automatically restart the server when you modify:

  • Python files in ccat_ops_db_api/

  • Router files

  • Schema definitions

  • Configuration files

Note

Hot reload only works when running uvicorn directly, not in Docker containers. For Docker development, mount your code as a volume.

Stopping and Cleaning Up#

Docker Compose#

# Stop services (keeps data)
docker compose stop

# Stop and remove containers (keeps volumes)
docker compose down

# Remove everything including volumes (⚠️ deletes data)
docker compose down -v

Individual Containers#

# Stop containers
docker stop ccat-api ccat-postgres ccat-redis

# Remove containers
docker rm ccat-api ccat-postgres ccat-redis

# Remove volumes (⚠️ deletes data)
docker volume rm ccat-postgres-data ccat-redis-data

Common Issues and Solutions#

Port Already in Use#

If port 8000, 5432, or 6379 is already in use:

# Find process using port
lsof -i :8000

# Kill the process
kill -9 <PID>

# Or use different ports
docker run -p 8001:8000 ...

Database Connection Refused#

If the API can’t connect to the database:

  1. Verify PostgreSQL container is running: docker ps

  2. Check network connectivity: docker network inspect ccat-network

  3. Verify environment variables match database credentials

  4. Check database logs: docker logs ccat-postgres

Redis Connection Errors#

If Redis connection fails:

  1. Verify Redis container is running: docker ps

  2. Test connection: docker exec ccat-redis redis-cli ping

  3. Check REDIS_HOST environment variable

  4. Verify Redis is accessible: telnet localhost 6379

Container Won’t Start#

If containers fail to start:

  1. Check logs: docker logs <container-name>

  2. Verify the image exists: docker images

  3. Check for port conflicts: docker ps -a

  4. Remove and recreate: docker rm <container> && docker run ...

Next Steps#

Now that you have the API running locally: