Installation#

This guide walks you through installing the CCAT Operations Database API on your local machine for development and testing.

System Requirements#

  • Python: 3.9 or higher

  • Operating System: Linux, macOS, or Windows with WSL

  • Memory: At least 4GB RAM available

  • Disk Space: At least 2GB free space

Required Services#

The API requires two external services:

  1. PostgreSQL (version 12 or higher)

    • Used for persistent data storage

    • Can be run locally or via Docker

  2. Redis (version 6 or higher)

    • Used for transaction buffering and caching

    • Can be run locally or via Docker

Step 1: Clone the Repository#

First, clone the repository from GitHub:

git clone https://github.com/ccatobs/ops-db-api.git
cd ops-db-api

Step 2: Create a Virtual Environment#

Create and activate a Python virtual environment:

# Create virtual environment
python3 -m venv venv

# Activate on Linux/macOS
source venv/bin/activate

# Activate on Windows (WSL)
source venv/bin/activate

Step 3: Install Dependencies#

Install the API package and its dependencies:

# Install in development mode
pip install -e .

# Or install from requirements.txt
pip install -r requirements.txt

This will install:

  • FastAPI and Uvicorn (web framework and server)

  • SQLAlchemy (database ORM)

  • Redis client (for buffering and caching)

  • PostgreSQL driver (asyncpg)

  • Pydantic (data validation)

  • Jose (JWT tokens)

  • And other dependencies

Step 4: Set Up Environment Variables#

Copy the example environment file:

cp env.example.txt .env

Edit the .env file with your configuration:

# Site Configuration
SITE_NAME=development
SITE_TYPE=main  # Use "main" for development, "secondary" for observatory simulation

# Main Database (for writes)
MAIN_DB_HOST=localhost
MAIN_DB_PORT=5432
MAIN_DB_USER=ccat_ops_user
MAIN_DB_PASSWORD=your_secure_password
MAIN_DB_NAME=ccat_ops_db

# Local Database (for reads, same as main in development)
LOCAL_DB_HOST=localhost
LOCAL_DB_PORT=5432
LOCAL_DB_USER=ccat_ops_user
LOCAL_DB_PASSWORD=your_secure_password
LOCAL_DB_NAME=ccat_ops_db

# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0

# Authentication
SECRET_KEY=your-secret-key-here-change-in-production
GITHUB_CLIENT_ID=your-github-oauth-client-id
GITHUB_CLIENT_SECRET=your-github-oauth-client-secret

# Transaction Buffering
TRANSACTION_BUFFER_SIZE=1000
TRANSACTION_RETRY_ATTEMPTS=3
TRANSACTION_RETRY_DELAY=5
BACKGROUND_PROCESSING_INTERVAL=1.0

# LSN Tracking
LSN_TRACKING_ENABLED=true
LSN_CHECK_INTERVAL=0.1
LSN_TIMEOUT=30

Note

For local development, you can use SITE_TYPE=main which disables transaction buffering and writes directly to the database.

Step 5: Set Up PostgreSQL#

Option A: Using Docker#

Run PostgreSQL in a Docker container:

docker run -d \
  --name ccat-postgres \
  -e POSTGRES_USER=ccat_ops_user \
  -e POSTGRES_PASSWORD=your_secure_password \
  -e POSTGRES_DB=ccat_ops_db \
  -p 5432:5432 \
  postgres:14

Option B: Using Local PostgreSQL#

If you have PostgreSQL installed locally:

# Create database and user
sudo -u postgres psql -c "CREATE USER ccat_ops_user WITH PASSWORD 'your_secure_password';"
sudo -u postgres psql -c "CREATE DATABASE ccat_ops_db OWNER ccat_ops_user;"

Initialize the Database Schema#

The database schema is managed by the ccat_ops_db package. Install and initialize it:

# Install the ops-db package
pip install git+https://github.com/ccatobs/ops-db.git

# Run migrations (if available) or initialize schema
# Check the ops-db repository for specific instructions

Step 6: Set Up Redis#

Option A: Using Docker#

Run Redis in a Docker container:

docker run -d \
  --name ccat-redis \
  -p 6379:6379 \
  redis:7

Option B: Using Local Redis#

If you have Redis installed locally:

# On Ubuntu/Debian
sudo apt-get install redis-server
sudo systemctl start redis

# On macOS with Homebrew
brew install redis
brew services start redis

Verify Redis is running:

redis-cli ping
# Should return: PONG

Step 7: Verify Installation#

Check that everything is installed correctly:

# Verify Python package
python -c "import ccat_ops_db_api; print('API package installed')"

# Check database connectivity
python -c "from ccat_ops_db import get_database_url; print('Database package available')"

# Test Redis connection
redis-cli ping

Step 8: Start the API Server#

Start the development server:

uvicorn ccat_ops_db_api.main:app --reload --port 8000

You should see output like:

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [12345] using StatReload
INFO:     Started server process [12346]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

Visit http://localhost:8000/docs in your browser to see the interactive API documentation.

Alternative: Docker Compose#

For the simplest setup, see Running Locally with Docker Compose for instructions on using Docker Compose to run the entire stack.

Troubleshooting#

Database Connection Errors#

If you see database connection errors:

  1. Verify PostgreSQL is running: pg_isready -h localhost

  2. Check credentials in .env match your database setup

  3. Ensure the database exists: psql -h localhost -U ccat_ops_user -l

Redis Connection Errors#

If you see Redis connection errors:

  1. Verify Redis is running: redis-cli ping

  2. Check REDIS_HOST and REDIS_PORT in .env

  3. Try connecting manually: redis-cli -h localhost -p 6379

Import Errors#

If you see import errors:

  1. Ensure virtual environment is activated

  2. Reinstall dependencies: pip install -e .

  3. Check Python version: python --version (should be 3.9+)

Next Steps#

Now that you have the API installed, proceed to: