Configuration#

✓
Documentation Verified Last checked: 2025-11-25 Reviewer: Christof Buchbender

The ops-db package uses DynaConf for flexible, environment-based configuration management. This allows you to configure database connections, credentials, and other settings without modifying code.

Overview#

Configuration in ops-db works through three layers:

  1. Default settings in settings.toml - Base configuration values

  2. Environment groups - Override defaults for specific environments (development, production, etc.)

  3. Environment variables - Override any setting at runtime via CCAT_OPS_DB_* prefixed variables

The active environment is selected using the ENV_FOR_DYNACONF environment variable.

How DynaConf Works in ops-db#

DynaConf is initialized in ccat_ops_db/config/config.py with the following settings:

  • envvar_prefix: "CCAT_OPS_DB" - All environment variables must be prefixed with this

  • environments: True - Enables environment-based configuration groups

  • load_dotenv: True - Automatically loads .env files if present

  • default_env: "default" - The default environment if none is specified

This means you can override any setting by setting an environment variable like:

export CCAT_OPS_DB_DATABASE_TYPE="postgresql"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="my-db-server"

Selecting an Environment#

The active environment determines which section from settings.toml is loaded. Set it using the ENV_FOR_DYNACONF environment variable:

# Use the development environment
export ENV_FOR_DYNACONF=development

# Use the production environment
export ENV_FOR_DYNACONF=production

# Use the default environment (SQLite for local development)
export ENV_FOR_DYNACONF=default
# or simply unset it:
unset ENV_FOR_DYNACONF

Available environments in settings.toml:

  • default - SQLite database for local development

  • development - PostgreSQL with Docker service names

  • development-ccat - PostgreSQL with replica service names

  • local - PostgreSQL for local development

  • staging-mekleth - Staging environment on mekleth server

  • staging-batleth - Staging environment on batleth server

  • docker-compose - PostgreSQL for Docker Compose setups

  • production - Production PostgreSQL server

  • production-ccat - Production with replica services

Environment Variables#

You can override any setting from settings.toml using environment variables. The variable name is constructed by:

  1. Prefixing with CCAT_OPS_DB_

  2. Converting the setting name to uppercase

  3. Replacing dots with underscores

Examples:

# Override database type
export CCAT_OPS_DB_DATABASE_TYPE="postgresql"

# Override PostgreSQL host
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="db.example.com"

# Override PostgreSQL port
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PORT="5433"

# Override PostgreSQL user
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="myuser"

# Override PostgreSQL password
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="mypassword"

# Override PostgreSQL database name
export CCAT_OPS_DB_DATABASE_POSTGRESQL_DATABASE="my_database"

# Override SQLite database path
export CCAT_OPS_DB_DATABASE_SQLITE_DATABASE="/path/to/database.sqlite"

For nested settings (like status.PENDING), use double underscores:

export CCAT_OPS_DB_STATUS__PENDING="waiting"

Database Configuration#

The database connection is controlled by the following settings:

Database Type#

Set database_type to one of: sqlite, mysql, or postgresql.

export CCAT_OPS_DB_DATABASE_TYPE="postgresql"

PostgreSQL Settings#

When using PostgreSQL, configure these settings:

  • database_postgresql_host - Database server hostname or IP

  • database_postgresql_port - Database server port (default: 5432)

  • database_postgresql_user - Database username

  • database_postgresql_password - Database password

  • database_postgresql_database - Database name

Example:

export CCAT_OPS_DB_DATABASE_TYPE="postgresql"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="localhost"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PORT="5432"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="ccat"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="secret"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_DATABASE="ccat_ops_db"

SQLite Settings#

When using SQLite, configure:

  • database_sqlite_database - Path to the SQLite database file

Example:

export CCAT_OPS_DB_DATABASE_TYPE="sqlite"
export CCAT_OPS_DB_DATABASE_SQLITE_DATABASE="/tmp/my_database.sqlite"

MySQL Settings#

When using MySQL, configure:

  • database_mysql_host - Database server hostname or IP

  • database_mysql_port - Database server port (default: 3306)

  • database_mysql_user - Database username

  • database_mysql_password - Database password

  • database_mysql_database - Database name

Example:

export CCAT_OPS_DB_DATABASE_TYPE="mysql"
export CCAT_OPS_DB_DATABASE_MYSQL_HOST="localhost"
export CCAT_OPS_DB_DATABASE_MYSQL_PORT="3306"
export CCAT_OPS_DB_DATABASE_MYSQL_USER="ccat"
export CCAT_OPS_DB_DATABASE_MYSQL_PASSWORD="secret"
export CCAT_OPS_DB_DATABASE_MYSQL_DATABASE="ccat_ops_db"

Practical Examples#

Local Development with SQLite#

For quick local development, use the default SQLite configuration:

# No configuration needed - uses defaults from settings.toml
python -c "from ccat_ops_db import init_ccat_ops_db; session, engine = init_ccat_ops_db()"

Or explicitly set the environment:

export ENV_FOR_DYNACONF=default
python your_script.py

Local Development with PostgreSQL#

To use PostgreSQL locally:

export ENV_FOR_DYNACONF=local
# Or override specific settings:
export CCAT_OPS_DB_DATABASE_TYPE="postgresql"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="localhost"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="ccat"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="mypassword"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_DATABASE="ccat_ops_db"

python your_script.py

Connecting to a Remote Database#

Override the host and credentials for a remote database:

export ENV_FOR_DYNACONF=production
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="db.example.com"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="remote_user"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="remote_password"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PORT="5432"

python your_script.py

Using a .env File#

Create a .env file in your project root (DynaConf will automatically load it):

# .env file
ENV_FOR_DYNACONF=development
CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST=localhost
CCAT_OPS_DB_DATABASE_POSTGRESQL_USER=ccat
CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD=secret
CCAT_OPS_DB_DATABASE_POSTGRESQL_DATABASE=ccat_ops_db

Then run your script:

python your_script.py

Switching Environments in a Script#

You can also override settings programmatically in Python:

import os
from ccat_ops_db import init_ccat_ops_db

# Set environment before importing settings
os.environ['ENV_FOR_DYNACONF'] = 'development'
os.environ['CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST'] = 'custom-host'

# Now initialize - will use the overridden settings
session, engine = init_ccat_ops_db()

Docker Compose Environment#

For Docker Compose setups:

export ENV_FOR_DYNACONF=docker-compose
python your_script.py

This uses the service names from your docker-compose.yml (e.g., postgres, redis).

How Settings Are Resolved#

DynaConf resolves settings in this order (highest priority first):

  1. Environment variables (CCAT_OPS_DB_*) - Highest priority

  2. Environment group from settings.toml (e.g., [development])

  3. Default values from [default] section in settings.toml

This means:

  • Environment variables always override file-based settings

  • Environment groups override default settings

  • Default settings are used as fallback

Example resolution:

# settings.toml has:
# [default]
# database_postgresql_host = "localhost"
#
# [development]
# database_postgresql_host = "postgres"

# If ENV_FOR_DYNACONF=development and no env vars:
# → Uses "postgres" from [development]

# If you set:
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="custom-host"
# → Uses "custom-host" regardless of environment

Checking Current Configuration#

To see what configuration is active, you can inspect the settings object:

from ccat_ops_db.config.config import ccat_ops_db_settings

print(f"Database type: {ccat_ops_db_settings.DATABASE_TYPE}")
print(f"PostgreSQL host: {ccat_ops_db_settings.DATABASE_POSTGRESQL_HOST}")
print(f"PostgreSQL user: {ccat_ops_db_settings.DATABASE_POSTGRESQL_USER}")
print(f"PostgreSQL database: {ccat_ops_db_settings.DATABASE_POSTGRESQL_DATABASE}")

# Check current environment
print(f"Current environment: {ccat_ops_db_settings.current_env}")

Common Configuration Patterns#

Development Workflow#

For development, create a .env file with your local settings:

ENV_FOR_DYNACONF=local
CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST=localhost
CCAT_OPS_DB_DATABASE_POSTGRESQL_USER=dev_user
CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD=dev_password

Production Deployment#

For production, use environment variables set by your deployment system:

export ENV_FOR_DYNACONF=production
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="${DB_HOST}"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="${DB_USER}"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="${DB_PASSWORD}"

Testing Different Configurations#

Switch between configurations easily:

# Test with SQLite
ENV_FOR_DYNACONF=default python test_script.py

# Test with local PostgreSQL
ENV_FOR_DYNACONF=local python test_script.py

# Test with staging
ENV_FOR_DYNACONF=staging-mekleth python test_script.py

Troubleshooting#

Configuration Not Taking Effect#

If your configuration changes aren’t being applied:

  1. Check environment variable names: They must be prefixed with CCAT_OPS_DB_ and use uppercase

  2. Verify ENV_FOR_DYNACONF: Make sure it’s set to the correct environment name

  3. Check for typos: Setting names are case-sensitive

  4. Restart your Python process: Settings are loaded at import time

Example debug session:

# Check what environment is active
python -c "from ccat_ops_db.config.config import ccat_ops_db_settings; print(ccat_ops_db_settings.current_env)"

# Check a specific setting
python -c "from ccat_ops_db.config.config import ccat_ops_db_settings; print(ccat_ops_db_settings.DATABASE_POSTGRESQL_HOST)"

Connection Errors#

If you’re getting database connection errors:

  1. Verify credentials: Check username, password, and database name

  2. Check network access: Ensure the host is reachable

  3. Verify port: Default PostgreSQL port is 5432

  4. Check database exists: The database must exist before connecting

Example connection test:

from ccat_ops_db import init_ccat_ops_db
from ccat_ops_db.config.config import ccat_ops_db_settings

# Print configuration
print(f"Connecting to: {ccat_ops_db_settings.DATABASE_POSTGRESQL_HOST}:{ccat_ops_db_settings.DATABASE_POSTGRESQL_PORT}")
print(f"Database: {ccat_ops_db_settings.DATABASE_POSTGRESQL_DATABASE}")
print(f"User: {ccat_ops_db_settings.DATABASE_POSTGRESQL_USER}")

# Try to connect
try:
    session, engine = init_ccat_ops_db()
    print("Connection successful!")
except Exception as e:
    print(f"Connection failed: {e}")