# Configuration

```{eval-rst}
.. verified:: 2025-11-25
   :reviewer: Christof Buchbender
```

The ops-db package uses [DynaConf](https://www.dynaconf.com/) for flexible, environment-based configuration management. This allows you to configure database connections, credentials, and other settings without modifying code.

## Overview

Configuration in ops-db works through three layers:

1. **Default settings** in `settings.toml` - Base configuration values
2. **Environment groups** - Override defaults for specific environments (development, production, etc.)
3. **Environment variables** - Override any setting at runtime via `CCAT_OPS_DB_*` prefixed variables

The active environment is selected using the `ENV_FOR_DYNACONF` environment variable.

## How DynaConf Works in ops-db

DynaConf is initialized in `ccat_ops_db/config/config.py` with the following settings:

- **envvar_prefix**: `"CCAT_OPS_DB"` - All environment variables must be prefixed with this
- **environments**: `True` - Enables environment-based configuration groups
- **load_dotenv**: `True` - Automatically loads `.env` files if present
- **default_env**: `"default"` - The default environment if none is specified

This means you can override any setting by setting an environment variable like:

```bash
export CCAT_OPS_DB_DATABASE_TYPE="postgresql"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="my-db-server"
```

## Selecting an Environment

The active environment determines which section from `settings.toml` is loaded. Set it using the `ENV_FOR_DYNACONF` environment variable:

```bash
# Use the development environment
export ENV_FOR_DYNACONF=development

# Use the production environment
export ENV_FOR_DYNACONF=production

# Use the default environment (SQLite for local development)
export ENV_FOR_DYNACONF=default
# or simply unset it:
unset ENV_FOR_DYNACONF
```

Available environments in `settings.toml`:

- `default` - SQLite database for local development
- `development` - PostgreSQL with Docker service names
- `development-ccat` - PostgreSQL with replica service names
- `local` - PostgreSQL for local development
- `staging-mekleth` - Staging environment on mekleth server
- `staging-batleth` - Staging environment on batleth server
- `docker-compose` - PostgreSQL for Docker Compose setups
- `production` - Production PostgreSQL server
- `production-ccat` - Production with replica services

## Environment Variables

You can override any setting from `settings.toml` using environment variables. The variable name is constructed by:

1. Prefixing with `CCAT_OPS_DB_`
2. Converting the setting name to uppercase
3. Replacing dots with underscores

Examples:

```bash
# Override database type
export CCAT_OPS_DB_DATABASE_TYPE="postgresql"

# Override PostgreSQL host
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="db.example.com"

# Override PostgreSQL port
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PORT="5433"

# Override PostgreSQL user
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="myuser"

# Override PostgreSQL password
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="mypassword"

# Override PostgreSQL database name
export CCAT_OPS_DB_DATABASE_POSTGRESQL_DATABASE="my_database"

# Override SQLite database path
export CCAT_OPS_DB_DATABASE_SQLITE_DATABASE="/path/to/database.sqlite"
```

For nested settings (like `status.PENDING`), use double underscores:

```bash
export CCAT_OPS_DB_STATUS__PENDING="waiting"
```

## Database Configuration

The database connection is controlled by the following settings:

### Database Type

Set `database_type` to one of: `sqlite`, `mysql`, or `postgresql`.

```bash
export CCAT_OPS_DB_DATABASE_TYPE="postgresql"
```

### PostgreSQL Settings

When using PostgreSQL, configure these settings:

- `database_postgresql_host` - Database server hostname or IP
- `database_postgresql_port` - Database server port (default: 5432)
- `database_postgresql_user` - Database username
- `database_postgresql_password` - Database password
- `database_postgresql_database` - Database name

Example:

```bash
export CCAT_OPS_DB_DATABASE_TYPE="postgresql"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="localhost"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PORT="5432"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="ccat"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="secret"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_DATABASE="ccat_ops_db"
```

### SQLite Settings

When using SQLite, configure:

- `database_sqlite_database` - Path to the SQLite database file

Example:

```bash
export CCAT_OPS_DB_DATABASE_TYPE="sqlite"
export CCAT_OPS_DB_DATABASE_SQLITE_DATABASE="/tmp/my_database.sqlite"
```

### MySQL Settings

When using MySQL, configure:

- `database_mysql_host` - Database server hostname or IP
- `database_mysql_port` - Database server port (default: 3306)
- `database_mysql_user` - Database username
- `database_mysql_password` - Database password
- `database_mysql_database` - Database name

Example:

```bash
export CCAT_OPS_DB_DATABASE_TYPE="mysql"
export CCAT_OPS_DB_DATABASE_MYSQL_HOST="localhost"
export CCAT_OPS_DB_DATABASE_MYSQL_PORT="3306"
export CCAT_OPS_DB_DATABASE_MYSQL_USER="ccat"
export CCAT_OPS_DB_DATABASE_MYSQL_PASSWORD="secret"
export CCAT_OPS_DB_DATABASE_MYSQL_DATABASE="ccat_ops_db"
```

## Practical Examples

### Local Development with SQLite

For quick local development, use the default SQLite configuration:

```bash
# No configuration needed - uses defaults from settings.toml
python -c "from ccat_ops_db import init_ccat_ops_db; session, engine = init_ccat_ops_db()"
```

Or explicitly set the environment:

```bash
export ENV_FOR_DYNACONF=default
python your_script.py
```

### Local Development with PostgreSQL

To use PostgreSQL locally:

```bash
export ENV_FOR_DYNACONF=local
# Or override specific settings:
export CCAT_OPS_DB_DATABASE_TYPE="postgresql"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="localhost"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="ccat"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="mypassword"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_DATABASE="ccat_ops_db"

python your_script.py
```

### Connecting to a Remote Database

Override the host and credentials for a remote database:

```bash
export ENV_FOR_DYNACONF=production
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="db.example.com"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="remote_user"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="remote_password"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PORT="5432"

python your_script.py
```

### Using a .env File

Create a `.env` file in your project root (DynaConf will automatically load it):

```bash
# .env file
ENV_FOR_DYNACONF=development
CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST=localhost
CCAT_OPS_DB_DATABASE_POSTGRESQL_USER=ccat
CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD=secret
CCAT_OPS_DB_DATABASE_POSTGRESQL_DATABASE=ccat_ops_db
```

Then run your script:

```bash
python your_script.py
```

### Switching Environments in a Script

You can also override settings programmatically in Python:

```python
import os
from ccat_ops_db import init_ccat_ops_db

# Set environment before importing settings
os.environ['ENV_FOR_DYNACONF'] = 'development'
os.environ['CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST'] = 'custom-host'

# Now initialize - will use the overridden settings
session, engine = init_ccat_ops_db()
```

### Docker Compose Environment

For Docker Compose setups:

```bash
export ENV_FOR_DYNACONF=docker-compose
python your_script.py
```

This uses the service names from your `docker-compose.yml` (e.g., `postgres`, `redis`).

## How Settings Are Resolved

DynaConf resolves settings in this order (highest priority first):

1. **Environment variables** (`CCAT_OPS_DB_*`) - Highest priority
2. **Environment group** from `settings.toml` (e.g., `[development]`)
3. **Default values** from `[default]` section in `settings.toml`

This means:

- Environment variables always override file-based settings
- Environment groups override default settings
- Default settings are used as fallback

Example resolution:

```bash
# settings.toml has:
# [default]
# database_postgresql_host = "localhost"
#
# [development]
# database_postgresql_host = "postgres"

# If ENV_FOR_DYNACONF=development and no env vars:
# → Uses "postgres" from [development]

# If you set:
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="custom-host"
# → Uses "custom-host" regardless of environment
```

## Checking Current Configuration

To see what configuration is active, you can inspect the settings object:

```python
from ccat_ops_db.config.config import ccat_ops_db_settings

print(f"Database type: {ccat_ops_db_settings.DATABASE_TYPE}")
print(f"PostgreSQL host: {ccat_ops_db_settings.DATABASE_POSTGRESQL_HOST}")
print(f"PostgreSQL user: {ccat_ops_db_settings.DATABASE_POSTGRESQL_USER}")
print(f"PostgreSQL database: {ccat_ops_db_settings.DATABASE_POSTGRESQL_DATABASE}")

# Check current environment
print(f"Current environment: {ccat_ops_db_settings.current_env}")
```

## Common Configuration Patterns

### Development Workflow

For development, create a `.env` file with your local settings:

```bash
ENV_FOR_DYNACONF=local
CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST=localhost
CCAT_OPS_DB_DATABASE_POSTGRESQL_USER=dev_user
CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD=dev_password
```

### Production Deployment

For production, use environment variables set by your deployment system:

```bash
export ENV_FOR_DYNACONF=production
export CCAT_OPS_DB_DATABASE_POSTGRESQL_HOST="${DB_HOST}"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_USER="${DB_USER}"
export CCAT_OPS_DB_DATABASE_POSTGRESQL_PASSWORD="${DB_PASSWORD}"
```

### Testing Different Configurations

Switch between configurations easily:

```bash
# Test with SQLite
ENV_FOR_DYNACONF=default python test_script.py

# Test with local PostgreSQL
ENV_FOR_DYNACONF=local python test_script.py

# Test with staging
ENV_FOR_DYNACONF=staging-mekleth python test_script.py
```

## Troubleshooting

### Configuration Not Taking Effect

If your configuration changes aren't being applied:

1. **Check environment variable names**: They must be prefixed with `CCAT_OPS_DB_` and use uppercase
2. **Verify ENV_FOR_DYNACONF**: Make sure it's set to the correct environment name
3. **Check for typos**: Setting names are case-sensitive
4. **Restart your Python process**: Settings are loaded at import time

Example debug session:

```bash
# Check what environment is active
python -c "from ccat_ops_db.config.config import ccat_ops_db_settings; print(ccat_ops_db_settings.current_env)"

# Check a specific setting
python -c "from ccat_ops_db.config.config import ccat_ops_db_settings; print(ccat_ops_db_settings.DATABASE_POSTGRESQL_HOST)"
```

### Connection Errors

If you're getting database connection errors:

1. **Verify credentials**: Check username, password, and database name
2. **Check network access**: Ensure the host is reachable
3. **Verify port**: Default PostgreSQL port is 5432
4. **Check database exists**: The database must exist before connecting

Example connection test:

```python
from ccat_ops_db import init_ccat_ops_db
from ccat_ops_db.config.config import ccat_ops_db_settings

# Print configuration
print(f"Connecting to: {ccat_ops_db_settings.DATABASE_POSTGRESQL_HOST}:{ccat_ops_db_settings.DATABASE_POSTGRESQL_PORT}")
print(f"Database: {ccat_ops_db_settings.DATABASE_POSTGRESQL_DATABASE}")
print(f"User: {ccat_ops_db_settings.DATABASE_POSTGRESQL_USER}")

# Try to connect
try:
    session, engine = init_ccat_ops_db()
    print("Connection successful!")
except Exception as e:
    print(f"Connection failed: {e}")
```

## Related Documentation

- {doc}`../api_reference/core` - Database initialization functions
- {doc}`../concepts/overview` - Database concepts and design
- [DynaConf Documentation](https://www.dynaconf.com/) - Complete DynaConf reference