# Deep Dive

This section provides detailed technical documentation on the core systems and implementations within ops-db-api.

```{contents} Table of Contents
:depth: 2
:local: true
```

## Overview

The deep dive documentation covers:

- **Transaction Buffering**: Complete buffering system architecture
- **Authentication**: Unified auth implementation details
- **Routers**: Endpoint organization and implementation
- **Data Flow**: Complete request/response patterns
- **Caching Strategy**: Redis caching patterns and policies

These guides are intended for developers who need to:

- Understand implementation details
- Extend the system with new features
- Debug complex issues
- Optimize performance

## Deep Dive Sections

```{toctree}
:maxdepth: 2

transaction-buffering/index
authentication/index
routers/index
data-flow
caching-strategy
```

### Transaction Buffering

The transaction buffering system is the core innovation that enables reliable operations despite network issues.

Topics covered:

- System lifecycle and architecture
- Transaction builder and dependencies
- Redis buffering and queue management
- Background processing patterns
- LSN-based replication tracking
- Smart query merging
- Read buffer for mutable updates

See {doc}`transaction-buffering/index` for details.

### Authentication

Unified authentication supporting both interactive users and automation.

Topics covered:

- Unified authentication interface
- GitHub OAuth flow implementation
- API token generation and management
- Role-based access control
- Permission system

See {doc}`authentication/index` for details.

### Routers

Endpoint organization, categorization, and implementation patterns.

Topics covered:

- UI-focused routers
- Operations-focused routers
- Shared routers
- Request/response patterns

See {doc}`routers/index` for details.

### Data Flow

Complete data flow examples from request to response.

See {doc}`data-flow` for examples.

### Caching Strategy

Redis caching patterns, TTL strategies, and invalidation.

See {doc}`caching-strategy` for details.

## Prerequisites

Before diving into these topics, you should be familiar with:

- FastAPI framework basics
- SQLAlchemy ORM
- Redis data structures
- PostgreSQL replication concepts
- Async Python (asyncio)

Recommended reading order:

1. **Start with philosophy**: {doc}`../philosophy/index` - Understand the "why"
2. **Then architecture**: {doc}`../architecture/index` - Understand the "what"
3. **Then deep dives**: This section - Understand the "how"
4. **Finally tutorials**: {doc}`../tutorials/index` - Apply the knowledge

## Code References

All deep dive documentation includes extensive code references using `literalinclude` directives that point to actual implementation files.

**Key source directories**:

- `ccat_ops_db_api/transaction_buffering/` - Buffering system
- `ccat_ops_db_api/auth/` - Authentication
- `ccat_ops_db_api/routers/` - API endpoints
- `ccat_ops_db_api/dependencies.py` - Dependency injection
- `ccat_ops_db_api/main.py` - Application setup

## Conventions

### Code Examples

Code examples are shown in two ways:

1. **Inline examples** - Simplified, pedagogical code
2. **literalinclude** - Actual production code with line references

```python
# Inline example (simplified)
def simple_example():
    return "hello"
```

```{literalinclude} ../../ccat_ops_db_api/main.py
:language: python
:lines: 1-10
```

### Diagrams

**Mermaid diagrams** for:

- Sequence diagrams (request flows)
- Architecture diagrams (system components)
- State machines (transaction lifecycle)

**ASCII diagrams** for:

- Simple flows
- Tree structures

### Cross-References

Extensive cross-references connect related topics:

- `:doc:` - Links to other documentation pages
- `:ref:` - Links to specific sections
- `:meth:` - Links to methods (when using Sphinx autodoc)

## Testing Deep Dive Concepts

Each deep dive topic includes suggestions for hands-on testing:

**Transaction Buffering**:

```bash
# Set as secondary site
export SITE_TYPE=secondary

# Monitor buffer
watch -n 1 curl -s http://localhost:8000/buffer-stats
```

**Authentication**:

```bash
# Test JWT
curl -H "Authorization: Bearer $JWT_TOKEN" http://localhost:8000/auth/me

# Test API token
curl -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/auth/me
```

**Caching**:

```bash
# Monitor Redis
redis-cli MONITOR
```

## Debugging Tips

Enable verbose logging:

```bash
export LOG_LEVEL=DEBUG
uvicorn ccat_ops_db_api.main:app --reload --log-level debug
```

Use API health endpoints:

```bash
curl http://localhost:8000/health
curl http://localhost:8000/buffer-stats
curl http://localhost:8000/api/site/info
```

Inspect Redis directly:

```bash
redis-cli
> LLEN site:observatory:transaction_buffer
> LRANGE site:observatory:transaction_buffer 0 -1
```

## Next Steps

Choose a deep dive topic:

- {doc}`transaction-buffering/index` - Start with the buffering system
- {doc}`authentication/index` - Learn authentication implementation
- {doc}`routers/index` - Understand endpoint organization
- {doc}`data-flow` - See complete request/response flows
- {doc}`caching-strategy` - Optimize with caching