101 lines
5.2 KiB
Markdown
101 lines
5.2 KiB
Markdown
# ADR-006: Data Storage Strategy (SQLite vs PostgreSQL)
|
||
|
||
**Status:** Proposed
|
||
**Date:** 2026-08-02
|
||
**Author:** AI Assistant (following AGENTS.md, ADR-002, ADR-003, ADR-004, ADR-005)
|
||
**Related ADRs:** [ADR-001](./ADR-001.md), [ADR-003](./ADR-003-tech-stack.md), [ADR-005](./ADR-005-admin-moderation-analytics.md), [ADR-007](./ADR-007-redis-usage.md)
|
||
|
||
## Context
|
||
|
||
The system needs persistent storage for:
|
||
|
||
- Chat messages and history (public rooms and private DMs).
|
||
- User accounts, roles, and ban lists.
|
||
- Moderation audit log (immutable record of admin and moderator actions).
|
||
- Playlist metadata and schedule overrides.
|
||
- System configuration (including Redis settings adjustable via admin UI per ADR-005/ADR-007).
|
||
|
||
From ADR-003 we prefer operational simplicity. ADR-005 requires a robust audit trail. We must decide between **SQLite** (embedded, zero-config) and **PostgreSQL** (full-featured, scalable) and document how the admin interface can view and adjust storage-related configuration.
|
||
|
||
## Decision Drivers
|
||
|
||
1. **Operational Simplicity** – Minimize infrastructure (no separate DB server for SQLite).
|
||
2. **Performance & Concurrency** – Chat and moderation may have high write volume.
|
||
3. **Backup and Recovery** – Easy point-in-time recovery and replication.
|
||
4. **Admin Capabilities** – Admin UI must be able to view storage stats, run vacuum/analyze, and switch between SQLite and PostgreSQL (with migration path).
|
||
5. **Auditability** – Moderation log must be immutable and queryable.
|
||
6. **Cost** – Avoid managed database costs during early stages.
|
||
|
||
## Decision
|
||
|
||
**Primary storage for v1: SQLite** (embedded in the chat and admin services).
|
||
|
||
**Migration Path**: Provide a clear upgrade path to PostgreSQL when concurrency or replication needs exceed SQLite capabilities (target v2 or when concurrent users > 500).
|
||
|
||
**Admin Interface Capabilities** (extension of ADR-005):
|
||
|
||
- "System Configuration" tab showing current storage engine, database size, connection stats, and vacuum status.
|
||
- Buttons to run `VACUUM`, `ANALYZE`, or trigger a backup.
|
||
- Configuration form to switch connection string (SQLite file path vs PostgreSQL URL) and restart services gracefully.
|
||
- Audit log of all configuration changes.
|
||
|
||
**Technical Details**:
|
||
|
||
- Use SQLite with WAL mode for better concurrency.
|
||
- Use GORM or sqlx (Go) or Prisma (if Node.js fallback) for ORM/migrations.
|
||
- Store large chat history in a separate table with retention policy (e.g., 30 days).
|
||
- Moderation log as an append-only table with triggers for immutability.
|
||
|
||
**Folder Placement** (per ADR-002):
|
||
|
||
- Database models and migrations in `backend/shared/db/` or `backend/chat/internal/db/`.
|
||
- Admin UI pages for storage management in `frontend/admin/system/`.
|
||
|
||
## Rationale
|
||
|
||
- SQLite aligns with ADR-003's emphasis on operational simplicity and low resource usage for the initial deployment on a single Hetzner VPS.
|
||
- It eliminates the need for a separate database container in `docker-compose.yml` during early development.
|
||
- The admin interface can provide visibility and basic maintenance operations, fulfilling the requirement from the previous TODO plan.
|
||
- PostgreSQL can be introduced later with a migration script when scale demands it (replication, advanced querying, full-text search on chat).
|
||
|
||
## Consequences
|
||
|
||
**Positive**:
|
||
|
||
- Zero-config database for quick local development and small-scale production.
|
||
- Simple backups (just copy the `.db` file).
|
||
- Admin UI can directly manage and monitor storage without external tools.
|
||
|
||
**Negative / Risks**:
|
||
|
||
- SQLite has limited concurrency compared to PostgreSQL (WAL mode helps but is not a full replacement).
|
||
- Large chat history or analytics queries may become slow – plan retention policy and archiving.
|
||
- Migration to PostgreSQL will require downtime or dual-write strategy (documented in this ADR).
|
||
|
||
**Alternatives Considered**:
|
||
|
||
- PostgreSQL from day one – rejected for added operational complexity (separate service, connection pooling, managed service cost).
|
||
- Pure key-value store (e.g., BoltDB) – rejected because SQL is better for audit logs and complex moderation queries.
|
||
- Managed database (Neon, Supabase) – rejected to maintain full self-hosted independence per ADR-001.
|
||
|
||
## Implementation Plan
|
||
|
||
1. Create this ADR-006 (done).
|
||
2. Add SQLite support in `backend/shared/db/` with GORM migrations.
|
||
3. Implement admin UI pages for storage monitoring and configuration (linked to ADR-005).
|
||
4. Add configuration reload capability without full restart where possible.
|
||
5. Write migration script from SQLite to PostgreSQL.
|
||
6. Update `docker-compose.yml` to optionally include PostgreSQL.
|
||
7. Add tests for data access patterns and admin configuration changes.
|
||
|
||
**Next ADR**: ADR-007 (Redis usage and admin configuration capabilities).
|
||
|
||
## References
|
||
|
||
- ADR-001: Chat persistence, moderation log, monitoring requirements.
|
||
- ADR-003: Preference for operational simplicity and Go backend.
|
||
- ADR-005: Admin interface for user management, moderation, and analytics – this ADR extends it with storage and Redis config capabilities.
|
||
- arc42 documentation: Cross-cutting Concepts (data management, monitoring) and Risks & Technical Debts.
|
||
|
||
This ADR ensures all data storage decisions are documented before code is written, maintaining full traceability.
|