linearvoid/docs/architecture/adr/ADR-006-data-storage.md

ADR-006: Data Storage Strategy (SQLite vs PostgreSQL)

Status: Proposed
Date: 2026-08-02
Author: AI Assistant (following AGENTS.md, ADR-002, ADR-003, ADR-004, ADR-005)
Related ADRs: ADR-001, ADR-003, ADR-005, ADR-007

Context

The system needs persistent storage for:

• Chat messages and history (public rooms and private DMs).
• User accounts, roles, and ban lists.
• Moderation audit log (immutable record of admin and moderator actions).
• Playlist metadata and schedule overrides.
• System configuration (including Redis settings adjustable via admin UI per ADR-005/ADR-007).

From ADR-003 we prefer operational simplicity. ADR-005 requires a robust audit trail. We must decide between SQLite (embedded, zero-config) and PostgreSQL (full-featured, scalable) and document how the admin interface can view and adjust storage-related configuration.

Decision Drivers

1. Operational Simplicity – Minimize infrastructure (no separate DB server for SQLite).
2. Performance & Concurrency – Chat and moderation may have high write volume.
3. Backup and Recovery – Easy point-in-time recovery and replication.
4. Admin Capabilities – Admin UI must be able to view storage stats, run vacuum/analyze, and switch between SQLite and PostgreSQL (with migration path).
5. Auditability – Moderation log must be immutable and queryable.
6. Cost – Avoid managed database costs during early stages.

Decision

Primary storage for v1: SQLite (embedded in the chat and admin services).

Migration Path: Provide a clear upgrade path to PostgreSQL when concurrency or replication needs exceed SQLite capabilities (target v2 or when concurrent users > 500).

Admin Interface Capabilities (extension of ADR-005):

• "System Configuration" tab showing current storage engine, database size, connection stats, and vacuum status.
• Buttons to run VACUUM, ANALYZE, or trigger a backup.
• Configuration form to switch connection string (SQLite file path vs PostgreSQL URL) and restart services gracefully.
• Audit log of all configuration changes.

Technical Details:

• Use SQLite with WAL mode for better concurrency.
• Use GORM or sqlx (Go) or Prisma (if Node.js fallback) for ORM/migrations.
• Store large chat history in a separate table with retention policy (e.g., 30 days).
• Moderation log as an append-only table with triggers for immutability.

Folder Placement (per ADR-002):

• Database models and migrations in backend/shared/db/ or backend/chat/internal/db/.
• Admin UI pages for storage management in frontend/admin/system/.

Rationale

• SQLite aligns with ADR-003's emphasis on operational simplicity and low resource usage for the initial deployment on a single Hetzner VPS.
• It eliminates the need for a separate database container in docker-compose.yml during early development.
• The admin interface can provide visibility and basic maintenance operations, fulfilling the requirement from the previous TODO plan.
• PostgreSQL can be introduced later with a migration script when scale demands it (replication, advanced querying, full-text search on chat).

Consequences

Positive:

• Zero-config database for quick local development and small-scale production.
• Simple backups (just copy the .db file).
• Admin UI can directly manage and monitor storage without external tools.

Negative / Risks:

• SQLite has limited concurrency compared to PostgreSQL (WAL mode helps but is not a full replacement).
• Large chat history or analytics queries may become slow – plan retention policy and archiving.
• Migration to PostgreSQL will require downtime or dual-write strategy (documented in this ADR).

Alternatives Considered:

• PostgreSQL from day one – rejected for added operational complexity (separate service, connection pooling, managed service cost).
• Pure key-value store (e.g., BoltDB) – rejected because SQL is better for audit logs and complex moderation queries.
• Managed database (Neon, Supabase) – rejected to maintain full self-hosted independence per ADR-001.

Implementation Plan

1. Create this ADR-006 (done).
2. Add SQLite support in backend/shared/db/ with GORM migrations.
3. Implement admin UI pages for storage monitoring and configuration (linked to ADR-005).
4. Add configuration reload capability without full restart where possible.
5. Write migration script from SQLite to PostgreSQL.
6. Update docker-compose.yml to optionally include PostgreSQL.
7. Add tests for data access patterns and admin configuration changes.

Next ADR: ADR-007 (Redis usage and admin configuration capabilities).

References

• ADR-001: Chat persistence, moderation log, monitoring requirements.
• ADR-003: Preference for operational simplicity and Go backend.
• ADR-005: Admin interface for user management, moderation, and analytics – this ADR extends it with storage and Redis config capabilities.
• arc42 documentation: Cross-cutting Concepts (data management, monitoring) and Risks & Technical Debts.

This ADR ensures all data storage decisions are documented before code is written, maintaining full traceability.