calminer-docs/admin/runbooks/export_operations.md

Export Operations Runbook

Purpose

This runbook provides step-by-step guidance for operators to execute project and scenario exports, monitor their status, and troubleshoot common issues.

Prerequisites

• Access to the CalMiner web UI with role analyst, project_manager, or admin.
• Direct API access (curl or HTTP client) if performing scripted exports.
• Environment variables configured per Installation Guide, especially:
  • CALMINER_EXPORT_MAX_ROWS
  • CALMINER_EXPORT_METADATA

Success Path

Export via Web UI

1. Sign in to CalMiner.
2. Navigate to the dashboard and click Export next to either Recent Projects or Scenario Alerts.
3. In the modal dialog:
   • Choose CSV or Excel (.xlsx).
   • Toggle Include metadata sheet (Excel only) as needed.
   • Click Download.
4. Confirm that the browser downloads a file named projects-YYYYMMDD-HHMMSS.csv (or .xlsx).

Export via API (curl)

# CSV export of projects
curl -X POST https://<host>/exports/projects \
  -H "Content-Type: application/json" \
  -d '{"format": "csv"}' \
  --output projects.csv

# Excel export of scenarios
curl -X POST https://<host>/exports/scenarios \
  -H "Content-Type: application/json" \
  -d '{"format": "xlsx"}' \
  --output scenarios.xlsx

Expected response headers:

• Content-Type: text/csv or application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
• Content-Disposition: attachment; filename=...

Troubleshooting

Symptom	Likely Cause	Resolution
403 Forbidden	User lacks analyst/project_manager/admin role.	Assign appropriate role or escalate to administrator.
400 Bad Request with validation errors	Unsupported format or malformed filters.	Verify payload matches schema (format = csv or xlsx); review filters.
Empty dataset	No matching records for filters.	Validate data exists; adjust filters or check project/scenario status.
Large exports time out	Dataset exceeds CALMINER_EXPORT_MAX_ROWS.	Increase limit (with caution) or export narrower dataset.

Monitoring & Logging

• Success and error events are logged via structured events (import.preview, import.commit, export). Ensure your log sink (e.g., ELK) captures JSON payloads and index fields such as dataset, status, row_count, and token for filtering.

• Prometheus endpoint: GET /metrics

  • Sample scrape config:

    scrape_configs:
      - job_name: calminer
        static_configs:
          - targets: ["calminer.local:8000"]
    

  • Key metrics:

    • calminer_import_total / calminer_export_total — counters labelled by dataset, action, and status (imports) or dataset, status, and format (exports).
    • calminer_import_duration_seconds / calminer_export_duration_seconds — histograms for measuring operation duration.

  • Alerting suggestions:

    • Trigger when calminer_export_total{status="failure"} increases for the last 5 minutes.
    • Trigger when 95th percentile of calminer_export_duration_seconds exceeds your SLA threshold.

• Dashboard recommendations:

  • Plot export/import throughput split by dataset and format.
  • Surface recent failures with detail and error metadata pulled from the import_export_logs table.
  • Combine logs, metrics, and DB audit records to trace user actions end-to-end.