jobs/README-Docker.md

Jobs App - Docker Deployment

This application is a Craigslist job scraper with a Flask web interface.

Quick Start with Docker

Prerequisites

• Docker
• Docker Compose

Deployment

1. Clone the repository and navigate to the project directory

   cd /path/to/jobs-app/jobs
   

2. Start the application

   docker-compose up --build -d
   

3. Wait for services to be ready (about 30 seconds)

   # Check if the app is running
   curl http://localhost:8000
   

4. Access the application

   • Main app: http://localhost:8000
   • Admin interface: http://localhost:8000/admin/users
     • Username: admin
     • Password: M11ffpgm.
   • Scraper interface: http://localhost:8000/scrape-page

Docker Architecture

Services

• jobs-app: Flask application with Gunicorn WSGI server
• mysql: MySQL 8.0 database

Ports

• 8000: Flask application
• 3306: MySQL database (exposed for external access if needed)

Volumes

• mysql_data: Persistent MySQL data storage

Configuration

Environment Variables

• FLASK_ENV: Set to production
• FLASK_SECRET: Secret key for Flask sessions (required)
• APT_CACHER_NG: Optional URL for apt-cacher-ng proxy to speed up package downloads (e.g., http://192.168.88.14:3142)

Database Configuration

The database configuration is in config/settings.json:

{
  "database": {
    "mysql": {
      "host": "mysql",
      "user": "jobs",
      "password": "jobdb",
      "database": "jobs",
      "port": 3306
    }
  }
}

Useful Commands

# View logs
docker-compose logs -f

# Stop services
docker-compose down

# Restart services
docker-compose restart

# Rebuild and restart
docker-compose up --build

# View running containers
docker-compose ps

# Execute commands in the app container
docker-compose exec jobs-app bash

# Check database
docker-compose exec mysql mysql -u jobs -p jobs

Production Considerations

1. Security:

   • Change default passwords in docker-compose.yml
   • Use environment variables for secrets
   • Configure proper firewall rules

2. Scaling:

   • Adjust Gunicorn workers in gunicorn.conf.py
   • Consider using a reverse proxy (nginx)
   • Implement proper logging and monitoring

3. Database:

   • Use external MySQL for production
   • Configure backups
   • Set up connection pooling

4. Networking:

   • Use proper domain names
   • Configure SSL/TLS
   • Set up load balancing if needed

Troubleshooting

Common Issues

1. Port conflicts: Change ports in docker-compose.yml
2. Database connection: Ensure MySQL container is healthy
3. Memory issues: Increase Docker memory limits
4. Permission issues: Check file permissions in mounted volumes

Logs

# Application logs
docker-compose logs jobs-app

# Database logs
docker-compose logs mysql

# All logs
docker-compose logs

Development

For development with hot reload:

# Run in development mode
docker-compose -f docker-compose.dev.yml up --build

Create docker-compose.dev.yml:

version: "3.8"
services:
  jobs-app:
    build: .
    ports:
      - "8000:8000"
    environment:
      - FLASK_ENV=development
    volumes:
      - .:/app
    command: ["flask", "run", "--host", "0.0.0.0", "--port", "8000"]

Coolify Deployment

This application can be deployed on Coolify using Docker Compose. Coolify provides additional features and management capabilities for containerized applications.

Coolify Prerequisites

• Coolify instance (self-hosted or cloud)
• Git repository accessible to Coolify

Coolify-Specific Configuration

1. Environment Variables

Coolify automatically detects environment variables in your docker-compose.yml and provides a UI to manage them. Use the following syntax for better integration:

services:
  jobs-app:
    environment:
      # Required variables (will show red border if empty)
      - FLASK_SECRET=${FLASK_SECRET:?}

      # Required with default (prefilled but editable)
      - FLASK_ENV=${FLASK_ENV:?production}

      # Optional with default
      - GUNICORN_WORKERS=${GUNICORN_WORKERS:-4}

2. Coolify Magic Environment Variables

Leverage Coolify's dynamic environment variables:

services:
  jobs-app:
    environment:
      # Generate FQDN for the application
      - SERVICE_FQDN_JOBS_APP

      # Generate secure password for admin user
      - ADMIN_PASSWORD=${SERVICE_PASSWORD_ADMIN:?M11ffpgm.}

      # Generate database credentials
      - DB_USER=${SERVICE_USER_DB:?jobs}
      - DB_PASSWORD=${SERVICE_PASSWORD_DB:?jobdb}

3. Storage Configuration

Coolify supports advanced storage options:

services:
  jobs-app:
    volumes:
      # Create empty directories
      - type: bind
        source: ./cache
        target: /app/cache
        is_directory: true

      - type: bind
        source: ./logs
        target: /app/logs
        is_directory: true

  mysql:
    volumes:
      # Persistent database storage
      - mysql_data:/var/lib/mysql

4. Health Checks and Service Management

services:
  jobs-app:
    # Exclude from health checks if needed
    exclude_from_hc: false

    # Labels for Coolify management and Traefik routing
    labels:
      - coolify.managed=true
      - traefik.enable=true
      - "traefik.http.routers.jobs-app.rule=Host(`${SERVICE_FQDN_JOBS_APP:-localhost}`)"
      - traefik.http.routers.jobs-app.entryPoints=https
      - "traefik.http.routers.jobs-app.middlewares=https-redirect"
      - "traefik.http.middlewares.https-redirect.redirectscheme.scheme=https"

5. Database Configuration for Coolify

Update your config/settings.json to use Coolify environment variables:

{
  "database": {
    "mysql": {
      "host": "mysql",
      "user": "${DB_USER:-jobs}",
      "password": "${DB_PASSWORD:-jobdb}",
      "database": "jobs",
      "port": 3306
    }
  }
}

Complete Coolify docker-compose.yml

Here's a complete docker-compose.yml optimized for Coolify:

version: "3.8"

services:
  jobs-app:
    build: .
    ports:
      - "8000:8000"
    environment:
      # Required environment variables
      - FLASK_SECRET=${FLASK_SECRET:?}
      - FLASK_ENV=${FLASK_ENV:?production}

      # Coolify magic variables
      - SERVICE_FQDN_JOBS_APP
      - ADMIN_PASSWORD=${SERVICE_PASSWORD_ADMIN:?M11ffpgm.}
      - DB_USER=${SERVICE_USER_DB:?jobs}
      - DB_PASSWORD=${SERVICE_PASSWORD_DB:?jobdb}

      # Optional configuration
      - GUNICORN_WORKERS=${GUNICORN_WORKERS:-4}
      - APT_CACHER_NG=${APT_CACHER_NG}
    volumes:
      - type: bind
        source: ./cache
        target: /app/cache
        is_directory: true
      - type: bind
        source: ./logs
        target: /app/logs
        is_directory: true
    depends_on:
      - mysql
    labels:
      - coolify.managed=true
      - traefik.enable=true
      - "traefik.http.routers.jobs-app.rule=Host(`${SERVICE_FQDN_JOBS_APP:-localhost}`)"
      - traefik.http.routers.jobs-app.entryPoints=https
      - "traefik.http.routers.jobs-app.middlewares=https-redirect"
      - "traefik.http.middlewares.https-redirect.redirectscheme.scheme=https"
    networks:
      - jobs-network
    restart: unless-stopped

  mysql:
    image: mysql:8.0
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD:?rootpassword}
      - MYSQL_DATABASE=jobs
      - MYSQL_USER=${DB_USER:-jobs}
      - MYSQL_PASSWORD=${DB_PASSWORD:-jobdb}
    ports:
      - "3306:3306"
    volumes:
      - mysql_data:/var/lib/mysql
      - ./mysql-init:/docker-entrypoint-initdb.d
    networks:
      - jobs-network

volumes:
  mysql_data:

networks:
  jobs-network:
    driver: bridge

Coolify Deployment Steps

1. Connect Repository: Link your Git repository to Coolify
2. Create Service Stack: Choose "Docker Compose" as the build pack
3. Configure Environment Variables: Set required variables in Coolify's UI:
   • FLASK_SECRET: Generate a secure random string
   • FLASK_ENV: Set to "production"
   • MYSQL_ROOT_PASSWORD: Set a secure password
4. Deploy: Coolify will automatically build and deploy your application
5. Access: Use the generated FQDN to access your application

Coolify Benefits

• Automatic SSL: HTTPS certificates are automatically managed
• Environment Management: Easy variable management through UI
• Monitoring: Built-in logging and health monitoring
• Scaling: Easy horizontal scaling
• Backups: Automated backup capabilities
• Security: Isolated networks and secure defaults

Troubleshooting Coolify Deployments

1. Environment Variables: Check that all required variables are set in Coolify's UI
2. Build Logs: Review build logs for any compilation errors
3. Network Issues: Ensure services can communicate within the stack
4. Storage Permissions: Verify volume permissions are correct
5. FQDN Configuration: Check that the generated FQDN is accessible

Common Coolify Errors

Error: "The scheme domain.sslip.io isn't valid. It should be either http, https"

• Cause: Incorrect Traefik router configuration using full URLs instead of hostnames
• Solution: Use SERVICE_FQDN_* variables (which contain just the domain) in Host() rules, not SERVICE_URL_* variables

Example:

# Correct
- "traefik.http.routers.app.rule=Host(`${SERVICE_FQDN_APP:-localhost}`)"

# Incorrect
- "traefik.http.routers.app.rule=Host(`${SERVICE_URL_APP:-localhost}`)"

Container fails to start with permission denied

• Cause: Volume mount permissions in Coolify environment
• Solution: Ensure is_directory: true is set for bind mounts that need directory creation

For more information, visit the Coolify Docker Compose documentation.