HOPS - Homelab Orchestration Provisioning Script

HOPS is a comprehensive, automated deployment solution for popular homelab applications. It simplifies the process of setting up and managing Docker-based services including media servers, download clients, monitoring tools, and more.

⚠️ Important Notices

🚧 Development Status

HOPS is under active development and should be considered beta software. While core functionality is stable and tested, new features and improvements are being added regularly.

🖥️ Platform Testing Status

• Linux (Ubuntu/Debian/Mint): ✅ Extensively tested and stable
• macOS: ✅ Recently improved with comprehensive compatibility fixes
• Windows (WSL2): ⚠️ Limited testing - Basic functionality works but may have undiscovered issues

💾 Data Safety & Backups

IMPORTANT: Always maintain regular backups of your data before using HOPS.

• Backup your media and configuration files before installation
• Test in a non-production environment first if possible
• HOPS is not responsible for any data loss that may occur during installation or operation
• Docker operations can be destructive - ensure you understand what services you're installing

📋 Recommended Backup Strategy

# Before running HOPS, backup important directories
sudo tar -czf homelab-backup-$(date +%Y%m%d).tar.gz /path/to/your/media /path/to/your/configs

# After installation, backup HOPS configuration
sudo tar -czf hops-config-backup-$(date +%Y%m%d).tar.gz ~/hops ~/.config/hops

🆕 What's New in v3.2.0

Major macOS Compatibility Improvements

• 🍎 Enhanced macOS Support: Comprehensive fixes for macOS installation and operation
• 🔐 Keychain Integration: Proper Docker authentication with macOS keychain
• 👤 User Directory Fixes: All directories now use actual user home instead of root
• 🚀 Docker Desktop Integration: Improved Docker Desktop startup and management
• ⚡ Better Error Handling: Enhanced error messages and troubleshooting for macOS

New Features

• 🌐 Caddy Support: Added Caddy reverse proxy as a service option (configuration not included)

Bug Fixes

• 🔧 Fixed password generation: Resolved shuf command and encoding issues on macOS
• 🐳 Fixed container creation: Resolved Docker Compose working directory issues
• 🏥 Fixed healthchecks: Improved Jellyseerr and other service health monitoring
• 📁 Fixed file permissions: Proper ownership of config and media directories

Previous in v3.1.0-beta

• 🔐 Encrypted Secret Management: All passwords and sensitive data now encrypted with AES-256
• 🛡️ Input Validation: Comprehensive validation preventing injection attacks
• ⚡ Privilege Separation: Root operations separated from user operations
• 📌 Pinned Versions: All container images use specific versions, not latest

New Architecture

• 📚 Modular Libraries: Shared code organized in lib/ directory
• 🔧 Enhanced Error Handling: Better error messages and recovery mechanisms
• 🎯 Improved Service Definitions: Standardized service generation with validation
• 📖 Documentation: Complete CLAUDE.md for development guidance
• 🍎 Cross-Platform Support: Native support for Linux, macOS, and Windows (WSL2) with automatic dependency installation

Installation Methods

• 🚀 New Secure Installer: sudo ./setup - Recommended method
• ⚙️ Manual Installation: Separate privileged and user operations
• 🔄 Legacy Support: Original hops.sh still fully supported

🎯 What is HOPS?

HOPS (Homelab Orchestration Provisioning Script) automates the deployment of a complete homelab infrastructure using Docker Compose. It provides an intuitive menu-driven interface for selecting, configuring, and managing services with enterprise-grade features like:

• Automated dependency resolution
• Security hardening and firewall configuration
• Service health monitoring
• Rollback capabilities on failure
• Comprehensive logging
• User-friendly management interface

✨ Key Features

🚀 Easy Installation

• One-command installation process
• Automatic Docker installation and configuration
• Interactive service selection
• Intelligent dependency resolution
• NEW: Privilege separation for enhanced security

🔒 Security First

• Automatic firewall configuration
• Secure password generation with encryption
• File permission hardening
• Network isolation
• NEW: AES-256 encrypted secret storage
• NEW: Comprehensive input validation
• NEW: Pinned container versions

📊 Management & Monitoring

• Real-time service status monitoring
• Centralized log viewing
• Easy service management (start/stop/restart)
• Health checks and service verification
• NEW: Modular architecture with shared libraries

🔄 Reliability

• Error handling with automatic rollback
• Service dependency management
• Port conflict detection
• System requirements validation
• NEW: Enhanced error handling with detailed context

📱 Supported Services

📺 Media Management (*arr Stack)

• Sonarr - TV show management
• Radarr - Movie management
• Lidarr - Music management
• Readarr - eBook/audiobook management
• Bazarr - Subtitle management
• Prowlarr - Indexer management
• Tdarr - Media transcoding
• Huntarr - Missing media discovery and automation

⬇️ Download Clients

• qBittorrent - Feature-rich BitTorrent client
• Transmission - Lightweight BitTorrent client
• NZBGet - Efficient Usenet downloader
• SABnzbd - Popular Usenet client

🎞️ Media Servers

• Jellyfin - Open-source media server
• Plex - Popular media server platform
• Emby - Feature-rich media server
• Jellystat - Jellyfin statistics and monitoring

🎛️ Request Management

• Overseerr - Media request management for Plex
• Jellyseerr - Media request management for Jellyfin
• Ombi - Media request platform

🔒 Reverse Proxy & Security

• Traefik - Modern reverse proxy with automatic SSL
• Nginx Proxy Manager - Easy-to-use reverse proxy
• Caddy - Automatic HTTPS web server (configuration not included)
• Authelia - Authentication and authorization server

📈 Monitoring & Management

• Portainer - Docker container management
• Uptime Kuma - Service monitoring
• Watchtower - Automatic container updates

🔧 System Requirements

Minimum Requirements

• OS:
  • Linux: Ubuntu 20.04+, Debian 11+, or Linux Mint 20+
  • macOS: 11.0+ (Big Sur) with Intel or Apple Silicon
  • Windows: 10/11 with WSL2 (Ubuntu 20.04+ distribution) ⚠️ Limited testing
• RAM: 2GB (4GB+ recommended)
• Storage: 10GB free space (more for media)
• CPU: 2 cores recommended
• Network: Internet connection required

Prerequisites

Linux:

• Root/sudo access
• x86_64 architecture
• Internet connection

macOS:

• Admin access (sudo privileges)
• Intel (x86_64) or Apple Silicon (ARM64)
• Internet connection
• Homebrew will be installed automatically if not present
• Docker Desktop will be installed automatically if not present

Windows:

• Windows 10 (build 19041+) or Windows 11
• WSL2 enabled with Ubuntu 20.04+ distribution
• Docker Desktop with WSL2 backend
• Admin access to install prerequisites
• x86_64 processor with virtualization support
• Hyper-V and virtualization enabled in BIOS

🪟 Windows Installation (WSL2)

HOPS runs on Windows through WSL2 (Windows Subsystem for Linux) with excellent compatibility and performance. This approach leverages the full Linux environment within Windows.

Prerequisites Setup

1. Enable WSL2:

# Run in PowerShell as Administrator
wsl --install
# Restart computer when prompted

2. Install Ubuntu Distribution:

# Install Ubuntu 22.04 LTS (recommended)
wsl --install Ubuntu-22.04
# Set up username and password when prompted

3. Install Docker Desktop:

• Download from Docker Desktop for Windows
• Enable WSL2 integration during installation
• Ensure "Use WSL 2 based engine" is checked in Docker settings

HOPS Installation on WSL2

⚠️ Note: WSL2 support has limited testing. Please proceed with caution and ensure you have backups.

1. Open WSL2 Terminal:

# Launch Ubuntu from Start Menu or run:
wsl -d Ubuntu-22.04

2. Install HOPS (same as Linux):

# Clone inside WSL2 filesystem (important for performance)
cd ~
git clone https://github.com/skiercm/hops.git
cd hops
chmod +x hops install uninstall setup

# Run installation
sudo ./setup

⚠️ Important Notes for Windows Users

File Location: Always run HOPS from the WSL2 filesystem (~/hops/) for optimal performance. Avoid running from /mnt/c/ (Windows drives).

Media Access: Your Windows media folders can be accessed at:

• C:\Users\YourName\ → /mnt/c/Users/YourName/
• External drives → /mnt/d/, /mnt/e/, etc.

Docker Integration: Services will be accessible from both Windows and WSL2:

• Web interfaces work from Windows browsers
• File shares accessible from Windows Explorer via \\wsl.localhost\Ubuntu-22.04\home\username\hops\

Performance: WSL2 provides 95% of native Linux performance when files are stored in the WSL2 filesystem.

🚀 Quick Start

1. Download HOPS

git clone https://github.com/skiercm/hops.git
cd hops
chmod +x hops install uninstall setup

2. Run Installation (New Improved Method)

# Option 1: Use the new secure installation wrapper
sudo ./setup

# Option 2: Manual two-phase installation
sudo ./privileged-setup  # Run as root
./user-operations generate <services>  # Run as user
./user-operations deploy  # Run as user

# Option 3: Legacy installation (still supported)
sudo ./hops.sh

3. Follow the Interactive Setup

• Select your desired services
• Configure directories and timezone
• Choose security options
• Wait for automated deployment

4. Access Your Services

The installer will provide URLs for all deployed services:

📱 Access your services at:
   ● Jellyfin     http://192.168.1.100:8096
   ● Sonarr       http://192.168.1.100:8989
   ● Radarr       http://192.168.1.100:7878
   ● Portainer    http://192.168.1.100:9000

📁 Default Directory Structure

~/hops/                       # Main deployment directory
├── docker-compose.yml        # Service definitions
├── .env                      # Environment variables
└── logs/                     # Application logs

/opt/appdata/                 # Application configurations
├── jellyfin/
├── sonarr/
├── radarr/
└── ...

/mnt/media/                   # Media storage
├── movies/
├── tv/
├── music/
└── downloads/

🎛️ Management Interface

HOPS includes a comprehensive management interface accessible through the main script:

sudo ./hops.sh

Available Options:

1. Install HOPS - Deploy new services
2. Uninstall HOPS - Complete removal with options
3. Manage Services - Start/stop/restart services
4. Service Status - Real-time service monitoring
5. Access Information - Get service URLs and credentials
6. View Logs - Centralized log viewing
7. Help & Documentation - Built-in help system

🔧 Advanced Configuration

Environment Variables

Configuration is now stored encrypted for enhanced security:

# NEW: Encrypted secret management
./lib/secrets.sh init                    # Initialize secret management
./lib/secrets.sh create                  # Create encrypted environment
./lib/secrets.sh update DOMAIN example.com  # Update values
./lib/secrets.sh get PUID               # Get values
./lib/secrets.sh list                   # List all keys

# Legacy: Plaintext configuration in ~/hops/.env
PUID=1000                     # User ID
PGID=1000                     # Group ID
TZ=America/New_York          # Timezone

# Directory Configuration
DATA_ROOT=/mnt/media         # Media storage
CONFIG_ROOT=/opt/appdata     # App configurations

# Security (now auto-generated and encrypted)
DEFAULT_ADMIN_PASSWORD=...   # Generated secure password
DEFAULT_DB_PASSWORD=...      # Database password

# Optional: Custom domain
DOMAIN=yourdomain.com
ACME_EMAIL=admin@yourdomain.com

Service-Specific Configuration

Caddy Configuration

Important: HOPS provides the Caddy container but does not include Caddyfile configuration. Users must provide their own Caddyfile.

# Create Caddy configuration directory
mkdir -p ~/hops/config/caddy

# Create your Caddyfile (example)
cat > ~/hops/config/caddy/Caddyfile << 'EOF'
# Example Caddyfile - customize as needed
example.com {
    reverse_proxy jellyfin:8096
}

api.example.com {
    reverse_proxy overseerr:5055
}
EOF

# Caddy will automatically handle HTTPS certificates
# Documentation: https://caddyserver.com/docs/

Configuration Scope: HOPS installs and runs the Caddy container with proper volume mounts, but all routing, SSL, and proxy configuration is the user's responsibility.

Service Management Commands

# NEW: User operations script (runs without sudo)
./user-operations status        # View service status
./user-operations logs <service> # View service logs
./user-operations deploy        # Deploy services
./user-operations stop          # Stop all services

# Legacy: Direct Docker Compose commands
cd ~/hops
docker compose ps                       # View running services
docker compose logs -f [service-name]  # View logs
docker compose restart [service-name]  # Restart specific service
docker compose pull && docker compose up -d  # Update all services
docker compose down                     # Stop all services

New Architecture

HOPS v3.1.0-beta introduces a modular architecture with shared libraries:

hops/
├── lib/                    # NEW: Shared libraries
│   ├── common.sh          # Logging, UI, utilities
│   ├── system.sh          # System validation
│   ├── docker.sh          # Docker operations
│   ├── security.sh        # Security utilities
│   ├── validation.sh      # Input validation
│   ├── secrets.sh         # Secret management
│   └── privileges.sh      # Privilege management
├── setup                  # NEW: Installation wrapper
├── privileged-setup       # NEW: Root-only operations
├── user-operations        # NEW: User operations
├── services-improved      # NEW: Enhanced service definitions
└── hops.sh               # Legacy main script (still supported)

🔒 Security Features

Automatic Security Hardening

• Firewall Configuration: Automatic UFW rules for service ports
• Secure Passwords: Cryptographically secure password generation
• File Permissions: Restrictive permissions on sensitive files
• Network Isolation: Docker network segregation
• SSL/TLS: Automatic certificate management with Traefik
• NEW: AES-256 encrypted secret storage with master key management
• NEW: Comprehensive input validation preventing injection attacks
• NEW: Privilege separation (root vs user operations)
• NEW: Pinned container versions preventing supply chain attacks

Post-Installation Security

1. Manage Encrypted Secrets: Use ./lib/secrets.sh for secure password management
2. Configure Reverse Proxy: Set up Traefik or Nginx Proxy Manager
3. Enable Authentication: Configure Authelia for additional security
4. Regular Updates: Use Watchtower for automatic updates
5. Security Auditing: Use ./lib/security.sh for security checks

🆘 Troubleshooting

Common Issues

Port Conflicts

# Check for port conflicts
sudo lsof -i :PORT_NUMBER

# View HOPS service status
sudo ./hops.sh
# Select option 4: Service Status

Service Won't Start

# Check service logs
cd ~/hops
docker compose logs [service-name]

# Restart service
docker compose restart [service-name]

Permission Issues

# Fix ownership of data directories
sudo chown -R $USER:$USER /mnt/media /opt/appdata

Log Locations

• Installation Logs: /var/log/hops/
• Service Logs: docker compose logs [service-name]
• System Logs: journalctl -u docker

Getting Help

1. Check the built-in help: sudo ./hops.sh → Option 7
2. Review logs in /var/log/hops/
3. Verify Docker status: systemctl status docker
4. Check service health: docker compose ps

🔄 Backup and Recovery

Backup Important Data

# Backup configurations
sudo tar -czf hops-config-backup.tar.gz /opt/appdata

# Backup compose files
cp ~/hops/.env ~/hops/docker-compose.yml /backup/location/

Recovery

# Restore configurations
sudo tar -xzf hops-config-backup.tar.gz -C /

# Redeploy services
cd ~/hops
docker compose up -d

📊 Performance Tuning

For Low-Resource Systems

• Start with fewer services initially
• Monitor resource usage with Portainer
• Consider using lightweight alternatives (Transmission vs qBittorrent)

For High-Performance Systems

• Enable GPU transcoding in Jellyfin/Plex
• Use SSD storage for application data
• Configure multiple download clients for redundancy

🤝 Contributing

We welcome contributions! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

Development Setup

git clone https://github.com/skiercm/hops.git
cd hops

# Test syntax validation
bash -n lib/*.sh
bash -n *.sh

# Test service definitions
./services-improved list
./services-improved generate jellyfin

# Test new installation method
sudo ./setup

# Test legacy method
sudo ./hops.sh

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• LinuxServer.io for excellent Docker images
• Docker for containerization platform
• The Servarr Team for the *arr applications
• Jellyfin Project for the open-source media server
• All the amazing open-source projects that make HOPS possible

📞 Support

HOPS Support

• Documentation: Check this README and built-in help
• Issues: Report HOPS bugs via GitHub Issues
• Community: Join discussions in GitHub Discussions

Service-Specific Support

⚠️ Important: If you encounter issues with a specific service (configuration, features, bugs), please reach out to the respective service developers directly using the links below. HOPS only handles deployment automation - the individual services are maintained by their respective teams.

📺 Media Management (*arr Stack)

• Sonarr: github.com/Sonarr/Sonarr - TV series management
• Radarr: github.com/Radarr/Radarr - Movie collection manager
• Lidarr: github.com/Lidarr/Lidarr - Music collection manager
• Readarr: github.com/Readarr/Readarr - E-book manager ⚠️ Project retired
• Bazarr: github.com/morpheus65535/bazarr - Subtitle management
• Prowlarr: github.com/Prowlarr/Prowlarr - Indexer manager
• Tdarr: github.com/HaveAGitGat/Tdarr - Media transcoding
• Huntarr: github.com/plexguide/Huntarr.io - Missing media discovery

⬇️ Download Clients

• qBittorrent: github.com/qbittorrent/qBittorrent - BitTorrent client
• Transmission: github.com/transmission/transmission - BitTorrent client
• NZBGet: github.com/nzbget/nzbget - Usenet downloader
• SABnzbd: github.com/sabnzbd/sabnzbd - Usenet downloader

🎞️ Media Servers

• Jellyfin: github.com/jellyfin/jellyfin - Free media server
• Plex: github.com/plexinc/pms-docker - Docker container repo
• Emby: github.com/MediaBrowser/Emby - Personal media server

🎛️ Request Management

• Overseerr: github.com/sct/overseerr - Media requests for Plex
• Jellyseerr: github.com/fallenbagel/jellyseerr - Media requests for Jellyfin/Emby/Plex
• Ombi: github.com/Ombi-app/Ombi - Media request platform
• Jellystat: github.com/CyferShepard/Jellystat - Jellyfin statistics

🔒 Network & Security

• Traefik: github.com/traefik/traefik - Modern reverse proxy
• Nginx Proxy Manager: github.com/NginxProxyManager/nginx-proxy-manager - Nginx proxy management
• Authelia: github.com/authelia/authelia - Authentication & SSO

📈 Monitoring & Management

• Portainer: github.com/portainer/portainer - Container management
• Watchtower: github.com/containrrr/watchtower - Automatic updates
• Uptime Kuma: github.com/louislam/uptime-kuma - Uptime monitoring

When to Contact HOPS vs Service Developers

Contact HOPS for:

• Installation/deployment issues
• Docker Compose generation problems
• Cross-platform compatibility issues
• Script errors or automation failures

Contact Service Developers for:

• Service configuration help
• Feature requests for individual services
• Bugs within the service itself
• Service-specific documentation

---

Made with ❤️ for the homelab community

HOPS - Making homelab deployment simple, secure, and reliable.