canadabot/hops
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reorganize documentation for better user experience

Browse Source 
- Streamline main README to essential information only
- Create comprehensive INSTALLATION.md with platform-specific guides
- Add detailed SERVICES.md with complete service catalog and support links
- Add ADVANCED.md with configuration, security, and troubleshooting
- Add CHANGELOG.md with complete version history and migration guides

This reorganization reduces main README length by 90% while improving
discoverability and maintenance of detailed information.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Stephen Klein
2025-07-18 18:35:13 -04:00
parent b81779784b
commit b114bd54c8
5 changed files with 1659 additions and 574 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ADVANCED.md
+715
View File
@@ -0,0 +1,715 @@
# Advanced Configuration & Troubleshooting
This guide covers advanced HOPS configuration, troubleshooting, security features, and system administration.
## 🔧 Advanced Configuration
### Environment Variables
HOPS uses both encrypted secret management and traditional environment files for configuration.
#### Encrypted Secret Management (v3.1.0+)
```bash
# Initialize secret management
./lib/secrets.sh init
# Create encrypted environment
./lib/secrets.sh create
# Update values
./lib/secrets.sh update DOMAIN example.com
./lib/secrets.sh update ACME_EMAIL admin@example.com
# Retrieve values
./lib/secrets.sh get PUID
./lib/secrets.sh get DEFAULT_ADMIN_PASSWORD
# List all keys
./lib/secrets.sh list
# Backup encrypted secrets
./lib/secrets.sh backup /backup/location/
```
#### Traditional Environment File (~/hops/.env)
```bash
# User and Group Configuration
PUID=1000 # User ID
PGID=1000 # Group ID
TZ=America/New_York # Timezone
# Directory Configuration
DATA_ROOT=/mnt/media # Media storage (Linux)
CONFIG_ROOT=/opt/appdata # App configurations (Linux)
# macOS paths (automatically set)
DATA_ROOT=/Users/username/hops/media
CONFIG_ROOT=/Users/username/hops/config
# Security (auto-generated and encrypted)
DEFAULT_ADMIN_PASSWORD=... # Generated secure password
DEFAULT_DB_PASSWORD=... # Database password
JELLYFIN_PASSWORD=... # Service-specific passwords
# Optional: Custom Domain Configuration
DOMAIN=yourdomain.com
ACME_EMAIL=admin@yourdomain.com
# Optional: Service Overrides
JELLYFIN_PORT=8096
SONARR_PORT=8989
RADARR_PORT=7878
```
### Service-Specific Configuration
#### Reverse Proxy Setup
**Traefik Configuration:**
```bash
# Traefik automatically configured with labels
# Custom configuration in ~/hops/config/traefik/
# Example dynamic configuration
mkdir -p ~/hops/config/traefik/dynamic
cat > ~/hops/config/traefik/dynamic/middleware.yml << 'EOF'
http:
middlewares:
default-headers:
headers:
frameDeny: true
sslRedirect: true
browserXssFilter: true
contentTypeNosniff: true
forceSTSHeader: true
stsIncludeSubdomains: true
stsPreload: true
stsSeconds: 31536000
EOF
```
**Nginx Proxy Manager:**
- Access admin interface on port 81
- Default credentials: `admin@example.com` / `changeme`
- Configure SSL certificates through web interface
**Caddy Configuration:**
```bash
# Create Caddy configuration directory
mkdir -p ~/hops/config/caddy
# Create custom Caddyfile
cat > ~/hops/config/caddy/Caddyfile << 'EOF'
# Global options
{
email your-email@domain.com
# Optional: Use custom CA
# acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}
# Main domain
yourdomain.com {
reverse_proxy jellyfin:8096
# Custom headers
header {
# Security headers
Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
Referrer-Policy "strict-origin-when-cross-origin"
}
}
# Subdomain routing
sonarr.yourdomain.com {
reverse_proxy sonarr:8989
}
radarr.yourdomain.com {
reverse_proxy radarr:7878
}
# Internal services (authentication required)
portainer.yourdomain.com {
reverse_proxy portainer:9000
# Optional: IP allowlist
@internal {
remote_ip 192.168.1.0/24 10.0.0.0/8
}
handle @internal {
reverse_proxy portainer:9000
}
handle {
respond "Access denied" 403
}
}
EOF
```
#### Authelia Integration
```bash
# Authelia configuration
mkdir -p ~/hops/config/authelia
# Example configuration (simplified)
cat > ~/hops/config/authelia/configuration.yml << 'EOF'
theme: dark
default_redirection_url: https://yourdomain.com
server:
host: 0.0.0.0
port: 9091
log:
level: warn
authentication_backend:
file:
path: /config/users_database.yml
password:
algorithm: argon2id
access_control:
default_policy: deny
rules:
- domain: jellyfin.yourdomain.com
policy: bypass
- domain: "*.yourdomain.com"
policy: one_factor
session:
name: authelia_session
domain: yourdomain.com
regulation:
max_retries: 3
ban_time: 10m
storage:
local:
path: /config/db.sqlite3
notifier:
filesystem:
filename: /config/notification.txt
EOF
```
### Service Management Commands
#### User Operations Script (No Sudo Required)
```bash
# Service status and control
./user-operations status # View all service status
./user-operations status jellyfin # View specific service
./user-operations logs jellyfin # View service logs
./user-operations logs jellyfin -f # Follow logs
# Deployment operations
./user-operations deploy # Deploy all services
./user-operations stop # Stop all services
./user-operations restart # Restart all services
./user-operations restart jellyfin # Restart specific service
# Update operations
./user-operations update # Update all containers
./user-operations update jellyfin # Update specific container
```
#### Direct Docker Compose Commands
```bash
cd ~/hops
# Service management
docker compose ps # List services
docker compose up -d # Start all services
docker compose down # Stop all services
docker compose restart # Restart all services
docker compose restart jellyfin # Restart specific service
# Logs and monitoring
docker compose logs # View all logs
docker compose logs -f jellyfin # Follow specific service logs
docker compose logs --tail=100 sonarr # Last 100 lines
# Updates and maintenance
docker compose pull # Pull new images
docker compose up -d --force-recreate # Recreate containers
docker compose down && docker compose up -d # Full restart
# Resource monitoring
docker stats # Real-time resource usage
docker system df # Disk usage
docker system prune # Clean unused data
```
### New Modular Architecture
HOPS v3.1.0+ introduces a modular library system:
```
hops/
├── lib/ # Shared libraries
│ ├── common.sh # Logging, UI, utilities
│ ├── system.sh # OS detection, requirements
│ ├── docker.sh # Docker operations
│ ├── security.sh # Security functions
│ ├── validation.sh # Input validation
│ ├── secrets.sh # Secret management
│ └── privileges.sh # Privilege separation
├── setup # Main installation wrapper
├── privileged-setup # Root-only operations
├── user-operations # User operations
├── services-improved # Enhanced service definitions
└── hops.sh # Legacy main script
```
#### Using Library Functions
```bash
# Source libraries in custom scripts
source lib/common.sh
source lib/system.sh
# Use logging functions
log "INFO" "Starting custom operation"
warning "This is a warning message"
error_exit "Fatal error occurred"
# Use system functions
detect_os
check_system_requirements 2 10 # 2GB RAM, 10GB disk
# Use validation functions
validate_port "8080"
validate_domain "example.com"
```
## 🔒 Security Features & Hardening
### Automatic Security Hardening
HOPS automatically implements several security measures:
#### Firewall Configuration (Linux)
```bash
# UFW rules automatically created
sudo ufw status
# Manual firewall management
sudo ufw allow 8096/tcp comment "Jellyfin"
sudo ufw allow 9000/tcp comment "Portainer"
sudo ufw delete allow 8096/tcp # Remove rule
```
#### File Permissions
```bash
# Automatic permission hardening
# Secrets: 600 (owner read/write only)
# Configs: 644 (owner write, group/other read)
# Scripts: 755 (executable)
# Manual permission fixes
chmod 600 ~/hops/.env
chmod 644 ~/hops/docker-compose.yml
chmod 755 ~/hops/user-operations
```
#### Network Isolation
```bash
# Docker network isolation
docker network ls | grep homelab
docker network inspect homelab
# View network configuration
docker compose config | grep network
```
### Security Auditing
```bash
# Run security audit
./lib/security.sh audit
# Check for security issues
./lib/security.sh check-passwords
./lib/security.sh check-permissions
./lib/security.sh check-firewall
# Security recommendations
./lib/security.sh recommendations
```
### SSL/TLS Configuration
#### Traefik SSL
Traefik automatically handles SSL certificates with Let's Encrypt:
```bash
# Check certificate status
docker compose logs traefik | grep -i certificate
# Manual certificate renewal (if needed)
docker compose restart traefik
```
#### Custom SSL Certificates
```bash
# For custom certificates, place in:
# ~/hops/config/traefik/certs/
# - yourdomain.com.crt
# - yourdomain.com.key
# Update Traefik configuration to use custom certs
```
## 🆘 Troubleshooting
### Common Issues & Solutions
#### Docker Issues
**Docker Not Starting:**
```bash
# Check Docker status
systemctl status docker
# Restart Docker
sudo systemctl restart docker
# Check Docker logs
journalctl -u docker --since "1 hour ago"
# Reinstall Docker (Linux)
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
newgrp docker
```
**Docker Compose Errors:**
```bash
# Validate compose file
docker compose config
# Check for syntax errors
docker compose config --quiet
# Force recreate containers
docker compose up -d --force-recreate
```
#### Service-Specific Issues
**Service Won't Start:**
```bash
# Check service logs
docker compose logs [service-name]
# Check container status
docker compose ps
# Restart service
docker compose restart [service-name]
# Check port conflicts
sudo lsof -i :[port-number]
```
**Permission Issues:**
```bash
# Fix ownership (Linux)
sudo chown -R $USER:$USER /opt/appdata
sudo chown -R $USER:$USER /mnt/media
sudo chown -R $USER:$USER ~/hops
# Fix ownership (macOS)
sudo chown -R $USER:$USER ~/hops/config
sudo chown -R $USER:$USER ~/hops/media
# Check PUID/PGID values
id $USER # Should match PUID/PGID in .env
```
**Database Issues:**
```bash
# Reset service database (example: Sonarr)
docker compose down sonarr
rm -rf ~/hops/config/sonarr/sonarr.db* # macOS
rm -rf /opt/appdata/sonarr/sonarr.db* # Linux
docker compose up -d sonarr
```
#### Network Issues
**Can't Access Services:**
```bash
# Check if services are running
docker compose ps
# Check port mapping
docker compose port jellyfin 8096
# Check firewall (Linux)
sudo ufw status
# Check local firewall (macOS)
# System Preferences → Security & Privacy → Firewall
# Find container IP
docker inspect jellyfin | grep IPAddress
```
**Reverse Proxy Issues:**
```bash
# Check proxy logs
docker compose logs traefik
docker compose logs nginx-proxy-manager
# Verify DNS resolution
nslookup yourdomain.com
# Check certificate status
openssl s_client -connect yourdomain.com:443 -servername yourdomain.com
```
#### Platform-Specific Issues
**macOS Issues:**
```bash
# Docker Desktop not starting
open /Applications/Docker.app
# Homebrew issues
brew doctor
brew update && brew upgrade
# Fix keychain authentication
security unlock-keychain ~/Library/Keychains/login.keychain
```
**Windows/WSL2 Issues:**
```bash
# WSL2 not starting
wsl --shutdown
wsl -d Ubuntu-22.04
# Docker Desktop WSL2 integration
# Check Docker Desktop → Settings → Resources → WSL Integration
# File permission issues
# Ensure files are in WSL2 filesystem, not /mnt/c/
```
### Log Analysis
#### Log Locations
- **Installation Logs**:
- Linux: `/var/log/hops/`
- macOS: `/usr/local/var/log/hops/`
- Windows: `~/hops/logs/`
- **Service Logs**: `docker compose logs [service-name]`
- **System Logs**: `journalctl -u docker`
#### Log Analysis Commands
```bash
# HOPS installation logs
sudo tail -f /var/log/hops/hops-main-*.log
# Service logs with filtering
docker compose logs jellyfin | grep -i error
docker compose logs --since="1h" sonarr
docker compose logs --tail=100 radarr
# System resource logs
dmesg | grep -i memory
journalctl --since="1 hour ago" | grep docker
```
### Recovery Procedures
#### Service Recovery
```bash
# Stop problematic service
docker compose stop [service-name]
# Remove container (keeps data)
docker compose rm [service-name]
# Recreate service
docker compose up -d [service-name]
# Full service reset (destroys data)
docker compose down [service-name]
rm -rf /path/to/service/config
docker compose up -d [service-name]
```
#### Complete System Recovery
```bash
# Stop all services
docker compose down
# Backup current state
sudo tar -czf hops-backup-$(date +%Y%m%d).tar.gz ~/hops /opt/appdata
# Clean Docker system
docker system prune -a
docker volume prune
# Restart from backup
cd ~/hops
docker compose up -d
# Or reinstall HOPS
sudo ./uninstall
sudo ./setup
```
## 📊 Performance Tuning
### Resource Monitoring
```bash
# Container resource usage
docker stats
# System resource usage
htop
iotop
free -h
df -h
# Service-specific monitoring
docker compose exec portainer sh
# Access Portainer for detailed monitoring
```
### Optimization Strategies
#### For Low-Resource Systems (2-4GB RAM)
```bash
# Recommended minimal stack
# - Jellyfin (media server)
# - qBittorrent (download)
# - Sonarr (TV management)
# - Portainer (monitoring)
# Resource limits in docker-compose.yml
services:
jellyfin:
mem_limit: 1g
cpus: '2'
sonarr:
mem_limit: 512m
cpus: '1'
```
#### For High-Performance Systems (8GB+ RAM)
```bash
# Full stack deployment
# Enable GPU transcoding
# Use multiple download clients
# Add monitoring stack
# GPU support (Linux only)
# Intel GPU passthrough automatically configured
devices:
- /dev/dri:/dev/dri
```
#### Storage Optimization
```bash
# Use SSD for application data
# HDD for media storage
# Separate Docker volumes
# Example optimized storage layout
/opt/appdata -> SSD
/mnt/media -> HDD array
~/hops -> SSD
```
### Update Management
```bash
# Automated updates with Watchtower
# Configure update schedule
WATCHTOWER_SCHEDULE=0 0 2 * * * # 2 AM daily
# Manual update process
docker compose pull
docker compose up -d
# Rollback to previous version
docker compose down
docker tag service:latest service:backup
docker pull service:previous
docker compose up -d
```
## 🔧 Advanced Features
### Custom Service Definitions
```bash
# Add custom services to docker-compose.yml
services:
custom-service:
image: custom/image:latest
container_name: custom-service
environment:
- PUID=${PUID}
- PGID=${PGID}
- TZ=${TZ}
volumes:
- ${CONFIG_ROOT}/custom:/config
- ${DATA_ROOT}:/data
ports:
- "8999:8999"
restart: unless-stopped
networks:
- homelab
```
### Backup Automation
```bash
# Automated backup script
#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/backup/hops"
# Create backup directory
mkdir -p "$BACKUP_DIR"
# Backup configurations
tar -czf "$BACKUP_DIR/config-$DATE.tar.gz" /opt/appdata
# Backup compose files
cp ~/hops/.env ~/hops/docker-compose.yml "$BACKUP_DIR/"
# Backup encrypted secrets
./lib/secrets.sh backup "$BACKUP_DIR/secrets-$DATE.enc"
# Clean old backups (keep 7 days)
find "$BACKUP_DIR" -name "*.tar.gz" -mtime +7 -delete
```
### Development & Testing
```bash
# Development setup
git clone https://github.com/skiercm/hops.git
cd hops
# Syntax validation
bash -n lib/*.sh
bash -n *.sh
# Test service definitions
./services-improved list
./services-improved generate jellyfin
# Test installation in VM/container
# Use test environment before production deployment
```
---
For installation guides, see [INSTALLATION.md](INSTALLATION.md).
For service information, see [SERVICES.md](SERVICES.md).
CHANGELOG.md
+216
View File
@@ -0,0 +1,216 @@
# Changelog
All notable changes to HOPS will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [3.2.0] - 2024-07-18
### Added
- **Caddy Support**: Added Caddy reverse proxy as a service option
- **Enhanced macOS Compatibility**: Comprehensive improvements for macOS installation and operation
- **Docker Desktop Integration**: Improved Docker Desktop startup and management on macOS
- **Keychain Integration**: Proper Docker authentication with macOS keychain on macOS
### Fixed
- **User Directory Fixes**: All directories now use actual user home instead of root on macOS
- **Password Generation**: Resolved `shuf` command and encoding issues on macOS
- **Container Creation**: Fixed Docker Compose working directory and execution context issues
- **Healthcheck Improvements**: Enhanced service health monitoring, particularly for Jellyseerr
- **File Permissions**: Proper ownership of all directories and files across platforms
- **Docker Compose Warnings**: Resolved version warnings and compatibility issues
### Changed
- **macOS File Structure**: Improved directory layout using user home instead of system directories
- **Error Handling**: Enhanced error messages and troubleshooting information for macOS
- **Documentation**: Updated platform-specific installation and configuration guides
### Security
- **Secure Authentication**: Enhanced Docker authentication methods on macOS
- **File Ownership**: Improved file permission management across all platforms
## [3.1.0-beta] - 2024-06-15
### Added
- **Encrypted Secret Management**: All passwords and sensitive data now encrypted with AES-256
- **Input Validation**: Comprehensive validation preventing injection attacks
- **Privilege Separation**: Root operations separated from user operations for enhanced security
- **Pinned Container Versions**: All container images use specific versions, not `latest`
- **Modular Architecture**: Shared code organized in `lib/` directory for better maintainability
- **Cross-Platform Support**: Native support for Linux, macOS, and Windows (WSL2)
- **Enhanced Error Handling**: Better error messages and recovery mechanisms
- **Improved Service Definitions**: Standardized service generation with validation
- **Complete Documentation**: Added `CLAUDE.md` for development guidance
### Changed
- **Installation Methods**: New secure installer `setup` script as recommended method
- **Service Management**: New `user-operations` script for non-privileged service management
- **Architecture**: Modular library system replacing monolithic scripts
- **Security Model**: Clear separation between privileged and user operations
### Security
- **AES-256 Encryption**: All secrets stored encrypted with master key management
- **Input Sanitization**: Comprehensive validation preventing code injection
- **Container Security**: Pinned versions preventing supply chain attacks
- **Privilege Minimization**: Reduced root access requirements
## [3.0.0] - 2024-05-01
### Added
- **Cross-Platform Support**: Full support for Linux, macOS, and Windows (WSL2)
- **Automatic Dependency Installation**: Docker and system requirements installed automatically
- **Platform Detection**: Intelligent OS detection and platform-specific optimizations
- **Enhanced Service Catalog**: Expanded service definitions with health checks
- **Comprehensive Logging**: Detailed logging system for troubleshooting
- **Service Health Monitoring**: Built-in health checks for all services
- **Rollback Capabilities**: Automatic rollback on deployment failure
### Changed
- **Installation Process**: Streamlined installation with better user experience
- **Directory Structure**: Platform-appropriate directory layouts
- **Service Definitions**: Standardized Docker Compose patterns
- **Error Handling**: Improved error messages and recovery procedures
### Fixed
- **Port Conflict Detection**: Better handling of port conflicts
- **Permission Issues**: Improved file permission management
- **Service Dependencies**: Enhanced dependency resolution
## [2.1.0] - 2024-03-15
### Added
- **Huntarr Support**: Missing media discovery and automation
- **Jellystat Support**: Jellyfin statistics and monitoring
- **Watchtower Integration**: Automatic container updates
- **Enhanced Monitoring**: Improved service status monitoring
- **Backup Utilities**: Built-in backup and recovery tools
### Changed
- **Service Management**: Improved start/stop/restart functionality
- **Log Viewing**: Enhanced centralized log viewing
- **Configuration Management**: Better environment variable handling
### Fixed
- **Memory Usage**: Optimized resource usage for low-resource systems
- **Startup Issues**: Resolved service startup race conditions
- **Network Configuration**: Fixed Docker network isolation issues
## [2.0.0] - 2024-02-01
### Added
- **Management Interface**: Comprehensive web-based management
- **Security Hardening**: Automatic firewall configuration and secure passwords
- **Service Templates**: Standardized service definitions
- **Real-time Monitoring**: Live service status and resource monitoring
- **User Interface**: Menu-driven installation and management
### Changed
- **Architecture**: Complete rewrite with modular design
- **Installation**: Simplified one-command installation
- **Configuration**: Centralized configuration management
### Breaking Changes
- **Directory Structure**: New standardized directory layout
- **Configuration Format**: Updated environment variable structure
- **Service Names**: Standardized container and service naming
## [1.2.0] - 2024-01-15
### Added
- **Authelia Support**: Multi-factor authentication and SSO
- **Nginx Proxy Manager**: Alternative reverse proxy option
- **Enhanced SSL**: Automatic SSL certificate management
- **Service Discovery**: Automatic service registration
### Fixed
- **Traefik Configuration**: Improved reverse proxy setup
- **SSL Issues**: Resolved certificate generation problems
- **Network Routing**: Fixed internal service communication
## [1.1.0] - 2023-12-01
### Added
- **Traefik Integration**: Automatic reverse proxy with SSL
- **Service Categories**: Organized services by function
- **Dependency Management**: Automatic service dependency resolution
- **Health Checks**: Service health monitoring and restart
### Changed
- **Service Definitions**: Improved Docker Compose templates
- **Network Configuration**: Enhanced Docker networking
## [1.0.0] - 2023-11-01
### Added
- **Initial Release**: Core HOPS functionality
- **Service Support**: Basic *arr stack, download clients, and media servers
- **Docker Integration**: Docker Compose based deployment
- **Linux Support**: Ubuntu/Debian/Mint support
- **Basic Management**: Simple service management interface
### Features
- **Automated Installation**: One-command deployment
- **Service Selection**: Interactive service selection
- **Basic Security**: Firewall rules and secure passwords
- **Directory Management**: Automatic directory creation and permissions
---
## Version Support
- **v3.2.x**: Current stable release with full platform support
- **v3.1.x**: Beta features, limited support
- **v3.0.x**: Legacy support for critical bugs only
- **v2.x and earlier**: No longer supported
## Upgrade Path
### From v3.1.x to v3.2.0
```bash
# Backup current installation
sudo tar -czf hops-backup-$(date +%Y%m%d).tar.gz ~/hops /opt/appdata
# Pull latest version
cd ~/hops
git pull origin main
# Run upgrade
sudo ./setup --upgrade
```
### From v3.0.x to v3.2.0
```bash
# Major version upgrade requires fresh installation
# Backup data first
sudo ./uninstall --keep-data
sudo ./setup
```
### From v2.x to v3.2.0
```bash
# Migration script available
sudo ./migrate-from-v2.sh
```
## Migration Notes
### v3.2.0 Changes
- **macOS Users**: Directory structure has changed, migration handled automatically
- **Caddy Users**: Manual Caddyfile configuration required
- **Configuration**: Encrypted secrets now default for new installations
### v3.1.0 Changes
- **Security**: All passwords moved to encrypted storage
- **Architecture**: New modular library system
- **Privileges**: Installation process now uses privilege separation
### v3.0.0 Changes
- **Cross-Platform**: New platform detection and configuration
- **Directories**: Platform-specific directory structures
- **Services**: Updated service definitions and health checks
---
For detailed upgrade instructions, see [INSTALLATION.md](INSTALLATION.md).
For breaking changes and migration help, see [ADVANCED.md](ADVANCED.md).
INSTALLATION.md
+365
View File
@@ -0,0 +1,365 @@
# Installation Guide
This guide provides detailed installation instructions for all supported platforms.
## 🔧 System Requirements
### Minimum Requirements
- **RAM**: 2GB (4GB+ recommended)
- **Storage**: 10GB free space (more for media storage)
- **CPU**: 2 cores recommended
- **Network**: Internet connection required
### Platform-Specific Requirements
#### Linux (Ubuntu/Debian/Mint)
- **OS**: Ubuntu 20.04+, Debian 11+, or Linux Mint 20+
- **Architecture**: x86_64
- **Access**: Root/sudo privileges
- **Dependencies**: Automatically installed
#### macOS
- **OS**: macOS 11.0+ (Big Sur)
- **Architecture**: Intel (x86_64) or Apple Silicon (ARM64)
- **Access**: Admin privileges (sudo)
- **Dependencies**: Homebrew and Docker Desktop installed automatically
#### Windows (WSL2)
- **OS**: Windows 10 (build 19041+) or Windows 11
- **WSL**: WSL2 enabled with Ubuntu 20.04+ distribution
- **Docker**: Docker Desktop with WSL2 backend
- **Access**: Admin access for initial setup
- **Architecture**: x86_64 with virtualization support
- **BIOS**: Hyper-V and virtualization enabled
⚠️ **Note**: Windows support has limited testing. Proceed with caution and ensure backups.
## 🐧 Linux Installation
### Quick Install
```bash
# Download HOPS
git clone https://github.com/skiercm/hops.git
cd hops
chmod +x hops install uninstall setup
# Run installation
sudo ./setup
```
### Manual Installation (Advanced)
```bash
# Two-phase installation for enhanced security
sudo ./privileged-setup # Root operations
./user-operations generate <services> # User operations
./user-operations deploy # Deploy services
# Legacy method (still supported)
sudo ./hops.sh
```
### What Gets Installed
- Docker Engine (via official Docker script)
- Docker Compose
- Required system packages
- UFW firewall rules (automatic)
- Service directories and permissions
### Directory Structure
```
~/hops/ # Main deployment directory
├── docker-compose.yml # Service definitions
├── .env # Environment variables
└── logs/ # Application logs
/opt/appdata/ # Application configurations
├── jellyfin/
├── sonarr/
└── [other services]/
/mnt/media/ # Media storage
├── movies/
├── tv/
├── music/
└── downloads/
```
## 🍎 macOS Installation
### Prerequisites Setup
HOPS automatically handles dependency installation on macOS, but you can pre-install if desired:
```bash
# Optional: Install Homebrew (will be done automatically)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Optional: Install Docker Desktop (will be done automatically)
brew install --cask docker
```
### HOPS Installation
```bash
# Download HOPS
git clone https://github.com/skiercm/hops.git
cd hops
chmod +x hops install uninstall setup
# Run installation with admin privileges
sudo ./setup
```
### macOS-Specific Features
- **Docker Desktop Integration**: Automatic installation and startup
- **Keychain Authentication**: Secure Docker authentication
- **User Directory Structure**: All files in user home directory
- **Automatic Dependency Resolution**: Homebrew packages installed as needed
### Directory Structure (macOS)
```
~/hops/ # Main deployment directory
├── docker-compose.yml # Service definitions
├── .env # Environment variables
├── logs/ # Application logs
├── config/ # Application configurations
│ ├── jellyfin/
│ ├── sonarr/
│ └── [other services]/
└── media/ # Media storage
├── movies/
├── tv/
├── music/
└── downloads/
```
### Important Notes for macOS
- **GPU Acceleration**: Not available (Docker containers cannot access macOS GPU)
- **Firewall**: Manual configuration required (automatic UFW setup skipped)
- **File Permissions**: Uses user's home directory for better compatibility
- **Performance**: Excellent performance on both Intel and Apple Silicon
## 🪟 Windows Installation (WSL2)
Windows support uses WSL2 (Windows Subsystem for Linux) for excellent Linux compatibility.
### Step 1: Enable WSL2
```powershell
# Run in PowerShell as Administrator
wsl --install
# Restart computer when prompted
```
### Step 2: Install Ubuntu Distribution
```powershell
# Install Ubuntu 22.04 LTS (recommended)
wsl --install Ubuntu-22.04
# Follow prompts to create username and password
```
### Step 3: Install Docker Desktop
1. Download [Docker Desktop for Windows](https://docs.docker.com/desktop/install/windows-install/)
2. Enable WSL2 integration during installation
3. Ensure "Use WSL 2 based engine" is enabled in Docker settings
4. Enable integration with your Ubuntu distribution
### Step 4: Install HOPS
```bash
# Launch Ubuntu from Start Menu or run: wsl -d Ubuntu-22.04
cd ~
git clone https://github.com/skiercm/hops.git
cd hops
chmod +x hops install uninstall setup
# Run installation (same as Linux)
sudo ./setup
```
### Windows-Specific Considerations
#### File System Performance
- **Always run HOPS from WSL2 filesystem** (`~/hops/`) for optimal performance
- **Avoid Windows drives** (`/mnt/c/`) as they have significant performance penalties
- WSL2 provides 95% of native Linux performance when using WSL2 filesystem
#### Media Access
Your Windows media can be accessed from WSL2:
- `C:\Users\YourName\` → `/mnt/c/Users/YourName/`
- External drives → `/mnt/d/`, `/mnt/e/`, etc.
#### Service Access
- **Web interfaces**: Accessible from Windows browsers using WSL2 IP or localhost
- **File shares**: Available in Windows Explorer via `\\wsl.localhost\Ubuntu-22.04\home\username\hops\`
- **Network**: Services are accessible from both Windows and WSL2
#### Directory Structure (Windows/WSL2)
```
# WSL2 filesystem (recommended location)
~/hops/ # Main deployment directory
├── docker-compose.yml # Service definitions
├── .env # Environment variables
└── logs/ # Application logs
/opt/appdata/ # Application configurations
├── jellyfin/
├── sonarr/
└── [other services]/
/mnt/media/ # Media storage (can link to Windows drives)
├── movies/ -> /mnt/d/Movies/
├── tv/ -> /mnt/d/TV/
├── music/ -> /mnt/d/Music/
└── downloads/
```
## 🔧 Installation Options
### Option 1: Secure Installation (Recommended)
```bash
sudo ./setup
```
- **Best Practice**: Separates privileged and user operations
- **Enhanced Security**: Minimizes root access time
- **User-Friendly**: Guided interactive setup
### Option 2: Manual Two-Phase Installation
```bash
sudo ./privileged-setup # Root-only operations
./user-operations generate [services] # Select and generate services
./user-operations deploy # Deploy services
```
- **Advanced Users**: Full control over each phase
- **Automation**: Can be scripted for multiple deployments
- **Security**: Clear separation of privileged operations
### Option 3: Legacy Installation
```bash
sudo ./hops.sh
```
- **Compatibility**: Original installation method
- **Full-Featured**: Complete management interface
- **Reliability**: Extensively tested method
## 🔍 Post-Installation Verification
### Check Service Status
```bash
# Via HOPS management interface
sudo ./hops.sh
# Select option 4: Service Status
# Via user operations
./user-operations status
# Direct Docker commands
cd ~/hops
docker compose ps
```
### Access Service URLs
After installation, HOPS provides URLs for all services:
```
📱 Access your services at:
● Jellyfin http://192.168.1.100:8096
● Sonarr http://192.168.1.100:8989
● Radarr http://192.168.1.100:7878
● Portainer http://192.168.1.100:9000
```
### Verify File Permissions
```bash
# Check directory ownership
ls -la ~/hops/
ls -la /opt/appdata/ (Linux) or ~/hops/config/ (macOS)
ls -la /mnt/media/ (Linux) or ~/hops/media/ (macOS)
```
## 🆘 Troubleshooting Installation
### Common Issues
#### Docker Installation Failed
```bash
# Check Docker status
systemctl status docker
# Restart Docker
sudo systemctl restart docker
# Reinstall Docker (Linux)
curl -fsSL https://get.docker.com | sh
```
#### Permission Denied Errors
```bash
# Fix directory ownership
sudo chown -R $USER:$USER ~/hops/
sudo chown -R $USER:$USER /opt/appdata/ (Linux)
sudo chown -R $USER:$USER ~/hops/config/ (macOS)
```
#### Port Conflicts
```bash
# Check what's using a port
sudo lsof -i :PORT_NUMBER
# Kill conflicting process
sudo kill -9 PID
```
#### WSL2 Issues (Windows)
```bash
# Restart WSL2
wsl --shutdown
wsl -d Ubuntu-22.04
# Check Docker Desktop WSL2 integration
# Go to Docker Desktop Settings → Resources → WSL Integration
```
### Log Locations
- **Installation Logs**:
- Linux: `/var/log/hops/`
- macOS: `/usr/local/var/log/hops/`
- Windows: `~/hops/logs/`
- **Service Logs**: `docker compose logs [service-name]`
- **System Logs**: `journalctl -u docker`
### Getting Help
1. **Built-in Help**: `sudo ./hops.sh` → Option 7
2. **Check Logs**: Review installation logs for errors
3. **Verify Prerequisites**: Ensure all system requirements are met
4. **Docker Status**: Confirm Docker is running and accessible
5. **GitHub Issues**: Report persistent issues with logs
## 🔄 Backup Strategy
### Before Installation
```bash
# Backup existing media and configs
sudo tar -czf homelab-backup-$(date +%Y%m%d).tar.gz \
/path/to/your/media \
/path/to/your/configs
```
### After Installation
```bash
# Backup HOPS configuration
sudo tar -czf hops-config-backup-$(date +%Y%m%d).tar.gz \
~/hops \
/opt/appdata (Linux) or ~/hops/config (macOS)
```
## 📊 Performance Optimization
### During Installation
- **SSD Storage**: Install on SSD for better performance
- **Sufficient RAM**: Ensure adequate memory for selected services
- **Network Speed**: Faster internet improves Docker image downloads
### After Installation
- **Monitor Resources**: Use Portainer to monitor CPU, RAM, and disk usage
- **Optimize Services**: Start with fewer services, add more gradually
- **Storage Configuration**: Use dedicated drives for media storage
---
For additional help, see [ADVANCED.md](ADVANCED.md) for configuration and troubleshooting details.
README.md
+63 -574
View File
@@ -4,620 +4,109 @@
[![Version](https://img.shields.io/badge/Version-3.2.0-blue.svg)]() [![Version](https://img.shields.io/badge/Version-3.2.0-blue.svg)]()
[![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows-orange.svg)]() [![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows-orange.svg)]()
**HOPS** is a comprehensive, automated deployment solution for popular homelab applications. It simplifies the process of setting up and managing Docker-based services including media servers, download clients, monitoring tools, and more. **HOPS** is a comprehensive automation tool for deploying homelab infrastructure using Docker Compose. Deploy and manage popular homelab services including media servers, download clients, monitoring tools, and more through an intuitive menu-driven interface.
## ⚠️ Important Notices ## ⚠️ Important: Beta Software
**HOPS is beta software**. Always backup your data before installation and test in non-production environments first.
### 🚧 Development Status **Platform Status:**
**HOPS is under active development** and should be considered beta software. While core functionality is stable and tested, new features and improvements are being added regularly. - **Linux**: ✅ Stable and extensively tested
- **macOS**: ✅ Recently improved with v3.2.0 fixes
### 🖥️ Platform Testing Status - **Windows (WSL2)**: ⚠️ Limited testing
- **Linux (Ubuntu/Debian/Mint)**: ✅ Extensively tested and stable
- **macOS**: ✅ Recently improved with comprehensive compatibility fixes
- **Windows (WSL2)**: ⚠️ **Limited testing** - Basic functionality works but may have undiscovered issues
### 💾 Data Safety & Backups
**IMPORTANT**: Always maintain regular backups of your data before using HOPS.
- **Backup your media and configuration files** before installation
- **Test in a non-production environment** first if possible
- **HOPS is not responsible for any data loss** that may occur during installation or operation
- **Docker operations can be destructive** - ensure you understand what services you're installing
### 📋 Recommended Backup Strategy
```bash
# Before running HOPS, backup important directories
sudo tar -czf homelab-backup-$(date +%Y%m%d).tar.gz /path/to/your/media /path/to/your/configs
# After installation, backup HOPS configuration
sudo tar -czf hops-config-backup-$(date +%Y%m%d).tar.gz ~/hops ~/.config/hops
```
## 🆕 What's New in v3.2.0 ## 🆕 What's New in v3.2.0
- **Enhanced macOS Support**: Docker Desktop integration, keychain authentication, user directory fixes
### Major macOS Compatibility Improvements - **Caddy Support**: Added reverse proxy option (user provides configuration)
- **🍎 Enhanced macOS Support**: Comprehensive fixes for macOS installation and operation - **Bug Fixes**: Password generation, container creation, healthchecks, file permissions
- **🔐 Keychain Integration**: Proper Docker authentication with macOS keychain - **Security**: Encrypted secret management, input validation, privilege separation
- **👤 User Directory Fixes**: All directories now use actual user home instead of root
- **🚀 Docker Desktop Integration**: Improved Docker Desktop startup and management
- **⚡ Better Error Handling**: Enhanced error messages and troubleshooting for macOS
### New Features
- **🌐 Caddy Support**: Added Caddy reverse proxy as a service option (configuration not included)
### Bug Fixes
- **🔧 Fixed password generation**: Resolved `shuf` command and encoding issues on macOS
- **🐳 Fixed container creation**: Resolved Docker Compose working directory issues
- **🏥 Fixed healthchecks**: Improved Jellyseerr and other service health monitoring
- **📁 Fixed file permissions**: Proper ownership of config and media directories
### Previous in v3.1.0-beta
- **🔐 Encrypted Secret Management**: All passwords and sensitive data now encrypted with AES-256
- **🛡️ Input Validation**: Comprehensive validation preventing injection attacks
- **⚡ Privilege Separation**: Root operations separated from user operations
- **📌 Pinned Versions**: All container images use specific versions, not `latest`
### New Architecture
- **📚 Modular Libraries**: Shared code organized in `lib/` directory
- **🔧 Enhanced Error Handling**: Better error messages and recovery mechanisms
- **🎯 Improved Service Definitions**: Standardized service generation with validation
- **📖 Documentation**: Complete `CLAUDE.md` for development guidance
- **🍎 Cross-Platform Support**: Native support for Linux, macOS, and Windows (WSL2) with automatic dependency installation
### Installation Methods
- **🚀 New Secure Installer**: `sudo ./setup` - Recommended method
- **⚙️ Manual Installation**: Separate privileged and user operations
- **🔄 Legacy Support**: Original `hops.sh` still fully supported
## 🎯 What is HOPS?
HOPS (Homelab Orchestration Provisioning Script) automates the deployment of a complete homelab infrastructure using Docker Compose. It provides an intuitive menu-driven interface for selecting, configuring, and managing services with enterprise-grade features like:
- **Automated dependency resolution**
- **Security hardening and firewall configuration**
- **Service health monitoring**
- **Rollback capabilities on failure**
- **Comprehensive logging**
- **User-friendly management interface**
## ✨ Key Features ## ✨ Key Features
### 🚀 **Easy Installation** - **🚀 Easy Installation**: One-command setup with automatic Docker installation
- One-command installation process - **🔒 Security First**: Encrypted secrets, firewall configuration, input validation
- Automatic Docker installation and configuration - **📊 Management**: Real-time monitoring, centralized logs, service control
- Interactive service selection - **🔄 Reliability**: Error handling, rollback capabilities, dependency management
- Intelligent dependency resolution - **🌐 Cross-Platform**: Linux, macOS, and Windows (WSL2) support
- **NEW**: Privilege separation for enhanced security
### 🔒 **Security First**
- Automatic firewall configuration
- Secure password generation with encryption
- File permission hardening
- Network isolation
- **NEW**: AES-256 encrypted secret storage
- **NEW**: Comprehensive input validation
- **NEW**: Pinned container versions
### 📊 **Management & Monitoring**
- Real-time service status monitoring
- Centralized log viewing
- Easy service management (start/stop/restart)
- Health checks and service verification
- **NEW**: Modular architecture with shared libraries
### 🔄 **Reliability**
- Error handling with automatic rollback
- Service dependency management
- Port conflict detection
- System requirements validation
- **NEW**: Enhanced error handling with detailed context
## 📱 Supported Services ## 📱 Supported Services
### 📺 Media Management (*arr Stack) **Media Management**: Sonarr, Radarr, Lidarr, Readarr, Bazarr, Prowlarr, Tdarr, Huntarr
- **Sonarr** - TV show management **Download Clients**: qBittorrent, Transmission, NZBGet, SABnzbd
- **Radarr** - Movie management **Media Servers**: Jellyfin, Plex, Emby, Jellystat
- **Lidarr** - Music management **Request Management**: Overseerr, Jellyseerr, Ombi
- **Readarr** - eBook/audiobook management **Reverse Proxy**: Traefik, Nginx Proxy Manager, Caddy, Authelia
- **Bazarr** - Subtitle management **Monitoring**: Portainer, Uptime Kuma, Watchtower
- **Prowlarr** - Indexer management
- **Tdarr** - Media transcoding
- **Huntarr** - Missing media discovery and automation
### ⬇️ Download Clients [View complete service list with support links →](SERVICES.md)
- **qBittorrent** - Feature-rich BitTorrent client
- **Transmission** - Lightweight BitTorrent client
- **NZBGet** - Efficient Usenet downloader
- **SABnzbd** - Popular Usenet client
### 🎞️ Media Servers
- **Jellyfin** - Open-source media server
- **Plex** - Popular media server platform
- **Emby** - Feature-rich media server
- **Jellystat** - Jellyfin statistics and monitoring
### 🎛️ Request Management
- **Overseerr** - Media request management for Plex
- **Jellyseerr** - Media request management for Jellyfin
- **Ombi** - Media request platform
### 🔒 Reverse Proxy & Security
- **Traefik** - Modern reverse proxy with automatic SSL
- **Nginx Proxy Manager** - Easy-to-use reverse proxy
- **Caddy** - Automatic HTTPS web server (*configuration not included*)
- **Authelia** - Authentication and authorization server
### 📈 Monitoring & Management
- **Portainer** - Docker container management
- **Uptime Kuma** - Service monitoring
- **Watchtower** - Automatic container updates
## 🔧 System Requirements ## 🔧 System Requirements
### Minimum Requirements **Minimum**: 2GB RAM, 10GB storage, 2 CPU cores, internet connection
- **OS**:
- **Linux**: Ubuntu 20.04+, Debian 11+, or Linux Mint 20+
- **macOS**: 11.0+ (Big Sur) with Intel or Apple Silicon
- **Windows**: 10/11 with WSL2 (Ubuntu 20.04+ distribution) ⚠️ *Limited testing*
- **RAM**: 2GB (4GB+ recommended)
- **Storage**: 10GB free space (more for media)
- **CPU**: 2 cores recommended
- **Network**: Internet connection required
### Prerequisites **Supported Platforms:**
- **Linux**: Ubuntu 20.04+, Debian 11+, Linux Mint 20+ (x86_64, sudo access)
- **macOS**: 11.0+ Big Sur, Intel/Apple Silicon (admin access, auto-installs Docker Desktop)
- **Windows**: 10/11 with WSL2 + Ubuntu 20.04+ (limited testing, requires Docker Desktop)
**Linux:** [View detailed installation guides →](INSTALLATION.md)
- Root/sudo access
- x86_64 architecture
- Internet connection
**macOS:**
- Admin access (sudo privileges)
- Intel (x86_64) or Apple Silicon (ARM64)
- Internet connection
- Homebrew will be installed automatically if not present
- Docker Desktop will be installed automatically if not present
**Windows:**
- Windows 10 (build 19041+) or Windows 11
- WSL2 enabled with Ubuntu 20.04+ distribution
- Docker Desktop with WSL2 backend
- Admin access to install prerequisites
- x86_64 processor with virtualization support
- Hyper-V and virtualization enabled in BIOS
## 🪟 Windows Installation (WSL2)
HOPS runs on Windows through WSL2 (Windows Subsystem for Linux) with excellent compatibility and performance. This approach leverages the full Linux environment within Windows.
### Prerequisites Setup
**1. Enable WSL2:**
```powershell
# Run in PowerShell as Administrator
wsl --install
# Restart computer when prompted
```
**2. Install Ubuntu Distribution:**
```powershell
# Install Ubuntu 22.04 LTS (recommended)
wsl --install Ubuntu-22.04
# Set up username and password when prompted
```
**3. Install Docker Desktop:**
- Download from [Docker Desktop for Windows](https://docs.docker.com/desktop/install/windows-install/)
- Enable WSL2 integration during installation
- Ensure "Use WSL 2 based engine" is checked in Docker settings
### HOPS Installation on WSL2
⚠️ **Note**: WSL2 support has limited testing. Please proceed with caution and ensure you have backups.
**1. Open WSL2 Terminal:**
```bash
# Launch Ubuntu from Start Menu or run:
wsl -d Ubuntu-22.04
```
**2. Install HOPS (same as Linux):**
```bash
# Clone inside WSL2 filesystem (important for performance)
cd ~
git clone https://github.com/skiercm/hops.git
cd hops
chmod +x hops install uninstall setup
# Run installation
sudo ./setup
```
### ⚠️ Important Notes for Windows Users
**File Location:** Always run HOPS from the WSL2 filesystem (`~/hops/`) for optimal performance. Avoid running from `/mnt/c/` (Windows drives).
**Media Access:** Your Windows media folders can be accessed at:
- `C:\Users\YourName\` → `/mnt/c/Users/YourName/`
- External drives → `/mnt/d/`, `/mnt/e/`, etc.
**Docker Integration:** Services will be accessible from both Windows and WSL2:
- Web interfaces work from Windows browsers
- File shares accessible from Windows Explorer via `\\wsl.localhost\Ubuntu-22.04\home\username\hops\`
**Performance:** WSL2 provides 95% of native Linux performance when files are stored in the WSL2 filesystem.
## 🚀 Quick Start ## 🚀 Quick Start
### 1. Download HOPS
```bash ```bash
# 1. Download HOPS
git clone https://github.com/skiercm/hops.git git clone https://github.com/skiercm/hops.git
cd hops cd hops
chmod +x hops install uninstall setup chmod +x hops install uninstall setup
```
### 2. Run Installation (New Improved Method) # 2. Run installation
```bash
# Option 1: Use the new secure installation wrapper
sudo ./setup sudo ./setup
# Option 2: Manual two-phase installation # 3. Follow interactive setup to select services
sudo ./privileged-setup # Run as root
./user-operations generate <services> # Run as user
./user-operations deploy # Run as user
# Option 3: Legacy installation (still supported) # 4. Access your services
# URLs will be provided after installation
```
**Directory Structure:**
- `~/hops/` - Main deployment (docker-compose.yml, .env, logs)
- `/opt/appdata/` (Linux) or `~/hops/config/` (macOS) - App configs
- `/mnt/media/` (Linux) or `~/hops/media/` (macOS) - Media storage
## 🎛️ Management
```bash
# Access management interface
sudo ./hops.sh sudo ./hops.sh
# Service operations (no sudo required)
./user-operations status # View service status
./user-operations logs <service> # View logs
./user-operations deploy # Deploy services
./user-operations stop # Stop all services
``` ```
### 3. Follow the Interactive Setup [View advanced configuration and troubleshooting →](ADVANCED.md)
- Select your desired services
- Configure directories and timezone
- Choose security options
- Wait for automated deployment
### 4. Access Your Services
The installer will provide URLs for all deployed services:
```
📱 Access your services at:
● Jellyfin http://192.168.1.100:8096
● Sonarr http://192.168.1.100:8989
● Radarr http://192.168.1.100:7878
● Portainer http://192.168.1.100:9000
```
## 📁 Default Directory Structure
```
~/hops/ # Main deployment directory
├── docker-compose.yml # Service definitions
├── .env # Environment variables
└── logs/ # Application logs
/opt/appdata/ # Application configurations
├── jellyfin/
├── sonarr/
├── radarr/
└── ...
/mnt/media/ # Media storage
├── movies/
├── tv/
├── music/
└── downloads/
```
## 🎛️ Management Interface
HOPS includes a comprehensive management interface accessible through the main script:
```bash
sudo ./hops.sh
```
### Available Options:
1. **Install HOPS** - Deploy new services
2. **Uninstall HOPS** - Complete removal with options
3. **Manage Services** - Start/stop/restart services
4. **Service Status** - Real-time service monitoring
5. **Access Information** - Get service URLs and credentials
6. **View Logs** - Centralized log viewing
7. **Help & Documentation** - Built-in help system
## 🔧 Advanced Configuration
### Environment Variables
Configuration is now stored encrypted for enhanced security:
```bash
# NEW: Encrypted secret management
./lib/secrets.sh init # Initialize secret management
./lib/secrets.sh create # Create encrypted environment
./lib/secrets.sh update DOMAIN example.com # Update values
./lib/secrets.sh get PUID # Get values
./lib/secrets.sh list # List all keys
# Legacy: Plaintext configuration in ~/hops/.env
PUID=1000 # User ID
PGID=1000 # Group ID
TZ=America/New_York # Timezone
# Directory Configuration
DATA_ROOT=/mnt/media # Media storage
CONFIG_ROOT=/opt/appdata # App configurations
# Security (now auto-generated and encrypted)
DEFAULT_ADMIN_PASSWORD=... # Generated secure password
DEFAULT_DB_PASSWORD=... # Database password
# Optional: Custom domain
DOMAIN=yourdomain.com
ACME_EMAIL=admin@yourdomain.com
```
### Service-Specific Configuration
#### Caddy Configuration
**Important**: HOPS provides the Caddy container but **does not include Caddyfile configuration**. Users must provide their own Caddyfile.
```bash
# Create Caddy configuration directory
mkdir -p ~/hops/config/caddy
# Create your Caddyfile (example)
cat > ~/hops/config/caddy/Caddyfile << 'EOF'
# Example Caddyfile - customize as needed
example.com {
reverse_proxy jellyfin:8096
}
api.example.com {
reverse_proxy overseerr:5055
}
EOF
# Caddy will automatically handle HTTPS certificates
# Documentation: https://caddyserver.com/docs/
```
**Configuration Scope**: HOPS installs and runs the Caddy container with proper volume mounts, but all routing, SSL, and proxy configuration is the user's responsibility.
### Service Management Commands
```bash
# NEW: User operations script (runs without sudo)
./user-operations status # View service status
./user-operations logs <service> # View service logs
./user-operations deploy # Deploy services
./user-operations stop # Stop all services
# Legacy: Direct Docker Compose commands
cd ~/hops
docker compose ps # View running services
docker compose logs -f [service-name] # View logs
docker compose restart [service-name] # Restart specific service
docker compose pull && docker compose up -d # Update all services
docker compose down # Stop all services
```
### New Architecture
HOPS v3.1.0-beta introduces a modular architecture with shared libraries:
```
hops/
├── lib/ # NEW: Shared libraries
│ ├── common.sh # Logging, UI, utilities
│ ├── system.sh # System validation
│ ├── docker.sh # Docker operations
│ ├── security.sh # Security utilities
│ ├── validation.sh # Input validation
│ ├── secrets.sh # Secret management
│ └── privileges.sh # Privilege management
├── setup # NEW: Installation wrapper
├── privileged-setup # NEW: Root-only operations
├── user-operations # NEW: User operations
├── services-improved # NEW: Enhanced service definitions
└── hops.sh # Legacy main script (still supported)
```
## 🔒 Security Features
### Automatic Security Hardening
- **Firewall Configuration**: Automatic UFW rules for service ports
- **Secure Passwords**: Cryptographically secure password generation
- **File Permissions**: Restrictive permissions on sensitive files
- **Network Isolation**: Docker network segregation
- **SSL/TLS**: Automatic certificate management with Traefik
- **NEW**: AES-256 encrypted secret storage with master key management
- **NEW**: Comprehensive input validation preventing injection attacks
- **NEW**: Privilege separation (root vs user operations)
- **NEW**: Pinned container versions preventing supply chain attacks
### Post-Installation Security
1. **Manage Encrypted Secrets**: Use `./lib/secrets.sh` for secure password management
2. **Configure Reverse Proxy**: Set up Traefik or Nginx Proxy Manager
3. **Enable Authentication**: Configure Authelia for additional security
4. **Regular Updates**: Use Watchtower for automatic updates
5. **Security Auditing**: Use `./lib/security.sh` for security checks
## 🆘 Troubleshooting
### Common Issues
#### Port Conflicts
```bash
# Check for port conflicts
sudo lsof -i :PORT_NUMBER
# View HOPS service status
sudo ./hops.sh
# Select option 4: Service Status
```
#### Service Won't Start
```bash
# Check service logs
cd ~/hops
docker compose logs [service-name]
# Restart service
docker compose restart [service-name]
```
#### Permission Issues
```bash
# Fix ownership of data directories
sudo chown -R $USER:$USER /mnt/media /opt/appdata
```
### Log Locations
- **Installation Logs**: `/var/log/hops/`
- **Service Logs**: `docker compose logs [service-name]`
- **System Logs**: `journalctl -u docker`
### Getting Help
1. Check the built-in help: `sudo ./hops.sh` → Option 7
2. Review logs in `/var/log/hops/`
3. Verify Docker status: `systemctl status docker`
4. Check service health: `docker compose ps`
## 🔄 Backup and Recovery
### Backup Important Data
```bash
# Backup configurations
sudo tar -czf hops-config-backup.tar.gz /opt/appdata
# Backup compose files
cp ~/hops/.env ~/hops/docker-compose.yml /backup/location/
```
### Recovery
```bash
# Restore configurations
sudo tar -xzf hops-config-backup.tar.gz -C /
# Redeploy services
cd ~/hops
docker compose up -d
```
## 📊 Performance Tuning
### For Low-Resource Systems
- Start with fewer services initially
- Monitor resource usage with Portainer
- Consider using lightweight alternatives (Transmission vs qBittorrent)
### For High-Performance Systems
- Enable GPU transcoding in Jellyfin/Plex
- Use SSD storage for application data
- Configure multiple download clients for redundancy
## 🤝 Contributing
We welcome contributions! Please:
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request
### Development Setup
```bash
git clone https://github.com/skiercm/hops.git
cd hops
# Test syntax validation
bash -n lib/*.sh
bash -n *.sh
# Test service definitions
./services-improved list
./services-improved generate jellyfin
# Test new installation method
sudo ./setup
# Test legacy method
sudo ./hops.sh
```
## 📄 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## 🙏 Acknowledgments
- **LinuxServer.io** for excellent Docker images
- **Docker** for containerization platform
- **The Servarr Team** for the *arr applications
- **Jellyfin Project** for the open-source media server
- All the amazing open-source projects that make HOPS possible
## 📞 Support ## 📞 Support
### HOPS Support **HOPS Issues**: [GitHub Issues](https://github.com/skiercm/hops/issues) | [Discussions](https://github.com/skiercm/hops/discussions)
- **Documentation**: Check this README and built-in help
- **Issues**: Report HOPS bugs via [GitHub Issues](https://github.com/skiercm/hops/issues)
- **Community**: Join discussions in [GitHub Discussions](https://github.com/skiercm/hops/discussions)
### Service-Specific Support **Service Issues**: Contact individual service developers (links in [SERVICES.md](SERVICES.md))
**⚠️ Important**: If you encounter issues with a specific service (configuration, features, bugs), please reach out to the respective service developers directly using the links below. HOPS only handles deployment automation - the individual services are maintained by their respective teams. **Contact HOPS for**: Installation/deployment issues, Docker Compose problems, cross-platform issues
**Contact Service Developers for**: Service configuration, features, service-specific bugs
#### 📺 Media Management (*arr Stack) ## 📄 Documentation
- **Sonarr**: [github.com/Sonarr/Sonarr](https://github.com/Sonarr/Sonarr) - TV series management
- **Radarr**: [github.com/Radarr/Radarr](https://github.com/Radarr/Radarr) - Movie collection manager
- **Lidarr**: [github.com/Lidarr/Lidarr](https://github.com/Lidarr/Lidarr) - Music collection manager
- **Readarr**: [github.com/Readarr/Readarr](https://github.com/Readarr/Readarr) - E-book manager ⚠️ *Project retired*
- **Bazarr**: [github.com/morpheus65535/bazarr](https://github.com/morpheus65535/bazarr) - Subtitle management
- **Prowlarr**: [github.com/Prowlarr/Prowlarr](https://github.com/Prowlarr/Prowlarr) - Indexer manager
- **Tdarr**: [github.com/HaveAGitGat/Tdarr](https://github.com/HaveAGitGat/Tdarr) - Media transcoding
- **Huntarr**: [github.com/plexguide/Huntarr.io](https://github.com/plexguide/Huntarr.io) - Missing media discovery
#### ⬇️ Download Clients - [INSTALLATION.md](INSTALLATION.md) - Detailed installation guides for all platforms
- **qBittorrent**: [github.com/qbittorrent/qBittorrent](https://github.com/qbittorrent/qBittorrent) - BitTorrent client - [SERVICES.md](SERVICES.md) - Complete service list with support links
- **Transmission**: [github.com/transmission/transmission](https://github.com/transmission/transmission) - BitTorrent client - [ADVANCED.md](ADVANCED.md) - Configuration, troubleshooting, security
- **NZBGet**: [github.com/nzbget/nzbget](https://github.com/nzbget/nzbget) - Usenet downloader - [CHANGELOG.md](CHANGELOG.md) - Version history and changes
- **SABnzbd**: [github.com/sabnzbd/sabnzbd](https://github.com/sabnzbd/sabnzbd) - Usenet downloader
#### 🎞️ Media Servers ## 📄 License
- **Jellyfin**: [github.com/jellyfin/jellyfin](https://github.com/jellyfin/jellyfin) - Free media server
- **Plex**: [github.com/plexinc/pms-docker](https://github.com/plexinc/pms-docker) - Docker container repo
- **Emby**: [github.com/MediaBrowser/Emby](https://github.com/MediaBrowser/Emby) - Personal media server
#### 🎛️ Request Management MIT License - see [LICENSE](LICENSE) file for details.
- **Overseerr**: [github.com/sct/overseerr](https://github.com/sct/overseerr) - Media requests for Plex
- **Jellyseerr**: [github.com/fallenbagel/jellyseerr](https://github.com/fallenbagel/jellyseerr) - Media requests for Jellyfin/Emby/Plex
- **Ombi**: [github.com/Ombi-app/Ombi](https://github.com/Ombi-app/Ombi) - Media request platform
- **Jellystat**: [github.com/CyferShepard/Jellystat](https://github.com/CyferShepard/Jellystat) - Jellyfin statistics
#### 🔒 Network & Security
- **Traefik**: [github.com/traefik/traefik](https://github.com/traefik/traefik) - Modern reverse proxy
- **Nginx Proxy Manager**: [github.com/NginxProxyManager/nginx-proxy-manager](https://github.com/NginxProxyManager/nginx-proxy-manager) - Nginx proxy management
- **Authelia**: [github.com/authelia/authelia](https://github.com/authelia/authelia) - Authentication & SSO
#### 📈 Monitoring & Management
- **Portainer**: [github.com/portainer/portainer](https://github.com/portainer/portainer) - Container management
- **Watchtower**: [github.com/containrrr/watchtower](https://github.com/containrrr/watchtower) - Automatic updates
- **Uptime Kuma**: [github.com/louislam/uptime-kuma](https://github.com/louislam/uptime-kuma) - Uptime monitoring
### When to Contact HOPS vs Service Developers
**Contact HOPS** for:
- Installation/deployment issues
- Docker Compose generation problems
- Cross-platform compatibility issues
- Script errors or automation failures
**Contact Service Developers** for:
- Service configuration help
- Feature requests for individual services
- Bugs within the service itself
- Service-specific documentation
--- ---
**Made with ❤️ for the homelab community** **Made with ❤️ for the homelab community**
*HOPS - Making homelab deployment simple, secure, and reliable.*
SERVICES.md
+300
View File
@@ -0,0 +1,300 @@
# Supported Services
HOPS supports a comprehensive collection of homelab services across multiple categories. All services use LinuxServer.io containers for consistency and reliability.
## 📺 Media Management (*arr Stack)
### Sonarr - TV Series Management
**Purpose**: Automatic TV show downloading and management
**Default Port**: 8989
**Support**: [github.com/Sonarr/Sonarr](https://github.com/Sonarr/Sonarr)
**Documentation**: [wiki.servarr.com/sonarr](https://wiki.servarr.com/sonarr)
### Radarr - Movie Management
**Purpose**: Automatic movie downloading and management
**Default Port**: 7878
**Support**: [github.com/Radarr/Radarr](https://github.com/Radarr/Radarr)
**Documentation**: [wiki.servarr.com/radarr](https://wiki.servarr.com/radarr)
### Lidarr - Music Management
**Purpose**: Automatic music downloading and management
**Default Port**: 8686
**Support**: [github.com/Lidarr/Lidarr](https://github.com/Lidarr/Lidarr)
**Documentation**: [wiki.servarr.com/lidarr](https://wiki.servarr.com/lidarr)
### Readarr - eBook Management
**Purpose**: Automatic eBook and audiobook downloading
**Default Port**: 8787
**Support**: [github.com/Readarr/Readarr](https://github.com/Readarr/Readarr)
**Documentation**: [wiki.servarr.com/readarr](https://wiki.servarr.com/readarr)
**Status**: ⚠️ Project retired, limited support
### Bazarr - Subtitle Management
**Purpose**: Automatic subtitle downloading for movies and TV
**Default Port**: 6767
**Support**: [github.com/morpheus65535/bazarr](https://github.com/morpheus65535/bazarr)
**Documentation**: [wiki.bazarr.media](https://wiki.bazarr.media)
### Prowlarr - Indexer Management
**Purpose**: Centralized indexer management for *arr applications
**Default Port**: 9696
**Support**: [github.com/Prowlarr/Prowlarr](https://github.com/Prowlarr/Prowlarr)
**Documentation**: [wiki.servarr.com/prowlarr](https://wiki.servarr.com/prowlarr)
### Tdarr - Media Transcoding
**Purpose**: Automated media transcoding and health checking
**Default Port**: 8265
**Support**: [github.com/HaveAGitGat/Tdarr](https://github.com/HaveAGitGat/Tdarr)
**Documentation**: [docs.tdarr.io](https://docs.tdarr.io)
### Huntarr - Missing Media Discovery
**Purpose**: Automated discovery and addition of missing media
**Default Port**: 7879
**Support**: [github.com/plexguide/Huntarr.io](https://github.com/plexguide/Huntarr.io)
**Documentation**: Community-driven
## ⬇️ Download Clients
### qBittorrent - BitTorrent Client
**Purpose**: Feature-rich BitTorrent client with web interface
**Default Port**: 8080
**Support**: [github.com/qbittorrent/qBittorrent](https://github.com/qbittorrent/qBittorrent)
**Documentation**: [github.com/qbittorrent/qBittorrent/wiki](https://github.com/qbittorrent/qBittorrent/wiki)
### Transmission - Lightweight BitTorrent
**Purpose**: Simple, lightweight BitTorrent client
**Default Port**: 9091
**Support**: [github.com/transmission/transmission](https://github.com/transmission/transmission)
**Documentation**: [transmissionbt.com](https://transmissionbt.com)
### NZBGet - Usenet Downloader
**Purpose**: Efficient Usenet binary newsreader
**Default Port**: 6789
**Support**: [github.com/nzbget/nzbget](https://github.com/nzbget/nzbget)
**Documentation**: [nzbget.net/documentation](https://nzbget.net/documentation)
### SABnzbd - Usenet Client
**Purpose**: Popular web-based Usenet client
**Default Port**: 8081
**Support**: [github.com/sabnzbd/sabnzbd](https://github.com/sabnzbd/sabnzbd)
**Documentation**: [sabnzbd.org/wiki](https://sabnzbd.org/wiki)
## 🎞️ Media Servers
### Jellyfin - Open Source Media Server
**Purpose**: Free, open-source media server and entertainment hub
**Default Port**: 8096
**Support**: [github.com/jellyfin/jellyfin](https://github.com/jellyfin/jellyfin)
**Documentation**: [jellyfin.org/docs](https://jellyfin.org/docs)
**Features**: No licensing fees, privacy-focused, extensive format support
### Plex - Media Server Platform
**Purpose**: Popular media server with premium features
**Default Port**: 32400
**Support**: [github.com/plexinc/pms-docker](https://github.com/plexinc/pms-docker)
**Documentation**: [support.plex.tv](https://support.plex.tv)
**Features**: Remote access, premium features with Plex Pass
### Emby - Personal Media Server
**Purpose**: Feature-rich personal media server
**Default Port**: 8097
**Support**: [github.com/MediaBrowser/Emby](https://github.com/MediaBrowser/Emby)
**Documentation**: [emby.media/support](https://emby.media/support)
**Features**: Premium features available, family-friendly
### Jellystat - Jellyfin Statistics
**Purpose**: Advanced statistics and monitoring for Jellyfin
**Default Port**: 3000
**Support**: [github.com/CyferShepard/Jellystat](https://github.com/CyferShepard/Jellystat)
**Documentation**: GitHub repository
**Requirements**: Requires Jellyfin server
## 🎛️ Request Management
### Overseerr - Plex Request Management
**Purpose**: Media discovery and request management for Plex
**Default Port**: 5055
**Support**: [github.com/sct/overseerr](https://github.com/sct/overseerr)
**Documentation**: [docs.overseerr.dev](https://docs.overseerr.dev)
**Integration**: Plex, Sonarr, Radarr
### Jellyseerr - Jellyfin Request Management
**Purpose**: Media requests for Jellyfin, Emby, and Plex
**Default Port**: 5056
**Support**: [github.com/fallenbagel/jellyseerr](https://github.com/fallenbagel/jellyseerr)
**Documentation**: [docs.jellyseerr.dev](https://docs.jellyseerr.dev)
**Integration**: Jellyfin, Emby, Plex, Sonarr, Radarr
### Ombi - Media Request Platform
**Purpose**: User-friendly media request and discovery platform
**Default Port**: 3579
**Support**: [github.com/Ombi-app/Ombi](https://github.com/Ombi-app/Ombi)
**Documentation**: [docs.ombi.app](https://docs.ombi.app)
**Integration**: Plex, Emby, Jellyfin
## 🔒 Reverse Proxy & Security
### Traefik - Modern Reverse Proxy
**Purpose**: Automatic reverse proxy with SSL certificate management
**Default Ports**: 80, 443, 8080 (dashboard)
**Support**: [github.com/traefik/traefik](https://github.com/traefik/traefik)
**Documentation**: [doc.traefik.io/traefik](https://doc.traefik.io/traefik)
**Features**: Automatic SSL, service discovery, load balancing
### Nginx Proxy Manager - Easy Reverse Proxy
**Purpose**: User-friendly web interface for Nginx reverse proxy
**Default Ports**: 80, 443, 81 (admin)
**Support**: [github.com/NginxProxyManager/nginx-proxy-manager](https://github.com/NginxProxyManager/nginx-proxy-manager)
**Documentation**: [nginxproxymanager.com/guide](https://nginxproxymanager.com/guide)
**Features**: Web GUI, SSL automation, access lists
### Caddy - Automatic HTTPS Web Server
**Purpose**: Modern web server with automatic HTTPS
**Default Ports**: 80, 443, 2019 (admin)
**Support**: [github.com/caddyserver/caddy](https://github.com/caddyserver/caddy)
**Documentation**: [caddyserver.com/docs](https://caddyserver.com/docs)
**Note**: ⚠️ **Configuration not included** - Users must provide their own Caddyfile
### Authelia - Authentication & Authorization
**Purpose**: Multi-factor authentication and single sign-on
**Default Port**: 9091
**Support**: [github.com/authelia/authelia](https://github.com/authelia/authelia)
**Documentation**: [authelia.com/integration](https://www.authelia.com/integration)
**Features**: 2FA, LDAP, OAuth2, OIDC
## 📈 Monitoring & Management
### Portainer - Container Management
**Purpose**: Web-based Docker container management interface
**Default Port**: 9000
**Support**: [github.com/portainer/portainer](https://github.com/portainer/portainer)
**Documentation**: [docs.portainer.io](https://docs.portainer.io)
**Features**: Container management, stack deployment, monitoring
### Uptime Kuma - Service Monitoring
**Purpose**: Self-hosted uptime monitoring tool
**Default Port**: 3001
**Support**: [github.com/louislam/uptime-kuma](https://github.com/louislam/uptime-kuma)
**Documentation**: [github.com/louislam/uptime-kuma/wiki](https://github.com/louislam/uptime-kuma/wiki)
**Features**: Multiple protocols, notifications, status pages
### Watchtower - Automatic Updates
**Purpose**: Automatic Docker container updating
**Default Port**: None (background service)
**Support**: [github.com/containrrr/watchtower](https://github.com/containrrr/watchtower)
**Documentation**: [containrrr.dev/watchtower](https://containrrr.dev/watchtower)
**Features**: Scheduled updates, notifications, selective updating
## 🔧 Service Dependencies
### Common Dependencies
Most services depend on:
- **Docker** and **Docker Compose**
- **Shared network** (`homelab` network)
- **Volume mounts** for configuration and data
- **Environment variables** (PUID, PGID, TZ)
### Integration Patterns
#### Media Management Workflow
1. **Prowlarr** → Manages indexers for all *arr services
2. **Sonarr/Radarr** → Monitors for new releases
3. **Download Client** (qBittorrent/Transmission) → Downloads content
4. **Media Server** (Jellyfin/Plex) → Serves content to users
5. **Request System** (Overseerr/Jellyseerr) → User requests interface
#### Security & Access
1. **Reverse Proxy** (Traefik/Nginx) → External access with SSL
2. **Authelia** → Authentication layer
3. **Firewall** → Network security (UFW on Linux)
#### Monitoring Stack
1. **Portainer** → Container management
2. **Uptime Kuma** → Service monitoring
3. **Watchtower** → Automatic updates
## ⚠️ Important Service Notes
### Caddy Configuration
HOPS provides the Caddy container but **does not include Caddyfile configuration**. Users must:
1. Create their own Caddyfile in `~/hops/config/caddy/`
2. Configure reverse proxy rules
3. Set up SSL certificates (automatic with proper domain configuration)
**Example minimal Caddyfile:**
```
example.com {
reverse_proxy jellyfin:8096
}
radarr.example.com {
reverse_proxy radarr:7878
}
```
### GPU Support
- **Linux**: Intel GPU support via `/dev/dri` passthrough
- **macOS**: No GPU acceleration available
- **Windows**: Limited GPU support in WSL2
### Service Health Checks
All web-based services include health checks for:
- Service startup verification
- Automatic restart on failure
- Status monitoring integration
## 🆘 Service-Specific Support
### When to Contact Service Developers
**Contact individual service developers for:**
- Service configuration help
- Feature requests
- Bugs within the service itself
- Service-specific documentation
- Advanced service setup
**Contact HOPS for:**
- Docker Compose generation issues
- Service deployment problems
- Cross-platform compatibility
- Installation and automation issues
### Getting Service Help
1. **Check service documentation** (links provided above)
2. **Review service GitHub issues** for known problems
3. **Check service community forums** (Reddit, Discord, etc.)
4. **Consult LinuxServer.io documentation** for container-specific issues
5. **Submit issues to appropriate repositories** with proper logs and details
### Common Service Issues
#### Permission Problems
```bash
# Fix ownership for Linux
sudo chown -R $USER:$USER /opt/appdata/[service-name]
# Fix ownership for macOS
sudo chown -R $USER:$USER ~/hops/config/[service-name]
```
#### Service Won't Start
```bash
# Check service logs
docker compose logs [service-name]
# Restart service
docker compose restart [service-name]
```
#### Configuration Issues
Most services store configuration in:
- **Linux**: `/opt/appdata/[service-name]/`
- **macOS**: `~/hops/config/[service-name]/`
- **Windows**: `/opt/appdata/[service-name]/` (in WSL2)
---
For installation help, see [INSTALLATION.md](INSTALLATION.md).
For advanced configuration, see [ADVANCED.md](ADVANCED.md).