Deployment Overview of DeepSeek-R1:70B on Server¶

Prerequisites and Basic Requirements¶

To ensure the successful deployment and operation of the application, the following requirements must be met:

• Operating System: Ubuntu (recommended).

• Privileges: Root or sudo access is required for all installation steps.

• Hardware Acceleration: NVIDIA GPU with CUDA support is required for hardware acceleration in Docker containers.

• Ports:

• 80 (HTTP): Used for ACME challenges and redirection to HTTPS.

• 443 (HTTPS): Primary access port for the web interface.

• 11434: Internal Ollama API service.

• 8080: Localhost communication between Nginx and Open WebUI.

FQDN of the final panel on the hostkey.in domain¶

The application is accessible via a specific subdomain template configured within the environment:

Application installation process¶

The installation follows a multi-stage process involving system package management, model deployment, and container orchestration:

1. System Dependencies: The system is updated to include curl and ca-certificates.

2. Ollama Installation:

3. The Ollama service is installed via the official shell script (https://ollama.com/install.sh).

4. A dedicated ollama system user is created.

5. Systemd configuration is modified to allow remote connections (OLLAMA_HOST=0.0.0.0) and enable Flash Attention (LLAMA_FLASH_ATTENTION=1).

6. Model Deployment: The deepseek-r1:70b model is pulled directly into the Ollama service.

7. Web Interface Deployment: A Docker container for Open WebUI is deployed using the CUDA-enabled image to provide a graphical interface for the models.

8. Reverse Proxy Setup: An Nginx instance is configured via Docker Compose to handle SSL termination and proxy traffic from port 443 to the local web interface on port 8080.

Access Rights and Security¶

• Firewall: Only ports 80 and 443 are exposed externally for web access.

• Service Isolation: The Ollama service is configured with OLLAMA_ORIGINS=* to allow communication from the Open WebUI container running on the host network.

• SSL/TLS: SSL certificates are managed via Certbot within a dedicated Nginx container, ensuring all traffic is encrypted.

Databases¶

The application uses Docker volumes for persistent data storage:

• open-webui: Stores application data and user information for the web interface.

• nginx_secrets: Manages SSL/TLS certificates.

Docker Containers and Their Deployment¶

The deployment utilizes two primary containers to manage the web interface and the secure proxy layer.

Open WebUI Container¶

• Image: ghcr.io/open-webui/open-webui:cuda

• Network Mode: host

• Environment Variables:

• ENV: dev

• OLLAMA_BASE_URLS: http://127.0.0.1:11434

• Volumes: open-webui:/app/backend/data

• Restart Policy: always

• Hardware Access: Direct access to the host GPU is enabled via device requests.

Nginx Proxy Container¶

• Image: jonasal/nginx-certbot:latest

• Network Mode: host

• Environment Variables:

• CERTBOT_EMAIL: [email protected]

• Volumes:

• nginx_secrets:/etc/letsencrypt

• /data/nginx/user_conf.d:/etc/nginx/user_conf.d

• Restart Policy: unless-stopped

Custom Scripts and Additional Setup¶

The deployment performs several non-standard configuration steps to ensure seamless integration:

• Systemd Overrides: A custom override file is created at /etc/systemd/system/ollama.service.d/override.conf to configure environment variables for the Ollama service.

• Nginx Configuration Injection: The deployment modifies Nginx virtual host files in /data/nginx/user_conf.d/ to redirect HTTPS traffic specifically to http://127.0.0.1:8080.

• ACME Webroot Setup: A directory structure is created inside the running Nginx container (/var/www/certbot/.well-known/acme-challenge) to facilitate SSL certificate renewal.

Application Update Instructions¶

To update the main application (Open WebUI), perform the following steps:

1. Pull the latest image:

   docker pull ghcr.io/open-webui/open-webui:cuda
   

2. Restart the container to apply changes:

   docker compose up -d
   

   (Note: This assumes the deployment directory for the web interface is used; otherwise, use docker restart open-webui)

Location of configuration files and data¶

Available ports for connection¶

• HTTPS: 443 (via the configured FQDN).

• HTTP: 80 (for redirection and ACME challenges).

Starting and Stopping the application¶

Open WebUI¶

• Start/Restart: The container is set to always restart. To manually restart:

  docker restart open-webui
  

Nginx Proxy¶

• Reload Configuration:

  docker exec nginx-nginx-1 nginx -s reload
  

• Restart Service:

  cd /root/nginx && docker compose restart