Deployment Overview of DeepSeek-R1:14B on Server¶

Prerequisites and Basic Requirements¶

To ensure a successful deployment, the server must meet the following requirements:

• Operating System: Ubuntu (recommended).

• Privileges: Root or sudo access is required for package installation and service management.

• Hardware Support: NVIDIA GPU support is required for CUDA acceleration.

• Network/Ports:

• Port 80 (HTTP): Used for ACME certificate challenges and redirection to HTTPS.

• Port 443 (HTTPS): The primary access port for the web interface.

• Port 11434: Local communication between Open WebUI and Ollama.

FQDN of the final panel on the hostkey.in domain¶

The application is accessible via a specific subdomain template based on your server ID.

Application installation process¶

The deployment follows a multi-stage installation method involving system packages, binary scripts, and container orchestration:

1. System Dependencies: The system is updated, and essential utilities such as curl and ca-certificates are installed via the package manager.

2. Ollama Installation: The official Ollama installation script is executed to install the backend engine. A dedicated ollama system user is created for security.

3. Systemd Configuration: An override configuration is applied to the ollama service to allow remote connections (0.0.0.0) and enable Flash Attention (LLAMA_FLASH_ATTENTION=1).

4. Model Retrieval: The deepseek-r1:14b model is automatically pulled into the Ollama local storage.

5. Container Deployment: The Open WebUI interface is deployed as a Docker container with GPU support enabled.

6. Reverse Proxy Setup: An Nginx instance is configured via Docker Compose to handle SSL termination and proxy traffic to the web interface.

Docker Containers and Their Deployment¶

The deployment utilizes two primary containerized services:

Open WebUI¶

• Image Name: ghcr.io/open-webui/open-webui:cuda

• Ports: Uses host networking (network_mode: host).

• Volumes: open-webui:/app/backend/data (Persistent data storage).

• Environment Variables:

• ENV: dev

• OLLAMA_BASE_URLS: http://127.0.0.1:11434

• Restart Policy: always

• Hardware Access: Direct access to NVIDIA GPU capabilities.

Nginx (Proxy)¶

• Image Name: jonasal/nginx-certbot:latest

• Ports: Uses host networking (network_mode: host).

• Volumes:

• nginx_secrets:/etc/letsencrypt (SSL certificates).

• /data/nginx/user_conf.d:/etc/nginx/user_conf.d (Custom configuration files).

• Restart Policy: unless-stopped

Custom Scripts and Additional Setup¶

The installation performs several non-standard configuration steps to ensure the environment is optimized for LLM workloads:

• Ollama Service Override: A custom systemd drop-in file is created at /etc/systemd/system/ollama.service.d/override.conf to configure environmental variables required for web interface communication and performance optimization.

• Nginx Configuration Injection: The deployment dynamically modifies Nginx configuration files in /data/nginx/user_conf.d/ to set up proxy_pass directives, ensuring traffic is correctly routed from the HTTPS port to the local Open WebUI instance on port 8080.

• ACME Webroot Setup: A specific directory structure (/.well-known/acme-challenge/) is manually initialized inside the Nginx container to facilitate automated SSL certificate renewal.

Application Update Instructions¶

To update the main application (Open WebUI), perform the following steps:

1. Navigate to the directory containing your Docker configurations.

2. Execute the following command to pull the latest image and restart the service:

   docker compose pull && docker compose up -d
   

Location of configuration files and data¶

• Nginx Configurations: /data/nginx/user_conf.d/

• Nginx Compose File: /root/nginx/compose.yml

• Ollama Service Overrides: /etc/systemd/system/ollama.service.d/

• Open WebUI Data Volume: Managed via Docker volumes (check with docker volume ls).

Available ports for connection¶

• HTTPS Access: 443

• HTTP Redirect: 80

Proxy Servers¶

The application uses an Nginx container acting as a reverse proxy. It handles:

• SSL/TLS termination via Certbot.

• Automatic redirection from HTTP to HTTPS.

• Proxying requests to the Open WebUI service running on http://127.0.0.1:8080.