Deployment Overview of DeepSeek-R1:14B on Server¶

Prerequisites and Basic Requirements¶

To ensure the successful deployment of the application, the following requirements must be met:

• Operating System: Ubuntu (recommended).

• Privileges: Root or sudo access is required for package installation and service management.

• Hardware Acceleration: NVIDIA GPU support is required for CUDA-enabled container execution.

• Ports:

• 80/tcp: For HTTP traffic and ACME challenges (SSL).

• 443/tcp: For secure HTTPS traffic.

• 11434/tcp: Internal Ollama service communication.

• 8080/tcp: Local upstream connection for Open WebUI.

FQDN of the final panel on the hostkey.in domain¶

The application is accessible via a specific subdomain template:

Application installation process¶

The installation follows a multi-stage process involving system package management, model preparation, and container orchestration:

1. System Dependencies: The system is updated to include essential utilities such as curl and ca-certificates.

2. Ollama Installation:

3. The official Ollama installation script is executed via curl.

4. A dedicated ollama system user is created.

5. Systemd service overrides are configured to allow remote access (OLLAMA_HOST=0.0.0.0) and enable Flash Attention (LLAMA_FLASH_ATTENTION=1).

6. Model Download: The deepseek-r1:14b model is pulled directly into the Ollama local storage.

7. Container Deployment: The Open WebUI interface is deployed as a Docker container using the CUDA-optimized image.

Access Rights and Security¶

• Firewall/Network: The application utilizes network_mode: host for the primary web interface to facilitate direct communication with local services.

• SSL/TLS: SSL certificates are managed via Certbot within an Nginx environment, ensuring all external traffic is encrypted over port 443.

• Service Isolation: A systemd override is applied to Ollama to manage its execution environment and security parameters.

Databases¶

The application uses a Docker volume for persistent data storage:

• Volume Name: open-webui

• Mount Path: /app/backend/data inside the container.

Docker Containers and Their Deployment¶

The deployment consists of two primary containerized components:

Open WebUI¶

• Image: ghcr.io/open-webui/open-webui:cuda

• Ports: Uses host network mode (access via standard web ports).

• Environment Variables:

• ENV: dev

• OLLAMA_BASE_URLS: http://127.0.0.1:11434

• Volumes: open-webui:/app/backend/data

• Restart Policy: always

• Hardware Access: GPU capabilities are enabled for hardware acceleration.

Nginx Proxy (Certbot)¶

• Image: jonasal/nginx-certbot:latest

• Ports: Uses host network mode.

• Environment Variables:

• CERTBOT_EMAIL: [email protected]

• Volumes:

• nginx_secrets:/etc/letsencrypt

• /data/nginx/user_conf.d:/etc/nginx/user_conf.d

• Restart Policy: unless-stopped

Custom Scripts and Additional Setup¶

The following configuration steps are performed during the setup:

• Nginx Configuration Generation: A custom Nginx configuration is generated to proxy HTTPS traffic from port 443 to the local Open WebUI service running on port 8080.

• ACME Challenge Setup: An HTTP server block is configured on port 80 specifically to handle .well-known/acme-challenge requests for automated SSL certificate renewal.

• Systemd Configuration: Custom override files are placed in /etc/systemd/system/ollama.service.d/override.conf to configure environment variables for the Ollama service.

Application Update Instructions¶

To update the main application interface, execute the following commands:

docker compose -f /root/nginx/compose.yml pull && docker compose -f /root/nginx/compose.yml up -d

Note: To update the underlying AI model, use the Ollama CLI:

ollama pull deepseek-r1:14b

Location of configuration files and data¶

Available ports for connection¶

• HTTPS: 443 (via Nginx proxy)

• HTTP: 80 (for SSL redirection and ACME)

Starting and Stopping the application¶

Open WebUI¶

The container is configured to restart automatically. To manage it manually:

docker stop open-webui
docker start open-webui

Nginx Proxy¶

To reload the proxy configuration after changes:

docker exec nginx-nginx-1 nginx -s reload