Deployment Overview of DeepSeek-R1:70B on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a Linux server running Ubuntu with root privileges. The following system components must be present:

• Operating System: Ubuntu (version compatible with apt package manager)

• Privileges: Root access (sudo or direct root shell)

• Required Packages: curl, ca-certificates

• Network: Access to the internet for downloading images and models

• Hardware: GPU support is required for the open-webui container to utilize CUDA capabilities

FQDN of the Final Panel¶

The application is accessible via the following Fully Qualified Domain Name (FQDN) format:

• deepseek<Server ID>.hostkey.in:443

Replace <Server ID> with the specific identifier assigned to the server instance. The service listens on port 443 for HTTPS traffic.

File and Directory Structure¶

The deployment utilizes the following directory structure for configuration, data, and certificates:

• /root/nginx/: Contains the Docker Compose configuration for the proxy and Certbot.

• /root/nginx/compose.yml: The Docker Compose file defining the Nginx and Certbot services.

• /data/nginx/user_conf.d/: Directory containing custom Nginx virtual host configurations.

• deepseek<Server ID>.hostkey.in.conf: HTTPS virtual host configuration.

• deepseek<Server ID>.hostkey.in.http.conf: HTTP virtual host configuration for ACME challenges.

• /data/nginx/nginx-certbot.env: Environment file for the Nginx-Certbot container.

• /var/www/certbot/: Webroot directory inside the Nginx container for Let's Encrypt ACME challenges.

• /etc/systemd/system/ollama.service: Systemd service file for Ollama.

• /etc/systemd/system/ollama.service.d/override.conf: Systemd drop-in configuration for Ollama environment variables.

Application Installation Process¶

The application stack consists of Ollama for model inference and Open WebUI for the user interface.

1. Ollama Installation:

2. The Ollama service is installed via the official installation script.

3. The deepseek-r1:70b model is pulled and stored locally.

4. The service is configured to listen on 0.0.0.0 and allow all origins.

5. Open WebUI Deployment:

6. The open-webui container is deployed using the ghcr.io/open-webui/open-webui:cuda image.

7. The container runs in host network mode.

8. It is configured to connect to the local Ollama instance at http://127.0.0.1:11434.

9. GPU resources are requested via device capabilities.

Access Rights and Security¶

• User Accounts: A system user named ollama is created to manage the Ollama service.

• Firewall: The Nginx proxy handles external traffic on port 443 (HTTPS) and port 80 (HTTP for redirection).

• Internal Communication:

• Open WebUI communicates with Ollama via 127.0.0.1:11434.

• Nginx proxies traffic to Open WebUI via 127.0.0.1:8080.

• SSL/TLS: SSL certificates are managed automatically by Certbot within the Nginx container.

Databases¶

• Storage: Open WebUI stores its data in a Docker volume named open-webui mounted at /app/backend/data inside the container.

• Connection: No external database connection is configured; the application uses local storage within the container volume.

Docker Containers and Their Deployment¶

Two primary Docker components are deployed:

1. Nginx-Certbot:

2. Image: jonasal/nginx-certbot:latest

3. Network Mode: host

4. Volumes:

   • nginx_secrets (external) mounted at /etc/letsencrypt

   • /data/nginx/user_conf.d mounted at /etc/nginx/user_conf.d

5. Environment: Configured via /data/nginx/nginx-certbot.env and [email protected].

6. Open WebUI:

7. Image: ghcr.io/open-webui/open-webui:cuda

8. Name: open-webui

9. Network Mode: host

10. Restart Policy: always

11. Volumes: open-webui mounted at /app/backend/data

12. Environment Variables:

    • ENV: dev

    • OLLAMA_BASE_URLS: http://127.0.0.1:11434

13. Device Requests: GPU capabilities enabled.

Proxy Servers¶

Nginx acts as the reverse proxy and SSL terminator for the application.

• Configuration Location: /data/nginx/user_conf.d/deepseek<Server ID>.hostkey.in.conf

• HTTPS Configuration:

• Listens on port 443.

• Proxies requests to http://127.0.0.1:8080.

• Sets the Host header to $host.

• HTTP Configuration:

• Listens on port 80.

• Handles ACME challenges at /.well-known/acme-challenge/.

• Redirects all other traffic to HTTPS (301 redirect).

• Certificate Management: Certbot runs within the Nginx container to issue and renew certificates automatically.

Permission Settings¶

• Nginx Directory: /root/nginx is owned by root:root with mode 0755.

• Compose File: /root/nginx/compose.yml is owned by root:root with mode 0644.

• Nginx Config Files: Files in /data/nginx/user_conf.d/ are owned by root:root with mode 0644.

• Systemd Override: /etc/systemd/system/ollama.service.d/override.conf is owned by root:root.

Location of Configuration Files and Data¶

Available Ports for Connection¶

• Port 80: HTTP (Redirects to HTTPS, handles ACME challenges).

• Port 443: HTTPS (Main entry point for the application).

• Port 11434: Ollama API (Internal only, bound to 127.0.0.1).

• Port 8080: Open WebUI (Internal only, bound to 127.0.0.1).

Starting, Stopping, and Updating¶

• Ollama Service:

• Start: systemctl start ollama

• Stop: systemctl stop ollama

• Restart: systemctl restart ollama

• Reload Daemon: systemctl daemon-reload (required after modifying systemd overrides)

• Nginx-Certbot Container:

• Start/Restart: docker compose up -d (executed from /root/nginx)

• Reload Nginx: docker exec nginx-nginx-1 nginx -s reload

• Test Configuration: docker exec nginx-nginx-1 nginx -t

• Open WebUI Container:

• Start/Restart: docker restart open-webui

• Stop: docker stop open-webui

• Update Image: docker pull ghcr.io/open-webui/open-webui:cuda followed by docker restart open-webui