Deployment Overview of Open WebUI with DeepSeek-R1:14B on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a server running Ubuntu with the following specifications: - Operating System: Ubuntu (compatible with apt package manager). - Privileges: Root access or sudo privileges are required for system configuration and service management. - Network: The server must have access to the internet to download Docker images, Ollama binaries, and the DeepSeek-R1:14B model. - Ports: - Port 80: Required for HTTP traffic and ACME challenge validation. - Port 443: Required for HTTPS traffic. - Port 8080: Used internally by Open WebUI. - Port 11434: Used internally by the Ollama service.

File and Directory Structure¶

The application utilizes the following directory structure for configuration and data storage: - /root/nginx: Contains the Docker Compose configuration for the proxy server. - /root/nginx/compose.yml: The Docker Compose file defining the Nginx and Certbot services. - /data/nginx/user_conf.d: Directory containing custom Nginx virtual host configurations. - /data/nginx/nginx-certbot.env: Environment file for Nginx Certbot settings. - /etc/systemd/system/ollama.service: The main systemd unit file for Ollama. - /etc/systemd/system/ollama.service.d/override.conf: Systemd drop-in configuration for Ollama environment variables. - /var/www/certbot: Webroot directory inside the Nginx container used for ACME challenge validation.

Application Installation Process¶

The installation involves setting up the Ollama backend, pulling the specific AI model, and deploying the Open WebUI frontend via Docker.

Install System Prerequisites: The system ensures the presence of curl and ca-certificates packages.
Install Ollama: The Ollama service is installed using the official installation script:
```
curl -fsSL https://ollama.com/install.sh | sh
```
This creates the ollama system user and installs the binary to /usr/local/bin/ollama.
Configure Ollama Service: A systemd override is created at /etc/systemd/system/ollama.service.d/override.conf to set the following environment variables:
- OLLAMA_HOST=0.0.0.0
- OLLAMA_ORIGINS=*
- LLAMA_FLASH_ATTENTION=1
Download the AI Model: The DeepSeek-R1:14B model is pulled into the Ollama registry:
```
ollama pull deepseek-r1:14b
```
Deploy Open WebUI: The Open WebUI application is deployed as a Docker container using the ghcr.io/open-webui/open-webui:cuda image. The container is configured with:
- Network mode: host
- Environment variable ENV set to dev.
- Environment variable OLLAMA_BASE_URLS set to http://127.0.0.1:11434.
- A named volume open-webui mounted to /app/backend/data for data persistence.
- GPU device requests enabled for hardware acceleration.

Docker Containers and Their Deployment¶

Two primary Docker containers are managed in this deployment:

Open WebUI Container:
- Name: open-webui
- Image: ghcr.io/open-webui/open-webui:cuda
- Restart Policy: always
- Network Mode: host
- Volumes: open-webui:/app/backend/data
- Device Requests: GPU capabilities are requested for inference acceleration.
Nginx and Certbot Container:
- Image: jonasal/nginx-certbot:latest
- Restart Policy: unless-stopped
- Network Mode: host
- Volumes:
- nginx_secrets (external) mounted to /etc/letsencrypt.
- /data/nginx/user_conf.d mounted to /etc/nginx/user_conf.d.
- Environment:
- CERTBOT_EMAIL is set to [email protected].
- Additional settings are loaded from /data/nginx/nginx-certbot.env.

Proxy Servers¶

The deployment utilizes an Nginx container with Certbot integration to handle SSL termination and reverse proxying.

Configuration Location: The Nginx configuration is managed via Docker Compose located at /root/nginx/compose.yml.
Virtual Host Configuration:
HTTPS Configuration: Located in /data/nginx/user_conf.d/<prefix><server_id>.hostkey.in.conf.
- The proxy_pass directive is set to http://127.0.0.1:8080 to forward traffic to the Open WebUI container.
- The Host header is preserved using proxy_set_header Host $host.
HTTP Configuration: Located in /data/nginx/user_conf.d/<prefix><server_id>.hostkey.in.http.conf.
- Listens on port 80.
- Serves the ACME challenge files from /var/www/certbot/.well-known/acme-challenge/.
- Redirects all other traffic to HTTPS using a 301 redirect.
SSL Management: Certbot is integrated within the Nginx container to automatically obtain and renew SSL certificates for the domain.

Starting, Stopping, and Updating¶

The services are managed using systemd for Ollama and Docker Compose for the proxy and application containers.

Ollama Service:
The service is enabled to start on boot.

To restart the service after configuration changes:

systemctl daemon-reload
systemctl restart ollama

Nginx and Certbot:
The container stack is started using Docker Compose from the /root/nginx directory:
```
docker compose up -d
```
To test the Nginx configuration inside the container:
```
docker exec nginx-nginx-1 nginx -t
```

To reload Nginx after configuration changes:

docker exec nginx-nginx-1 nginx -s reload

Open WebUI:
The container is managed by Docker and set to restart automatically.

To update the image, pull the latest version and recreate the container:

docker pull ghcr.io/open-webui/open-webui:cuda
docker restart open-webui

Access Rights and Security¶

System User: The ollama system user is created to run the Ollama service.
Firewall: The deployment assumes the server firewall allows traffic on ports 80 and 443 for web access. Internal communication occurs over localhost (127.0.0.1) on ports 8080 and 11434.
SSL: All external traffic is encrypted via HTTPS using Let's Encrypt certificates managed by Certbot.
Origins: The Ollama service is configured with OLLAMA_ORIGINS=* to allow cross-origin requests from the web interface.

Permission Settings¶

Nginx Directory: The /root/nginx directory is owned by root with permissions 0755.
Compose File: The /root/nginx/compose.yml file is owned by root with permissions 0644.
Nginx Config Directory: The /data/nginx/user_conf.d directory is mounted into the container and must be accessible by the Nginx process.
ACME Webroot: The directory /var/www/certbot/.well-known/acme-challenge is created inside the Nginx container to store challenge files for certificate validation.