Deployment Overview of Open WebUI with Ollama on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a Linux server running Ubuntu with root privileges. The system must have Docker installed and configured to support GPU acceleration for the open-webui container. The following components are required:

• Operating System: Ubuntu

• Privileges: Root access

• Domain: hostkey.in

• Ports:

• 443 (HTTPS external access)

• 8080 (Internal Open WebUI service)

• 11434 (Internal Ollama service)

FQDN of the Final Panel¶

The application is accessible via the following Fully Qualified Domain Name (FQDN) format:

• llama<Server ID>.hostkey.in:443

Replace <Server ID> with the specific identifier assigned to the server instance.

File and Directory Structure¶

The deployment utilizes the following directory structure for configuration, data, and certificates:

• /root/nginx/: Contains the Docker Compose configuration for the proxy.

• /root/nginx/compose.yml: The Docker Compose file for the Nginx and Certbot services.

• /data/nginx/user_conf.d/: Contains custom Nginx configuration files for the specific domain.

• /data/nginx/nginx-certbot.env: Environment variables for the Certbot service.

• /etc/systemd/system/ollama.service: Systemd service file for Ollama.

• /etc/systemd/system/ollama.service.bak: Backup of the original Ollama service file.

Application Installation Process¶

The application stack consists of Ollama for model inference and Open WebUI for the user interface.

1. Ollama Installation:

   • Ollama is installed using the official installation script.

   • The ollama system user is created.

   • The ollama service is configured with specific environment variables:

     • OLLAMA_HOST=0.0.0.0

     • OLLAMA_ORIGINS=*

     • LLAMA_FLASH_ATTENTION=1

   • The llama3.3 model is pulled and loaded into the system.

2. Open WebUI Deployment:

   • The Open WebUI application is deployed as a Docker container using the ghcr.io/open-webui/open-webui:cuda image.

   • The container is configured to expose port 8080.

   • It utilizes a named volume open-webui for backend data persistence.

   • The container connects to the local Ollama instance via http://host.docker.internal:11434.

Access Rights and Security¶

• Firewall: External access is restricted to port 443 (HTTPS) via the Nginx proxy. Internal services on ports 8080 and 11434 are not directly exposed to the public internet.

• Users: The ollama system user is created to manage the Ollama service.

• Restrictions: The Nginx proxy handles SSL termination and forwards traffic to the internal application.

Databases¶

• Storage: Open WebUI stores its data in a Docker named volume labeled open-webui.

• Location: The data is persisted at /app/backend/data within the container, mapped to the host's Docker volume storage.

• Connection: No external database connection is required; the application uses an internal SQLite database managed within the container volume.

Docker Containers and Their Deployment¶

Two primary Docker components are deployed:

1. Open WebUI Container:

   • Image: ghcr.io/open-webui/open-webui:cuda

   • Command:

     docker run -d -p 8080:8080 --gpus all \
       --add-host=host.docker.internal:host-gateway \
       -v open-webui:/app/backend/data \
       --name open-webui \
       -e ENV='dev' \
       -e OLLAMA_BASE_URLS='http://host.docker.internal:11434' \
       --restart always ghcr.io/open-webui/open-webui:cuda
     

   • Restart Policy: always

2. Nginx and Certbot Container:

   • Image: jonasal/nginx-certbot:latest

   • Deployment Method: Docker Compose

   • Location: /root/nginx/compose.yml

   • Configuration:

   • Uses network_mode: host.

   • Mounts nginx_secrets volume for Let's Encrypt certificates.

   • Mounts /data/nginx/user_conf.d for custom site configurations.

Proxy Servers¶

The deployment uses Nginx with Certbot for SSL termination and reverse proxying.

• Proxy Configuration:

• The Nginx container listens on the host network.

• Custom configuration for the domain is located in /data/nginx/user_conf.d/llama<Server ID>.hostkey.in.conf.

• The proxy forwards requests from the root path / to the internal Open WebUI service at http://127.0.0.1:8080.

• SSL/TLS:

• Managed automatically by Certbot within the Nginx container.

• Certificates are stored in the nginx_secrets volume at /etc/letsencrypt.

• Email: Certificate renewal notifications are sent to [email protected].

Permission Settings¶

• Nginx Directory: /root/nginx is owned by root with permissions 0755.

• Compose File: /root/nginx/compose.yml is owned by root with permissions 0644.

• Ollama Service: The ollama service runs under the ollama system user.

Location of Configuration Files and Data¶

• Nginx Compose: /root/nginx/compose.yml

• Nginx Custom Config: /data/nginx/user_conf.d/llama<Server ID>.hostkey.in.conf

• Certbot Environment: /data/nginx/nginx-certbot.env

• Ollama Service: /etc/systemd/system/ollama.service

• Open WebUI Data: Docker volume open-webui (mapped to /app/backend/data inside the container).

Available Ports for Connection¶

• 443: HTTPS (Public access via Nginx proxy).

• 8080: HTTP (Internal Open WebUI service, proxied by Nginx).

• 11434: HTTP (Internal Ollama API, accessible only from the host or Docker network).

Starting, Stopping, and Updating¶

• Ollama Service:

• Start: systemctl start ollama

• Stop: systemctl stop ollama

• Restart: systemctl restart ollama

• Enable on boot: systemctl enable ollama

• Open WebUI Container:

• Start: docker start open-webui

• Stop: docker stop open-webui

• Restart: docker restart open-webui

• Update: Pull the latest image and recreate the container using the deployment command provided in the Docker Containers section.

• Nginx Proxy:

• Start/Restart: docker compose up -d (executed from /root/nginx)

• Stop: docker compose down (executed from /root/nginx)