Deployment Overview of Open WebUI and Ollama on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a Linux server running Ubuntu 22.04 or Ubuntu 24.04. Ubuntu 25.04 is not supported due to a lack of official CUDA support from NVIDIA for non-LTS releases. The system must have root privileges to install drivers, configure services, and manage containers.

• Operating System: Ubuntu 22.04 or Ubuntu 24.04.

• Privileges: Root access or sudo privileges.

• Hardware: NVIDIA GPU with compatible drivers for CUDA acceleration.

• Network: Access to the internet for downloading packages and models.

• Ports: Ports 80 and 443 must be open for the reverse proxy and SSL termination.

FQDN of the Final Panel¶

The application is accessible via the hostkey.in domain. The Fully Qualified Domain Name (FQDN) follows the format: ollama<Server ID>.hostkey.in

The service listens on port 443 for HTTPS traffic.

File and Directory Structure¶

The deployment utilizes specific directories for configuration, data storage, and certificates:

• Nginx Configuration: /data/nginx/user_conf.d/

• Nginx Compose File: /root/nginx/compose.yml

• Nginx Environment: /data/nginx/nginx-certbot.env

• Ollama Service Unit: /etc/systemd/system/ollama.service

• Ollama Backup: /etc/systemd/system/ollama.service.bak

• Docker Volumes:

• nginx_secrets: Stores Let's Encrypt certificates.

• open-webui: Stores application backend data.

Application Installation Process¶

The installation involves setting up the operating system, installing NVIDIA drivers and CUDA, configuring Docker for GPU support, and deploying the Ollama service and Open WebUI container.

1. System Preparation: The system is updated, and the kernel is upgraded to the HWE stack if running Ubuntu 22.04.

2. CUDA and Driver Installation:

   • GCC compiler is installed.

   • CUDA keyring and toolkit are installed based on the Ubuntu release version.

   • Environment variables for CUDA are added to ~/.bashrc.

   • The nouveau kernel module is removed, and NVIDIA drivers are initialized.

3. Docker GPU Support:

   • On Ubuntu 22.04, nvidia-docker2 is installed.

   • On Ubuntu 24.04, nvidia-container-toolkit is installed.

   • The Docker service is restarted to apply changes.

4. Ollama Installation:

   • The Ollama binary is installed via the official installation script.

   • A system user named ollama is created.

   • The default model qwen3:14b is pulled.

5. Service Configuration:

   • The ollama.service systemd unit is generated and enabled.

   • The service is restarted to apply the new configuration.

Access Rights and Security¶

Security is managed through the reverse proxy and container isolation.

• Firewall: Ports 80 and 443 are exposed for the Nginx container to handle incoming traffic.

• User Isolation: The Ollama service runs under the dedicated ollama system user and group.

• SSL/TLS: SSL certificates are managed automatically by the nginx-certbot container using Let's Encrypt.

• Network Restrictions: The Ollama service is configured to listen on 0.0.0.0 but is accessed externally only through the Nginx proxy.

Databases¶

The Open WebUI application stores its data within a Docker volume named open-webui. This volume maps to the container path /app/backend/data. No external database server is configured; data persistence is handled by the Docker volume.

Docker Containers and Their Deployment¶

Two primary Docker containers are deployed using docker compose:

1. Open WebUI:

   • Image: ghcr.io/open-webui/open-webui:main

   • Container Name: open-webui

   • Restart Policy: always

   • Volumes: Mounts the open-webui volume to /app/backend/data.

   • Hosts: Adds host.docker.internal pointing to the host gateway.

2. Nginx Proxy:

   • Image: jonasal/nginx-certbot:latest

   • Restart Policy: unless-stopped

   • Ports: Maps host ports 80 and 443 to container ports 80 and 443.

   • Volumes:

     • nginx_secrets mounted to /etc/letsencrypt.

     • Host directory /data/nginx/user_conf.d mounted to /etc/nginx/user_conf.d.

   • Dependencies: Depends on the open-webui service.

The deployment is managed via the docker compose command executed in the /root/nginx directory.

Proxy Servers¶

Nginx acts as the reverse proxy and SSL terminator for the application.

• Configuration Location: /data/nginx/user_conf.d/ollama<Server ID>.hostkey.in.conf

• Proxy Pass: Traffic is forwarded from the Nginx location block to the Open WebUI container at http://open-webui:8080.

• SSL Management: The nginx-certbot container handles certificate generation and renewal automatically.

• Environment: The proxy uses the email [email protected] for Let's Encrypt notifications.

Permission Settings¶

File and directory permissions are set as follows:

• Nginx Directory: /root/nginx is owned by root:root with mode 0755.

• Compose File: /root/nginx/compose.yml is owned by root:root with mode 0644.

• Ollama Service File: /etc/systemd/system/ollama.service is owned by root:root with mode 0644.

• Ollama User: The ollama user is a system user used to run the Ollama service.

Location of Configuration Files and Data¶

Key configuration and data files are located in the following paths:

• Nginx Compose: /root/nginx/compose.yml

• Nginx User Config: /data/nginx/user_conf.d/ollama<Server ID>.hostkey.in.conf

• Nginx Environment: /data/nginx/nginx-certbot.env

• Ollama Service: /etc/systemd/system/ollama.service

• Ollama Data: Managed by the Docker volume open-webui.

Available Ports for Connection¶

The following ports are available for external and internal connections:

• Port 443: HTTPS traffic for the Open WebUI interface via Nginx.

• Port 80: HTTP traffic for SSL redirection and Let's Encrypt validation.

• Port 3000: Internal port defined in configuration (not exposed externally).

• Port 8080: Internal port used by the Open WebUI container, proxied by Nginx.

Starting, Stopping, and Updating¶

Service management is handled via systemd for Ollama and Docker Compose for the web interface.

• Ollama Service:

• Start: systemctl start ollama

• Stop: systemctl stop ollama

• Restart: systemctl restart ollama

• Enable on Boot: systemctl enable ollama

• Docker Containers (Open WebUI and Nginx):

• Start/Update: docker compose up -d (executed in /root/nginx)

• Stop: docker compose down (executed in /root/nginx)

• View Logs: docker compose logs -f (executed in /root/nginx)