Deployment Overview of Open-WebUI with Gemma3:27B on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a Linux-based operating system, specifically Ubuntu, with root privileges to manage system services and Docker containers. The server must have access to the internet to pull Docker images and model weights. NVIDIA GPU hardware is required for optimal performance, along with the NVIDIA Container Toolkit installed to enable GPU acceleration within Docker containers.

The following ports must be available and open on the server firewall:

• Port 443 for HTTPS external access.

• Port 8080 for internal communication between the web interface and the backend.

• Port 11434 for the Ollama service internal API.

FQDN of the Final Panel¶

The application is accessible via the following Fully Qualified Domain Name (FQDN) format: gemma<Server ID>.hostkey.in:443

Replace <Server ID> with the specific identifier assigned to the server instance. The domain zone is configured as hostkey.in with the prefix gemma.

File and Directory Structure¶

The deployment utilizes the following directory structure for configuration files, data storage, and certificates:

Application Installation Process¶

The application stack consists of three main components: Ollama for model inference, Open-WebUI for the user interface, and Nginx with Certbot for reverse proxy and SSL termination.

1. Ollama Installation: The Ollama service is installed using the official installation script. The service is configured to listen on all network interfaces (0.0.0.0) and allows all origins.

2. Model Download: The gemma3:27b model is pulled directly into the Ollama service.

3. Docker and NVIDIA Setup: Docker is installed and configured to use the NVIDIA container runtime. The nvidia-container-toolkit and nvidia-container-runtime packages are installed to enable GPU passthrough.

4. Open-WebUI Deployment: The Open-WebUI container is launched with CUDA support, mapping the internal port 8080 and connecting to the Ollama service running on the host.

5. Proxy Deployment: An Nginx container with Certbot is deployed via Docker Compose to handle SSL certificates and route traffic to the internal services.

Access Rights and Security¶

The system relies on the root user for initial setup and management of Docker containers and systemd services. The ollama system user is created to manage the Ollama service.

Security measures include:

• SSL/TLS encryption is enforced via Nginx and Certbot for all external traffic on port 443.

• The Ollama service is configured with OLLAMA_ORIGINS=* to allow connections from the web interface.

• Docker containers are isolated, with specific volume mounts for data persistence.

Databases¶

The Open-WebUI application stores its data in a Docker volume named open-webui. This volume is mounted to /app/backend/data inside the container. No external database server is required; the application uses an internal SQLite database managed within the container's volume.

Docker Containers and Their Deployment¶

The deployment utilizes the following Docker containers:

• Open-WebUI:

  • Image: ghcr.io/open-webui/open-webui:cuda

  • Name: open-webui

  • Ports: Maps host port 8080 to container port 8080.

  • GPU: Configured with --gpus all for CUDA acceleration.

  • Environment Variables:

    • ENV: Set to dev.

    • OLLAMA_BASE_URLS: Set to http://host.docker.internal:11434.

  • Volumes: Mounts the open-webui volume to /app/backend/data.

  • Restart Policy: Set to always.

• Nginx-Certbot:

  • Image: jonasal/nginx-certbot:latest

  • Deployment Method: Managed via docker compose in the /root/nginx directory.

  • Network Mode: host.

  • Volumes:

    • nginx_secrets (external) mounted to /etc/letsencrypt.

    • /data/nginx/user_conf.d mounted to /etc/nginx/user_conf.d.

  • Environment: Uses [email protected] and loads configuration from /data/nginx/nginx-certbot.env.

Proxy Servers¶

Nginx acts as the reverse proxy for the application, handling SSL termination via Certbot. The proxy is configured to listen on port 443 for external traffic and route requests to the internal Open-WebUI service running on port 8080. The Nginx configuration is managed through a Docker Compose file located at /root/nginx/compose.yml. Custom domain configurations are stored in /data/nginx/user_conf.d.

Permission Settings¶

The following permission settings are applied to critical directories and files:

• /root/nginx: Directory permissions set to 0755, owned by root:root.

• /root/nginx/compose.yml: File permissions set to 0644, owned by root:root.

• /etc/docker/daemon.json: File permissions set to 0644, owned by root:root.

• /etc/systemd/system/ollama.service: Managed by systemd, typically owned by root.

Location of Configuration Files and Data¶

Configuration files and persistent data are located in the following paths:

• Nginx Compose: /root/nginx/compose.yml

• Nginx Environment: /data/nginx/nginx-certbot.env

• Docker Daemon Config: /etc/docker/daemon.json

• Ollama Service: /etc/systemd/system/ollama.service

• Open-WebUI Data: Docker volume open-webui (mapped to /app/backend/data inside the container).

• SSL Certificates: Docker volume nginx_secrets (mapped to /etc/letsencrypt inside the Nginx container).

Available Ports for Connection¶

The following ports are configured for the application:

Starting, Stopping, and Updating¶

Service management is handled through systemd for Ollama and Docker for the containers.

Ollama Service:

• Start: systemctl start ollama

• Stop: systemctl stop ollama

• Restart: systemctl restart ollama

• Enable on Boot: systemctl enable ollama

Open-WebUI Container:

• Start: docker start open-webui

• Stop: docker stop open-webui

• Restart: docker restart open-webui

• Update: Pull the latest image with docker pull ghcr.io/open-webui/open-webui:cuda and recreate the container.

Nginx-Certbot Container:

• Navigate to the configuration directory: cd /root/nginx

• Start/Restart: docker compose up -d

• Stop: docker compose down

• Update: Pull the latest image and run docker compose up -d.