Deployment Overview of Ollama and Open WebUI on Server¶

Prerequisites and Basic Requirements¶

To ensure the successful deployment of the AI Chatbot environment, the following requirements must be met:

Operating System: Ubuntu (22.04 or 24.04 recommended) or Debian. Note that Ubuntu 25.04 is not supported due to NVIDIA CUDA compatibility issues.
Privileges: Root or sudo access is required for system package installation and service configuration.
Hardware Requirement: An NVIDIA GPU is required for hardware acceleration.
Ports:
80/TCP: For HTTP (used by Nginx/Certbot).
443/TCP: For HTTPS (secure web access).

The application is accessible via a specific subdomain template:

Parameter	Value
Prefix	`ollama`
Domain	`hostkey.in`
Full template	`ollama{Server_ID_from_Invapi}.hostkey.in`

The application is installed using a combination of system-level scripts and Docker containers:

System Preparation: An installation script (install_script.sh) is executed to update the system, install gcc, and configure CUDA repositories.
NVIDIA Driver & CUDA Setup: The installer configures NVIDIA drivers and installs the CUDA toolkit. It also sets up necessary environment variables (PATH and LD_LIBRARY_PATH) in the user's .bashrc.
Docker Integration: Depending on the OS version, either nvidia-docker2 (for Ubuntu 22.04) or nvidia-container-toolkit (for Ubuntu 24.04) is installed to allow Docker containers to access the GPU.
Ollama Installation: The Ollama service is installed via the official installation script and configured as a systemd service (ollama.service) running under a dedicated ollama user.
Docker Deployment: A reverse proxy (Nginx) and the WebUI interface are deployed using Docker Compose.

User Management: A dedicated system user named ollama is created to run the Ollama service.
Firewall/Ports: The server exposes ports 80 and 443 via Nginx for secure web access.
Service Isolation: The WebUI runs in a containerized environment, communicating with the host's Ollama service via host.docker.internal.

The application uses Docker volumes for persistent storage of the WebUI data:

open-webui: A named volume used to store user data and configurations within the container.

The deployment utilizes two primary containers managed via a compose.yml file located in /root/nginx/.

Image: ghcr.io/open-webui/open-webui:main
Container Name: open-webui
Restart Policy: always
Volumes: open-webui:/app/backend/data
Network Configuration: Uses host.docker.internal to communicate with the host machine's Ollama service.

Image: jonasal/nginx-certbot:latest
Restart Policy: unless-stopped
Ports: 80:80, 443:443
Environment Variables:
[email protected]
Volumes:
nginx_secrets:/etc/letsencrypt (for SSL certificates)
/data/nginx/user_conf.d:/etc/nginx/user_conf.d (for custom Nginx configurations)

The following actions are performed during the setup process:

Initial Model Download: The script automatically executes ollama pull qwen3:14b to ensure a default model is available upon first launch.
Systemd Configuration: A custom systemd service file is created at /etc/systemd/system/ollama.service with specific environment variables including OLLAMA_HOST=0.0.0.0 and LLAMA_FLASH_ATTENTION=1.
Nginx Reverse Proxy Setup: The Nginx configuration in /data/nginx/user_conf.d/ is dynamically updated to proxy traffic from the domain to http://open-webui:8080.

To update the application components:

WebUI and Proxy: Navigate to /root/nginx/ and run:

docker compose pull && docker compose up -d

Ollama Service: The service is managed via systemd. To apply updates to the Ollama binary, restart the service:
```
sudo systemctl restart ollama
```

Component	Path / Volume
Nginx Compose File	`/root/nginx/compose.yml`
Nginx User Configs	`/data/nginx/user_conf.d/`
Ollama Service File	`/etc/systemd/system/ollama.service`
WebUI Persistent Data	`open-webui` (Docker Volume)