Skip to content

Deployment Overview of Qwen3-32B on Server

Prerequisites and Basic Requirements

The deployment requires a server running the Ubuntu operating system with root privileges or sudo access. The system must have Docker and Docker Compose installed to manage the application containers. Additionally, the server requires internet connectivity to download the Ollama installation script, the Qwen3:32B model, and the Open WebUI Docker image.

The following ports must be available and open on the server firewall:

  • Port 443 for HTTPS traffic.

  • Port 80 for SSL certificate validation (Let's Encrypt).

  • Port 8080 for internal communication between the reverse proxy and the Open WebUI container.

FQDN of the Final Panel

The application is accessible via the following Fully Qualified Domain Name (FQDN) format, where <ServerID> represents the unique identifier of the server:

qwen3-32b<ServerID>.hostkey.in

The service operates exclusively over HTTPS (port 443).

File and Directory Structure

The deployment organizes files and directories as follows:

  • Ollama Service Configuration: Located at /etc/systemd/system/ollama.service. A backup of the original configuration is stored at /etc/systemd/system/ollama.service.bak.

  • Nginx and Certbot Configuration:

  • Docker Compose file: /root/nginx/compose.yml

  • Environment variables: /data/nginx/nginx-certbot.env

  • Server-specific configuration: /data/nginx/user_conf.d/qwen3-32b<ServerID>.hostkey.in.conf

  • Let's Encrypt secrets volume: Mounted at /etc/letsencrypt within the Docker container.

  • Ollama Models: Stored in the system directory /usr/share/ollama/.ollama/models/.

  • Open WebUI Data: Persisted in the Docker volume named open-webui.

Application Installation Process

The application installation involves two primary components: the Ollama backend and the Open WebUI frontend.

  1. Ollama Installation:

    • The Ollama package is installed via the official installation script (curl -fsSL https://ollama.com/install.sh | sh).

    • The system user ollama is created if it does not exist.

    • The ollama.service unit is modified to include the following environment variables:

      • OLLAMA_HOST=0.0.0.0

      • OLLAMA_ORIGINS=*

      • LLAMA_FLASH_ATTENTION=1

    • The systemd daemon is reloaded, and the ollama service is restarted and enabled for startup.

    • The qwen3:32b model is pulled and stored locally.

  2. Open WebUI Deployment:

    • The Open WebUI container is launched using the ghcr.io/open-webui/open-webui:cuda image.

    • The container is configured with the environment variable OLLAMA_BASE_URLS pointing to http://host.docker.internal:11434 to communicate with the Ollama service.

    • GPU acceleration is enabled by passing the --gpus all flag.

Access Rights and Security

Access to the Ollama service is restricted to local network interfaces via 0.0.0.0 binding in the service configuration. The Open WebUI container runs in a Docker environment with network isolation, exposing port 8080 internally. All external traffic is routed through Nginx with SSL/TLS encryption managed by Certbot.

The ollama user is a system user with restricted privileges. The Nginx and Certbot containers run with host network mode (network_mode: host) to ensure proper DNS resolution and port binding.

Databases

The Open WebUI component utilizes an internal SQLite database for user sessions, chat history, and configuration. This database is stored within the Docker volume open-webui at /app/backend/data. No external database server is required or configured for this deployment.

Docker Containers and Their Deployment

Two primary Docker containers are deployed to run the application:

  1. Open WebUI Container:

    • Image: ghcr.io/open-webui/open-webui:cuda

    • Container Name: open-webui

    • Ports: Maps host port 8080 to container port 8080.

    • Volumes: Mounts the named volume open-webui to /app/backend/data.

    • Environment Variables:

      • ENV=dev

      • OLLAMA_BASE_URLS=http://host.docker.internal:11434

    • Device Access: Grants access to all GPUs via --gpus all.

    • Restart Policy: always.

  2. Nginx-Certbot Container:

    • Image: jonasal/nginx-certbot:latest

    • Network Mode: Host (network_mode: host).

    • Volumes:

      • Mounts nginx_secrets (external) to /etc/letsencrypt.

      • Mounts /data/nginx/user_conf.d to /etc/nginx/user_conf.d.

    • Environment Variables:

    • Restart Policy: unless-stopped.

The Nginx-Certbot container is managed via the Docker Compose file located at /root/nginx/compose.yml.

Proxy Servers

A reverse proxy is implemented using Nginx to handle SSL termination and route traffic to the Open WebUI application.

  • Nginx Configuration: The proxy is configured to listen on the FQDN qwen3-32b<ServerID>.hostkey.in.

  • SSL/TLS: SSL certificates are automatically generated and renewed using Certbot.

  • Routing:

  • The Nginx configuration file at /data/nginx/user_conf.d/qwen3-32b<ServerID>.hostkey.in.conf directs traffic from the root path / to the internal backend.

  • The proxy_pass directive is set to http://127.0.0.1:8080.

  • Certbot: The Docker Compose configuration uses an external volume named nginx_secrets to store certificate data.

Permission Settings

File and directory permissions are configured as follows:

  • /root/nginx: Owner root, Group root, Mode 0755.

  • /root/nginx/compose.yml: Owner root, Group root, Mode 0644.

  • /data/nginx: Required for Nginx configuration files and Let's Encrypt storage.

  • Ollama Service: The ollama service runs as the ollama system user.

Location of Configuration Files and Data

All critical configuration files and data stores are located in the following paths:

  • Nginx Compose: /root/nginx/compose.yml

  • Nginx Server Config: /data/nginx/user_conf.d/qwen3-32b<ServerID>.hostkey.in.conf

  • Nginx Environment: /data/nginx/nginx-certbot.env

  • Ollama Service: /etc/systemd/system/ollama.service

  • Ollama Model Data: /usr/share/ollama/.ollama/models/

  • Open WebUI Data: Docker volume open-webui

Available Ports for Connection

The following ports are utilized by the deployed application:

Port Protocol Purpose Scope
443 TCP HTTPS (Public Access) External
80 TCP HTTP (SSL Validation) External
8080 TCP Open WebUI Internal Internal
11434 TCP Ollama API Internal/Localhost

Starting, Stopping, and Updating

The application components are managed using systemctl for the Ollama service and docker compose for the Nginx proxy.

  • Ollama Service:

  • Start: systemctl start ollama

  • Stop: systemctl stop ollama

  • Restart: systemctl restart ollama

  • Enable on Boot: systemctl enable ollama

  • Nginx-Certbot Service:

  • Start: docker compose up -d (executed from /root/nginx)

  • Stop: docker compose down (executed from /root/nginx)

  • Update Configuration: Modify /root/nginx/compose.yml or the configuration files in /data/nginx/user_conf.d, then run docker compose up -d.

  • Open WebUI Container:

  • Start: docker start open-webui

  • Stop: docker stop open-webui

  • Restart: docker restart open-webui

  • Update Image: docker pull ghcr.io/open-webui/open-webui:cuda followed by docker stop open-webui && docker run -d ... (re-execution of the run command with updated image).

question_mark
Is there anything I can help you with?
question_mark
AI Assistant ×