Deployment Overview of Qwen3-Coder on Server¶

Prerequisites and Basic Requirements¶

To ensure the successful deployment of the application, the following requirements must be met:

Operating System: Ubuntu (recommended).
Privileges: Root or sudo access is required for all installation steps.
Hardware Acceleration: NVIDIA GPU with functional drivers and CUDA support is required for hardware acceleration.
Ports:
443/TCP: External HTTPS access.
8080/TCP: Internal application access.

FQDN of the final panel on the hostkey.in domain¶

The application is accessible via a specific subdomain template based on the server ID.

Parameter	Value
Prefix	`qwen3-coder`
Domain	`hostkey.in`
Full template	`qwen3-coder{Server_ID}.hostkey.in`

File and Directory Structure¶

The following directories are used for configuration, certificates, and application data:

/root/nginx: Contains the Nginx Docker Compose configuration.
/data/nginx/user_conf.d: Stores Nginx user configuration files.
/data/nginx/nginx-certbot.env: Environment file for Certbot.
/data/nginx/secrets: Managed volume for SSL certificates.
open-webui (Docker Volume): Persistent storage for the Open WebUI application data.

Application installation process¶

The installation follows a multi-stage process involving system-level driver configuration, model setup, and container orchestration:

NVIDIA Driver and CUDA Setup: The system is configured with NVIDIA drivers and the CUDA toolkit to enable GPU acceleration. This includes repairing potential package conflicts and ensuring the nvidia-smi utility is functional.
Ollama Installation:
The Ollama service is installed via the official installation script.
A dedicated ollama system user is created.
The ollama.service configuration is modified to allow remote access (OLLAMA_HOST=0.0.0.0) and enable Flash Attention.
Model Download: The qwen3-coder model is pulled via the Ollama CLI:
```
ollama pull qwen3-coder
```
Docker Environment Setup: Docker is installed, and the NVIDIA Container Runtime is configured to allow containers to access the host's GPU resources.

Access Rights and Security¶

Firewall: Only necessary ports (primarily 443) should be exposed externally.
Service Isolation: The application runs within a Docker container with restricted environment variables.
User Management: A system user ollama is created to manage the backend model service.

Databases¶

The application uses Ollama as the primary engine for managing and serving models. Data related to chat history and user settings is stored within the open-webui Docker volume.

Docker Containers and Their Deployment¶

The deployment utilizes two main containerized components:

Open WebUI Container¶

This container provides the web interface for interacting with the model.

Parameter	Value
Image	`ghcr.io/open-webui/open-webui:cuda`
Name	`open-webui`
Ports	`8080:8080`
Volumes	`open-webui:/app/backend/data`
Environment Variables	`ENV=dev`, `OLLAMA_BASE_URLS=http://host.docker.internal:11434`
Restart Policy	`always`
GPU Access	Enabled (`--gpus all`)

Nginx Proxy Container¶

This container handles SSL termination and reverse proxying via Certbot.

Parameter	Value
Image	`jonasal/nginx-certbot:latest`
Name	`nginx`
Network Mode	`host`
Volumes	`nginx_secrets:/etc/letsencrypt`, `/data/nginx/user_conf.d:/etc/nginx/user_conf.d`
Restart Policy	`unless-stopped`

Custom Scripts and Additional Setup¶

The following setup actions are performed during the installation:

NVIDIA Driver Repair: A script executes to resolve APT dependency conflicts, specifically targeting potential mismatches between Ubuntu open drivers and existing CUDA driver packages.
Systemd Configuration: The ollama.service is updated to include specific environment variables for network accessibility and performance optimization (LLAMA_FLASH_ATTENTION=1).
Nginx Proxy Configuration: A configuration file is dynamically generated in /data/nginx/user_conf.d/ that points the proxy pass to http://127.0.0.1:8080.

Application Update Instructions¶

To update the main application interface, use the following command to pull the latest image and restart the container:

docker compose -f /root/nginx/compose.yml up -d --pull always

Or, if managing the Open WebUI container manually:

docker pull ghcr.io/open-webui/open-webui:cuda && docker rm -f open-webui && [run command]

Location of configuration files and data¶

Nginx Configuration: /data/nginx/user_conf.d/
Certbot Secrets: Managed via the nginx_secrets Docker volume.
Ollama Models: /usr/share/ollama/.ollama/models/

Available ports for connection¶

HTTPS (External): 443
Web Interface (Internal): 8080

Starting and Stopping the application¶

To manage the proxy service:

cd /root/nginx && docker compose up -d # Start
docker compose down # Stop

To manage the Web UI container:

docker start open-webui # Start
docker stop open-webui # Stop