Deployment Overview of Qwen3-Coder on Server¶

Prerequisites and Basic Requirements¶

To ensure the successful deployment of the application, the following requirements must be met:

• Operating System: Ubuntu (recommended).

• Privileges: Root or sudo access is required for package installation and service management.

• Hardware Acceleration: NVIDIA GPU support is required for CUDA-enabled container execution.

• Network Ports:

• 443: External HTTPS traffic.

• 8080: Internal application traffic.

FQDN of the final panel on the hostkey.in domain¶

The application is accessible via a specific subdomain template based on the server ID:

Application installation process¶

The installation follows a multi-step process involving system package management, service configuration, and container orchestration:

1. Ollama Installation: The Ollama service is installed via the official shell script.

2. System Configuration:

3. A dedicated ollama user is created.

4. The ollama.service file is configured to allow remote connections (OLLAMA_HOST=0.0.0.0) and enable Flash Attention (LLAMA_FLASH_ATTENTION=1).

5. Model Deployment: The qwen3-coder model is pulled into the local Ollama repository.

6. Docker Setup: Necessary dependencies for Docker execution are installed via apt.

7. Container Orchestration: The Open WebUI container is deployed to provide a graphical interface for the model.

8. Reverse Proxy Configuration: An Nginx proxy is configured with SSL certificates and points traffic from port 443 to the internal application port 8080.

Docker Containers and Their Deployment¶

The deployment utilizes the following containers:

Open WebUI¶

• Image Name: ghcr.io/open-webui/open-webui:cuda

• Ports: 8080:8080

• Volumes: open-webui:/app/backend/data

• Environment Variables:

• ENV=dev

• OLLAMA_BASE_URLS=http://host.docker.internal:11434

• Restart Policy: always

• Additional Config: Uses --gpus all for hardware acceleration and --add-host=host.docker.internal:host-gateway to communicate with the host's Ollama service.

Nginx Proxy¶

• Image Name: jonasal/nginx-certbot:latest

• Network Mode: host

• Volumes:

• nginx_secrets:/etc/letsencrypt

• /data/nginx/user_conf.d:/etc/nginx/user_conf.d

• Restart Policy: unless-stopped

Custom Scripts and Additional Setup¶

The following actions are performed to prepare the environment:

• Ollama Service Modification: The systemd service for Ollama is modified to ensure it listens on all network interfaces, allowing the Docker container to communicate with the host.

• Model Pre-loading: An automated command ollama pull qwen3-coder is executed during installation to ensure the model is available immediately upon startup.

Application Update Instructions¶

To update the main application (Open WebUI), execute the following commands:

docker compose -f /root/nginx/compose.yml pull && docker compose -f /root/nginx/compose.yml up -d

Note: If you are managing the container manually via docker run, you must stop, remove, and re-run the container with the latest image.

Location of configuration files and data¶

Available ports for connection¶

• HTTPS: 443 (via Nginx proxy)

• Internal API: 8080 (Open WebUI)

• Ollama API: 11434 (Localhost)