Deployment Overview of Qwen3-Coder on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a Linux server running Ubuntu. The following components must be available on the system:

Root privileges or sudo access for system-wide installations.
Docker Engine installed and running with GPU support enabled.
Network access to ollama.com for downloading the installation script and models.
Network access to ghcr.io for pulling the Open WebUI container image.
Port 8080 must be open for the Open WebUI application.
Port 11434 must be accessible internally for the Ollama service.
Port 80 and 443 must be open for the Nginx reverse proxy and SSL certificate management.

File and Directory Structure¶

The application utilizes specific directories for configuration, data storage, and certificates:

/root/nginx/: Contains the Docker Compose configuration for the Nginx proxy.
/root/nginx/compose.yml: The Docker Compose file defining the Nginx and Certbot services.
/data/nginx/: Stores Nginx configuration files and environment variables.
/data/nginx/user_conf.d/: Contains custom Nginx server block configurations.
/data/nginx/nginx-certbot.env: Environment file for Nginx and Certbot settings.
/etc/systemd/system/ollama.service: Systemd unit file for the Ollama service.
/usr/share/ollama/.ollama/models/: Default storage location for Ollama models, including qwen3-coder.
/var/lib/docker/volumes/open-webui/: Docker volume storing Open WebUI backend data.

Application Installation Process¶

The deployment involves installing the Ollama service, pulling the Qwen3-Coder model, and running the Open WebUI container.

Install Ollama: The Ollama service is installed using the official installation script.
```
curl -fsSL https://ollama.com/install.sh | sh
```
Configure Ollama Service: The ollama.service file is updated to expose the service on all network interfaces and enable specific environment variables.
- OLLAMA_HOST is set to 0.0.0.0.
- OLLAMA_ORIGINS is set to *.
- LLAMA_FLASH_ATTENTION is set to 1.
Pull the Model: The qwen3-coder model is downloaded and stored locally.
```
ollama pull qwen3-coder
```

Deploy Open WebUI: The Open WebUI application is deployed as a Docker container with GPU acceleration.

docker run -d -p 8080:8080 --gpus all \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  -e ENV='dev' \
  -e OLLAMA_BASE_URLS='http://host.docker.internal:11434' \
  --restart always ghcr.io/open-webui/open-webui:cuda

Docker Containers and Their Deployment¶

Two primary Docker-based components are deployed: the Open WebUI application and the Nginx reverse proxy with Certbot.

Open WebUI Container¶

The Open WebUI container is configured with the following parameters: - Image: ghcr.io/open-webui/open-webui:cuda - Name: open-webui - Ports: Maps host port 8080 to container port 8080. - Volumes: Mounts the open-webui named volume to /app/backend/data. - Environment Variables: - ENV: Set to dev. - OLLAMA_BASE_URLS: Set to http://host.docker.internal:11434. - Restart Policy: Set to always. - GPU Access: Enabled via --gpus all. - Host Resolution: Adds a custom host entry for host.docker.internal pointing to the host gateway.

Nginx and Certbot Container¶

The Nginx proxy is managed via Docker Compose located at /root/nginx/compose.yml. - Image: jonasal/nginx-certbot:latest - Network Mode: Uses host networking. - Volumes: - Mounts the external nginx_secrets volume to /etc/letsencrypt. - Mounts /data/nginx/user_conf.d to /etc/nginx/user_conf.d. - Environment: - CERTBOT_EMAIL: Set to [email protected]. - Loads additional environment variables from /data/nginx/nginx-certbot.env. - Restart Policy: Set to unless-stopped.

Proxy Servers¶

The Nginx reverse proxy is configured to handle incoming traffic and manage SSL certificates via Certbot.

Configuration Location: Custom server configurations are stored in /data/nginx/user_conf.d/.
Proxy Pass: The Nginx configuration includes a location / block that forwards traffic to the Open WebUI application running on http://127.0.0.1:8080.
SSL Management: The nginx-certbot container automatically handles SSL certificate generation and renewal using Let's Encrypt.
Deployment: The proxy stack is started using the command:
```
docker compose up -d
```
This command is executed from the /root/nginx directory.

Starting, Stopping, and Updating¶

Ollama Service¶

The Ollama service is managed via systemd.

Restart Service:

systemctl daemon-reload
systemctl restart ollama

Enable Service:
```
systemctl enable ollama
```

Open WebUI Container¶

The Open WebUI container is managed via Docker commands.

Start/Restart: Since the container is configured with --restart always, it will automatically start on boot. To manually restart:
```
docker restart open-webui
```
Stop:
```
docker stop open-webui
```

Update: To update the container to the latest version, pull the new image and recreate the container:

docker pull ghcr.io/open-webui/open-webui:cuda
docker rm -f open-webui
docker run -d -p 8080:8080 --gpus all \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  -e ENV='dev' \
  -e OLLAMA_BASE_URLS='http://host.docker.internal:11434' \
  --restart always ghcr.io/open-webui/open-webui:cuda

Nginx Proxy Stack¶

The Nginx and Certbot stack is managed via Docker Compose.

Start/Restart:
```
cd /root/nginx
docker compose up -d
```
Stop:
```
cd /root/nginx
docker compose down
```