Deployment Overview of Qwen3-Coder on Server¶

Prerequisites and Basic Requirements¶

The deployment of the Qwen3-Coder application requires a Linux server running Ubuntu. The following conditions must be met before initiating the installation process:

• Operating System: Ubuntu (specific version not specified, but compatible with apt package manager).

• Privileges: Root access or sudo privileges are required to install system services, manage Docker containers, and configure firewall rules.

• Domain Configuration: The server must be configured to resolve the hostkey.in domain.

• Ports:

• Port 8080: Internal communication between the proxy and the Open WebUI application.

• Port 443: External HTTPS access for the final panel.

• Port 11434: Internal Ollama service port (bound to 0.0.0.0 for internal Docker networking).

FQDN of the Final Panel¶

The application is accessible via the following Fully Qualified Domain Name (FQDN) format on the hostkey.in domain:

• Format: qwen3-coder<Server ID>.hostkey.in:443

• The prefix qwen3-coder is hardcoded in the configuration.

• The <Server ID> placeholder must be replaced with the actual unique identifier of the server instance.

• The external port is 443 for secure HTTPS connections.

File and Directory Structure¶

The deployment utilizes specific directories for configuration files, data storage, and certificates:

• /root/nginx/: Contains the Docker Compose configuration for the Nginx proxy and Certbot.

• /root/nginx/compose.yml: Docker Compose file for the proxy service.

• /data/nginx/: Stores Nginx user configurations and environment variables.

• /data/nginx/user_conf.d/: Directory containing specific server block configurations.

  • qwen3-coder<Server ID>.hostkey.in.conf: Configuration file for the specific application instance.

• /data/nginx/nginx-certbot.env: Environment file for the Nginx-Certbot service.

• /etc/systemd/system/ollama.service: Systemd service unit file for the Ollama backend.

• /usr/share/ollama/.ollama/models/: Default storage location for the qwen3-coder model.

• /var/lib/docker/volumes/open-webui/_data: Persistent volume for Open WebUI backend data.

Application Installation Process¶

The application stack consists of three main components: Ollama (backend), Open WebUI (frontend), and Nginx with Certbot (proxy).

1. Ollama Installation:

2. The Ollama service is installed using the official installation script.

3. The ollama system user is created.

4. The ollama.service file is modified to include the following environment variables:

   • OLLAMA_HOST=0.0.0.0

   • OLLAMA_ORIGINS=*

   • LLAMA_FLASH_ATTENTION=1

5. The qwen3-coder model is pulled and stored locally.

6. Open WebUI Deployment:

7. The Open WebUI application is deployed as a Docker container using the ghcr.io/open-webui/open-webui:cuda image.

8. The container is named open-webui.

9. It maps the internal port 8080 to the host port 8080.

10. GPU acceleration is enabled via the --gpus all flag.

11. The container connects to the Ollama service at http://host.docker.internal:11434.

12. Proxy and SSL Configuration:

13. The Nginx proxy is deployed using Docker Compose located in /root/nginx/compose.yml.

14. The image used is jonasal/nginx-certbot:latest.

15. SSL certificates are managed automatically via Certbot using the email [email protected].

Access Rights and Security¶

• User Accounts: A dedicated system user named ollama is created for the Ollama service.

• Firewall: The configuration assumes that ports 8080 (internal) and 443 (external) are accessible. Port 80 is implicitly required for Let's Encrypt validation.

• Network Restrictions: The Ollama service is bound to 0.0.0.0 to allow connections from the Docker network, while the external access is restricted to the Nginx proxy on port 443.

• Environment Variables: The OLLAMA_ORIGINS variable is set to * to allow cross-origin requests from the Open WebUI container.

Databases¶

• The Open WebUI application uses a local persistent volume for its backend data.

• Storage Location: /var/lib/docker/volumes/open-webui/_data.

• Connection Method: The application accesses this data via the Docker volume mount -v open-webui:/app/backend/data.

• No external database server or specific database configuration files are defined in the provided context.

Docker Containers and Their Deployment¶

Two primary Docker components are deployed:

1. Open WebUI Container:

2. Command:

   docker run -d -p 8080:8080 --gpus all \
     --add-host=host.docker.internal:host-gateway \
     -v open-webui:/app/backend/data \
     --name open-webui \
     -e ENV='dev' \
     -e OLLAMA_BASE_URLS='http://host.docker.internal:11434' \
     --restart always ghcr.io/open-webui/open-webui:cuda
   

3. Restart Policy: always.

4. Volume: open-webui (named volume).

5. Nginx-Certbot Container:

6. Deployment Method: Docker Compose.

7. Location: /root/nginx/compose.yml.

8. Image: jonasal/nginx-certbot:latest.

9. Network Mode: host.

10. Volumes:

    • nginx_secrets (external) mounted to /etc/letsencrypt.

    • /data/nginx/user_conf.d mounted to /etc/nginx/user_conf.d.

Proxy Servers¶

The Nginx proxy handles SSL termination and routing for the application.

• Image: jonasal/nginx-certbot:latest.

• SSL Provider: Let's Encrypt (Certbot).

• Email for Certificates: [email protected].

• Configuration File: /data/nginx/user_conf.d/qwen3-coder<Server ID>.hostkey.in.conf.

• Proxy Pass: The proxy forwards requests from the root path / to the internal application at http://127.0.0.1:8080.

• Environment File: /data/nginx/nginx-certbot.env.

Permission Settings¶

• Nginx Directory: /root/nginx is owned by root:root with permissions 0755.

• Compose File: /root/nginx/compose.yml is owned by root:root with permissions 0644.

• Systemd Service: The ollama.service file is located in /etc/systemd/system/ and is managed by the root user.

• Docker Volumes: Docker manages permissions for the open-webui volume automatically based on the container's internal user configuration.

Location of Configuration Files and Data¶

Available Ports for Connection¶

• Port 443: External HTTPS access to the Qwen3-Coder panel.

• Port 8080: Internal HTTP access for the Open WebUI application (proxied by Nginx).

• Port 11434: Internal Ollama API port (accessible within the Docker network).

Starting, Stopping, and Updating¶

The services are managed via systemd for Ollama and Docker for the application containers.

• Ollama Service:

• Start: systemctl start ollama

• Stop: systemctl stop ollama

• Restart: systemctl restart ollama

• Enable on Boot: systemctl enable ollama

• Reload Daemon: systemctl daemon-reload

• Open WebUI Container:

• Start: docker start open-webui

• Stop: docker stop open-webui

• Restart: docker restart open-webui

• Update: Pull the new image and recreate the container using the docker run command with the --rm flag for the old container if necessary, or use docker pull followed by docker update.

• Nginx Proxy:

• Start/Restart: docker compose up -d executed from the /root/nginx directory.

• Stop: docker compose down executed from the /root/nginx directory.