Deployment Overview of Qwen36-35b on Server¶

Prerequisites and Basic Requirements¶

To ensure the stable operation of the application, the following system requirements must be met:

Operating System: Ubuntu (recommended for driver compatibility).
Privileges: Root or sudo access is required for package installation and service configuration.
Hardware Acceleration: NVIDIA GPU presence is highly recommended for model inference performance.
Network Ports:
443 (HTTPS): External access to the web interface.
8080 (HTTP): Internal communication between the proxy and the application.
11434: Local Ollama API port.

FQDN of the final panel on the hostkey.in domain¶

The application is accessible via a specific subdomain template based on the server ID.

Parameter	Value
Prefix	`qwen`
Domain	`hostkey.in`
Full template	`qwen{Server_ID_from_Invapi}.hostkey.in`

File and Directory Structure¶

The following directories are used for configuration, data persistence, and SSL management:

/root/nginx: Contains the Nginx Docker Compose configuration.
/data/nginx/user_conf.d: Stores Nginx user configuration files.
/data/nginx/nginx-certbot.env: Environment file for Certbot.
/var/lib/ollama: Default data directory for Ollama models (managed by the service).

Application Installation Process¶

The installation follows a multi-stage process involving system optimization, driver configuration, and container deployment:

System Preparation: The OS package cache is updated, and the HWE (Hardware Enablement) kernel is installed to ensure modern hardware support.
GPU Driver Setup: If an NVIDIA GPU is detected, ubuntu-drivers autoinstall is executed to install the recommended drivers. A system reboot is performed if drivers are updated.
Ollama Installation: The application uses a dedicated installation script which:
Detects the system architecture (x86_64 or ARM).
Downloads and installs the Ollama binary to /usr/local/bin.
Configures systemd to manage the ollama service.
Sets up necessary user permissions for GPU access.
Model Deployment: The specific model qwen3.6:35b is pulled via the Ollama CLI.
Web Interface Deployment: Open WebUI is deployed as a Docker container to provide the user interface.

Access Rights and Security¶

Firewall: Only necessary ports (443, 8080) are utilized for service communication.
Service User: The ollama service runs under a dedicated system user with restricted permissions.
SSL/TLS: SSL certificates are managed via Certbot to secure the external connection on port 443.

Databases¶

This deployment does not utilize an external standalone database; data is persisted within Docker volumes and local directories for the web interface and model storage.

Docker Containers and Their Deployment¶

The deployment utilizes two primary containerized components:

Nginx Proxy Container¶

Image: jonasal/nginx-certbot:latest
Ports: Host network mode (uses system ports).
Volumes:
nginx_secrets:/etc/letsencrypt
/data/nginx/user_conf.d:/etc/nginx/user_conf.d
Environment Variables: [email protected]
Restart Policy: unless-stopped

Open WebUI Container¶

Image: ghcr.io/open-webui/open-webui:main
Ports: Host network mode.
Volumes:
open-webui:/app/backend/data
Environment Variables:
OLLAMA_BASE_URL=http://127.0.0.1:11434
Restart Policy: always

Custom Scripts and Additional Setup¶

The following actions are performed during the setup process to ensure full functionality:

NVIDIA Driver/CUDA Configuration: A script automatically detects NVIDIA hardware and installs the appropriate CUDA drivers and kernel headers if they are missing.
Systemd Integration: The installation creates a ollama.service file in /etc/systemd/system/ to ensure the backend service starts automatically on boot.
Nginx Configuration Injection: A configuration update is performed to set the proxy_pass directive, routing traffic from the Nginx proxy to the local application port (8080).

Application Update Instructions¶

To update the main web interface component:

Navigate to the Nginx directory:
```
cd /root/nginx
```

Pull the latest images and restart the services:

docker compose pull && docker compose up -d

Location of Configuration Files and Data¶

Nginx Configs: /data/nginx/user_conf.d
WebUI Data: Managed via Docker volume open-webui.
SSL Certificates: Managed via Docker volume nginx_secrets.

Available Ports for Connection¶

Port	Usage	Access Type
`443`	Secure Web Interface (HTTPS)	External
`8080`	Nginx Proxy Backend	Internal
`11434`	Ollama API	Localhost

Starting and Stopping the Application¶

The application is managed via systemd for the backend and docker compose for the frontend/proxy.

Starting the Web Interface:
```
cd /root/nginx && docker compose up -d
```
Restarting the Backend (Ollama):
```
sudo systemctl restart ollama
```

Stopping the Application:

cd /root/nginx && docker compose down
sudo systemctl stop ollama

Proxy Servers¶

The application uses an Nginx proxy with Certbot integration. This setup handles SSL termination on port 443 and forwards requests to the Open WebUI container running on port 8080.