Deployment Overview of Phi-4-14b on Server¶

Prerequisites and Basic Requirements¶

Operating System: Ubuntu (compatible with apt package manager).
Privileges: Root access or sudo privileges are required to install system packages, manage Docker containers, and configure systemd services.
Domain Zone: hostkey.in.
Ports:
Internal application port: 3000 (internal reference).
External secure port: 443 (HTTPS via Nginx).
Ollama internal port: 11434.
Open WebUI internal port: 8080.

FQDN of the Final Panel¶

The application is accessible via the following Fully Qualified Domain Name format: phi<Server ID>.hostkey.in:443

Replace <Server ID> with the actual numeric identifier assigned to your server instance. The application operates over HTTPS on port 443.

File and Directory Structure¶

The deployment utilizes the following directory structure for configurations, certificates, and data:

/root/nginx: Contains the Docker Compose file for the Nginx proxy and Certbot.
/root/nginx/compose.yml: Docker Compose configuration for the Nginx and Certbot service.
/data/nginx/user_conf.d: Directory containing custom Nginx configuration files for the host.
/data/nginx/user_conf.d/phi<Server ID>.hostkey.in.conf: Specific Nginx configuration for the application proxy.
/data/nginx/nginx-certbot.env: Environment file for the Nginx/Certbot service.
/etc/systemd/system/ollama.service: Systemd service file for the Ollama service.
/etc/systemd/system/ollama.service.bak: Backup of the original Ollama service file.
open-webui (Docker Volume): Persistent storage for Open WebUI backend data (/app/backend/data).

Application Installation Process¶

The application stack consists of three main components: Ollama, Open WebUI, and an Nginx reverse proxy with SSL.

Ollama Installation:
- Ollama is installed via the official shell installer script.
- The ollama system user is created to manage the service.
- The ollama service is configured to listen on all interfaces (0.0.0.0).
- The phi4 model is pulled automatically during the setup.
Open WebUI Deployment:
- The Open WebUI container is deployed using the ghcr.io/open-webui/open-webui:cuda image.
- The container is configured to connect to the local Ollama instance via the host.docker.internal gateway.
- GPU resources are passed through to the container using the --gpus all flag.
Nginx and SSL Setup:
- A Docker Compose setup is used to run Nginx and Certbot for automatic SSL certificate management.
- The Nginx configuration is dynamically updated to proxy requests to the Open WebUI container.

Docker Containers and Their Deployment¶

The system uses Docker to run the web interface and the reverse proxy.

Open WebUI Container¶

Image: ghcr.io/open-webui/open-webui:cuda
Container Name: open-webui
Port Mapping: Host port 8080 mapped to container port 8080.
GPU Access: Enabled with --gpus all.
Network Mode: Uses host gateway resolution via --add-host=host.docker.internal:host-gateway.
Environment Variables:
ENV: dev
OLLAMA_BASE_URLS: http://host.docker.internal:11434
Restart Policy: always

Nginx and Certbot Container¶

Image: jonasal/nginx-certbot:latest
Network Mode: host
Volumes:
nginx_secrets (external) mapped to /etc/letsencrypt.
Host directory /data/nginx/user_conf.d mapped to /etc/nginx/user_conf.d.
Environment:
CERTBOT_EMAIL: [email protected]
Loads environment file from /data/nginx/nginx-certbot.env.

Proxy Servers¶

The deployment utilizes Nginx as a reverse proxy to handle SSL termination and route traffic to the internal application.

Nginx Image: jonasal/nginx-certbot:latest
SSL Management: Handled automatically by Certbot within the Docker container.
Domain Configuration:
The Nginx configuration file is located at /data/nginx/user_conf.d/phi<Server ID>.hostkey.in.conf.
The proxy_pass directive in the location / block is set to forward traffic to http://127.0.0.1:8080.
Configuration Path:
Compose file: /root/nginx/compose.yml.
Custom user config: /data/nginx/user_conf.d.

Starting, Stopping, and Updating¶

The system uses a combination of systemd for the Ollama service and Docker Compose for the proxy services.

Ollama Service¶

Start/Restart:
```
systemctl restart ollama
```
Enable on Boot:
```
systemctl enable ollama
```

Nginx and Certbot Stack¶

Start/Restart Containers:
```
cd /root/nginx
docker compose up -d
```
Stop Containers:
```
cd /root/nginx
docker compose down
```

Open WebUI Container¶

Restart Container:
```
docker restart open-webui
```

Remove and Re-create Container:

docker rm -f open-webui
docker run -d -p 8080:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui -e ENV='dev' -e OLLAMA_BASE_URLS='http://host.docker.internal:11434' --restart always ghcr.io/open-webui/open-webui:cuda

Available Ports for Connection¶

The following ports are utilized in the deployment:

Port	Protocol	Service	Description
443	HTTPS	Nginx	External secure access via `phi<Server ID>.hostkey.in`
8080	HTTP	Open WebUI	Internal access point for the web interface
11434	HTTP	Ollama	Internal API endpoint for the LLM inference engine
3000	HTTP	(Internal)	Referenced in configuration but not exposed externally

Access Rights and Security¶

System User: A dedicated system user ollama is created for the Ollama service.
Firewall/Network: The application is accessible externally only via HTTPS on port 443. Internal services (Ollama and Open WebUI) are bound to specific ports but proxied through Nginx.
Environment Variables:
OLLAMA_HOST: Set to 0.0.0.0 to allow connections from Docker containers.
OLLAMA_ORIGINS: Set to * to allow cross-origin requests.
LLAMA_FLASH_ATTENTION: Set to 1 to enable flash attention optimization.

Permission Settings¶

Nginx Directory: The /root/nginx directory is owned by root with 0755 permissions.
Compose File: The /root/nginx/compose.yml file is owned by root with 0644 permissions.
Nginx Config Directory: The /data/nginx/user_conf.d directory contains configuration files accessible by the Nginx Docker container via volume mount.
Certificates: SSL certificates are stored in the nginx_secrets Docker volume mounted at /etc/letsencrypt.