Deployment Overview of Phi-4-14b on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a Linux-based server environment with the following specifications:

Operating System: Ubuntu (compatible with apt package manager).
Privileges: Root access or sudo privileges are required to install system services and manage Docker containers.
Domain Configuration: The server must be configured to resolve the hostkey.in domain.
Ports:
Port 443 (HTTPS) for external access.
Port 8080 for the internal application interface.
Port 11434 for the Ollama backend service.

The application is accessible via the following Fully Qualified Domain Name (FQDN) format: phi<Server ID>.hostkey.in:443

Replace <Server ID> with the specific identifier assigned to the server instance. The service listens on port 443 for secure HTTPS connections.

The deployment utilizes the following directory structure for configuration, data, and certificates:

/root/nginx/: Contains the Docker Compose configuration for the proxy and SSL management.
/data/nginx/user_conf.d/: Stores custom Nginx configuration files for the specific domain.
/data/nginx/nginx-certbot.env: Environment file for the Certbot service.
/etc/systemd/system/ollama.service: Systemd unit file for the Ollama service.
/app/backend/data: Persistent volume mount point for Open WebUI data.

The application stack consists of the Ollama backend and the Open WebUI frontend, deployed as follows:

Ollama Installation:
- The Ollama service is installed using the official installation script.
- The phi4 model is pulled and loaded into the Ollama runtime.
- The service is configured to listen on all network interfaces (0.0.0.0) and enables flash attention optimization.
Open WebUI Deployment:
- The Open WebUI application is deployed as a Docker container.
- The container image used is ghcr.io/open-webui/open-webui:cuda.
- The container is configured to connect to the local Ollama instance running on the host.

The deployment utilizes two primary Docker components:

Open WebUI Container:
Name: open-webui
Image: ghcr.io/open-webui/open-webui:cuda
Ports: Maps host port 8080 to container port 8080.
GPU Access: Configured with --gpus all to utilize hardware acceleration.
Environment Variables:
- ENV: Set to dev.
- OLLAMA_BASE_URLS: Set to http://host.docker.internal:11434.
Volumes: Mounts the open-webui named volume to /app/backend/data.
Restart Policy: Set to always.
Nginx and Certbot Container:
Image: jonasal/nginx-certbot:latest
Network Mode: Host.
Volumes:
- nginx_secrets: Mounted to /etc/letsencrypt for SSL certificate storage.
- /data/nginx/user_conf.d: Mounted to /etc/nginx/user_conf.d for custom configurations.
Management: Deployed via docker compose located in /root/nginx/compose.yml.

The application is fronted by an Nginx reverse proxy managed through Docker Compose to handle SSL termination and domain routing.

Proxy Configuration:
The Nginx container is configured to proxy requests from the external domain to the internal application.
The proxy pass directive is set to http://127.0.0.1:8080.
The configuration file is located at /data/nginx/user_conf.d/phi<Server ID>.hostkey.in.conf.
SSL and Certbot:
SSL certificates are managed automatically by the nginx-certbot service.
The certificate email is configured as [email protected].
Certificates are stored in the nginx_secrets volume.

System Users: The ollama system user is created and utilized to run the Ollama service.
Firewall: External access is restricted to port 443 (HTTPS). Internal services communicate over localhost or the host network.
Service Restrictions: The Ollama service is configured with OLLAMA_ORIGINS=* to allow cross-origin requests from the web interface.

The Open WebUI application utilizes a local storage mechanism for its data.

Storage Location: Data is persisted in the Docker named volume open-webui, which maps to the container path /app/backend/data.
Connection Method: The application accesses this data directly via the mounted volume; no external database connection string is required.

File and directory permissions are set as follows:

/root/nginx/: Owned by root:root with mode 0755.
/root/nginx/compose.yml: Owned by root:root with mode 0644.
/data/nginx/user_conf.d/: Configured to allow the Nginx container to read custom configuration files.

The following ports are active and configured for the deployment:

Port	Protocol	Description
443	HTTPS	External secure access via Nginx proxy.
8080	HTTP	Internal access to Open WebUI (proxied).
11434	HTTP	Internal access to the Ollama API.
3000	HTTP	Internal port defined in configuration (not directly exposed).

The services are managed using systemd for the backend and docker compose for the proxy.

Ollama Service:
Start: systemctl start ollama
Stop: systemctl stop ollama
Restart: systemctl restart ollama
Enable on boot: systemctl enable ollama
Nginx Proxy:
Start/Update: docker compose up -d (executed from /root/nginx/)
Stop: docker compose down (executed from /root/nginx/)
Open WebUI Container:
Start: docker start open-webui
Stop: docker stop open-webui
Restart: docker restart open-webui
Update: Pull the latest image with docker pull ghcr.io/open-webui/open-webui:cuda and restart the container.