Deployment Overview of Open WebUI with DeepSeek-R1:70B on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a server running the Ubuntu operating system. The following conditions must be met before proceeding: - Root privileges or sudo access are required for all installation and configuration steps. - The system must have internet access to download packages, Docker images, and model weights. - The following ports must be available and open on the server: - Port 80 for HTTP traffic and ACME challenge validation. - Port 443 for HTTPS traffic. - Port 8080 for the internal Open WebUI service. - Port 11434 for the Ollama API service.

File and Directory Structure¶

The application utilizes specific directories for configuration, data storage, and certificates: - /root/nginx: Contains the Docker Compose configuration for the reverse proxy and SSL management. - /root/nginx/compose.yml: The Docker Compose file defining the Nginx and Certbot services. - /data/nginx/user_conf.d: Stores custom Nginx virtual host configuration files. - /data/nginx/nginx-certbot.env: Environment file containing settings for the Nginx-Certbot container. - /etc/systemd/system/ollama.service: The main systemd unit file for the Ollama service. - /etc/systemd/system/ollama.service.d/override.conf: Systemd drop-in configuration for Ollama environment variables. - /var/www/certbot: The webroot directory inside the Nginx container used for ACME challenge validation.

Application Installation Process¶

The deployment involves installing the Ollama runtime, pulling the specific AI model, and launching the Open WebUI interface via Docker.

1. Install Ollama: The Ollama package is installed using the official installation script.
2. Configure Ollama Service: The ollama systemd service is modified to listen on all network interfaces (0.0.0.0) and allow all origins. Flash attention is enabled via the LLAMA_FLASH_ATTENTION environment variable.
3. Pull Model: The deepseek-r1:70b model is downloaded and cached locally by the Ollama service.
4. Deploy Open WebUI: The Open WebUI application is deployed as a Docker container using the ghcr.io/open-webui/open-webui:cuda image. The container is configured to use the host network mode and access the GPU.

Access Rights and Security¶

Security is managed through a reverse proxy with SSL termination and specific service configurations: - SSL/TLS: HTTPS is enforced for all external traffic. SSL certificates are managed automatically using Let's Encrypt via the Certbot integration within the Nginx container. - Firewall: External access to the application is restricted to ports 80 and 443. Internal services (Ollama on 11434 and Open WebUI on 8080) are bound to 127.0.0.1 or accessed via the host network, preventing direct external exposure of the backend services. - User Accounts: The ollama system user is created to manage the Ollama service processes.

Docker Containers and Their Deployment¶

Two primary Docker containers are deployed to run the application stack:

1. Open WebUI Container:

   • Image: ghcr.io/open-webui/open-webui:cuda
   • Name: open-webui
   • Network Mode: host
   • Environment Variables:
   • ENV: dev
   • OLLAMA_BASE_URLS: http://127.0.0.1:11434
   • Volumes: A named volume open-webui is mounted to /app/backend/data for persistent data storage.
   • GPU Access: The container requests GPU capabilities to accelerate model inference.
   • Restart Policy: Configured to always.

2. Nginx-Certbot Container:

   • Image: jonasal/nginx-certbot:latest
   • Network Mode: host
   • Volumes:
   • nginx_secrets (external) mounted to /etc/letsencrypt for certificate storage.
   • /data/nginx/user_conf.d mounted to /etc/nginx/user_conf.d for custom configurations.
   • Environment: Uses an environment file located at /data/nginx/nginx-certbot.env and sets CERTBOT_EMAIL to [email protected].

Proxy Servers¶

The Nginx-Certbot container acts as the reverse proxy and SSL terminator for the application: - HTTPS Configuration: A virtual host is configured to proxy requests to the Open WebUI service running on http://127.0.0.1:8080. - Header Handling: The Host header is explicitly passed to the upstream service using proxy_set_header Host $host. - HTTP Redirect: An HTTP virtual host on port 80 is configured to redirect all traffic to HTTPS (301 redirect). - ACME Challenge: The HTTP server includes a location block for /.well-known/acme-challenge/ pointing to /var/www/certbot to facilitate automatic certificate issuance and renewal. - Configuration Files: Custom server configurations are stored in /data/nginx/user_conf.d with naming conventions based on the prefix and server ID.

Permission Settings¶

File and directory permissions are set to ensure proper access for the services: - /root/nginx: Owned by root:root with mode 0755. - /root/nginx/compose.yml: Owned by root:root with mode 0644. - /etc/systemd/system/ollama.service.d: Created with mode 0755. - /data/nginx/user_conf.d: Contains configuration files with mode 0644. - /var/www/certbot: The directory structure for ACME challenges is created inside the Nginx container with appropriate permissions for the web server process.

Starting, Stopping, and Updating¶

The services are managed using systemd for Ollama and Docker Compose for the proxy stack.

• Ollama Service:
• To restart the service: systemctl restart ollama
• To enable the service on boot: systemctl enable ollama

• The service is configured to reload the daemon and restart automatically after configuration changes.

• Nginx-Certbot Stack:

• To start or update the proxy stack: docker compose up -d executed from the /root/nginx directory.
• To test the Nginx configuration inside the container: docker exec nginx-nginx-1 nginx -t

• To reload Nginx configuration inside the container: docker exec nginx-nginx-1 nginx -s reload

• Open WebUI Container:

• The container is managed by Docker and configured with a restart: always policy, ensuring it starts automatically if the system reboots or the container crashes.