Deployment Overview of Hallo3 on Server¶

Prerequisites and Basic Requirements¶

The deployment of Hallo3 requires a server running Ubuntu 22.04 with root privileges. The system must have access to the internet to download dependencies, CUDA toolkits, and pretrained models. The following hardware and software components are mandatory:

Operating System: Ubuntu 22.04 LTS
Hardware: NVIDIA GPU with compatible drivers
Privileges: Root access (sudo)
Ports: Port 7860 must be open for the Gradio interface
Storage: Sufficient disk space for CUDA toolkits, Miniconda environments, and pretrained models

File and Directory Structure¶

The application and its supporting components are organized in the following directory structure:

/opt/hallo3: Main application directory containing the Hallo3 source code cloned from the repository.
/opt/hallo3/pretrained_models: Directory storing downloaded pretrained models from Hugging Face.
/opt/miniconda3: Installation directory for the Miniconda package manager.
/opt/miniconda3/envs/hallo: Conda environment specifically created for Hallo3 dependencies.
/root/nginx: Directory containing the Docker Compose configuration for the Nginx proxy.
/root/nginx/compose.yml: Docker Compose file defining the Nginx and Certbot services.
/data/nginx/user_conf.d: Directory containing custom Nginx configuration files for specific host keys.
/data/nginx/nginx-certbot.env: Environment file for Nginx Certbot configuration.

Application Installation Process¶

The Hallo3 application is installed manually on the host system using the following steps:

Install NVIDIA utilities and the CUDA toolkit:

sudo apt install nvidia-utils-550-server
sudo apt install cuda

Download and install Miniconda:

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -p /opt/miniconda3

Initialize Conda and add it to the system PATH:

source /opt/miniconda3/etc/profile.d/conda.sh
export PATH="/opt/miniconda3/bin:$PATH"

Clone the Hallo3 repository:

git clone https://github.com/fudan-generative-vision/hallo3 /opt/hallo3

Create and activate the Conda environment:

conda create -n hallo python=3.10 -y
conda activate hallo

Install Python dependencies:

pip install -r requirements.txt
pip install gradio
pip install huggingface-hub

Install system dependencies:
```
sudo apt install ffmpeg
```

Download pretrained models:

huggingface-cli download fudan-generative-ai/hallo3 --local-dir /opt/hallo3/pretrained_models

Docker Containers and Their Deployment¶

The Nginx reverse proxy and SSL certificate management are deployed using Docker Compose. The configuration is located in /root/nginx/compose.yml.

Service Name: nginx
Image: jonasal/nginx-certbot:latest
Restart Policy: unless-stopped
Network Mode: host
Environment Variables:
CERTBOT_EMAIL: Set to [email protected]
Volumes:
nginx_secrets: Mounted to /etc/letsencrypt for SSL certificate storage.
/data/nginx/user_conf.d: Mounted to /etc/nginx/user_conf.d for custom configurations.

To start the proxy services, execute the following command from the /root/nginx directory:

docker compose up -d

Proxy Servers¶

The Nginx container acts as a reverse proxy for the Hallo3 application. It handles SSL termination via Certbot and routes traffic to the application running on the host.

The proxy configuration is dynamically updated to forward requests to http://127.0.0.1:7860.
Custom host configurations are stored in /data/nginx/user_conf.d/ with filenames following the pattern {prefix}{server_id}.hostkey.in.conf.
The proxy_pass directive within the location block is configured to direct traffic to the local Gradio instance.

Starting, Stopping, and Updating¶

The Hallo3 application is started as a background process within the Conda environment.

To start the application:

source /opt/miniconda3/etc/profile.d/conda.sh
conda activate hallo
cd /opt/hallo3
python hallo3/app.py

The application listens on port 7860. To verify the application is running, check if the port is open:

netstat -tlnp | grep 7860

To update the application code, pull the latest changes from the repository:

cd /opt/hallo3
git pull origin main

After updating the code, restart the application process. To update the Nginx proxy configuration, modify the files in /data/nginx/user_conf.d/ and restart the Docker container:

cd /root/nginx
docker compose down
docker compose up -d