Deployment Overview of Hallo3 on Server¶

Prerequisites and Basic Requirements¶

The deployment of Hallo3 requires a Linux environment with specific hardware and software dependencies to ensure proper functionality of the GPU-accelerated application and the web interface.

Operating System: Ubuntu 22.04 LTS is required.
Privileges: Root access or sudo privileges are necessary for system-wide installations and service management.
Hardware: NVIDIA GPU with compatible drivers is mandatory for running the application.
Software Dependencies:
CUDA Toolkit
NVIDIA Drivers (version 550-server or compatible)
Docker Engine
Miniconda
FFmpeg
Network: The server must have outbound internet access to download repositories, models, and certificates.

FQDN of the Final Panel¶

The application is accessible via the hostkey.in domain using the following Fully Qualified Domain Name (FQDN) format:

Format: hallo<Server ID>.hostkey.in
Port: 443 (HTTPS)

The final address will appear as hallo<Server ID>.hostkey.in:443.

File and Directory Structure¶

The application and its supporting components are organized in the following directories on the host system:

Application Source: /opt/hallo3
Contains the Hallo3 repository cloned from GitHub.
Pretrained Models: /opt/hallo3/pretrained_models
Stores the downloaded models from Hugging Face.
Miniconda Installation: /opt/miniconda3
Contains the Python environment and the hallo conda environment.
Nginx Configuration: /root/nginx
Contains the Docker Compose file for the proxy service.
Nginx User Configuration: /data/nginx/user_conf.d
Stores the specific configuration file for the Hallo3 domain.
Nginx Secrets: /data/nginx
Stores SSL certificates and environment variables for the proxy.

Application Installation Process¶

The Hallo3 application is installed manually on the host system using the following steps:

System Preparation:
Update the system packages and install the Linux Hardware Enablement (HWE) stack for Ubuntu 22.04.
Install the GCC compiler required for CUDA installation.
CUDA and NVIDIA Driver Installation:
Download and install the CUDA repository keyring package.
Install the cuda-toolkit and nvidia-cuda-toolkit packages.
Configure environment variables (PATH and LD_LIBRARY_PATH) in ~/.bashrc to include CUDA binaries and libraries.
Load NVIDIA kernel modules and verify the driver using nvidia-smi.
Miniconda Setup:
Download the Miniconda installer script.
Install Miniconda to /opt/miniconda3 in batch mode.
Initialize Conda for the shell environment.
Hallo3 Repository and Environment:
Clone the Hallo3 repository from https://github.com/fudan-generative-vision/hallo3 to /opt/hallo3.
Create a new Conda environment named hallo with Python version 3.10.
Activate the hallo environment and install Python dependencies listed in requirements.txt and the gradio package.
Model Download:
Install the huggingface-hub package.
Download the pretrained models from fudan-generative-ai/hallo3 to the /opt/hallo3/pretrained_models directory.
Application Launch:
Start the application by executing python hallo3/app.py within the hallo Conda environment.
The application listens on port 7860.

Docker Containers and Their Deployment¶

The reverse proxy and SSL termination are managed via Docker containers using Docker Compose.

Compose File Location: /root/nginx/compose.yml
Service Name: nginx
Image: jonasal/nginx-certbot:latest
Restart Policy: unless-stopped
Network Mode: host
Volumes:
nginx_secrets (external volume) mounted at /etc/letsencrypt inside the container.
/data/nginx/user_conf.d mounted at /etc/nginx/user_conf.d inside the container.
Environment:
CERTBOT_EMAIL: Set to [email protected].
Environment variables are loaded from /data/nginx/nginx-certbot.env.

To start the proxy service, execute the following command in the /root/nginx directory:

docker compose up -d

Proxy Servers¶

The application is exposed to the internet via an Nginx container running with Certbot for automatic SSL certificate management.

Proxy Configuration: The Nginx container proxies requests from the external domain to the internal application.
Internal Target: http://127.0.0.1:7860
External Path: /
Internal Path: /
SSL/TLS: Managed automatically by Certbot within the Nginx container.
Configuration File: The specific proxy rule is located in /data/nginx/user_conf.d/hallo<Server ID>.hostkey.in.conf.
Proxy Pass Directive:
```
proxy_pass http://127.0.0.1:7860;
```

Available Ports for Connection¶

The following ports are utilized by the Hallo3 deployment:

Port	Protocol	Description
7860	TCP	Internal application port (Gradio interface)
443	TCP	External HTTPS port (Nginx proxy)

Starting, Stopping, and Updating¶

The Hallo3 application and the proxy service are managed using the following methods:

Starting the Application: The application is started by activating the Conda environment and running the Python script:
```
source /opt/miniconda3/etc/profile.d/conda.sh
conda activate hallo
cd /opt/hallo3
python hallo3/app.py
```
Starting the Proxy: The Nginx proxy is started using Docker Compose:
```
cd /root/nginx
docker compose up -d
```
Stopping the Proxy: To stop the proxy service:
```
cd /root/nginx
docker compose down
```

Updating the Application: To update the Hallo3 codebase, pull the latest changes from the repository and reinstall dependencies:

cd /opt/hallo3
git pull
source /opt/miniconda3/etc/profile.d/conda.sh
conda activate hallo
pip install -r requirements.txt

Updating the Proxy: To update the Nginx container image:

cd /root/nginx
docker compose pull
docker compose up -d