Deployment Overview of Kibana on Server¶

This document provides the technical deployment specifications for the Kibana and Elasticsearch stack, including a reverse proxy configuration using Nginx in Docker.

Prerequisites and Basic Requirements¶

The application requires an Ubuntu-based operating system with administrative (root) privileges. The following network ports must be open to allow proper communication and external access:

22/tcp: SSH access
80/tcp: HTTP (for SSL certificate challenges)
443/tcp: HTTPS (secure web access)
5601/tcp: Kibana service
9200/tcp: Elasticsearch API

FQDN of the final panel on the hostkey.in domain¶

The application is accessible via a specific subdomain template based on the server ID.

Parameter	Value
Prefix	`kibana`
Domain	`hostkey.in`
Full template	`kibana{Server_ID_from_Invapi}.hostkey.in`

File and Directory Structure¶

The deployment utilizes several directories for configuration, certificates, and logs:

/etc/elasticsearch/certs: Elasticsearch SSL certificates (CA, node certificate, and private key).
/etc/kibana: Kibana configuration files and CA certificates.
/root/elastic_certs: Temporary directory used during the certificate generation process.
/root/kibana-nginx-proxy: Directory containing Docker Compose files for the Nginx proxy.
/var/www/certbot: Web root for Certbot SSL challenges.

Application installation process¶

The deployment follows a multi-stage installation method:

System Dependencies: The system is updated, and required packages such as apt-transport-https, curl, gnupg, software-properties-common, and unzip are installed.
Repository Setup: The official Elastic GPG key is added to the system keyring, and the Elastic APT repository for version 8.x is configured.
Elasticsearch Installation: Elasticsearch (version 8.18.3) is installed via the package manager.
Certificate Generation: A Certificate Authority (CA) is generated using the elasticsearch-certutil tool. Node certificates are then generated and moved to /etc/elasticsearch/certs.
Kibana Installation: Kibana (version 8.18.3) is installed via the package manager. The configuration is updated to point to the local Elasticsearch instance using HTTPS.
Nginx & SSL Setup: Nginx is installed on the host, and Certbot is used to obtain a Let's Encrypt SSL certificate for the configured FQDN.

Access Rights and Security¶

Security is enforced through several mechanisms:

Firewall: ufw (Uncomplicated Firewall) is enabled with specific rules allowing only necessary ports (22, 80, 443, 5601, 9200).
Permissions: Sensitive files, such as Elasticsearch private keys and Kibana configuration files, are restricted using strict ownership (elasticsearch or kibana users) and permission modes (e.g., 0600 or 0640).
Encryption: All communication between Kibana and Elasticsearch is encrypted via SSL/TLS.

Databases¶

The application uses Elasticsearch as its primary data store.

Connection Method: Localhost connection via HTTPS (https://localhost:9200).
Storage Location: /var/lib/elasticsearch.
Security Settings: X-Pack security is enabled, requiring authentication for all requests.

Docker Containers and Their Deployment¶

The deployment includes a dedicated Nginx proxy running in a Docker container to handle SSL termination and reverse proxying to the Kibana service.

Nginx Proxy Container

Image: nginx:latest
Container Name: Defined by system variables
Ports: 80:80, 443:443
Volumes:
/etc/nginx/sites-enabled/{domain}.conf:/etc/nginx/conf.d/{domain}.conf:ro
/etc/letsencrypt:/etc/letsencrypt:ro
/var/www/certbot:/var/www/certbot:ro
Restart Policy: always
Network Mode: host

Custom Scripts and Additional Setup¶

The installation performs several automated configuration steps:

Credential Management: Elasticsearch passwords for the elastic and kibana_system users are reset to a known secure value and stored in /root/.kibana_passwords.
Certificate Provisioning: Automated generation of self-signed CA and node certificates using elasticsearch-certutil.
MOTD Configuration: A custom Message of the Day (MOTD) is created at /etc/motd to display connection information and credential locations upon login.

Application Update Instructions¶

To update the main application:

Elasticsearch & Kibana: Since these are installed via the APT package manager, use the standard system update commands:
```
sudo apt update
sudo apt upgrade elasticsearch kibana
```
Nginx Proxy: To update the Nginx proxy container, navigate to the deployment directory and run:
```
cd /root/kibana-nginx-proxy
docker compose pull && docker compose up -d
```

Location of configuration files and data¶

Component	Configuration Path	Data Path
Elasticsearch	`/etc/elasticsearch/elasticsearch.yml`	`/var/lib/elasticsearch`
Kibana	`/etc/kibana/kibana.yml`	`/var/log/kibana/kibana.log`

Available ports for connection¶

HTTPS (Web UI): 443
Elasticsearch API: 9200
Kibana Service: 5601

Starting and Stopping the application¶

The services are managed via systemd:

Starting Services:

sudo systemctl start elasticsearch
sudo systemctl start kibana
sudo systemctl start nginx

Stopping Services:

sudo systemctl stop elasticsearch
sudo systemctl stop kibana
sudo systemctl stop nginx

Proxy Servers¶

The application uses an Nginx reverse proxy running in a Docker container. This proxy handles SSL termination using certificates obtained via Certbot and forwards traffic to the Kibana service running on http://localhost:5601.