Deployment Overview of Kibana on Server¶

Prerequisites and Basic Requirements¶

The deployment requires a server running Ubuntu with the following specifications:

Operating System: Ubuntu (with universe repository enabled).
Privileges: Root access or sudo privileges are required for all installation and configuration steps.
Domain: A valid domain name (kibana_domain) must be configured and pointed to the server's IP address.
Ports: The following TCP ports must be open and accessible:
22 (SSH)
80 (HTTP for SSL redirection)
443 (HTTPS for secure access)
5601 (Kibana internal port)
9200 (Elasticsearch internal port)

The application utilizes the following directory structure for configuration, data, and certificates:

The deployment installs Elasticsearch and Kibana using the official Elastic APT repository for version 8.x.

Dependencies Installation: The system installs apt-transport-https, curl, gnupg, software-properties-common, and unzip.
Repository Configuration: The Elastic GPG key is added, and the https://artifacts.elastic.co/packages/8.x/apt repository is configured.
Elasticsearch Installation: The elasticsearch package is installed via apt.
Certificate Generation:
- A Certificate Authority (CA) is generated using elasticsearch-certutil ca.
- A node certificate with Subject Alternative Names (SAN) for localhost and 127.0.0.1 is generated.
- Certificates are extracted and moved to /etc/elasticsearch/certs/ with appropriate ownership (elasticsearch:elasticsearch).
Kibana Installation: The kibana package is installed via apt.
Nginx and Certbot Installation:
- nginx is installed.
- certbot and python3-certbot-nginx are installed to manage SSL certificates.

Security is enforced through TLS encryption, firewall rules, and user permissions.

Firewall (UFW): The Uncomplicated Firewall (ufw) is enabled and configured to allow traffic on ports 22, 80, 443, 5601, and 9200.
SSL/TLS:
Elasticsearch and Kibana communicate over HTTPS using self-signed certificates generated by elasticsearch-certutil.
Public access is secured via Let's Encrypt certificates managed by Certbot.
User Accounts:
The elastic user and kibana_system user are configured with passwords stored in /root/.kibana_passwords.
File permissions for private keys are set to 0600 and certificates to 0640.
Service Isolation: Both Elasticsearch and Kibana are configured to bind to 127.0.0.1, preventing direct external access to the application ports.

Elasticsearch serves as the backend database for Kibana.

Connection Method: Kibana connects to Elasticsearch via https://localhost:9200.
Authentication: Kibana authenticates using the kibana_system user.
Storage Location: Data is stored in /var/lib/elasticsearch.
Configuration Settings:
cluster.name: elastic-cluster
node.name: elastic-node-1
discovery.type: single-node
Security is enabled (xpack.security.enabled: true) with transport and HTTP SSL verification.
Disk watermark thresholds are set to 85% (low), 90% (high), and 95% (flood_stage).

A Docker container is deployed to run Nginx as a reverse proxy for Kibana.

Directory: The Docker Compose configuration is located in /root/kibana-nginx-proxy.
Compose File: compose.yml defines the nginx service.
Service Configuration:
Image: nginx:latest
Container Name: Defined by nginx_container_name.
Ports: Exposes 80 and 443 on the host.
Volumes:
- Mounts the Nginx configuration file from /etc/nginx/sites-enabled/.
- Mounts Let's Encrypt certificates from /etc/letsencrypt.
- Mounts the Certbot webroot from /var/www/certbot.
Network Mode: Uses host network mode.
Restart Policy: Set to always.

Nginx acts as the reverse proxy and SSL terminator for Kibana.

Configuration Location: The active configuration is linked in /etc/nginx/sites-enabled/.
HTTP Handling: Port 80 listens for requests and redirects all traffic to HTTPS (301 redirect).
HTTPS Handling: Port 443 handles secure connections using certificates from /etc/letsencrypt/live/.
Proxy Settings:
proxy_pass: Forwards requests to http://localhost:5601.
Headers: Host, X-Real-IP, X-Forwarded-For, and X-Forwarded-Proto are set to preserve client information.
ACME Challenge: The /.well-known/acme-challenge/ location is configured to serve files from /var/www/certbot for automatic certificate renewal.

File and directory permissions are strictly enforced to ensure security:

Elasticsearch Directories:
/usr/share/elasticsearch/logs: 0755, owned by elasticsearch:elasticsearch.
/etc/elasticsearch/certs/: 0750, owned by elasticsearch:elasticsearch.
Private keys (*.key): 0600.
Certificates (*.crt): 0640.
Kibana Directories:
/etc/kibana/: 0750, owned by kibana:kibana.
ca.crt: 0640.
Nginx Configuration:
/etc/nginx/sites-enabled/: 0644, owned by root:root.
Docker Compose Directory:
/root/kibana-nginx-proxy: 0755, owned by root:root.
Credentials File:
/root/.kibana_passwords: 0600, owned by root.

Services are managed using systemd for native installations and docker compose for the proxy container.

Elasticsearch:
Start: systemctl start elasticsearch
Stop: systemctl stop elasticsearch
Restart: systemctl restart elasticsearch
Enable on boot: systemctl enable elasticsearch
Kibana:
Start: systemctl start kibana
Stop: systemctl stop kibana
Restart: systemctl restart kibana
Enable on boot: systemctl enable kibana
Nginx:
Start: systemctl start nginx
Stop: systemctl stop nginx
Restart: systemctl restart nginx
Status Check: systemctl status nginx
Docker Proxy:
Start/Update: Navigate to /root/kibana-nginx-proxy and run docker compose up -d.
Stop: Run docker compose down in the same directory.