Ollama Installation¶

In this article

• Introduction to Ollama
• Updating Ollama on Linux
• Installing Language Models LLM
• Environment Variables

Introduction to Ollama¶

Ollama is a framework for running and managing large language models (LLMs) on local computing resources. It enables the loading and deployment of selected LLMs and provides access to them through an API.

System Requirements:

Installing Ollama on Linux¶

1. Download and install Ollama:

   curl -L https://ollama.com/download/ollama-linux-amd64 -o /usr/bin/ollama
   chmod +x /usr/bin/ollama
   

2. Create a group:

   sudo useradd -r -s /bin/false -m -d /usr/share/ollama ollama
   

3. Create the Ollama service:

   tee /usr/lib/systemd/system/ollama.service > /dev/null <<EOF
   [Unit]
   Description=Ollama Service
   After=network-online.target
   
   [Service]
   ExecStart=/usr/bin/ollama serve
   User=ollama
   Group=ollama
   Restart=always
   RestartSec=3
   Environment="OLLAMA_HOST=0.0.0.0" 
   Environment="OLLAMA_ORIGINS=*"
   
   [Install]
   WantedBy=default.target
   EOF
   

   For Nvidia GPUs, add Environment="OLLAMA_FLASH_ATTENTION=1" to improve token generation speed.

4. Enable and start the service:

   sudo systemctl daemon-reload
   sudo systemctl enable ollama
   sudo systemctl start ollama
   

Ollama will be accessible at http://127.0.0.1:11434 or http://<you_server_IP>:11434.

Updating Ollama on Linux¶

To update Ollama, you will need to re-download and install its binary package:

sudo curl -L https://ollama.com/download/ollama-linux-amd64 -o /usr/bin/ollama
sudo chmod +x /usr/bin/ollama

For ease of future updates, you can create a script ollama_update.sh (run as root or with sudo):

#!/bin/bash
systemctl stop ollama
sudo curl -L https://ollama.com/download/ollama-linux-amd64 -o /usr/bin/ollama
sudo chmod +x /usr/bin/ollama
systemctl start ollama

Installing Language Models LLM¶

You can find the list of available language models on this page.

To install a model, click on its name and then select the size and type of the model on the next page. Copy the installation command from the right-hand window and run it in your terminal/command line:

ollama run llama3

After downloading the model, restart the service:

service ollama restart

For more information about Ollama, you can read the developer documentation.

Environment Variables¶

Set this variables in Ollama service as Environment="VARIABLE=VALUE".