Run Ollama on a Hostperl VPS with Private AI Access

What you will set up

This tutorial shows you how to run Ollama on a fresh VPS, keep it private, and expose it through Nginx only after you verify the service locally. You will create a non-root sudo user, harden SSH, install Docker, run Ollama with persistent storage, add a reverse proxy, and test access from the server and your browser.

If you are choosing infrastructure for private AI workloads, a Hostperl VPS gives you control without handing the stack to a shared platform. For customers who want data, prompts, and model traffic under their own account, that matters more than marketing language.

For background on the broader decision, see Private AI Hosting on VPS: What Buyers Need in 2026. If you plan to add retrieval later, pgvector on a Hostperl VPS is the next step after this setup.

Connect to the VPS and check the operating system

On your local computer, connect as root first.

ssh root@203.0.113.10

That address is a documentation example. Replace 203.0.113.10 with the real public IP assigned to your Hostperl server. If your provider gave you a non-root SSH login, use that account first and keep the same IP.

On the VPS as root, detect the OS before you install anything.

cat /etc/os-release

You will use the Ubuntu/Debian commands if the output shows ID=ubuntu or ID=debian. Use the AlmaLinux/Rocky Linux commands if it shows ID=almalinux or ID=rocky.

Create a sudo user before you do the rest

Keep the root session open. Open a second terminal so you can test the new login before you lock anything down.

Ubuntu and Debian — on the VPS as root

adduser deploy
usermod -aG sudo deploy
mkdir -p /home/deploy/.ssh
chmod 700 /home/deploy/.ssh

This creates the deploy account and gives it sudo access. Next, copy your SSH public key from your local machine into authorized_keys.

cat > /home/deploy/.ssh/authorized_keys <<'EOF'
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIExampleKeyReplaceWithYourOwnKey
EOF
chown -R deploy:deploy /home/deploy/.ssh
chmod 600 /home/deploy/.ssh/authorized_keys

AlmaLinux and Rocky Linux — on the VPS as root

useradd -m -s /bin/bash deploy
passwd deploy
usermod -aG wheel deploy
mkdir -p /home/deploy/.ssh
chmod 700 /home/deploy/.ssh

Now add your public key, fix ownership, and lock permissions.

cat > /home/deploy/.ssh/authorized_keys <<'EOF'
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIExampleKeyReplaceWithYourOwnKey
EOF
chown -R deploy:deploy /home/deploy/.ssh
chmod 600 /home/deploy/.ssh/authorized_keys

On your local computer, open a second terminal and test the new account.

ssh deploy@203.0.113.10

You should land in the shell without a password prompt if your key is correct. Then verify sudo.

sudo -v
whoami
id

You want to see root for sudo validation and deploy in the user and group listing. Only after this works should you consider disabling root login or password authentication.

Update packages and install Docker

Private AI access works best when the runtime is easy to restart and update. Docker keeps the Ollama service isolated, and it fits the same pattern VPS users often rely on later for separate web apps or workers.

Ubuntu and Debian — on the VPS as the non-root sudo user

sudo apt update
sudo apt -y upgrade
sudo apt -y install ca-certificates curl gnupg
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release; echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker
sudo docker --version

AlmaLinux and Rocky Linux — on the VPS as the non-root sudo user

sudo dnf -y update
sudo dnf -y install dnf-plugins-core ca-certificates curl
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker
sudo docker --version

If Docker starts cleanly, the version command prints the installed release and the service stays active.

Run Ollama with persistent storage

Create a working directory and a Compose file. This keeps the model data on disk, which makes migration and recovery easier for support teams and agencies.

On the VPS as the non-root sudo user

sudo mkdir -p /opt/myapp/ollama
cd /opt/myapp/ollama
sudo nano compose.yml

Paste this file, then save and exit with Ctrl+O, Enter, and Ctrl+X.

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "127.0.0.1:11434:11434"
    volumes:
      - ollama_data:/root/.ollama

volumes:
  ollama_data:

Start it and confirm the container is healthy enough to answer local requests.

sudo docker compose up -d
sudo docker ps
curl http://127.0.0.1:11434/api/tags

The API may return an empty model list at first. That is fine. The important part is that the service responds locally.

Pull a model and test inference

Choose a small model first. That keeps the download manageable on a new VPS and gives you a quick sanity check before you commit more RAM or storage.

On the VPS as the non-root sudo user

sudo docker exec -it ollama ollama pull llama3.2:3b
sudo docker exec -it ollama ollama run llama3.2:3b "Write one short sentence about private AI access."

If the model pulls successfully, the second command should return a short text answer. If it hangs, check the container logs next.

sudo docker logs --tail 50 ollama

On smaller VPS plans, model downloads and inference can use most available memory. If the container exits, move to a larger plan before you spend time tuning around the limit. Hostperl customers who expect repeated model loads usually start with a VPS sized for extra RAM or move to dedicated server hosting once usage becomes steady.

Open the firewall only after local checks pass

Keep Ollama bound to localhost. You will publish it through Nginx, not directly to the internet. That avoids exposing the model API to the whole network.

Ubuntu and Debian — on the VPS as root or sudo user

sudo apt -y install ufw nginx
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

AlmaLinux and Rocky Linux — on the VPS as root or sudo user

sudo dnf -y install firewalld nginx
sudo systemctl enable --now firewalld
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

On SELinux-enabled systems, Nginx is allowed to proxy outbound HTTP after you set the right boolean later in this guide.

Put Nginx in front of the API

Reverse proxying keeps the Ollama port private while serving a predictable hostname over port 80 or 443. This is the pattern most customers want when internal teams or a small support workflow need access to one service.

On the VPS as the non-root sudo user, create a server block.

sudo nano /etc/nginx/conf.d/ollama.conf

Paste the file below, then save and exit.

server {
    listen 80;
    server_name server.example.com;

    location / {
        proxy_pass http://127.0.0.1:11434;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Test the config before reload.

sudo nginx -t
sudo systemctl reload nginx
sudo systemctl status nginx --no-pager

On AlmaLinux and Rocky Linux with SELinux, allow Nginx to make outbound proxy connections.

sudo setsebool -P httpd_can_network_connect 1

If you later move this service behind a certificate, you can pair it with the same approach used in Deploy PHP-FPM Behind Nginx on Hostperl VPS; the proxy pattern is the same even though the backend is different.

Add HTTPS with Let’s Encrypt

Point server.example.com to your VPS IP in DNS first. Then request a certificate.

Ubuntu and Debian — on the VPS as root or sudo user

sudo apt -y install certbot python3-certbot-nginx
sudo certbot --nginx -d server.example.com

AlmaLinux and Rocky Linux — on the VPS as root or sudo user

sudo dnf -y install certbot python3-certbot-nginx
sudo certbot --nginx -d server.example.com

After issuance, test renewal logic.

sudo certbot renew --dry-run

A successful dry run tells you the certificate will renew automatically before expiry.

Harden access and keep the service maintainable

At this point, you can disable password SSH login only if your second terminal still works with the deploy account. Keep root login available until you are sure you can recover the VPS.

On the VPS as root, edit SSH policy carefully.

sudo nano /etc/ssh/sshd_config

Add or update these lines.

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes

Test the syntax and reload SSH.

sudo sshd -t
sudo systemctl reload sshd

On Ubuntu and Debian, the service name is usually ssh. Use this if sshd does not match your system.

sudo systemctl reload ssh

For brute-force protection, Fail2Ban is a sensible extra layer.

Ubuntu and Debian

sudo apt -y install fail2ban
sudo systemctl enable --now fail2ban
sudo fail2ban-client status

AlmaLinux and Rocky Linux

sudo dnf -y install fail2ban
sudo systemctl enable --now fail2ban
sudo fail2ban-client status

You should see the jail service active. If you need to inspect suspicious login attempts later, use audit logs and journal entries rather than guessing.

sudo journalctl -u ssh --since "1 hour ago"
sudo journalctl -u nginx --since "1 hour ago"
sudo docker logs --tail 100 ollama

Final verification from server and client

On the VPS, confirm that every service is running and the ports are where you expect them.

sudo ss -tulpn | grep -E '(:80|:443|:11434)'
sudo systemctl is-active nginx
sudo docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'

On your local computer, test the public endpoint.

curl -I https://server.example.com
curl https://server.example.com/api/tags

Then run one real functional smoke test from your browser or terminal. A simple prompt through the API confirms the reverse proxy, TLS, and backend container are all working together.

curl https://server.example.com/api/generate -d '{"model":"llama3.2:3b","prompt":"Reply with one sentence confirming private AI access works."}'

If you reboot the VPS, the container should come back because the Compose service uses restart: unless-stopped and Docker starts on boot. Check that with:

sudo reboot

After reconnecting, run:

sudo systemctl is-enabled docker
sudo docker ps
sudo systemctl is-active nginx

Troubleshooting the failures you are most likely to see

If the model pull fails, diagnose storage and memory first.

free -h
df -h
sudo docker logs --tail 100 ollama

If memory is tight, the logs usually show the container being killed or the pull failing mid-stream. Add RAM, add swap, or choose a smaller model.

If Nginx returns 502, check the backend port and local API response.

curl http://127.0.0.1:11434/api/tags
sudo tail -n 50 /var/log/nginx/error.log

If the API works locally but not over HTTPS, the issue is usually the server block, DNS, or certificate. Re-run nginx -t and confirm the domain points to the VPS IP.

If SSH locks you out after hardening, use your provider console or rescue access, revert sshd_config, and re-test with a second open session before trying again.

FAQ

What size VPS do I need for Ollama?

For a light test setup, start with at least 2 vCPU, 4 GB RAM, and enough disk for the model files. If you want smoother inference or larger models, plan for more memory first.

Should I expose port 11434 to the internet?

No. Keep it bound to 127.0.0.1 and publish it through Nginx. That keeps the model API behind TLS and standard web controls.

Can I run this with a custom domain?

Yes. Point server.example.com to the VPS IP, issue a certificate with Certbot, and keep the upstream service private on localhost.

How do I update Ollama later?

Run sudo docker compose pull and sudo docker compose up -d inside /opt/myapp/ollama. Always check the logs after an update.