Deploy Private AI Hosting on a VPS in 2026

What you are building

This tutorial sets up private AI hosting on a fresh VPS with Docker, PostgreSQL and pgvector, plus Nginx as the reverse proxy. You end up with a locked-down host, a working application stack, and a clear way to confirm that your AI service stays private and restarts cleanly.

If you are choosing infrastructure for this kind of workload, a Hostperl VPS gives you the control you need without putting you on a shared platform. That matters when your model endpoint, embeddings, and API keys all need to stay under your own account.

If you plan to grow later, this setup also maps well to a dedicated server. A predictable directory layout and systemd-managed services make that move much easier.

What you will have at the end: SSH hardening, a non-root sudo user, Docker and Compose, PostgreSQL with pgvector, a sample private AI app, Nginx TLS-ready proxying, and verification commands from both the server and your client.

Connect to the VPS and check the OS

On your local computer: connect to the server first.

ssh root@203.0.113.10

203.0.113.10 is a reserved documentation example. Replace it with the real public IP assigned by your hosting provider.

If your provider gave you a default non-root account, the first login may look like this instead:

ssh deploy@203.0.113.10

After you log in, identify the operating system before you install anything.

On the VPS as root:

cat /etc/os-release

Look for ID=ubuntu, ID=debian, ID=almalinux, or ID=rocky. The next steps split where package names differ.

Create a non-root sudo user for private AI hosting

Do not build the stack as root. Keep the root session open until you verify the new login works.

Ubuntu and Debian

On the VPS as root:

adduser deploy

Set a strong password when prompted. Then give the account sudo access.

usermod -aG sudo deploy

Create the SSH directory and copy your public key from your local computer.

mkdir -p /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
cp /root/.ssh/authorized_keys /home/deploy/.ssh/authorized_keys
chown -R deploy:deploy /home/deploy/.ssh
chmod 600 /home/deploy/.ssh/authorized_keys

If your root account does not already have the right key, paste your public key from your local computer into /home/deploy/.ssh/authorized_keys with nano or vi.

AlmaLinux and Rocky Linux

On the VPS as root:

useradd -m -s /bin/bash deploy
passwd deploy

Then add the account to the wheel group, which provides sudo on RHEL-compatible systems.

usermod -aG wheel deploy

Now prepare SSH access.

mkdir -p /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
cp /root/.ssh/authorized_keys /home/deploy/.ssh/authorized_keys
chown -R deploy:deploy /home/deploy/.ssh
chmod 600 /home/deploy/.ssh/authorized_keys

Open a second terminal on your local computer and test the new account before you change any root login settings.

On your local computer:

ssh deploy@203.0.113.10

Then verify sudo.

On the VPS as the non-root sudo user:

sudo -v

If that succeeds, keep this session open and return to the root session only when you need it for admin tasks.

For a wider walkthrough of account handling, Hostperl also publishes a companion guide on creating a non-root sudo user on Linux VPS.

Update packages, time sync, and base tools

Private AI services are sensitive to stale packages and bad clocks. Update first, then install the tools you need.

Ubuntu and Debian

On the VPS as the non-root sudo user:

sudo apt update
sudo apt -y upgrade
sudo apt -y install ca-certificates curl gnupg git ufw fail2ban chrony

Confirm the installed versions and that time sync is active.

chronyc tracking

Expected result: a current time source and a small offset, not a large drift.

AlmaLinux and Rocky Linux

On the VPS as the non-root sudo user:

sudo dnf -y update
sudo dnf -y install ca-certificates curl git firewalld fail2ban chrony

Enable time synchronization and verify it.

sudo systemctl enable --now chronyd
chronyc tracking

Expected result: your server reports an active source and sensible offset values.

Set the hostname and add swap if needed

Use a clear hostname so logs and prompts make sense during support work.

On the VPS as root or the sudo user:

sudo hostnamectl set-hostname server.example.com
hostnamectl

server.example.com is a documentation hostname. Replace it with your real server name if you have one.

If the VPS has little RAM, add swap before you build and test the stack. Hostperl has a separate procedure for that in Add Swap on Ubuntu, Debian, AlmaLinux, and Rocky Linux.

Harden SSH and open the firewall safely

Make firewall changes before you disable any existing access path. That prevents lockouts.

Ubuntu and Debian with UFW

On the VPS as the non-root sudo user:

sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

OpenSSH stays allowed, and ports 80 and 443 are ready for Nginx and TLS.

AlmaLinux and Rocky Linux with firewalld

On the VPS as the non-root sudo user:

sudo systemctl enable --now firewalld
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

After the firewall is active, tighten SSH itself.

On the VPS as root: edit /etc/ssh/sshd_config.

nano /etc/ssh/sshd_config

Add or set these lines:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers deploy

Save and exit, then test the configuration before you reload SSH.

sshd -t

If there is no output, the syntax is valid.

On Ubuntu and Debian as root:

systemctl reload ssh

On AlmaLinux and Rocky Linux as root:

systemctl reload sshd

For admins who want a fuller security pass after deployment, Hostperl also covers Linux VPS sudo setup and support-friendly account workflows.

Install Docker and Docker Compose

Docker keeps the application layer clean and makes rollbacks simpler. The exact install path differs by distribution.

Ubuntu and Debian

On the VPS as the non-root sudo user:

sudo apt -y install docker.io docker-compose-plugin
sudo systemctl enable --now docker
sudo docker version
sudo docker compose version

You should see a working Docker Engine and the Compose plugin.

AlmaLinux and Rocky Linux

On the VPS as the non-root sudo user:

sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo systemctl enable --now docker
sudo docker version
sudo docker compose version

Add your user to the docker group so you do not need sudo for every Compose command.

sudo usermod -aG docker deploy

Log out and back in, then confirm group membership with id.

Build the private AI app with PostgreSQL and pgvector

This example uses PostgreSQL as the data store and pgvector for embeddings. It is a practical pattern for internal search, document lookup, and private AI assistants.

First, create the application directory.

On the VPS as the non-root sudo user:

sudo mkdir -p /opt/myapp
sudo chown -R deploy:deploy /opt/myapp
cd /opt/myapp
pwd

Now create the Compose file.

nano /opt/myapp/docker-compose.yml

Paste this file content:

services:
  db:
    image: pgvector/pgvector:pg16
    container_name: myapp-db
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: change_this_password
    volumes:
      - db_data:/var/lib/postgresql/data
    restart: unless-stopped

  app:
    image: ghcr.io/example/private-ai-app:latest
    container_name: myapp-app
    environment:
      DATABASE_URL: postgresql://myapp:change_this_password@db:5432/myapp
      APP_PORT: 8000
    depends_on:
      - db
    restart: unless-stopped
    expose:
      - "8000"

volumes:
  db_data:

Save and exit. Replace change_this_password with a long, unique password before you start the stack.

Start the containers and check their status.

cd /opt/myapp
sudo docker compose up -d
sudo docker compose ps

Expected result: both containers show Up.

Check the app logs if the container does not start.

sudo docker compose logs --tail=100 app
sudo docker compose logs --tail=100 db

For a deeper deployment pattern, Hostperl’s Git-based Docker deployments on VPS guide shows how teams keep this workflow repeatable.

Put Nginx in front of the app

Nginx handles TLS, host routing, and a cleaner public edge. It also gives you a simple place to add caching headers later.

Install Nginx

On the VPS as the non-root sudo user:

sudo apt -y install nginx

On AlmaLinux and Rocky Linux, use:

sudo dnf -y install nginx

Enable and start the service.

sudo systemctl enable --now nginx
sudo systemctl status nginx --no-pager

Create the reverse proxy config

On the VPS as root:

nano /etc/nginx/sites-available/private-ai

Use this configuration on Ubuntu and Debian:

server {
    listen 80;
    server_name example.com server.example.com;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Enable the site and test syntax.

ln -s /etc/nginx/sites-available/private-ai /etc/nginx/sites-enabled/private-ai
nginx -t

On AlmaLinux and Rocky Linux, place the same server block in /etc/nginx/conf.d/private-ai.conf, then run nginx -t.

Reload Nginx only after the syntax check passes.

systemctl reload nginx

Add TLS with Let's Encrypt

Point your domain to the VPS first. Then request a certificate. If DNS is not ready, this step will fail.

On Ubuntu and Debian as the non-root sudo user:

sudo apt -y install certbot python3-certbot-nginx
sudo certbot --nginx -d example.com -d server.example.com

On AlmaLinux and Rocky Linux as the non-root sudo user:

sudo dnf -y install certbot python3-certbot-nginx
sudo certbot --nginx -d example.com -d server.example.com

Choose the redirect option so HTTP sends visitors to HTTPS. Then test renewal.

sudo certbot renew --dry-run

If you want a broader look at web-server tradeoffs before choosing your public edge, see Nginx vs Apache for VPS hosting in 2026.

Verify PostgreSQL, pgvector, and the app

Now confirm that the database listens locally and the app answers through Nginx.

On the VPS as the non-root sudo user:

sudo docker exec -it myapp-db psql -U myapp -d myapp -c "SELECT extname FROM pg_extension;"
curl -I http://127.0.0.1:8000
curl -kI https://example.com

The database query should list vector if the image created the extension, and the curl commands should return HTTP headers rather than connection errors.

Check the listening ports too.

ss -tulpn | grep -E '(:80|:443|:8000)'

Expected result: Nginx listens on 80 and 443, and the app is bound only where you expect it.

Make the stack persistent after reboot

Restart the VPS once before you call the deployment complete. That catches service ordering problems early.

On the VPS as the non-root sudo user:

sudo reboot

Reconnect after the server comes back.

ssh deploy@203.0.113.10

Then confirm the services survived.

sudo systemctl status nginx --no-pager
sudo docker compose -f /opt/myapp/docker-compose.yml ps

Troubleshooting the most likely failures

• SSH login fails after hardening: run sshd -t from the console or rescue shell. If the output shows a syntax error, fix /etc/ssh/sshd_config and reload SSH only after the test passes.
• Port 80 or 443 is unreachable: run sudo ufw status verbose on Ubuntu/Debian or sudo firewall-cmd --list-all on AlmaLinux/Rocky. Add the missing web ports, then retry the client-side curl test.
• The app container keeps restarting: run sudo docker compose logs --tail=100 app. Check for bad environment variables, a wrong database password, or an image name that does not exist.
• Certbot refuses the domain: run dig +short example.com from your local computer. If it does not resolve to the VPS IP, fix DNS before trying again.
• Nginx returns 502: run curl -I http://127.0.0.1:8000. If that fails, the upstream app is not listening on the port that Nginx expects.

Why this setup fits Hostperl customers

Private AI hosting is usually not a research project. It is a customer workflow, an internal knowledge base, or an API that must stay under your control. A VPS gives you enough isolation for that job without unnecessary overhead.

When you need more RAM, more CPU, or a different region for latency, Hostperl can move you toward a larger Hostperl VPS or a dedicated platform without changing the way your stack is built. That reduces migration work later, which is exactly what support teams and small technical businesses need.

FAQ

Do I need a GPU for private AI hosting?
Not for this stack. Many private AI workloads start with API-based inference, embeddings, or lightweight internal assistants on CPU-only VPS plans.

Why use pgvector with PostgreSQL?
It keeps vectors and app data in one database, which simplifies backups, restores, and access control.

Can I use Apache or OpenLiteSpeed instead of Nginx?
Yes, but Nginx is the cleanest fit for a small reverse-proxy front end and TLS termination.

What should I monitor first?
Track container status, disk space, memory pressure, and certificate renewal. Those are the first things that usually break on a small VPS.

Can Hostperl help if I migrate from another host?
Yes. A clean VPS migration is easier when your app is already packaged, your DNS is documented, and your reverse proxy is simple.