RAG App Hosting on VPS with pgvector and Docker Compose
Start with the right RAG app hosting plan
RAG app hosting works best when you size the VPS for retrieval work, not just the web app. For private support bots, sales assistants, and internal search tools, a modest Hostperl VPS is usually enough at the start, especially if you keep embeddings, PostgreSQL, and the app on one machine.
If traffic grows or your vector index expands, move up before the plan feels cramped. Hostperl’s VPS hosting fits this kind of deployment because you can match CPU, RAM, and storage to actual usage.
This tutorial uses a straightforward stack: a Next.js or Node.js app, PostgreSQL with pgvector, and Docker Compose. You get a private setup you can inspect, move, and restore without waiting on a serverless provider to explain the invoice.
If you are comparing options, the context in Hostperl’s AI hosting buyer guide is useful before you build.
What you will deploy
You will deploy a small RAG stack on a fresh VPS:
- APP_USER: non-root Linux account for the app
- APP_DIR: application directory, for example
/opt/ragbot - DOMAIN_NAME: public hostname, for example
rag.example.com - APP_PORT: local app port, for example
3000 - DB_NAME, DB_USER, DB_PASSWORD: PostgreSQL credentials
You’ll also set up Docker Compose, Nginx, TLS, and a small smoke test so you can verify the app after reboot. If your team needs a private deployment with tighter security and cost control, this is the same pattern covered in Private AI app hosting on VPS.
1) Connect to the VPS and check the operating system
On your local computer, connect as root. Replace 203.0.113.10 with your server IP.
ssh root@203.0.113.10If your provider gives you a default user instead of root, use that account name in the same command. Keep this first session open until the new sudo user is working.
On the VPS as root, confirm the OS before installing packages.
cat /etc/os-releaseYou should see either Ubuntu/Debian or AlmaLinux/Rocky Linux. The next steps split by family.
2) Create a non-root admin user
Ubuntu and Debian
adduser adminuser
usermod -aG sudo adminuserReplace adminuser with your own admin account. The first command creates the user and prompts for a password. The second adds sudo access.
You should be able to run sudo -v later without errors.
AlmaLinux and Rocky Linux
useradd -m adminuser
passwd adminuser
usermod -aG wheel adminuserThe wheel group gives sudo rights on these systems. Set a password only if you need password login; SSH keys are better for day-to-day use.
Now add your SSH key.
mkdir -p /home/adminuser/.ssh
chmod 700 /home/adminuser/.ssh
cp /root/.ssh/authorized_keys /home/adminuser/.ssh/authorized_keys
chown -R adminuser:adminuser /home/adminuser/.ssh
chmod 600 /home/adminuser/.ssh/authorized_keysThese commands copy the current authorized key into the new account for testing. If you manage keys from your laptop, append the public key to authorized_keys instead.
Open a second terminal and test the login before you change root access.
On your local computer, verify the new account.
ssh adminuser@203.0.113.10On the VPS as the non-root sudo user, confirm sudo works.
sudo -v
whoamiYou should see no sudo error, and whoami should return adminuser. Only after this test should you consider disabling root password login later.
3) Install Docker, Compose, and Nginx
Ubuntu and Debian
sudo apt update
sudo apt install -y ca-certificates curl gnupg nginx docker.io docker-compose-plugin
sudo systemctl enable --now nginx dockerAlmaLinux and Rocky Linux
sudo dnf install -y dnf-plugins-core curl ca-certificates nginx docker-compose-plugin docker-cli
sudo systemctl enable --now nginx dockerIf Docker is not available from your default repository on your RHEL-family system, install the supported Docker repository package first, then repeat the install command. Run a quick version check so you know the binaries are present.
docker --version
docker compose version
nginx -vAt this point you have the tools needed for a private AI app stack. If you want a broader deployment comparison, Hostperl’s RAG, bots, and webhooks deployment guide covers the same hosting pattern from a customer-operations angle.
4) Create the application directory and secrets file
On the VPS as the non-root sudo user, create the app directory and a file for environment variables.
sudo mkdir -p /opt/ragbot
sudo chown -R adminuser:adminuser /opt/ragbot
cd /opt/ragbot
pwd
mkdir -p appNow create .env. Replace the sample values with your real passwords and domain.
cat > /opt/ragbot/.env <<'EOF'
DOMAIN_NAME=rag.example.com
APP_PORT=3000
DB_NAME=ragdb
DB_USER=raguser
DB_PASSWORD=change-this-long-password
POSTGRES_PASSWORD=change-this-stronger-password
EOF
chmod 600 /opt/ragbot/.envThat file should stay readable only by the app owner. It keeps the deployment easier to audit and keeps secrets out of shell history.
5) Add the Docker Compose stack
Use PostgreSQL with pgvector plus a small web app container. The app below is a simple Node.js example that stores and reads documents from Postgres; you can replace it with Next.js, NestJS, or FastAPI later.
If your team is comparing deployment platforms, Hostperl’s private AI hosting cost and latency guide explains why this kind of self-managed stack stays predictable.
On the VPS as the non-root sudo user, create docker-compose.yml.
cat > /opt/ragbot/docker-compose.yml <<'EOF'
services:
db:
image: pgvector/pgvector:pg16
container_name: ragbot-db
restart: unless-stopped
env_file: .env
environment:
POSTGRES_DB: ${DB_NAME}
POSTGRES_USER: ${DB_USER}
POSTGRES_PASSWORD: ${DB_PASSWORD}
volumes:
- db_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${DB_USER} -d ${DB_NAME}"]
interval: 10s
timeout: 5s
retries: 5
app:
image: node:22-alpine
container_name: ragbot-app
restart: unless-stopped
working_dir: /app
command: sh -c "npm install express pg && node server.js"
env_file: .env
environment:
PORT: ${APP_PORT}
DATABASE_URL: postgresql://${DB_USER}:${DB_PASSWORD}@db:5432/${DB_NAME}
volumes:
- ./app:/app
depends_on:
db:
condition: service_healthy
ports:
- "127.0.0.1:${APP_PORT}:${APP_PORT}"
volumes:
db_data:
EOFThe loopback bind keeps the app private behind Nginx. That is the right default for support bots and sales workflows where you do not want the container exposed directly on the public IP.
6) Add a small RAG-style Node.js app
Create a minimal server that stores a document and returns a search result. It is not a full vector pipeline, but it proves the hosting path, database connection, and deployment flow.
On the VPS as the non-root sudo user, create /opt/ragbot/app/server.js.
cat > /opt/ragbot/app/server.js <<'EOF'
const express = require('express');
const { Pool } = require('pg');
const app = express();
app.use(express.json());
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const port = process.env.PORT || 3000;
(async () => {
await pool.query('CREATE EXTENSION IF NOT EXISTS vector');
await pool.query(`CREATE TABLE IF NOT EXISTS docs (
id SERIAL PRIMARY KEY,
title TEXT NOT NULL,
body TEXT NOT NULL
)`);
})();
app.get('/healthz', async (_req, res) => {
try {
await pool.query('SELECT 1');
res.json({ ok: true });
} catch (err) {
res.status(500).json({ ok: false, error: err.message });
}
});
app.post('/docs', async (req, res) => {
const { title, body } = req.body;
const result = await pool.query('INSERT INTO docs(title, body) VALUES ($1, $2) RETURNING id', [title, body]);
res.json({ id: result.rows[0].id });
});
app.get('/docs', async (_req, res) => {
const result = await pool.query('SELECT id, title, body FROM docs ORDER BY id DESC LIMIT 10');
res.json(result.rows);
});
app.listen(port, () => console.log(`RAG app listening on ${port}`));
EOFOpenAI, local embeddings, pgvector, or Qdrant can sit behind this same pattern. For a deeper vector-db deployment example, link this guide with Hostperl’s pgvector and Docker Compose tutorial.
7) Start the stack and confirm it runs
On the VPS as the non-root sudo user, bring up the containers.
cd /opt/ragbot
docker compose up -d
docker compose psYou should see both containers in a running state. If the app exits, inspect logs immediately.
docker compose logs --tail=50 app
docker compose logs --tail=50 dbNext, test the app from the VPS itself.
curl -s http://127.0.0.1:3000/healthz
curl -s -X POST http://127.0.0.1:3000/docs -H 'Content-Type: application/json' -d '{"title":"Pricing note","body":"Reply with the 2026 renewal price and next steps."}'
curl -s http://127.0.0.1:3000/docsYou should get {"ok":true}, then a document ID, then the stored document list. That confirms the container, database, and app path are working before you add the reverse proxy.
8) Put Nginx in front of the app
On the VPS as the non-root sudo user, create an Nginx site file. Replace rag.example.com with your real domain.
sudo tee /etc/nginx/conf.d/ragbot.conf > /dev/null <<'EOF'
server {
listen 80;
server_name rag.example.com;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
EOF
sudo nginx -t
sudo systemctl reload nginxThe nginx -t check should report syntax is ok. Only reload after that. If you are new to reverse proxies, Hostperl’s Next.js on VPS deployment guide uses the same front-end pattern.
9) Open the firewall and issue TLS
Ubuntu and Debian
sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable
sudo ufw statusAlmaLinux and Rocky Linux
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
sudo firewall-cmd --list-allThen request a certificate. Make sure DNS already points rag.example.com to the VPS IP.
sudo apt install -y certbot python3-certbot-nginx || true
sudo certbot --nginx -d rag.example.comOn AlmaLinux or Rocky, install the matching Certbot package from EPEL or your preferred repository first, then run the same Certbot command. When it finishes, open https://rag.example.com in a browser. The certificate should be valid and the app should load through Nginx.
10) Verify reboot persistence, logs, and rollback path
Restart the VPS or at least test the service states after a reload. You want to know the stack comes back on its own.
docker compose ps
systemctl status nginx --no-pager
journalctl -u nginx -n 20 --no-pagerFrom your local computer, run a final check against the public domain.
curl -I https://rag.example.com
curl -s https://rag.example.com/healthzIf you need to roll back after a bad app push, the cleanest move is to update the files in /opt/ragbot/app, then run docker compose up -d again. For bigger changes, keep a copy of the previous docker-compose.yml and .env so you can restore the last known good state quickly.
Troubleshooting the usual failures
Nginx returns 502
Check whether the app container is listening on the local port.
docker compose logs --tail=50 app
curl -s http://127.0.0.1:3000/healthzIf the health check fails, fix the container error first. If the app works locally, confirm Nginx points to 127.0.0.1:3000.
PostgreSQL extension error
If CREATE EXTENSION IF NOT EXISTS vector fails, verify the image tag is pgvector-enabled.
docker compose logs --tail=50 db
docker image ls | grep pgvectorReplace the database image with a pgvector image if you used a plain Postgres tag.
Certbot cannot issue the certificate
DNS or firewall is usually the problem.
dig +short rag.example.com
sudo ss -tlnp | grep ':80\|:443'
If the IP is wrong, fix the DNS record and wait for propagation. If port 80 is closed, re-open the firewall rule before retrying Certbot.
If you are launching a private RAG app, Hostperl gives you the control you need without making the setup heavier than it should be. Start with a right-sized Hostperl VPS, then follow the same deployment pattern you used here for support bots, sales assistants, or internal search tools.
For teams that want to compare deployment styles before moving production traffic, the AI hosting buyer guide is the fastest way to sanity-check your next step.
FAQ
Should I use pgvector, Qdrant, or Chroma?
Use pgvector if you already rely on PostgreSQL and want one backup path. Qdrant fits larger vector-heavy workloads. Chroma is fine for prototypes, but most production teams prefer clearer operational control.
Can I host a private AI app on one small VPS?
Yes, if traffic is light and your vector index is modest. Start small, watch memory and disk use, then scale before the host hits swap.
How do I keep secrets out of code?
Store API keys and database passwords in .env with strict permissions, or move them into a secrets manager later. Do not hardcode keys in the app files.
What should I watch for with APAC users?
Choose a datacenter close to your users if the workflow is latency-sensitive. For New Zealand and wider APAC teams, a nearby VPS usually feels faster than a distant serverless region.
How do I control AI hosting cost?
Cache repeated prompts, keep embeddings local, and use a small model for routing or classification. The cheapest request is the one you do not send to an external API.
