Deploy a Private RAG Stack on VPS in 2026
Start with the right VPS for a private RAG stack on VPS
A private RAG stack on VPS fits support bots, sales assistants, and internal search when you want control over data, latency, and spend. For many Hostperl customers, that means one VPS for the app and another for PostgreSQL, or a larger single VPS if you want to keep the first launch simple. If you want a quick sizing baseline before you start, Hostperl VPS plans fit that staged rollout better than serverless pricing that rises with traffic and token use.
This guide builds a small, auditable setup: Node.js API, PostgreSQL with pgvector, Redis for jobs and caching, Nginx for TLS, and Docker Compose for repeatable deployment. It is aimed at the real hosting buyer who needs a working bot, not a lab project.
Before you log in: placeholders and assumptions
Replace these values as you follow along: SERVER_IP is your VPS IP, for example 203.0.113.10; DOMAIN_NAME is the public name, for example bot.example.com; ADMIN_USER is your sudo account, for example hostperladmin; APP_DIR is the deployment folder, for example /opt/privaterag. This tutorial assumes Ubuntu 24.04/Debian 12 or AlmaLinux 9/Rocky Linux 9.
On your local computer, connect as root first so you can finish the initial setup safely.
ssh root@SERVER_IPIf your provider gives you a non-root default user, use that instead. Keep this first session open until the new sudo user is tested.
On the VPS as root, detect the OS before you install anything.
cat /etc/os-releaseYou should see either Ubuntu/Debian or AlmaLinux/Rocky details. The package commands below differ by family.
Create a sudo user and test a second SSH session
This is the step people skip and later regret. Create the admin account, add SSH keys, and prove the new login works before you touch root access.
Ubuntu and Debian
On the VPS as root, create the account and add it to sudo.
adduser ADMIN_USER
usermod -aG sudo ADMIN_USERNow create the SSH directory and authorize your public key. Replace the example key path with the file on your laptop.
install -d -m 700 -o ADMIN_USER -g ADMIN_USER /home/ADMIN_USER/.ssh
cat >> /home/ADMIN_USER/.ssh/authorized_keysPaste your public key, press Ctrl+D, then lock the permissions.
chown ADMIN_USER:ADMIN_USER /home/ADMIN_USER/.ssh/authorized_keys
chmod 600 /home/ADMIN_USER/.ssh/authorized_keysOn your local computer, open a second terminal and test the new login.
ssh ADMIN_USER@SERVER_IPThen confirm sudo works.
sudo -v
whoami
pwdYou want to see ADMIN_USER and a successful sudo timestamp. Keep the original root session open until this passes.
AlmaLinux and Rocky Linux
On the VPS as root, create the account and add it to wheel.
useradd -m ADMIN_USER
passwd ADMIN_USER
usermod -aG wheel ADMIN_USERSet up the SSH directory, then copy your public key in the same way.
install -d -m 700 -o ADMIN_USER -g ADMIN_USER /home/ADMIN_USER/.ssh
cat >> /home/ADMIN_USER/.ssh/authorized_keys
chown ADMIN_USER:ADMIN_USER /home/ADMIN_USER/.ssh/authorized_keys
chmod 600 /home/ADMIN_USER/.ssh/authorized_keysOn your local computer, test the new login and sudo access.
ssh ADMIN_USER@SERVER_IP
sudo -v
whoamiOnce that works, you can continue from the non-root shell.
Install Docker, Nginx, and the app runtime
This stack uses Docker Compose because it fits migrations, backups, and rollback better than ad hoc installs. It also makes support easier when you hand the server to another engineer later.
Ubuntu and Debian
On the VPS as the non-root sudo user, update packages and install Docker, Compose, Nginx, and utilities.
sudo apt update
sudo apt install -y ca-certificates curl gnupg git nginx
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo usermod -aG docker ADMIN_USERConfirm the versions.
docker --version
docker compose version
nginx -vOn AlmaLinux and Rocky Linux, use dnf, enable Docker’s repo, and install the same stack.
sudo dnf install -y dnf-plugins-core git nginx curl
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo systemctl enable --now docker nginx
sudo usermod -aG wheel ADMIN_USERCheck the runtime and service status.
docker --version
docker compose version
systemctl status docker --no-pager
systemctl status nginx --no-pagerBuild the private RAG app files
The example app exposes a small HTTP endpoint, reads secrets from .env, and connects to PostgreSQL with pgvector plus Redis for caching. Compared with hosted platforms like Vercel or Render, the practical difference is straightforward: you keep the database, logs, and environment under your own control, which matters for audit trails and regional latency.
On the VPS as the non-root sudo user, create the deployment directory.
sudo mkdir -p /opt/privaterag
sudo chown ADMIN_USER:ADMIN_USER /opt/privaterag
cd /opt/privateragCreate the Compose file.
nano compose.ymlPaste this content, then save with Ctrl+O, Enter, and exit with Ctrl+X.
services:
postgres:
image: pgvector/pgvector:pg16
environment:
POSTGRES_DB: ragdb
POSTGRES_USER: raguser
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
volumes:
- pgdata:/var/lib/postgresql/data
restart: unless-stopped
redis:
image: redis:7.4-alpine
command: ["redis-server", "--appendonly", "yes"]
volumes:
- redisdata:/data
restart: unless-stopped
app:
image: node:22-alpine
working_dir: /app
command: sh -c "npm install && node server.js"
env_file:
- .env
volumes:
- ./:/app
depends_on:
- postgres
- redis
restart: unless-stopped
ports:
- "127.0.0.1:3000:3000"
volumes:
pgdata:
redisdata:Create the environment file with restrictive permissions.
nano .envUse real values for secrets, then save and exit.
POSTGRES_PASSWORD=change-this-long-password
JWT_SECRET=change-this-too
APP_PORT=3000
DOMAIN_NAME=bot.example.com
NODE_ENV=productionLock the file down.
chmod 600 .envNow add a minimal Node.js app and health check.
cat > server.js <<'EOF'
const http = require('http');
const port = process.env.APP_PORT || 3000;
http.createServer((req, res) => {
if (req.url === '/health') {
res.writeHead(200, {'Content-Type': 'application/json'});
return res.end(JSON.stringify({status: 'ok', mode: process.env.NODE_ENV || 'dev'}));
}
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('private RAG stack ready');
}).listen(port, '0.0.0.0');
EOFStart the stack and confirm the containers come up cleanly.
docker compose up -d
docker compose psFor a real RAG deployment, this is where you would load embeddings, create a documents table with vector columns, and add your ingestion worker. Hostperl customers often begin with this exact base, then add queue workers, webhook handlers, and model calls after the first production smoke test.
If you want a fuller pattern for that layer, the deployment flow in RAG app hosting with pgvector and Docker Compose shows the database side in more depth. For private bot design choices, the broader buying guide AI bot hosting decisions explains why a VPS often beats serverless once you need stable costs and local data control.
Put Nginx in front and add TLS
Nginx keeps your app on localhost and handles HTTPS at the edge. That gives you one place for access logs, rate limiting, and cleanup during migrations.
On the VPS as the non-root sudo user, create the server block.
sudo nano /etc/nginx/sites-available/privaterag.confPaste this configuration, then save and exit.
server {
listen 80;
server_name bot.example.com;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}Enable it, test syntax, and reload only if the check passes.
sudo ln -s /etc/nginx/sites-available/privaterag.conf /etc/nginx/sites-enabled/privaterag.conf
sudo nginx -t
sudo systemctl reload nginxFor TLS, install Certbot and request a certificate after DNS points to the VPS. If you are still moving users from shared hosting or a previous platform, the DNS and SSL sequence in this DNS and SSL setup tutorial is the same sequence to follow: publish the record first, then request the certificate.
Ubuntu and Debian:
sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d bot.example.comAlmaLinux and Rocky Linux:
sudo dnf install -y certbot python3-certbot-nginx
sudo certbot --nginx -d bot.example.comAfter issuance, confirm the renewal timer.
sudo systemctl status certbot.timer --no-pagerVerify the app from both server and client
First check the local endpoint on the VPS.
curl -fsS http://127.0.0.1:3000/healthYou should get JSON with status: ok. Then test through Nginx.
curl -I https://bot.example.com
curl -fsS https://bot.example.com/healthFrom the server side, inspect service state and logs if anything looks wrong.
docker compose logs --tail=50
sudo journalctl -u nginx -n 50 --no-pagerOpen the site in a browser and submit one real smoke test: ask your bot a question that should hit your knowledge source, then confirm the response and the log entry line up. That is the part executives care about during launch, not just container uptime.
If you need a queue-backed worker later for webhooks, ingestion, or scheduled refreshes, keep it in the same Compose stack. That is usually enough for support bots and sales bots until you outgrow a single VPS or decide to split the database onto a dedicated host.
Basic security, rollback, and cost control
- Secrets: keep API keys in
.envand usechmod 600. Do not bake them into images. - Backups: run
pg_dumpnightly and copy archives off the server. Verify one restore before launch. - Rate limits: add Nginx limits if public traffic can trigger expensive model calls.
- Rollback: tag each image or keep a Git commit you can redeploy with
docker compose up -d. - Spend control: cache repeated answers, set token ceilings, and prefer a smaller model for routine routing.
For a customer-friendly backup plan, Hostperl’s VPS backup testing and restore plan is the right follow-up once your bot is live. It saves time later, especially when a knowledge base import or prompt change goes sideways.
If you want private AI deployment without the usual server sprawl, Hostperl can help you size the VPS, move the app, and keep the setup supportable. Start with a right-sized Hostperl VPS, then expand to dedicated capacity if your RAG index, webhook load, or regional traffic grows.
For support bots, customer-service automations, and APAC-facing apps, that mix usually gives you the best balance of latency, control, and cost.
Common problems and quick fixes
Docker won’t start: run systemctl status docker --no-pager. If it is disabled on AlmaLinux/Rocky, use sudo systemctl enable --now docker.
Nginx returns 502: check docker compose ps and docker compose logs app. If the app crashed, fix the startup command and run docker compose up -d again.
TLS issuance fails: run sudo nginx -t and confirm DNS points to SERVER_IP. If the domain is not resolving yet, wait for propagation before retrying Certbot.
Port is closed: check UFW or firewalld. On Ubuntu/Debian, use sudo ufw status and open only 80/443. On AlmaLinux/Rocky, use sudo firewall-cmd --list-all and add the public services before reloading.
That sequence is enough to launch a private RAG stack on VPS, keep the deployment auditable, and avoid the cost surprises that often show up on serverless platforms. If you need a cleaner migration path for the next app, Hostperl’s private AI hosting on VPS guide explains the tradeoffs in plain terms.
