RAG App Hosting on VPS with pgvector and Docker Compose

Start with the right hosting shape for your RAG app

RAG app hosting usually fits a VPS better than a serverless stack once you need private documents, steady latency, or a vector store that stays online between requests. For many Hostperl customers, that means one app server, one Postgres instance with pgvector, and one reverse proxy in front of both.

If you are comparing deployment options, our Hostperl VPS plans are a practical middle ground for support bots, sales assistants, and internal knowledge apps. You get full control over Docker, environment files, backups, and regional placement, which matters when your users are in New Zealand, Australia, or the wider APAC region.

This guide deploys a simple private RAG stack with Docker Compose: a Node.js web app, PostgreSQL with pgvector, Nginx, and TLS. If you want a broader decision guide first, see Hostperl's AI hosting buyer's guide and private AI hosting on VPS.

What you will deploy

• APP: a small Node.js RAG API that accepts a question and returns a retrieved answer.
• DB: PostgreSQL 16 with pgvector for embeddings and similarity search.
• Proxy: Nginx for TLS and public traffic.
• Runtime: Docker Compose, so you can restart the full stack cleanly after maintenance.

Use these placeholders throughout the tutorial:

• SERVER_IP: your VPS public IP, for example 203.0.113.10
• DOMAIN_NAME: the hostname you point at the VPS, for example ai.example.com
• ADMIN_USER: your non-root sudo user, for example hostperl
• APP_DIR: the app directory, for example /opt/ragapp
• APP_PORT: the internal app port, for example 3000

Connect to the VPS and detect the OS

Run the SSH command from your local computer. Keep the root session open until the new sudo user is tested successfully.

ssh root@SERVER_IP

You should land on the VPS as root. Next, identify the distribution so you use the right package commands.

cat /etc/os-release

Look for Ubuntu, Debian, AlmaLinux, or Rocky Linux. The setup below shows both command paths where they differ.

Create a non-root admin user

Do this on the VPS as root. If you already have a sudo user, you can skip to the Docker section, but many migrations start from a fresh server and need this step done safely.

Ubuntu and Debian

adduser hostperl
usermod -aG sudo hostperl
mkdir -p /home/hostperl/.ssh
chmod 700 /home/hostperl/.ssh
cp /root/.ssh/authorized_keys /home/hostperl/.ssh/authorized_keys
chown -R hostperl:hostperl /home/hostperl/.ssh
chmod 600 /home/hostperl/.ssh/authorized_keys

This creates hostperl, gives sudo access, and reuses your SSH key. You should see no errors. After that, open a second terminal on your local computer and test the new login.

ssh hostperl@SERVER_IP

Then confirm sudo works:

sudo -v
whoami

You should see hostperl and no password failure. Leave the original root session open for now.

AlmaLinux and Rocky Linux

useradd -m hostperl
passwd hostperl
usermod -aG wheel hostperl
mkdir -p /home/hostperl/.ssh
chmod 700 /home/hostperl/.ssh
cp /root/.ssh/authorized_keys /home/hostperl/.ssh/authorized_keys
chown -R hostperl:hostperl /home/hostperl/.ssh
chmod 600 /home/hostperl/.ssh/authorized_keys

AlmaLinux and Rocky use the wheel group for sudo access. Test the new login from a second terminal before changing any root access settings.

Install Docker, Compose, and Nginx

Run the matching block for your distribution. These commands install the runtime, the reverse proxy, and the firewall tools you need for a public app.

Ubuntu and Debian

sudo apt update
sudo apt install -y ca-certificates curl gnupg nginx ufw
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release; echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker "$USER"
sudo systemctl enable --now docker nginx

Log out and back in after the Docker group change so your user can run Compose without sudo. Check versions next.

docker --version
docker compose version
nginx -v

AlmaLinux and Rocky Linux

sudo dnf install -y dnf-plugins-core nginx firewalld
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker nginx firewalld
sudo usermod -aG docker "$USER"

For SELinux, keep the defaults unless you hit a permissions issue later. Check the versions now so you know the install completed cleanly.

docker --version
docker compose version
nginx -v
getenforce

Open only the ports you need

For a public RAG app, you only need SSH, HTTP, and HTTPS open. Add the new rules before you consider changing any existing access policy.

Ubuntu and Debian with UFW

sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

You should see OpenSSH, 80/tcp, and 443/tcp allowed.

AlmaLinux and Rocky Linux with firewalld

sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
sudo firewall-cmd --list-services

That should list ssh, http, and https.

Create the RAG app files

Switch to your sudo user if you are still in the root shell, then create the app directory. Use one terminal for editing and another for testing.

sudo mkdir -p /opt/ragapp
sudo chown -R hostperl:hostperl /opt/ragapp
cd /opt/ragapp
pwd

Now create the Compose file. This example uses PostgreSQL 16 and a small Node app that reads embeddings from the same network.

nano /opt/ragapp/docker-compose.yml

services:
  db:
    image: pgvector/pgvector:pg16
    container_name: rag-db
    restart: unless-stopped
    environment:
      POSTGRES_DB: ragdb
      POSTGRES_USER: raguser
      POSTGRES_PASSWORD: change_this_db_password
    volumes:
      - dbdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U raguser -d ragdb"]
      interval: 10s
      timeout: 5s
      retries: 5

  app:
    image: node:20-bookworm-slim
    container_name: rag-app
    restart: unless-stopped
    working_dir: /app
    command: sh -c "npm install && node server.js"
    environment:
      DATABASE_URL: postgresql://raguser:change_this_db_password@db:5432/ragdb
      APP_PORT: 3000
      OPENAI_API_KEY: replace_with_your_key
      EMBEDDING_MODEL: text-embedding-3-small
    volumes:
      - ./app:/app
    depends_on:
      db:
        condition: service_healthy
    ports:
      - "127.0.0.1:3000:3000"

volumes:
  dbdata:

Save the file with Ctrl+O, press Enter, then exit with Ctrl+X. The app is bound only to localhost, so Nginx will be the public entry point.

Create the app source next.

mkdir -p /opt/ragapp/app
nano /opt/ragapp/app/package.json

{
  "name": "ragapp",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "express": "^4.21.2",
    "pg": "^8.13.1"
  }
}

Then create the server file.

nano /opt/ragapp/app/server.js

import express from 'express';
import pg from 'pg';

const app = express();
app.use(express.json());

const port = process.env.APP_PORT || 3000;
const db = new pg.Pool({ connectionString: process.env.DATABASE_URL });

async function init() {
  await db.query('CREATE EXTENSION IF NOT EXISTS vector');
  await db.query(`CREATE TABLE IF NOT EXISTS documents (
    id SERIAL PRIMARY KEY,
    title TEXT NOT NULL,
    content TEXT NOT NULL,
    embedding vector(1536)
  )`);
}

app.get('/health', async (_, res) => {
  await db.query('SELECT 1');
  res.json({ ok: true });
});

app.post('/ingest', async (req, res) => {
  const { title, content } = req.body;
  await db.query('INSERT INTO documents (title, content) VALUES ($1, $2)', [title, content]);
  res.json({ stored: true });
});

app.post('/ask', async (req, res) => {
  const { question } = req.body;
  const result = await db.query('SELECT title, content FROM documents ORDER BY id DESC LIMIT 3');
  res.json({ question, sources: result.rows });
});

await init();
app.listen(port, '0.0.0.0', () => console.log(`RAG app listening on ${port}`));

This is a minimal retrieval pipeline. It stores documents and returns recent sources, which is enough to prove the hosting path before you add real embeddings or a vector index.

Create a private environment file and keep it readable only by you and root.

nano /opt/ragapp/.env

chmod 600 /opt/ragapp/.env

Put secrets in .env only if you also reference them from Compose. If you later move to Next.js, FastAPI, or NestJS, the same file pattern still works.

Bring up the stack and check the database

Run Compose from /opt/ragapp. The first start pulls images and creates the Postgres volume.

cd /opt/ragapp
docker compose up -d
docker compose ps

You want both services marked as running. Then confirm the database container accepts connections and the extension exists.

docker compose exec db psql -U raguser -d ragdb -c "\dx"
docker compose logs --tail=50 db

Look for vector in the extension list. If the app container restarts, check its logs next.

docker compose logs --tail=100 app

Put Nginx in front of the app

Create a reverse proxy so your domain serves HTTPS traffic while the app keeps listening on localhost. This is the part most migration projects miss on the first pass.

sudo nano /etc/nginx/sites-available/ragapp

server {
    listen 80;
    server_name DOMAIN_NAME;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Replace DOMAIN_NAME with your real hostname. Save, exit, then test the syntax before enabling the site.

sudo ln -s /etc/nginx/sites-available/ragapp /etc/nginx/sites-enabled/ragapp
sudo nginx -t
sudo systemctl reload nginx

On AlmaLinux and Rocky, use the same config file path under /etc/nginx/conf.d/ragapp.conf instead of the sites-available layout. After saving it, run the same nginx -t check and reload.

Issue TLS and finish the public check

Once DNS points at the VPS, request a certificate. If DNS is still propagating, wait until DOMAIN_NAME resolves to SERVER_IP from your local machine.

curl -I http://DOMAIN_NAME

You should see a 200 or 301 response from Nginx. Then install Certbot and issue the certificate.

Ubuntu and Debian

sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d DOMAIN_NAME

AlmaLinux and Rocky Linux

sudo dnf install -y certbot python3-certbot-nginx
sudo certbot --nginx -d DOMAIN_NAME

Accept the redirect option so HTTP goes to HTTPS. After that, verify the site from your local computer.

curl -I https://DOMAIN_NAME
curl https://DOMAIN_NAME/health

You should get a TLS response and a JSON health payload containing {"ok":true}.

Test one real RAG workflow

Now run a simple ingest and retrieval test. This gives you a concrete smoke test before you hand the app to a team or client.

curl -X POST https://DOMAIN_NAME/ingest \
  -H 'Content-Type: application/json' \
  -d '{"title":"Support policy","content":"Refunds are handled within two business days."}'

curl -X POST https://DOMAIN_NAME/ask \
  -H 'Content-Type: application/json' \
  -d '{"question":"How long do refunds take?"}'

The second call should return the stored source. That proves the app, proxy, and database are working together.

Make the deployment supportable

This is where private AI hosting pays off. Save a copy of your Compose file, .env template, and Nginx config in your backup set. Hostperl customers often keep a restore note that includes the domain, DB password location, and the exact Compose command used during the last deployment.

For more on keeping the budget under control, see Private AI hosting on VPS: cost, security, and latency. If you are comparing a VPS against an external platform, the tradeoffs are also covered in AI hosting buyer's guide.

Common failures and quick fixes

• Nginx shows 502: run docker compose ps and docker compose logs --tail=50 app. If the app failed to start, check the database password in docker-compose.yml.
• Certbot cannot find the domain: run dig +short DOMAIN_NAME from your local machine. If it does not return SERVER_IP, fix DNS first.
• Database extension missing: run docker compose exec db psql -U raguser -d ragdb -c "CREATE EXTENSION IF NOT EXISTS vector;".
• Permission error in AlmaLinux: check getenforce and sudo ausearch -m avc -ts recent. If SELinux blocks Nginx, use the default paths and keep files under standard locations.

Keep the app ready for production use

Before you hand this to a support team or sales team, make sure backups run, logs are readable, and a reboot brings the stack back cleanly. A VPS works well here because you control the full path, from retrieval data to restart order. For teams that want managed capacity with the same level of operational control, Hostperl's managed VPS hosting is a practical starting point.

One final check from the server:

docker compose restart
sleep 10
docker compose ps
sudo systemctl status nginx --no-pager

And one final check from your browser or terminal:

curl -s https://DOMAIN_NAME/health

If that returns healthy after a restart, your RAG app hosting setup is ready for real traffic, a migration, or a private client rollout.

FAQ

Is pgvector enough for a first RAG deployment?

Yes. For many support bots and internal knowledge apps, PostgreSQL with pgvector is simpler to operate than adding a separate vector database.

Should I use Qdrant or Chroma instead?

Use them if your retrieval layer needs their specific filtering or scaling behavior. For a first private deployment, pgvector is usually easier to back up and restore.

Can I host a Next.js or FastAPI front end on the same VPS?

Yes, as long as you keep the app behind Nginx and size the VPS for your database, workers, and traffic. Start small, then scale once your logs and latency show you need more headroom.

How do I reduce AI hosting cost?

Cache common answers, trim embedding batch size, use smaller models for routine classification, and watch your VPS CPU, RAM, and storage growth each week.