Deploy a Private RAG App on Debian 12 VPS

What you are building

This tutorial shows you how to deploy a private RAG app on a fresh Debian 12 VPS, from the first SSH login through HTTPS verification and rollback checks. You will run a small production-style stack: Nginx as the reverse proxy, Docker Compose for the app layer, PostgreSQL for metadata, and Qdrant for vector search.

The goal is not a demo. You are setting up a service that can survive routine updates, restarts, and failed deployments without exposing your data or making recovery painful. If you need a larger VPS as your workload grows, Hostperl VPS hosting is the right starting point for this kind of build: Hostperl VPS hosting.

This guide targets Debian 12 because it is the least-covered OS track for this run, and it fits the stack cleanly. Debian 13 works with the same architecture, but this tutorial uses Debian 12 commands and package names.

Architecture and decision points

A private RAG app has three parts that need separate care. The web app handles authentication and requests, PostgreSQL stores application records and job state, and Qdrant stores vectors for retrieval.

Keeping them in separate containers makes upgrades and rollback much easier.

You will also keep the service private behind Nginx. That lets you terminate TLS at the edge, enforce rate limits later, and keep the app port closed to the public internet.

For teams comparing storage and memory footprints before launch, this sizing guide for private AI infrastructure helps you choose a VPS that is not too small for indexing and retrieval.

Use this design if you need controlled access, predictable update windows, and a clean way to restore data after a bad release. If your workload is customer-facing and tied to uptime, this is safer than exposing the app directly on a public port.

1) Connect to the VPS and identify Debian

On your local computer, open your first SSH session with the example address below.

ssh root@203.0.113.10

203.0.113.10 is a reserved documentation address. Replace it with the real public IP assigned to your Hostperl server.

If your provider gave you a default non-root login, use that account instead:

ssh deploy@203.0.113.10

Once connected, identify the operating system before you install anything.

On the VPS as root:

cat /etc/os-release

You should see Debian 12 details such as ID=debian and VERSION_ID="12". If you are on Debian 13, the steps still apply, but package versions may differ slightly.

2) Create a non-root administrator and keep root open

Do not disable root access until you have tested the new account in a second terminal. That avoids lockout while you are still setting up SSH keys and sudo access.

On the VPS as root, create the account, add sudo access, and set a password you can use once for the first login test:

adduser deploy

Follow the prompts and choose a strong password. Then grant sudo rights:

usermod -aG sudo deploy

Now prepare SSH access. If your public key is already on your local computer, copy it into place with ssh-copy-id from the local machine:

ssh-copy-id deploy@203.0.113.10

That command appends your local public key to /home/deploy/.ssh/authorized_keys. If you need to do it manually, use this safer sequence on the VPS:

sudo -iu deploy mkdir -p /home/deploy/.ssh
sudo -iu deploy chmod 700 /home/deploy/.ssh
sudo -iu deploy touch /home/deploy/.ssh/authorized_keys
sudo -iu deploy chmod 600 /home/deploy/.ssh/authorized_keys

Paste your public key into /home/deploy/.ssh/authorized_keys with a text editor, then set ownership:

chown -R deploy:deploy /home/deploy/.ssh

On your local computer, open a second terminal and test the new login before changing any SSH policy:

ssh deploy@203.0.113.10

After login, verify sudo access:

sudo -v

If that works, keep both sessions open and continue in the root shell for the remaining server setup.

3) Update the system, time sync, and basic hardening

On the VPS as root, update package lists and install the basic tools you will use throughout the build:

apt update
apt -y full-upgrade
apt -y install ca-certificates curl gnupg git ufw fail2ban chrony

Debian uses apt, and chrony keeps the system clock accurate. That matters for TLS, log correlation, and token expiry.

Enable time sync and check the status:

systemctl enable --now chrony
chronyc tracking

You should see a stable time source and a small offset.

Set a clean hostname if needed:

hostnamectl set-hostname server.example.com

Use server.example.com as the example hostname in this tutorial and replace it with your own fully qualified name if you have one.

4) Build a safe firewall rule set

Open SSH first, then add HTTP and HTTPS. Do not remove SSH access until you have tested the new account and confirmed the firewall allows it.

On the VPS as root:

ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw --force enable
ufw status verbose

You should see OpenSSH, 80/tcp, and 443/tcp listed as allowed. The app port will stay private on the loopback interface only.

For customers who later need a tighter perimeter, Hostperl’s network and VPS support teams can help align the firewall with your launch window and DNS cutover. That is one reason many small teams keep application hosting on a managed VPS rather than scattering services across multiple hosts.

5) Install Docker and Docker Compose on Debian

This stack uses Docker because it gives you a clean rollback path. You can stop one Compose file and start another without changing the host packages.

On the VPS as root, install Docker from Debian packages:

apt -y install docker.io docker-compose-plugin
systemctl enable --now docker
docker --version
docker compose version

If Docker starts correctly, the version commands should print current package versions and the service should be active.

Add your non-root administrator to the Docker group so you can manage the stack without root:

usermod -aG docker deploy

Log out and back in as deploy so group membership takes effect.

6) Create the application layout and environment file

On the VPS as the non-root sudo user, create the application directory structure:

sudo mkdir -p /opt/myapp/{app,nginx,postgres,qdrant}
sudo chown -R deploy:deploy /opt/myapp
cd /opt/myapp
pwd

Now create the environment file that will hold secrets and connection strings:

nano /opt/myapp/.env

Paste this content into the file:

POSTGRES_DB=myapp
POSTGRES_USER=myapp
POSTGRES_PASSWORD=change_this_to_a_long_unique_password
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
QDRANT_URL=http://qdrant:6333
APP_PORT=8000
APP_SECRET_KEY=change_this_to_another_long_unique_secret

Save and exit. In nano, press Ctrl+O, Enter, then Ctrl+X.

Restrict the file so only the owner can read it:

chmod 600 /opt/myapp/.env

7) Add a simple private RAG app container

For a production tutorial, the app image should be predictable. This example uses a small Python API container with a health endpoint and placeholders for retrieval logic.

Replace it with your actual application code later, but keep the deployment pattern.

On the VPS as the non-root sudo user, create the app file:

nano /opt/myapp/app/Dockerfile

Paste this Dockerfile:

FROM python:3.12-slim
WORKDIR /app
RUN pip install --no-cache-dir fastapi uvicorn[standard] psycopg[binary] httpx
COPY app.py /app/app.py
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Save and exit, then create the app source file:

nano /opt/myapp/app/app.py

Paste this minimal private RAG app skeleton:

from fastapi import FastAPI

app = FastAPI()

@app.get("/health")
def health():
    return {"status": "ok"}

@app.get("/")
def root():
    return {"message": "private RAG app online"}

This tutorial keeps the code small so the hosting pattern stays in focus. In a real deployment, your retrieval layer would query PostgreSQL and Qdrant and return an answer assembled from your indexed documents.

8) Define PostgreSQL, Qdrant, and the app in Compose

On the VPS as the non-root sudo user, create the Compose file:

nano /opt/myapp/compose.yml

Paste this content:

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - ./postgres:/var/lib/postgresql/data
    restart: unless-stopped

  qdrant:
    image: qdrant/qdrant:v1.13.1
    volumes:
      - ./qdrant:/qdrant/storage
    restart: unless-stopped

  app:
    build: ./app
    env_file:
      - .env
    depends_on:
      - postgres
      - qdrant
    ports:
      - "127.0.0.1:8000:8000"
    restart: unless-stopped

Save and exit. The app port binds only to 127.0.0.1, so nobody can reach it directly from the internet.

Build and start the stack:

cd /opt/myapp
docker compose up -d --build
docker compose ps

The containers should show Up. If one fails, inspect it immediately rather than continuing.

9) Check logs and fix container startup issues

On the VPS as the non-root sudo user, check the app logs if the stack does not come up cleanly:

docker compose logs --no-color --tail=100 app

docker compose logs --no-color --tail=100 postgres

docker compose logs --no-color --tail=100 qdrant

If PostgreSQL reports authentication errors, confirm that the password in .env matches the Compose file. If Qdrant fails, check disk space with df -h and permissions on /opt/myapp/qdrant.

A useful Hostperl reference for release hygiene is Docker Compose Release Readiness for VPS Launches. It pairs well with this deployment pattern because it treats the host and the Compose stack as one release unit.

10) Put Nginx in front and keep the app private

Nginx will handle public traffic, TLS, and request forwarding to the private app. The app stays on localhost.

On the VPS as root, install Nginx:

apt -y install nginx
systemctl enable --now nginx
nginx -v

Create a site configuration:

nano /etc/nginx/sites-available/myapp

Paste this configuration:

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Enable the site and test syntax before reloading:

ln -s /etc/nginx/sites-available/myapp /etc/nginx/sites-enabled/myapp
nginx -t
systemctl reload nginx

nginx -t must report that the configuration is successful. If it fails, fix the file before reloading.

11) Add Let’s Encrypt TLS

Once DNS for example.com points to your VPS, request a certificate. Replace the domain with your real hostname before you run the command.

On the VPS as root:

apt -y install certbot python3-certbot-nginx
certbot --nginx -d example.com

Choose the redirect option when prompted so HTTP moves to HTTPS. Certbot will update Nginx and install an auto-renewal timer.

Confirm renewal plumbing now, not after launch:

systemctl list-timers | grep certbot
certbot renew --dry-run

If the dry run succeeds, your certificate renewal path is ready.

12) Harden SSH after the new login is verified

Only do this after you have confirmed the deploy account can log in and use sudo. Keep the root session open until the end of this step.

On the VPS as root, edit SSH settings:

nano /etc/ssh/sshd_config

Set or add these lines:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes

Test the SSH daemon configuration first:

sshd -t

If there is no output, the config syntax is valid. Then reload SSH:

systemctl reload ssh

Open a new terminal and confirm you can still connect as deploy. Do not close the original root session until that test passes.

13) Add Fail2Ban for noisy login attempts

On the VPS as root, create a small jail override for SSH:

nano /etc/fail2ban/jail.d/sshd.local

Paste this content:

[sshd]
enabled = true
maxretry = 3
findtime = 10m
bantime = 1h

Save the file and start the service:

systemctl enable --now fail2ban
systemctl restart fail2ban
fail2ban-client status sshd

You should see the SSH jail active. This is a practical safeguard for public-facing admin ports.

14) Verify the stack from the server and from a client

On the VPS as the non-root sudo user, confirm the services and listening ports:

docker compose ps
ss -ltnp | grep -E '(:80|:443|:8000)'
journalctl -u nginx -n 50 --no-pager

You should see Nginx listening on 80 and 443, while the app stays bound to localhost on 8000.

On your local computer, test the public site:

curl -I https://example.com
curl https://example.com/health

Successful output should show a valid HTTPS response and a JSON health payload. If TLS is wrong, check the certificate name and the DNS record.

Run one functional smoke test from the server itself as well:

curl http://127.0.0.1:8000/health

If that works locally but not externally, the problem is almost always Nginx, DNS, or firewall rules.

15) Reboot test and persistence check

On the VPS as root, confirm the service survives a reboot:

reboot

Reconnect after the host returns, then check:

systemctl is-active nginx
systemctl is-active docker
systemctl is-active fail2ban

Log in as deploy and confirm the Compose stack resumed:

cd /opt/myapp
docker compose ps

If the containers do not restart, verify that the restart: unless-stopped policy is still present in compose.yml.

Rollback and recovery path

If a new release breaks the app, stop the stack, restore the previous Compose file, and restart cleanly. That is the benefit of keeping the app, database, and vector store isolated.

On the VPS as the non-root sudo user:

cd /opt/myapp
docker compose down
cp compose.yml compose.yml.broken.$(date +%F-%H%M%S)
# restore your last known-good compose.yml here
docker compose up -d --build
docker compose ps

If the database files become corrupted, restore from a volume snapshot or your last validated backup. For teams that use PostgreSQL heavily, Hostperl’s PostgreSQL logical backup guide for Debian 12 is a good companion to this setup. If you prefer point-in-time recovery, this PITR recovery tutorial shows the restore pattern in a production context.

Troubleshooting the most likely failures

Nginx returns 502 Bad Gateway

Run:

systemctl status nginx --no-pager
curl -I http://127.0.0.1:8000
journalctl -u nginx -n 100 --no-pager

If the local curl fails, the app container is down. Rebuild the stack with docker compose up -d --build. If Nginx logs show upstream connection refused, confirm the proxy target is 127.0.0.1:8000.

Certbot fails to issue a certificate

Run:

dig +short example.com
ss -ltnp | grep ':80'
certbot certificates

If DNS does not point to the VPS or port 80 is blocked, fix that first. Certbot needs public reachability for the domain you requested.

Docker containers do not start after reboot

Run:

systemctl status docker --no-pager
docker compose ps
docker compose logs --tail=50

If the Docker service is inactive, enable it with systemctl enable --now docker. If the app container keeps exiting, inspect the application logs and the environment file.

Wrap-up

You now have a private RAG app running on Debian 12 with a locked-down public edge, Docker-based rollback, PostgreSQL storage, Qdrant vector search, and TLS in front of the app. That is a practical production baseline, not a lab exercise.

For teams planning their next deployment window, Hostperl VPS hosting gives you the control needed for containers, reverse proxies, and private application workloads without losing the support path that matters during cutover week. If you are also comparing launch-readiness practices, this release checklist for VPS deployments is worth pairing with the stack you just built.

FAQ

Can I run this on Ubuntu or AlmaLinux?
Yes. The app architecture is portable, but the package manager, firewall, and SSH service names change. Debian 12 is used here because this tutorial is written for the Debian track.

Why keep Qdrant and PostgreSQL in separate containers?
It makes backup, restore, and failure isolation much simpler. If the vector store needs a rebuild, your metadata database still survives.

Should I expose port 8000 to the internet?
No. Keep it on localhost and let Nginx handle public traffic. That reduces your attack surface and makes TLS easier to manage.

What is the fastest way to test a broken deployment?
Start with docker compose ps, then check each container log with docker compose logs. After that, test curl http://127.0.0.1:8000/health locally and curl -I https://example.com from your client.