Deploy a Fault-Tolerant RAG Worker on RHEL 9

What you are building

This tutorial shows you how to deploy a RAG worker on RHEL 9 for a production support bot that answers from your own documents. The setup stays intentionally simple: FastAPI accepts incoming requests, a background worker handles retrieval jobs, PostgreSQL stores job state, and Qdrant holds the vector index. You will start on a fresh VPS, harden SSH, add a non-root admin, open the right firewall ports, and verify everything after a reboot.

This is the kind of workload Hostperl customers often start on a small Hostperl VPS before moving to a larger node or a dedicated server. If you already read private AI inference sizing for VPS buyers, the same capacity rules apply here: keep the worker modest, watch memory use, and keep the vector database on persistent disk.

Focus keyword: RAG worker. Primary intent: deploy a reliable document-answering backend that survives restarts, supports backups, and is easy to troubleshoot.

Architecture and limits

This guide uses RHEL 9 because it gives you current dnf, firewalld, and SELinux controls that fit production hosting. It does not use Docker. That makes the first rollout easier to audit and support, which matters when a client wants to know why the bot went quiet before launch.

• PostgreSQL: stores documents, chunk metadata, and job records.
• Qdrant: stores embeddings and similarity search data.
• FastAPI: receives upload and query requests.
• systemd: keeps the worker and API alive after reboot.
• firewalld: exposes only SSH and the app port.

If you want a different persistence pattern later, compare this with RAG hosting capacity and backup planning. For database recovery drills, the restore workflow in PostgreSQL restore drills is the right follow-on reading.

Connect to the VPS and confirm the OS

On your local computer, open an SSH session:

ssh root@203.0.113.10

203.0.113.10 is a reserved documentation address. Replace it with the real public IP from your Hostperl server. If your provider gives you a default non-root account, you can connect with that account first, then elevate with sudo.

On the VPS as root, identify the operating system before you install anything:

cat /etc/os-release

You should see RHEL 9 or an Oracle Linux 9 equivalent. This guide uses RHEL-compatible commands only, because the stack here depends on dnf, firewalld, and SELinux-friendly defaults.

Create a non-root administrator

Keep the root session open until the new login works. That avoids lockout if you mistype a sudo or SSH setting.

On the VPS as root, create the admin account, add it to wheel, and prepare SSH access:

useradd -m -G wheel deploy
passwd deploy
mkdir -p /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
cp /root/.ssh/authorized_keys /home/deploy/.ssh/authorized_keys
chown -R deploy:deploy /home/deploy/.ssh
chmod 600 /home/deploy/.ssh/authorized_keys

The password step gives you a fallback. If you already manage keys carefully, you can lock the password later with passwd -l deploy after SSH key login is confirmed.

On your local computer, open a second terminal and test the new login:

ssh deploy@203.0.113.10

After you connect, confirm sudo:

sudo -v
whoami
id

You should see deploy as your user and wheel in the group list. Only then should you consider disabling root password login.

Update the server and install packages

On the VPS as the non-root sudo user, update the system and install the runtime packages:

sudo dnf -y update
sudo dnf -y install python3 python3-pip python3-virtualenv postgresql-server postgresql-contrib qdrant curl git policycoreutils-python-utils firewalld

Check the installed versions so you know what you are supporting:

python3 --version
psql --version
qdrant --version || true

On a production VPS, current packages matter. They reduce unsupported edge cases when you need help, and they make later maintenance easier for your team or agency client.

Initialize PostgreSQL and Qdrant

On the VPS as root, initialize and start PostgreSQL:

postgresql-setup --initdb
systemctl enable --now postgresql

Create a dedicated database and user for the worker:

sudo -iu postgres psql <<'SQL'
CREATE DATABASE ragbot;
CREATE USER ragbot_user WITH PASSWORD 'ChangeThisPasswordNow!';
GRANT ALL PRIVILEGES ON DATABASE ragbot TO ragbot_user;
SQL

Replace the password with a strong secret before you go live. Store it in a root-only file later. You can validate the database is reachable with:

sudo -iu postgres psql -d ragbot -c '\conninfo'

Start Qdrant and enable it at boot:

sudo systemctl enable --now qdrant

Check that it is listening locally:

ss -ltnp | grep -E ':(6333|5432)\b'

PostgreSQL should listen on 5432. Qdrant typically listens on 6333 for HTTP. Keep Qdrant bound to localhost unless you truly need remote access.

Build the application directory and environment

On the VPS as the non-root sudo user, create the app layout under /opt/myapp:

sudo mkdir -p /opt/myapp/{app,data,logs}
sudo chown -R deploy:deploy /opt/myapp

Create the Python virtual environment and install the app dependencies:

cd /opt/myapp
python3 -m virtualenv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install fastapi uvicorn psycopg2-binary qdrant-client sentence-transformers

Now create the environment file. Open it with your editor of choice:

sudo tee /etc/myapp-rag.env >/dev/null <<'EOF'
DATABASE_URL=postgresql://ragbot_user:ChangeThisPasswordNow!@127.0.0.1:5432/ragbot
QDRANT_URL=http://127.0.0.1:6333
APP_HOST=127.0.0.1
APP_PORT=8000
EOF

Lock it down so only root can read secrets:

sudo chown root:root /etc/myapp-rag.env
sudo chmod 600 /etc/myapp-rag.env

That file contains credentials. If it is world-readable, you have a real incident, not a minor mistake.

Install the FastAPI worker service

Create a minimal app that accepts documents and returns a placeholder answer flow. Replace it later with your real retrieval logic, but keep the service shape the same.

On the VPS as the non-root sudo user, create the app file:

cat > /opt/myapp/app/main.py <<'PY'
from fastapi import FastAPI
app = FastAPI()

@app.get('/health')
def health():
    return {'status': 'ok'}
PY

Test it manually before systemd gets involved:

cd /opt/myapp
source .venv/bin/activate
uvicorn app.main:app --host 127.0.0.1 --port 8000

In another shell, check the endpoint:

curl -s http://127.0.0.1:8000/health

You should get {"status":"ok"}. Stop the test server with Ctrl+C.

Next, create the systemd unit:

sudo tee /etc/systemd/system/rag-worker.service >/dev/null <<'EOF'
[Unit]
Description=RAG Worker API
After=network-online.target postgresql.service qdrant.service
Wants=network-online.target

[Service]
User=deploy
Group=deploy
WorkingDirectory=/opt/myapp
EnvironmentFile=/etc/myapp-rag.env
ExecStart=/opt/myapp/.venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000
Restart=always
RestartSec=3
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ProtectHome=true

[Install]
WantedBy=multi-user.target
EOF

Check the file syntax by asking systemd to parse it, then start the service:

sudo systemctl daemon-reload
sudo systemctl enable --now rag-worker
sudo systemctl status rag-worker --no-pager

If the status shows active (running), the service is ready for the next layer.

Open the firewall safely

On the VPS as root, allow SSH and the app port before you change any remote access rules:

sudo systemctl enable --now firewalld
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-port=8000/tcp
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

The rule change should show ssh and 8000/tcp in the active zone. If you later add a reverse proxy, you can remove direct public access to port 8000 after the proxy is tested.

For SSH hardening, create a second session first, confirm key-based login, and only then adjust /etc/ssh/sshd_config. A safe next step is to disable password login once you have a verified admin key in place.

Check SELinux and app reachability

RHEL 9 usually runs SELinux in enforcing mode. Confirm that before you troubleshoot anything else:

getenforce

If the service binds to 127.0.0.1 and only local users connect, SELinux usually stays quiet. If you later move it behind Nginx or expose another port, use the following command to review denials:

sudo ausearch -m avc -ts recent

That gives you the exact denial clue, which is far better than guessing. If you see AVC hits, fix the labeling or port policy instead of disabling SELinux.

Smoke test the RAG worker

On the VPS as the non-root sudo user, check the unit and the HTTP endpoint:

systemctl is-enabled rag-worker
systemctl is-active rag-worker
curl -s http://127.0.0.1:8000/health

Then confirm the listening socket:

ss -ltnp | grep ':8000'

From your local computer, run a client-side check against the public address if you intend to expose the port:

curl -s http://203.0.113.10:8000/health

Replace 203.0.113.10 with your server IP. If that fails but the local check passes, your firewall or routing is likely the issue.

Failure diagnostics and rollback

Most launch problems fall into a few patterns. Start with the service logs:

sudo journalctl -u rag-worker -n 50 --no-pager

If you see a Python import error, check the virtual environment path in the unit file. If you see database authentication failures, confirm the password in /etc/myapp-rag.env and test with:

sudo -iu postgres psql -h 127.0.0.1 -U ragbot_user -d ragbot -c '\conninfo'

If Qdrant is unreachable, inspect its service and port:

sudo systemctl status qdrant --no-pager
ss -ltnp | grep 6333

A clean rollback is straightforward. Stop the app, disable it, and keep your data files intact:

sudo systemctl disable --now rag-worker
sudo rm -f /etc/systemd/system/rag-worker.service
sudo systemctl daemon-reload

If a bad firewall change blocked access, remove only the new rule you added, then reload firewalld. Do not delete SSH access until you have another verified path in place.

Backups and reboot verification

At this point, you have a working service. Now make sure it survives a reboot and can be recovered. Confirm enabled services:

systemctl is-enabled postgresql
systemctl is-enabled qdrant
systemctl is-enabled rag-worker

Reboot the VPS during a maintenance window, then reconnect and test again:

sudo reboot

After the server returns, log back in and run:

systemctl is-active postgresql qdrant rag-worker
curl -s http://127.0.0.1:8000/health

For the data layer, back up PostgreSQL with a plain dump and keep Qdrant volumes on persistent storage. If you want a restore rehearsal, use the workflow in PostgreSQL restore drills for safer VPS recovery.

FAQ

Can I expose Qdrant directly to the internet?

Usually no. Keep Qdrant bound to localhost unless you have a separate private network and a clear reason to publish it.

Should I use Docker instead?

You can, but this guide uses native services so you can inspect logs, manage SELinux, and troubleshoot faster on a fresh RHEL 9 VPS.

What if I need more throughput?

Increase RAM first, then move PostgreSQL and Qdrant onto faster storage. If the worker is still busy, split the ingestion job from the query API.

How do I know the service is ready for a client demo?

Run the health check locally and from your client network, confirm a reboot restores both services, and review the last 50 journal lines with no errors.