RHEL 9 Private AI API Gateway with Nginx and SELinux

Use case and deployment plan

A private AI API gateway gives you one controlled entry point in front of a model server, vector service, or internal inference app. On RHEL 9, that usually means Nginx handling TLS and request limits, SELinux keeping the reverse proxy confined, and firewalld exposing only the ports you actually need. This setup works well when you want private model access without placing the backend on the public network.

In this tutorial, you will build a production-ready private AI API gateway on RHEL 9. The example backend listens on 127.0.0.1:8000, and Nginx publishes it on port 443 after you add a certificate. If you need hosting capacity for the backend itself, a Hostperl VPS fits smaller inference services well, while larger or GPU-backed deployments belong on enterprise dedicated hosting or dedicated server hosting.

This guide uses RHEL 9 because SELinux and firewalld are part of the normal production baseline. The same pattern also fits Oracle Linux 9 and Rocky Linux 9 with the same package manager and service model.

Before you start

You will need a fresh RHEL 9 server with root access, a domain name pointed at the server, and a backend AI service already running locally on port 8000. If your backend lives in Docker or another process manager, keep it bound to 127.0.0.1 so the public network never reaches it directly.

If you are setting up a new account for ongoing work, create the deploy user first and keep your root session open until SSH and sudo are confirmed. For background on safe server access and exposure control, see SSH hardening without lockouts and server security with UFW and Fail2Ban.

Connect to the server and confirm the OS

On your local computer, connect with the documentation IP first, then replace it with your real server IP.

ssh root@203.0.113.10

203.0.113.10 is a reserved documentation address. Replace it with the public IP assigned by your hosting provider. If your server uses a non-root default account, connect with that account instead and keep the same IP pattern.

On the VPS as root, confirm the operating system before you install anything.

cat /etc/os-release

You should see RHEL 9, Rocky Linux 9, or Oracle Linux 9 details. If the output shows a different platform, stop and follow the correct distribution path for that host.

Create the deploy account and test sudo

On the VPS as root, create the non-root administrator, add it to the sudo-capable group, and lock down the password after key-based access is in place.

useradd -m -s /bin/bash deploy
passwd deploy
usermod -aG wheel deploy

The first command creates the account, the second sets a temporary password only if you need one, and the third grants sudo via the wheel group. If you use SSH keys only, you can skip setting a permanent password and disable password login later.

On your local computer, copy your SSH key to the new account.

ssh-copy-id deploy@203.0.113.10

Replace 203.0.113.10 with your real server IP. If ssh-copy-id is unavailable on your workstation, copy the public key manually into /home/deploy/.ssh/authorized_keys.

On the VPS as root, fix ownership and permissions after creating the key directory by hand.

mkdir -p /home/deploy/.ssh
chown -R deploy:deploy /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
chmod 600 /home/deploy/.ssh/authorized_keys

Now open a second terminal from your computer and test the new login.

ssh deploy@203.0.113.10

On the VPS as the non-root sudo user, confirm sudo access.

sudo -v
whoami
id

You should see root prompts for sudo validation, then deploy as the current user and wheel in the group list. Keep the root session open until this test succeeds.

Update packages and install the gateway stack

On the VPS as root, refresh package metadata and install Nginx plus the SELinux utilities needed to label proxy connections correctly.

dnf update -y
dnf install -y nginx policycoreutils-python-utils firewalld curl

RHEL-family systems ship with SELinux enforcing by default in many deployments. The utilities you install here let Nginx connect to a local backend without disabling SELinux.

Check the installed Nginx version before you continue.

nginx -v

You should see a current RHEL 9 build line. If package installation fails, check repository access before moving forward.

Prepare the AI backend for local-only access

Your backend should bind to 127.0.0.1:8000. That keeps the inference service private and leaves Nginx as the only public-facing entry point. If you are running a Python app, a Node service, or a model server, make sure its listen address is not 0.0.0.0.

Verify that something is listening on the expected port.

ss -ltnp | grep ':8000'

You should see a process bound to localhost. If you do not, start your backend now and only then proceed to the reverse proxy layer.

If you are building the backend on the same server, the surrounding app pattern in RAG application hosting with PostgreSQL is a useful reference for keeping app services private behind a proxy.

Configure Nginx as the private AI API gateway

On the VPS as root, create a dedicated Nginx site file for the gateway.

cat > /etc/nginx/conf.d/private-ai-gateway.conf <<'EOF'
server {
    listen 80;
    server_name server.example.com;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 5s;
        proxy_send_timeout 300s;
        proxy_read_timeout 300s;
    }
}
EOF

server.example.com is the example hostname. Replace it with your real DNS name before you test or reload Nginx. The long read timeout matters for model inference endpoints that stream or take several seconds to answer.

Test the configuration before restarting Nginx.

nginx -t

If the syntax is correct, Nginx will report success. Then start and enable the service.

systemctl enable --now nginx

Check that Nginx is active.

systemctl status nginx --no-pager

Open the firewall safely

On the VPS as root, open HTTP first so you can verify the gateway before adding TLS. This avoids locking yourself out if certificate setup is not ready yet.

systemctl enable --now firewalld
firewall-cmd --permanent --add-service=http
firewall-cmd --reload
firewall-cmd --list-all

You should see http in the allowed services list. After TLS is in place, add HTTPS before deciding whether to keep HTTP for redirect-only use.

Allow Nginx to talk to the local backend under SELinux

SELinux blocks proxy connections to local services unless you explicitly allow them. That is the right default for a private AI API gateway, because it prevents the web server from reaching arbitrary ports.

On the VPS as root, permit Nginx to connect to network services locally.

setsebool -P httpd_can_network_connect on

If your backend binds only to localhost, this boolean is usually sufficient. You can confirm the current SELinux mode with:

getenforce

The expected output is Enforcing. If you see AVC denials later, inspect them instead of disabling SELinux.

Test the gateway from the server and client

On the VPS as root, test the upstream directly and then through Nginx.

curl -i http://127.0.0.1:8000/
curl -i http://127.0.0.1/

The first command confirms the backend, and the second confirms that Nginx can proxy to it. If your API has a health endpoint such as /health, use that path instead of the root path for a cleaner test.

On your local computer, run the same test against the public hostname or IP.

curl -i http://server.example.com/

Replace server.example.com with the real domain. You should see the backend response coming back through Nginx, not a direct backend error.

Enable TLS for public access

For production traffic, add Let's Encrypt after DNS resolves correctly. If your domain is already pointed at the server, install Certbot and issue the certificate.

dnf install -y certbot python3-certbot-nginx
certbot --nginx -d server.example.com

Replace the hostname with your real domain. Certbot will update the Nginx site and can add the HTTP-to-HTTPS redirect during issuance. Accept the redirect if you want all traffic forced to TLS.

Confirm certificate renewal is scheduled.

systemctl list-timers | grep certbot

Then add HTTPS to the firewall.

firewall-cmd --permanent --add-service=https
firewall-cmd --reload

You can leave HTTP open if you are using it only for redirects. If you prefer tighter exposure, remove the HTTP service only after HTTPS works from an external client.

Add basic API protection

A private AI gateway should not behave like an open public endpoint. Start with request limits so a burst of traffic does not exhaust the backend.

On the VPS as root, extend the server block to limit request rates per client IP. Edit the file and add these lines inside the server block, above location /:

cat > /etc/nginx/conf.d/private-ai-gateway.conf <<'EOF'
limit_req_zone $binary_remote_addr zone=ai_limit:10m rate=5r/s;

server {
    listen 80;
    server_name server.example.com;

    location / {
        limit_req zone=ai_limit burst=20 nodelay;
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 5s;
        proxy_send_timeout 300s;
        proxy_read_timeout 300s;
    }
}
EOF

Test the syntax again before reloading.

nginx -t
systemctl reload nginx

If you use an API key header or bearer token in your backend, keep that secret in the application layer rather than Nginx. For request-level logging and diagnosis, Nginx logging for faster app troubleshooting gives a good pattern for reading proxy logs efficiently.

Watch logs and handle the common failures

On the VPS as root, inspect the web server logs after your first live tests.

journalctl -u nginx -n 50 --no-pager
journalctl -u firewalld -n 30 --no-pager

If the browser shows 502 Bad Gateway, the backend is not reachable or is returning too slowly. Check the upstream first:

ss -ltnp | grep ':8000'
curl -i http://127.0.0.1:8000/

If SELinux blocks the proxy, look for denials and then confirm the boolean is set.

ausearch -m avc -ts recent
getsebool httpd_can_network_connect

The expected clue is an AVC denial mentioning httpd and the backend port. If the boolean is off, turn it on again with setsebool -P httpd_can_network_connect on.

If Nginx returns 403 or a redirect loop, check the host name in the server block and the TLS redirect rules. A mismatched server_name is a common cause when a domain points to the server but the config still uses the sample hostname.

Smoke test the gateway like a real client

Now run one functional request that looks like production traffic. Use a prompt or payload your backend actually accepts. For a JSON API, the pattern looks like this:

curl -sS -X POST https://server.example.com/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"local-model","messages":[{"role":"user","content":"Say hello in one sentence."}]}'

Replace the endpoint, model name, and payload shape with your real backend contract. A successful response should come back through Nginx and TLS, proving that the proxy, backend, and DNS chain all work together.

Make it survive a reboot

On the VPS as root, confirm the services start automatically after reboot.

systemctl is-enabled nginx
systemctl is-enabled firewalld

Both should report enabled. If your backend runs as a systemd service, check that one too. Then reboot during a maintenance window and validate again after login.

reboot

After the server returns, reconnect and test Nginx plus the backend. This is the last check you want before sending traffic to the gateway.

Rollback and recovery

If a config change breaks the gateway, revert the last edit, test syntax, and reload. Keep a known-good copy before major changes.

cp /etc/nginx/conf.d/private-ai-gateway.conf /etc/nginx/conf.d/private-ai-gateway.conf.bak
# restore the previous version if needed
nginx -t
systemctl reload nginx

If you add TLS and want to back out quickly, remove the Certbot redirect first, then release the HTTPS-only firewall rule only after the site works on HTTP again. That sequence avoids a second outage during recovery.

FAQ

Why keep the backend on localhost?

It removes the attack surface from the public network. Nginx becomes the only exposed service, and SELinux can confine it tightly.

Can I use this with Docker?

Yes. Bind the container port to 127.0.0.1 on the host, then proxy to that local port from Nginx.

What if my AI app streams long responses?

Raise proxy_read_timeout and confirm your backend sends data before the timeout. A long timeout is normal for model generation.

Should I disable SELinux to make it easier?

No. Keep it enforcing and enable only the booleans or labels your gateway actually needs.

How do I know the gateway is ready for production?

You should have HTTPS, a working health check, rate limiting, enabled services after reboot, and no AVC denials in the logs during a test request.