In this tutorial, we'll learn how to install Kubernetes on Rocky Linux 10 with Cilium (eBPF), secrets encryption, kubeadm and containerd. Complete production guide.
Introduction
We build a Kubernetes cluster on Rocky Linux 10 LTS using kubeadm and containerd, then enhance it with eBPF-based networking through Cilium. Instead of relying on traditional iptables-based networking, we use Cilium to leverage eBPF in the Linux kernel for improved performance, observability, and security.
We also implement foundational hardening measures, including Pod Security enforcement and encryption of Kubernetes Secrets at rest. These steps ensure that sensitive data is protected inside etcd and that workloads follow security best practices by default.
Prerequisites
Before we begin, ensure we have the following:
- An Rocky Linux 10 on dedicated server or KVM VPS.
- Basic Linux Command Line Knowledge.
Environment Requirements
Minimum per control plane node:
- 2 CPU
- 2GB RAM (4GB recommended)
- 20GB disk
Step 1: Prepare Rocky Linux 10
Update system:
sudo dnf update -y
Disable swap (required):
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab
Kubernetes scheduler assumes memory is not swapped. If swap is on, pod scheduling becomes unreliable.
Load required kernel modules:
sudo modprobe overlay
sudo modprobe br_netfilter
Persist them:
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
Set sysctl values:
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system
These settings allow Kubernetes to manage network traffic between pods correctly.
Step 2: Install containerd
Kubernetes does not run containers by itself. It needs a runtime.
We use containerd because:
- It is lightweight
- It is recommended for production
- It works perfectly with kubeadm
Add Docker repo
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Now install containerd
sudo dnf install -y containerd
Generate default config:
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
Enable systemd cgroup driver:
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
Restart:
sudo systemctl enable --now containerd
Why change cgroup driver?
Rocky Linux 10 uses systemd. Kubernetes must match the same cgroup driver or we get instability.
Step 3: Install Kubernetes Packages
Create repository:
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.35/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.35/rpm/repodata/repomd.xml.key
EOF
Install:
sudo dnf install -y kubelet kubeadm kubectl
Enable kubelet:
sudo systemctl enable --now kubelet
Note: Only above commands need to execute on both master node and worker node. After this all below commands need to execute on master node only.
Step 4: Initialize Control Plane
Allow 6443 port, so that other worker node gets connected to master node:
firewall-cmd --permanent --add-port=6443/tcp
firewall-cmd --reload
Run on master node:
sudo kubeadm init
At the end, it gives a kubeadm join command. Save it.
After completion, configure kubectl:
mkdir -p $HOME/.kube
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Verify:
kubectl get nodes
Node will show NotReady until networking is installed.
Step 5: Join Worker Nodes
On worker nodes, run the join command displayed by kubeadm.
sudo kubeadm join 192.168.1.10:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>
After joining:
kubectl get nodes
Step 6: Install Cilium (eBPF Networking)
We use Cilium because:
- It uses eBPF (modern Linux kernel technology)
- Better performance than traditional iptables
- Built-in network security
Download CLI:
curl -L --remote-name https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz
sudo tar xzvf cilium-linux-amd64.tar.gz -C /usr/local/bin
Install Cilium:
cilium install
Check status:
cilium status
Nodes should now become Ready.
Now check nodes:
kubectl get nodes
They should become Ready.
What is eBPF doing here?
Instead of using iptables for networking, Cilium programs the Linux kernel directly using eBPF. This makes networking faster, scalable, and more secure.
Step 7: Enable Secrets Encryption at Rest
By default, Kubernetes stores secrets unencrypted in etcd. We now enable encryption properly.
Generate Encryption Key
head -c 32 /dev/urandom | base64
Copy the output. Create Encryption Configuration File
sudo nano /etc/kubernetes/encryption.yaml
Add following content:
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc:
keys:
- name: key1
secret: PASTE_BASE64_KEY_HERE
- identity: {}
Save and exit the file.
Set permissions:
sudo chmod 600 /etc/kubernetes/encryption.yaml
sudo chown root:root /etc/kubernetes/encryption.yaml
Mount Encryption File in API Server Static Pod
Edit:
sudo nano /etc/kubernetes/manifests/kube-apiserver.yaml
Add flag under command section:
- --encryption-provider-config=/etc/kubernetes/encryption.yaml
Add under volumeMounts:
volumeMounts:
- mountPath: /etc/kubernetes/encryption.yaml
name: encryption-config
readOnly: true
Add under volumes:
volumes:
- name: encryption-config
hostPath:
path: /etc/kubernetes/encryption.yaml
type: File
Save file.
Kubelet automatically restarts the API server.
Re-encrypt Existing Secrets
Run:
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
Now secrets are encrypted at rest.
Step 8: Apply Basic Security Hardening
Enable Pod Security (restricted mode):
kubectl label ns default \
pod-security.kubernetes.io/enforce=restricted
This prevents:
- Privileged containers
- Root containers
- Dangerous capabilities
Create default deny network policy:
sudo nano deny.yaml
Add following content:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: default
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
Apply:
kubectl apply -f deny.yaml
Verify Cluster Health
Check:
kubectl get nodes
kubectl get pods -A
cilium status
Troubleshoot
If you get any issue related to SELinux, execute following commands:
Label Kubernetes Directories Correctly
By default, Kubernetes directories like etcd data and PKI files need access inside container runtimes. SELinux will block them unless labeled.
Apply correct SELinux types:
sudo mkdir -p /var/lib/etcd
sudo mkdir -p /etc/kubernetes/pki
sudo chcon -R -t svirt_sandbox_file_t /var/lib/etcd
sudo chcon -R -t svirt_sandbox_file_t /etc/kubernetes/pki
This uses svirt_sandbox_file_t type so containers can read those directories. Without this, kubelet and etcd will fail to access PKI and data paths.
Allow Kubelet and Containerd to Work with SELinux
By default, containerd and kubelet need permissions to manage containers and writes under /var/lib/kubelet and runtime socket paths.
Ensure correct labels:
sudo chcon -R -t container_file_t /var/lib/kubelet
sudo chcon -R -t container_file_t /run/containerd/
Conclusion
We have successfully seen how to install Kubernetes on Rocky Linux 10 with Cilium. Modern networking and essential security controls enabled from the start. The cluster uses containerd for runtime stability, Cilium for high-performance eBPF networking, and encryption at rest to protect sensitive data stored in etcd.
More importantly, we have established a security-first baseline. Pod Security standards prevent insecure workloads from running. Network isolation policies restrict unnecessary communication. Secrets are no longer stored in plaintext. These are not optional enhancements; they are foundational practices for responsible infrastructure design.
