Complete Guide to Kubernetes Persistence Series Part 2: The Foundation — From Systemd...

In Part 1, we investigated why a Grafana ingress disappeared after GKE node upgrades. The fix was straightforward: use Helm-managed resources instead of manual kubectl apply.

But that raised a deeper question: How do controllers themselves survive pod evictions?

The answer is a hierarchical supervision model—each layer watches the layer above it, ensuring continuous operation despite failures.

In this post, we'll explore Layers 1-3. Part 3 covers Layer 4 and the complete resilience model.

At the very bottom of the stack is systemd, the init system running as PID 1 on most modern Linux distributions.

If kubelet crashes, systemd restarts it within 10 seconds. This is the foundation of Kubernetes resilience—the node agent is supervised by the operating system itself.

kubelet is the Kubernetes agent running on every node. It has two critical responsibilities:

kubelet watches a directory (typically /etc/kubernetes/manifests/) for pod manifests and runs them directly—no API server required.

This is how the control plane bootstraps itself. The API server can't schedule pods before it exists, so kubelet runs these components directly from files.

The control plane runs as static pods managed directly by kubelet:

Source: Dev.to