Every kubeadm-built Kubernetes cluster runs on a small private PKI, and almost all of it expires after one year. That single fact is behind one of the most common “my cluster died overnight” incidents in the real world — and it is exactly the kind of scenario the Certified Kubernetes Administrator (CKA) exam loves, because it lives at the intersection of Cluster Architecture, Installation & Configuration and the 30%-weighted Troubleshooting domain. You are handed a cluster where kubectl returns x509: certificate has expired, and you have a few minutes to bring it back.
This guide walks the whole topic from a practitioner’s point of view: where the certificates live, how to read their expiry, how to renew them with kubeadm, the subtle reason kubectl stops working even after you “renewed everything,” and the precise recovery sequence for a cluster that has already gone dark. If you want the broader troubleshooting mindset afterwards, the CKA troubleshooting guide is the companion piece; here we go deep on certificates specifically.
Why Certificates Matter So Much in Kubernetes
Kubernetes components do not trust each other by default. Every conversation between the API server, the kubelet, the controller manager, the scheduler, and etcd is mutually authenticated with TLS client certificates signed by a cluster certificate authority. When you run kubectl, your request is authenticated the same way: your kubeconfig carries a client certificate that the API server validates against its CA.
That design is excellent for security and brutal for anyone who forgets about expiry. kubeadm issues leaf certificates with a 365-day lifetime and a CA certificate that lasts 10 years. Nothing warns you as the year runs out. One morning the API server refuses connections, the kubelets go NotReady, and workloads that were already running keep running — until they need the control plane, at which point everything unravels.
For the exam, internalize this hierarchy:
| Layer | Examples | Default validity | Where it lives |
|---|---|---|---|
| Cluster CA | ca.crt / ca.key | 10 years | /etc/kubernetes/pki/ |
| Component leaf certs | apiserver.crt, apiserver-kubelet-client.crt, front-proxy-client.crt | 1 year | /etc/kubernetes/pki/ |
| etcd certs | etcd/server.crt, etcd/peer.crt, apiserver-etcd-client.crt | 1 year | /etc/kubernetes/pki/etcd/ |
| Admin & component kubeconfigs | admin.conf, controller-manager.conf, scheduler.conf, kubelet.conf | 1 year (embedded client cert) | /etc/kubernetes/ |
Where the PKI Actually Lives
On any control-plane node, look in two places:
# The certificates and keys themselves
ls -l /etc/kubernetes/pki/
ls -l /etc/kubernetes/pki/etcd/
# The kubeconfig files that embed client certs for the control-plane components
ls -l /etc/kubernetes/*.confYou will see files such as ca.crt, ca.key, apiserver.crt, apiserver.key, apiserver-kubelet-client.crt, front-proxy-ca.crt, sa.pub, and under etcd/, the etcd server, peer, and healthcheck certificates. These same paths are referenced in the static Pod manifests under /etc/kubernetes/manifests/ — the API server manifest, for example, mounts /etc/kubernetes/pki and points --tls-cert-file and --client-ca-file at these files. If you have worked through etcd backup and restore, you have already met /etc/kubernetes/pki/etcd/ — the same certificates that authenticate etcdctl are part of this PKI.
One distinction the exam expects you to keep straight: cluster PKI certificates are not the same as the user certificates you provision through the CertificateSigningRequest (CSR) API. The CSR workflow — generating a key, submitting a CertificateSigningRequest object, approving it, and building a kubeconfig — is how you onboard human users and their RBAC identity, and it is covered in the cluster access, kubeconfig & CSR guide. What we are doing here is managing the control-plane’s own certificates with kubeadm. Different problem, different tool.
Reading Expiry with kubeadm certs check-expiration
The single most important command in this topic is:
kubeadm certs check-expirationIt prints a table of every managed certificate and every embedded kubeconfig certificate, with the expiry date, the residual time, the CA that signed it, and — crucially — whether the certificate is externally managed:
CERTIFICATE EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
admin.conf Sep 14, 2027 09:12 UTC 364d no
apiserver Sep 14, 2027 09:12 UTC 364d no
apiserver-etcd-client Sep 14, 2027 09:12 UTC 364d no
apiserver-kubelet-client Sep 14, 2027 09:12 UTC 364d no
controller-manager.conf Sep 14, 2027 09:12 UTC 364d no
front-proxy-client Sep 14, 2027 09:12 UTC 364d no
scheduler.conf Sep 14, 2027 09:12 UTC 364d no
etcd-server Sep 14, 2027 09:12 UTC 364d no
...
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME
ca Sep 12, 2035 09:12 UTC 9y
etcd-ca Sep 12, 2035 09:12 UTC 9yThe EXTERNALLY MANAGED column tells you whether kubeadm can renew a certificate itself. If it says no, the CA private key (ca.key) is present on the node and kubeadm can re-sign. If it says yes, the cluster uses an external CA and you must renew through that external process — kubeadm will refuse.
If the cluster is already down and the API server will not start, kubeadm certs check-expiration still works because it reads certificates from disk — it does not need a healthy control plane. You can also verify a single certificate directly with OpenSSL, which is worth knowing when kubeadm is unavailable:
openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -enddate
# notAfter=Sep 14 09:12:00 2027 GMTRenewing Certificates with kubeadm certs renew
To renew everything in one shot:
kubeadm certs renew allThis re-signs every managed leaf certificate and rewrites the embedded certificates in admin.conf, controller-manager.conf, and scheduler.conf using the existing CA. You can also renew a single certificate when you know only one is the problem:
kubeadm certs renew apiserver
kubeadm certs renew admin.confRenewal writes new files to disk, but the running control-plane components do not pick them up automatically — they loaded their certificates at start-up. You must restart the control-plane static Pods. The clean way is to move the manifests out of the directory and back:
cd /etc/kubernetes/manifests/
mv kube-apiserver.yaml kube-controller-manager.yaml kube-scheduler.yaml etcd.yaml /tmp/
# wait a few seconds for the kubelet to notice the pods are gone
sleep 20
mv /tmp/kube-apiserver.yaml /tmp/kube-controller-manager.yaml /tmp/kube-scheduler.yaml /tmp/etcd.yaml .The kubelet watches /etc/kubernetes/manifests/, so removing a manifest stops the static Pod and putting it back starts a fresh one that reads the renewed certificates. Restarting the kubelet (systemctl restart kubelet) also works and is often faster to type under exam pressure.
The trap: kubectl still fails after renewal
Here is the mistake that costs people the question. You renew apiserver and the certs, restart the static Pods, and the cluster comes back — but your own kubectl still returns x509: certificate has expired. The reason: your personal kubeconfig at ~/.kube/config is a copy of admin.conf made when the cluster was first set up. Renewing admin.conf in /etc/kubernetes/ does not update the copy in your home directory. You must re-copy it:
kubeadm certs renew admin.conf # if not already covered by "renew all"
sudo cp -f /etc/kubernetes/admin.conf ~/.kube/config
sudo chown $(id -u):$(id -g) ~/.kube/configOnly after refreshing ~/.kube/config will kubectl get nodes succeed. On the exam, always finish a certificate task by re-copying admin.conf and running a quick kubectl get nodes to prove the fix.
The Automatic Path: Certificates Renew on Upgrade
There is a reason many clusters never hit expiry despite the one-year clock: kubeadm upgrade apply automatically renews all certificates as part of the upgrade. Kubernetes ships a new minor version roughly every four months and support windows are about a year, so an operator who upgrades on any sane cadence resets the certificate clock every time.
kubeadm upgrade apply v1.34.0 # renews certs as a side effectThis is why the certificate problem tends to bite neglected clusters — the ones nobody has upgraded in over a year. If you are comfortable with the cluster upgrade workflow, you already know one of the cleanest ways to keep certificates current. For the exam, remember the causal chain: no upgrades for a year → certificates silently expire → control plane fails.
The kubelet Certificate Is a Special Case
The kubelet has two certificates, and they behave differently from the control-plane leaf certs:
- The kubelet client certificate (used to authenticate the kubelet to the API server) is set up to auto-rotate by default on modern kubeadm clusters. The
kubelet.confno longer embeds a static certificate; instead it points at/var/lib/kubelet/pki/kubelet-client-current.pem, a symlink the kubelet rotates automatically before expiry whenrotateCertificates: trueis set. - The kubelet serving certificate (used when the API server or metrics-server talks to the kubelet) can also rotate but often requires the
kube-controller-managerto approve the serving CSRs (serverTLSBootstrap: true), which is whykubectl toporkubectl logssometimes fails on a fresh node until those CSRs are approved.
The practical exam takeaway: kubeadm certs renew handles the control-plane certificates and the *.conf kubeconfigs, but the kubelet client cert rotates on its own — you do not renew it with kubeadm. If a single node is NotReady after cert issues, check /var/lib/kubelet/pki/ and the kubelet logs (journalctl -u kubelet) rather than reaching for kubeadm certs renew.
Recovering a Cluster Whose Certificates Have Already Expired
This is the highest-value scenario to rehearse. The symptoms are unmistakable:
$ kubectl get nodes
Unable to connect to the server: x509: certificate has expired or is not yet valid:
current time ... is after ...The API server static Pod will be in a crash loop, and crictl ps on the node shows it restarting. Here is the recovery sequence, in order:
- Confirm it is a certificate problem. On the control-plane node, run
sudo kubeadm certs check-expiration. Expired leaf certs with a still-valid CA is the recoverable case. - Renew everything.
sudo kubeadm certs renew all. - Restart the control plane. Move the manifests out of
/etc/kubernetes/manifests/and back, orsudo systemctl restart kubelet, so the API server, controller manager, scheduler, and etcd reload the new certs. - Refresh your kubeconfig.
sudo cp -f /etc/kubernetes/admin.conf ~/.kube/configand fix ownership. - Verify.
kubectl get nodesandkubectl get pods -n kube-systemshould now respond, with the control-plane PodsRunning.
If check-expiration shows the CA itself has expired, you are in a far worse place — CA rotation is a disruptive, multi-step operation and effectively a rebuild for exam purposes. That will not happen on a normally aged cluster (the CA lasts 10 years), so if you see it, re-read the scenario; the intended fix is almost always leaf renewal.
A Summary Cheat Sheet
| Task | Command |
|---|---|
| List all certificate expiries | kubeadm certs check-expiration |
| Check one cert with OpenSSL | openssl x509 -in <cert> -noout -enddate |
| Renew everything | kubeadm certs renew all |
| Renew a single certificate | kubeadm certs renew apiserver |
| Reload certs into components | move manifests out/in, or systemctl restart kubelet |
| Fix your own kubectl | cp -f /etc/kubernetes/admin.conf ~/.kube/config |
| Keep certs fresh long-term | kubeadm upgrade apply <version> |
Practice on a Real Exam-Like Cluster
Certificate management is a muscle-memory skill. Reading that you should run kubeadm certs renew all is easy; doing it on a live node, restarting static Pods without panicking when kubectl briefly stops responding, and remembering to re-copy admin.conf — all inside a ticking timer — is a different skill that only repetition builds. The fastest way to build it is to break a cluster on purpose (back-date the clock, or let a lab cluster age) and recover it until the sequence is automatic.
Sailor.sh’s CKA Certification-Ready Mock Exam Bundle puts you in front of scenario-based, performance-style questions that mirror the real exam’s pacing, including cluster-lifecycle tasks like this one. Pair the mocks with the 30-day CKA study plan and the broader CKA exam guide for 2026 so certificate management becomes one more reliable, fast win rather than the question that eats your clock.
Frequently Asked Questions
How long are kubeadm certificates valid?
Leaf certificates (API server, kubelet client, etcd, the *.conf kubeconfigs) are valid for one year by default. The cluster CA certificate is valid for 10 years. This is why clusters that go a year without an upgrade suddenly fail.
Why does kubectl still say the certificate expired after I renewed everything?
Because your ~/.kube/config is a copy of /etc/kubernetes/admin.conf made at install time. kubeadm certs renew updates admin.conf, not your home-directory copy. Re-run cp -f /etc/kubernetes/admin.conf ~/.kube/config after renewing.
Do I need to restart the control plane after renewing certificates?
Yes. The API server, controller manager, scheduler, and etcd read their certificates at start-up and will keep using the expired ones until restarted. Move the static Pod manifests out of /etc/kubernetes/manifests/ and back, or restart the kubelet.
Does renewing certificates require the CA private key?
Yes. kubeadm certs renew re-signs certificates using /etc/kubernetes/pki/ca.key. If the cluster uses an external CA (the EXTERNALLY MANAGED column shows yes, and ca.key is absent), kubeadm cannot renew and you must renew through the external CA.
How do I stop certificates from expiring in the first place?
Upgrade the cluster at least once a year. kubeadm upgrade apply renews all managed certificates automatically, so a normal upgrade cadence keeps the PKI fresh without any manual renewal.
Is the kubelet certificate renewed by kubeadm too?
The kubelet client certificate auto-rotates by default via /var/lib/kubelet/pki/kubelet-client-current.pem and is not something you renew with kubeadm. kubeadm certs renew covers the control-plane leaf certificates and the *.conf kubeconfigs.
Conclusion
Certificate management rewards understanding the moving parts more than memorization: a one-year clock on the leaf certs, a kubeadm certs check-expiration to read the state, a kubeadm certs renew all to re-sign, a static-Pod restart to reload, and — the step everyone forgets — a re-copy of admin.conf so your own kubectl works again. Layer on the knowledge that upgrades renew certs automatically and that the kubelet client cert rotates itself, and you can walk into the exam ready to recover an expired cluster in the two or three minutes it deserves. Practice the sequence until it is boring, and a scenario that panics most candidates becomes a guaranteed handful of points.