Most of what the CNCF exams test is creating things — a Deployment here, a NetworkPolicy there. But a large share of the points, and almost all of the troubleshooting questions, come down to a different skill: reaching inside a cluster that already exists and interacting with a Pod that is already running. You need to open a shell in a container, pull a config file out of it, hit a service that has no external route, or attach a debugger to a container that ships without a shell at all.
Four kubectl subcommands cover that entire surface: exec, cp, port-forward, and debug. They show up across the CKA, CKAD, and CKS exams — and if you are chasing all five certifications on the KubeAstronaut path, you will use them in every single one. This guide makes each command mechanical: how it actually works, the flags that matter, the traps that cost people time, and the exact command shapes to reach for when the clock is running.
If you only need the raw syntax, keep the CKA kubectl cheat sheet open in a second tab. This article is the deeper “why and when” behind those commands.
The Mental Model: Four Ways Into a Running Pod
Before the syntax, fix the model. A Pod is a set of containers sharing a network namespace and (optionally) volumes. To interact with one you are always doing one of four things:
| Command | What it does | Typical exam use |
|---|---|---|
kubectl exec | Runs a process inside an existing container | Open a shell, check a file, run a one-off command |
kubectl cp | Copies files between your machine and a container | Extract logs/config, seed a file into a Pod |
kubectl port-forward | Tunnels a local port to a Pod/Service port | Reach a ClusterIP-only app from your workstation |
kubectl debug | Attaches an ephemeral debug container to a Pod (or node) | Debug a distroless/crashed container with no shell |
The first three assume the target container is healthy enough to help you. The fourth exists precisely for when it is not. Knowing which tool the scenario calls for is half the battle — the exam rewards picking debug over exec the moment a container has no shell, and picking port-forward over editing a Service the moment the task says “verify the app responds.”
kubectl exec: Running Commands Inside a Container
kubectl exec runs a new process in an already-running container. The single most important detail is the -- separator: everything after it is the command to run inside the container, not a flag for kubectl.
# Run a one-off command and print its output
kubectl exec my-pod -- cat /etc/nginx/nginx.conf
# Open an interactive shell (-i keeps stdin open, -t allocates a TTY)
kubectl exec -it my-pod -- /bin/sh
# Prefer bash if the image has it, fall back to sh if it doesn't
kubectl exec -it my-pod -- /bin/bash-i (--stdin) keeps standard input open; -t (--tty) allocates a pseudo-terminal so line editing and prompts behave. You want both for an interactive shell and neither for a scripted one-off. A frequent mistake under time pressure is forgetting the --, which makes kubectl try to interpret cat or ls as its own arguments.
Multi-Container Pods: the -c flag
If a Pod has more than one container, exec targets the first one unless you say otherwise. Specify the container with -c:
kubectl exec -it my-pod -c sidecar -- /bin/shIf you omit -c on a multi-container Pod, Kubernetes picks the container listed first in the spec (and prints a note about the default). On the exam, always name the container explicitly when the Pod has a sidecar — it removes ambiguity and matches what the question is asking for. This is the same discipline you apply when reading multi-container patterns, covered in the multi-container Pod patterns guide.
Common exec patterns that earn points
# Check environment variables the app actually sees
kubectl exec my-pod -- env | sort
# Test in-cluster DNS from inside the Pod
kubectl exec -it my-pod -- nslookup my-service
# Verify a mounted ConfigMap/Secret landed where you expect
kubectl exec my-pod -- ls -l /etc/configThat DNS check is a classic CKA/CKAD troubleshooting move: if a Pod cannot reach a Service by name, resolving it from inside the Pod tells you instantly whether the problem is DNS, the Service selector, or the network. Pair it with the application troubleshooting workflow for the full decision tree.
kubectl cp: Copying Files In and Out
kubectl cp moves files between your local filesystem and a container. The syntax mirrors scp: source first, destination second, with the Pod side written as namespace/pod:path.
# Copy a file OUT of a Pod to your local machine
kubectl cp my-namespace/my-pod:/var/log/app.log ./app.log
# Copy a file INTO a Pod
kubectl cp ./seed-data.json my-namespace/my-pod:/tmp/seed-data.json
# Target a specific container in a multi-container Pod
kubectl cp ./file.txt my-pod:/tmp/file.txt -c sidecarThe detail that trips people up: kubectl cp requires the tar binary to exist inside the target container. Under the hood it streams a tar archive through exec. Distroless and scratch-based images usually have no tar, so cp fails with an error like tar: not found. When that happens on the exam, do not fight it — pivot. For pulling text out, kubectl exec my-pod -- cat /path/file > file works without tar. For richer extraction, an ephemeral debug container (below) can see the target’s filesystem.
Two more rules worth memorizing:
- Paths inside the container are interpreted relative to the container’s working directory if not absolute — always use absolute paths to avoid surprises.
- If you omit the namespace,
cpuses your current context namespace. Set it once withkubectl config set-context --current --namespace=<ns>at the start of a task so every subsequent command targets the right place.
kubectl port-forward: Reaching a Pod With No Route
Plenty of workloads have no Ingress and only a ClusterIP Service — perfectly normal, and untestable from your workstation unless you tunnel in. kubectl port-forward forwards a local port to a port on a Pod, Service, or Deployment through the API server. No NodePort, no LoadBalancer, no editing the app.
# Forward local 8080 to the Pod's container port 80
kubectl port-forward pod/my-pod 8080:80
# Forward to a Service (kubectl picks a backing Pod)
kubectl port-forward svc/my-service 8080:80
# Forward to a Deployment
kubectl port-forward deploy/my-deploy 8080:80
# Let kubectl choose a random free local port (: prefix)
kubectl port-forward svc/my-service :80The mapping is always LOCAL:REMOTE. Once it is running, curl http://localhost:8080 from another terminal hits the app. This is the fastest way to answer any “verify the application is serving traffic on port X” question without touching the Service type.
Things to know for the exam:
port-forwardruns in the foreground and holds the terminal until youCtrl+C. Run it in a background shell or a second terminal so you cancurlin the first.- Forwarding to a
svcstill connects to a single Pod behind it — it is not load-balanced. That is fine for verification. - To listen on all interfaces (not just loopback) add
--address 0.0.0.0, though loopback is what you want in almost every exam scenario.
If the app does have a Service and you are testing routing rather than the app itself, prefer an in-cluster test Pod (kubectl run tmp --image=busybox --rm -it -- wget -qO- my-service) so you exercise the real Service path. Choosing the right verification tool is a recurring exam theme — see the CKA troubleshooting guide for when to use each.
kubectl debug: Ephemeral Containers and Node Debugging
Here is the scenario the other three commands cannot solve: the container you need to inspect is built from a distroless or scratch base and has no shell, no tar, no tools — or it is stuck in CrashLoopBackOff and never stays up long enough to exec into. kubectl debug is the answer, and it is increasingly emphasized on all three exams because production images are shrinking to reduce attack surface.
kubectl debug uses ephemeral containers: a container you add to a running Pod at runtime, sharing that Pod’s namespaces, without restarting it. It is not part of the Pod spec and cannot be added by editing YAML — it is a runtime-only construct created through the API.
Debug a running Pod with a toolbox image
# Attach a busybox debug container that shares the target Pod
kubectl debug -it my-pod --image=busybox --target=my-app -- /bin/sh--imageis the debug toolbox (busybox, or a richer image withcurl,dig, etc.).--target=<container>shares the target container’s process namespace, so you can see its PIDs and, with the right image, its/proc— essential for inspecting a distroless app’s processes.- Because the ephemeral container shares the Pod’s network namespace,
localhostinside it reaches the app’s ports directly.
Debug a crashing Pod by copying it
If the Pod crashes too fast to attach to, create a copy with a changed command so it stays up:
# Make a copy of the Pod that runs a shell instead of the crashing command
kubectl debug my-pod -it --copy-to=my-pod-debug --image=busybox --share-processes -- sh
# Or copy and only swap the image of one container, keeping everything else
kubectl debug my-pod --copy-to=my-pod-debug --set-image=my-app=busybox--copy-to builds a new Pod you can poke at without disturbing (or waiting on) the original. Delete the copy when you are done.
Debug a node
kubectl debug also drops you onto a node’s host namespace via a privileged Pod — invaluable for CKA node-level troubleshooting (checking the kubelet, container runtime, or host filesystem):
# Get a shell on node's host, with the host filesystem under /host
kubectl debug node/my-node -it --image=busyboxThe node’s root filesystem is mounted at /host inside the debug Pod. This is how you inspect /etc/kubernetes, kubelet logs, or the container runtime socket without SSH access to the node.
exec vs debug — pick correctly
| Situation | Reach for |
|---|---|
| Container has a shell and is running | kubectl exec |
| Container is distroless / has no shell | kubectl debug (ephemeral container) |
| Container is in CrashLoopBackOff | kubectl debug --copy-to |
| You need host/node access | kubectl debug node/... |
For CKS specifically, know that ephemeral containers are a capability worth restricting: an attacker with rights to create them can attach a privileged toolbox to any Pod. RBAC controls this through the pods/ephemeralcontainers subresource — a detail that connects directly to the least-privilege principles in Kubernetes security best practices.
What About Logs and Metrics?
exec, cp, port-forward, and debug are about interacting. For observing a Pod without entering it, you reach for kubectl logs and kubectl top — which have their own mechanics (previous-container logs, the metrics-server dependency, streaming). Those are covered end to end in the cluster and application monitoring guide, and formatting their output cleanly is the subject of the kubectl output formatting guide. In a real troubleshooting flow you interleave both families: logs to see what broke, then exec/debug to find out why.
Exam-Day Speed Tips
- Set your namespace once.
kubectl config set-context --current --namespace=<ns>so you stop typing-non every command. - Alias
k=kubectl(usually pre-configured on CNCF exam terminals) and lean on it. - Background your port-forward. Append
&or use a second tmux pane so you cancurlimmediately. - Reach for
debugthe instantexecsays “no such file” for the shell — that is the distroless signal, not a mistake to retry. - Clean up debug copies (
kubectl delete pod my-pod-debug) so leftover objects don’t confuse a later task.
Practice Beats Memorization
These four commands are pure muscle memory: the exam gives you minutes, not hours, and the difference between passing and failing is whether kubectl debug -it my-pod --image=busybox --target=app -- sh flows from your fingers without thinking. That only comes from repetition on a real cluster.
Sailor.sh’s KubeAstronaut mock exam bundle is built around exactly this kind of hands-on, cross-certification practice — the same interaction-and-debugging tasks framed the way the CKA, CKAD, and CKS present them, so you build the reflexes once and carry them across all five CNCF exams. If you want a lighter warm-up first, the free Kubernetes practice lab for the CNCF exams is a good place to start drilling these commands.
Frequently Asked Questions
What is the difference between kubectl exec and kubectl debug?
kubectl exec runs a command inside an existing container, so it needs that container to have the binary you want (a shell, cat, etc.) and to be running. kubectl debug adds a brand-new ephemeral container to the Pod, letting you bring your own tools — which is the only option when the target container is distroless, has no shell, or is crashing too fast to exec into.
Why does kubectl cp fail with “tar: not found”?
kubectl cp streams files as a tar archive and relies on the tar binary existing inside the target container. Minimal images (distroless, scratch) omit tar, so the copy fails. Work around it with kubectl exec ... -- cat for single files, or attach an ephemeral debug container that has the tools you need.
Does kubectl port-forward load-balance across Pods?
No. Even when you forward to a Service (svc/...), the connection is pinned to a single backing Pod for the life of the forward. It is meant for local verification and debugging, not for distributing traffic. To exercise real Service load-balancing, test from inside the cluster instead.
Can I add an ephemeral container by editing the Pod YAML?
No. Ephemeral containers are a runtime-only feature created through the API (via kubectl debug); they are not part of the Pod’s declarative spec and cannot be applied with kubectl apply. This is by design — they are for live debugging, not for defining workloads.
How do I debug a Kubernetes node itself?
Use kubectl debug node/<node-name> -it --image=<toolbox>. Kubernetes schedules a privileged Pod onto that node and mounts the host’s root filesystem at /host, letting you inspect kubelet config, logs, and the container runtime without SSH.
Which exams test these commands?
All of them. exec and logs are foundational for the CKAD and CKA troubleshooting domains; debug and ephemeral containers appear on the CKA and increasingly the CKS (where restricting them via RBAC also matters); and port-forward shows up wherever you must verify an app with no external route. On the KubeAstronaut path you will use every one repeatedly.
Conclusion
The commands that let you create Kubernetes objects get most of the study time, but the commands that let you reach into a running cluster decide most of the troubleshooting points. exec runs commands inside a container, cp moves files across the boundary, port-forward tunnels to unrouted apps, and debug gives you tools and access when the container itself offers none. Learn which one each scenario calls for — especially the exec-to-debug pivot the moment a container has no shell — and drill the exact syntax until it is automatic. Do that, and the interaction-and-debugging questions on the CKA, CKAD, and CKS stop being a scramble and start being free points.