sre-skills · Guide

Debugging Kubernetes CrashLoopBackOff

CrashLoopBackOff isn't an error in itself. It means the container keeps exiting and Kubernetes is waiting longer and longer (10s, 20s, 40s … up to 5 minutes) before restarting it. The real error is in why it exits. This checklist finds it.

1. Confirm the context and find the pod

kubectl config current-context
kubectl get pods -n <ns> -o wide | grep -v Running

Make sure you're in the right cluster before you do anything else. The RESTARTS column tells you how long it's been looping.

2. Read the logs of the crashed container

kubectl logs <pod> -n <ns> -c <container> --previous

This is the most commonly missed step. Without --previous you see the logs of the current attempt, which is often empty because it just started. The previous instance holds the stack trace or the "missing env var" message.

3. Decode the exit code

kubectl describe pod <pod> -n <ns>   # look at "Last State: Terminated", Reason and Exit Code
Exit codeMeaningLook at
0The process finished successfullyThe container's command runs once and exits. A Deployment expects a long-running process. Use a Job instead, or fix the command.
1 (or other app codes)Application errorThe logs from step 2: config, missing secret, failed DB connection, failed migration
126 / 127Command not executable / not foundcommand/args in the manifest, the entrypoint in the image, the shell form vs exec form
137SIGKILLReason: OOMKilled means the memory limit is too low or there's a leak. Without OOMKilled, it's usually the liveness probe killing the container.
139SegfaultNative libraries, an architecture mismatch (an arm64 image on amd64 nodes)
143SIGTERMSomething asked it to stop: probes, or the node draining

4. Check the usual suspects

5. Get a shell when the logs aren't enough

You can't exec into a container that keeps dying. Override the command in a copy of the pod instead (this creates a new pod, so don't do it in prod without thinking):

kubectl debug <pod> -n <ns> -it --copy-to=<pod>-debug --container=<container> -- sh

Then run the original entrypoint by hand and watch it fail.

6. Fix, verify, prevent

kubectl rollout status deploy/<name> -n <ns>
kubectl get pods -n <ns> -w

Prevention: set resource requests and limits from real usage, add a startupProbe for slow starters, and validate that referenced Secret and ConfigMap keys exist in CI before deploying.

Let Claude Code do this for you. The free sre-skills plugin includes a k8s-debug skill that works through this checklist using read-only commands and quotes the evidence it finds:

/plugin marketplace add polytheseus/sre-skills
/plugin install sre@sre-skills