50 Kubernetes Interview Questions and Answers for 2026 (Beginner to Advanced)

In two years of interviewing Kubernetes engineers, I've asked the same core questions to 200+ candidates. The engineers who land senior roles are not necessarily the ones who know the most — they are the ones who explain the WHY, not just the WHAT. They say "the reason kube-proxy uses iptables in O(n) rule traversal and not a hash table is..." rather than just "kube-proxy handles Services." This guide covers the 50 questions I use in real interviews, with the answers that separate senior candidates from junior ones.

I've structured each answer to match what I'm actually evaluating when I ask it. Junior engineers describe features. Mid-level engineers describe how features work. Senior engineers describe tradeoffs, failure modes, and why the designers made those choices. Read every "ideal answer approach" section carefully — that's where the promotion-level insight lives.

How to Use This Guide

Don't memorize these answers word for word. Interviewers can smell a rehearsed answer from two questions in. Instead, understand the mental model behind each answer. The goal is to be able to derive the answer on the spot from first principles. If you understand why etcd is a distributed key-value store that uses the Raft consensus algorithm, you can answer a dozen follow-up questions you've never seen before.

This guide is organized by topic area. If you have one week to prepare, do Architecture first, then Networking (the most common area where candidates collapse), then Operations. If you have two weeks, cover everything. For each section, run the examples in a real cluster — either a local kind or k3s cluster, or a managed cluster on a free trial. Theoretical knowledge without hands-on experience is always visible to an experienced interviewer.

Interview Preparation Strategy

The most effective preparation combines conceptual study with hands-on lab work. Read the official Kubernetes documentation for each component you study — it is surprisingly readable and precise. For lab work, use kind (Kubernetes in Docker) locally, or spin up a three-node cluster on any cloud provider. Run kubectl get events --sort-by=.lastTimestamp, break things intentionally, and read the logs.

For senior roles, interviewers expect you to have debugged real production issues. Prepare two or three specific war stories: a networking outage you diagnosed, an OOM kill pattern you solved, a cluster upgrade you planned. Specificity signals experience. "I've debugged networking issues" is worth nothing. "We had a CNI misconfiguration that caused intermittent pod-to-pod packet loss only on cross-AZ traffic, which we found by running tcpdump inside the container network namespace" is a senior-level answer.

Section 1: Architecture (Q1–Q10)

Q1: What are the components of the Kubernetes control plane and what does each do?

Answer: The control plane has four core components. The kube-apiserver is the single entry point for all cluster state changes; it validates and processes REST requests, then writes state to etcd. The etcd cluster is the distributed key-value store that holds all cluster state — the only stateful component in the control plane. The kube-scheduler watches for unscheduled pods (pods with no nodeName set) and assigns them to nodes based on resource availability, affinity rules, taints, and topology constraints. The kube-controller-manager runs a collection of reconciliation loops that drive actual state toward desired state — if you have 3 replicas desired and 2 running, the ReplicaSet controller creates a new pod. On cloud providers, a fourth component, the cloud-controller-manager, handles cloud-specific reconciliation like provisioning LoadBalancer Services and tagging nodes.

What the interviewer is testing: Whether you understand that the control plane is stateless except for etcd, and that all components communicate through the apiserver — no component talks to another directly.

Common wrong answer: Candidates list the components correctly but describe them as a monolith: "the control plane manages the cluster." Or they miss that the apiserver is the only component that reads and writes etcd directly.

Ideal answer approach: Draw the communication flow: all components watch the apiserver via the watch API. The controller-manager and scheduler never talk to each other. Everything is mediated through etcd via the apiserver. This demonstrates understanding of the level-triggered, reconciliation-loop design.

Q2: What is etcd and why does Kubernetes need it? What happens if etcd goes down?

Answer: etcd is a distributed, strongly consistent key-value store that uses the Raft consensus algorithm to guarantee that all writes are agreed upon by a quorum of members before being committed. Kubernetes needs it because every resource — pods, deployments, secrets, configmaps, service accounts — is stored as serialized protobuf in etcd. It is the source of truth for the entire cluster. If etcd goes down, the apiserver cannot serve reads or writes for most resources, controllers stop reconciling, and the scheduler stops placing pods. Existing running pods continue to run because kubelet does not depend on etcd directly — it manages containers based on its local state. However, no new pods can be scheduled and no changes can be made to any resources. For high availability, etcd should run as a three-node or five-node cluster; a three-node cluster can tolerate one failure (requires quorum of 2), and a five-node cluster can tolerate two failures.

What the interviewer is testing: Understanding of Raft quorum requirements, the difference between control-plane availability and data-plane availability, and why you should never run a single-node etcd in production.

Common wrong answer: "If etcd goes down, all pods stop running." This is wrong — pods already running on nodes continue because kubelet is autonomous.

Ideal answer approach: Mention the Raft quorum formula (floor(n/2)+1 nodes needed), the fact that even-numbered clusters are worse than odd-numbered ones (4 nodes is no better than 3 for fault tolerance), and that etcd should be on fast SSD storage because it is extremely write-latency sensitive.

Q3: What is the role of the kube-scheduler and how does it select a node for a pod?

Answer: The kube-scheduler watches for pods with no spec.nodeName set and assigns them to a node. The selection process has two phases: filtering (also called predicates) and scoring (also called priorities). In the filtering phase, it eliminates nodes that cannot run the pod — insufficient CPU or memory, node taints not tolerated, affinity rules not satisfied, and topology constraints not met. In the scoring phase, it ranks the remaining nodes using multiple weighted scoring functions. The default scoring includes factors like resource balance (LeastAllocated), spreading pods across nodes (PodTopologySpread), and affinity weight. The node with the highest score is selected. The scheduler then writes the nodeName field to the pod spec via the apiserver — it does not create the container, it just binds the pod to a node. The kubelet on that node then sees the pod in its watch stream and creates the containers.

What the interviewer is testing: Whether you understand that scheduling is a two-phase process and that the scheduler's only output is writing nodeName to a pod spec.

Common wrong answer: Conflating the scheduler with the kubelet, or saying it "starts the pod." The scheduler never touches containers.

Ideal answer approach: Mention the scheduler framework and extension points (Filter, Score, Reserve, Bind), and note that you can write custom scheduler plugins if the default scoring doesn't fit your workload (e.g., GPU bin-packing for ML workloads).

Q4: What does the kube-controller-manager do? Name at least 5 controllers it runs.

Answer: The kube-controller-manager is a single binary that runs multiple independent reconciliation loops, each watching specific resources via the apiserver watch API and reconciling actual state with desired state. Controllers it runs include: the ReplicaSet controller (ensures the correct number of pod replicas exist), the Deployment controller (manages rolling updates by creating/scaling ReplicaSets), the Node controller (monitors node health and evicts pods from unreachable nodes after a grace period), the Namespace controller (cleans up resources when a namespace is deleted), the ServiceAccount controller (creates default ServiceAccounts in new namespaces), the EndpointSlice controller (populates EndpointSlices for Services), the Job controller (manages pod lifecycle for batch jobs), and the CronJob controller (creates Jobs on a schedule). All of these run as goroutines inside a single process, but they are conceptually independent control loops.

What the interviewer is testing: Understanding of the controller pattern — the fundamental design principle of Kubernetes — and breadth of knowledge about what runs where.

Common wrong answer: Candidates name 2-3 controllers and stop. Or they describe controllers as running in separate pods when most are compiled into kube-controller-manager.

Ideal answer approach: Explain the reconciliation loop pattern: watch for desired state, observe actual state, compute the diff, act. This pattern is how every controller works, and it's also how custom operators work. Demonstrating you can extend this pattern shows architectural depth.

Q5: What is the difference between the control plane and a worker node?

Answer: The control plane runs the components that make global decisions about the cluster — apiserver, etcd, scheduler, and controller-manager. Worker nodes run the components that execute workloads: kubelet (manages pod lifecycle on the node), kube-proxy (programs iptables/IPVS rules for Service routing), and the container runtime (containerd, CRI-O). Pods run on worker nodes, not on control plane nodes in production (unless you remove the control-plane taint). The control plane is responsible for desired state; worker nodes are responsible for actual state. In managed clusters (EKS, GKE, AKS), the control plane is fully abstracted and you never interact with control plane nodes directly. In self-managed clusters, you typically have 3 control plane nodes for HA and any number of worker nodes.

What the interviewer is testing: Basic architecture clarity and whether you understand the desired-vs-actual state model.

Common wrong answer: Saying the control plane "runs" the workloads. It orchestrates them — the worker nodes run them.

Ideal answer approach: Mention the node-role.kubernetes.io/control-plane:NoSchedule taint that prevents regular pods from landing on control plane nodes, and that system components like CoreDNS use tolerations to schedule there.

Q6: What is the kubelet and what is its relationship with the container runtime?

Answer: The kubelet is the primary node agent. It watches the apiserver for pods assigned to its node (via spec.nodeName), and ensures the containers in those pods are running and healthy. The kubelet does not manage containers directly — it communicates with a container runtime through the Container Runtime Interface (CRI), a gRPC API. The container runtime (typically containerd) is responsible for pulling images, creating and managing container namespaces, and reporting container status back to the kubelet. This separation means Kubernetes is runtime-agnostic: you can swap containerd for CRI-O without changing the kubelet code. The kubelet also manages volume mounting, resource cgroup enforcement (CPU and memory limits), and runs the probes (liveness, readiness, startup) defined in the pod spec.

What the interviewer is testing: Understanding of the CRI abstraction and why it exists.

Common wrong answer: Saying "kubelet runs Docker." Docker was removed as a direct runtime in Kubernetes 1.24. Even before that, Docker was accessed through dockershim, a CRI translation layer. Modern clusters use containerd or CRI-O directly.

Ideal answer approach: Mention that the kubelet uses the CRI gRPC API (RuntimeService and ImageService), that containerd uses a shim architecture (containerd-shim-runc) which means containers survive a containerd restart, and that this is why a kubelet crash doesn't kill running containers.

Q7: What is the cloud controller manager and when is it needed?

Answer: The cloud controller manager (CCM) runs cloud-provider-specific control loops that were originally embedded in the kube-controller-manager. It handles three main responsibilities: the Node controller (checks cloud provider APIs to verify a node still exists after it becomes unreachable), the Route controller (configures routes in the cloud network for pod CIDR blocks), and the Service controller (creates, updates, and deletes cloud load balancers in response to LoadBalancer-type Services). It is only needed when running Kubernetes on a cloud provider — bare-metal or on-prem clusters without cloud infrastructure do not run a CCM. The motivation for separating it was to let cloud providers ship their own CCM at their own release cadence instead of waiting for upstream Kubernetes releases.

What the interviewer is testing: Whether you understand how Kubernetes integrates with cloud providers and the history of the in-tree vs out-of-tree provider split.

Common wrong answer: "It's not needed, Kubernetes handles the cloud natively." Without CCM, a LoadBalancer Service just sits in <pending> state forever.

Ideal answer approach: Mention that managed Kubernetes services (EKS, GKE) include the CCM automatically and you never see it directly, but on self-managed clusters (kubeadm on EC2) you need to deploy it separately and configure node IAM roles with the right permissions.

Q8: Explain the concept of a watch in Kubernetes. How do components stay in sync?

Answer: The apiserver exposes a watch API that lets clients receive a stream of events (ADDED, MODIFIED, DELETED) for any resource. This is built on HTTP long-polling (HTTP/1.1) or HTTP/2 server push. Every Kubernetes controller uses a SharedInformer, which is a cached, thread-safe watch implementation that stores a local copy of resource state in memory (the informer cache). The SharedInformer re-establishes the watch automatically on disconnection, using the resourceVersion field to resume from where it left off without replaying events that were already processed. When a change occurs, the controller receives an event and pushes the affected object's key into a work queue. Worker goroutines dequeue the key, look up the current object state from the informer cache, and reconcile. This is why Kubernetes is called level-triggered rather than edge-triggered: the controller always reconciles to the current desired state, not the delta, so missed events are harmless.

What the interviewer is testing: Deep understanding of the controller internals — a strong signal for senior and staff engineers.

Common wrong answer: Describing polling ("components check the apiserver every N seconds"). Kubernetes uses watches, not polling, for most communication.

Ideal answer approach: Explain the level-triggered vs edge-triggered distinction: if the controller crashes and misses 100 MODIFIED events, on restart it just reads the current state and reconciles to it. This makes the system self-healing by design.

Q9: What is leader election in Kubernetes and why does it matter for HA clusters?

Answer: Leader election ensures that only one instance of a controller runs active reconciliation at a time, even when multiple replicas of that controller exist. Kubernetes implements leader election using a Lease resource (in the coordination.k8s.io API group). The active leader continuously renews a Lease object, and standby replicas watch that Lease. If the leader fails to renew within a configurable timeout (typically 15 seconds with a 10-second renewal period), a standby acquires the Lease and becomes the new leader. This pattern prevents split-brain scenarios where two controller instances could simultaneously create conflicting resources. Both the kube-scheduler and kube-controller-manager use leader election, which is why you can run 3 replicas of each for HA without them conflicting with each other.

What the interviewer is testing: Whether you understand distributed systems concepts and how they apply to Kubernetes HA design.

Common wrong answer: "The standby controllers are just backups and don't do anything." Technically correct, but misses the mechanism and the reason it matters for correctness.

Ideal answer approach: Note that custom operators you write should also implement leader election if they manage state — the controller-runtime library makes this easy with theLeaderElection: true flag. Also worth mentioning that leader election adds latency to failover — during the 15-second timeout window, no reconciliation happens.

Q10: If you run kubectl apply and the pod doesn't appear, walk me through the full flow from your command to a running container.

Answer: kubectl apply sends a PATCH or PUT request to the apiserver with the desired manifest. The apiserver authenticates (mTLS client cert or bearer token), authorizes (RBAC check), runs admission controllers (mutating webhooks first, then validating webhooks, then built-in validations), and writes the object to etcd. The Deployment controller's watch fires — it sees the new Deployment, creates a ReplicaSet, and creates Pod objects (setting spec.containers but not spec.nodeName). The scheduler watches for pods without a nodeName, runs filtering and scoring, and writes the selected node name back to the pod spec via the apiserver. The kubelet on that node's watch fires — it sees a pod bound to its node. The kubelet calls the container runtime via CRI to pull the image, create network namespaces (CNI plugin invoked here), set up cgroups, and start the container. The kubelet reports pod status back to the apiserver. You can observe every step with kubectl get events --sort-by=.lastTimestamp.

What the interviewer is testing: The ability to trace the full request path — the "10,000-foot to ground-level" question that exposes gaps in architectural understanding.

Common wrong answer: Skipping admission controllers, skipping the ReplicaSet layer, or saying the scheduler "starts" the pod.

Ideal answer approach: Mention that each step is asynchronous and event-driven, not a synchronous chain. If the pod doesn't appear, the first debugging step is kubectl describe deployment to check if the ReplicaSet was created, then kubectl describe pod to check scheduler and kubelet events.

Section 2: Pods and Workloads (Q11–Q20)

Q11: What is the difference between a Pod, ReplicaSet, and Deployment?

Answer: A Pod is the smallest deployable unit — it wraps one or more containers that share a network namespace and can share volumes. Pods are ephemeral; if a pod dies, nothing recreates it automatically. A ReplicaSet ensures a specified number of identical pod replicas are running at all times by watching for pod deletions and creating replacements. However, a ReplicaSet has no concept of rollout history or update strategy. A Deployment manages ReplicaSets to enable declarative rolling updates and rollbacks: when you update a Deployment's pod template, it creates a new ReplicaSet, scales it up, and scales the old one down according to your rolling-update strategy. You can roll back to a previous Deployment revision, which reactivates the corresponding old ReplicaSet. In practice, you almost never create a ReplicaSet directly — Deployments are the standard for stateless workloads.

What the interviewer is testing: Understanding the layering of abstractions and why each exists.

Common wrong answer: "A ReplicaSet and Deployment are basically the same thing." They're not — the rollout and rollback history that a Deployment provides is operationally critical.

Ideal answer approach: Explain that kubectl rollout history deployment/my-app shows the ReplicaSet history, and kubectl rollout undo just scales the previous ReplicaSet back up. This makes the abstraction concrete.

Q12: What is a DaemonSet and when would you use it vs a Deployment?

Answer: A DaemonSet ensures that exactly one pod runs on every node (or every node that matches a selector). When a new node joins the cluster, the DaemonSet controller automatically schedules a pod on it. When a node is removed, the pod is garbage collected. Use a DaemonSet for node-level concerns: log collectors (Fluentd, Filebeat), metrics agents (Prometheus Node Exporter, Datadog Agent), networking components (CNI plugins, kube-proxy itself is a DaemonSet), and security agents. Use a Deployment for application workloads where you want a specific replica count spread across nodes. The key distinction is "one per node" vs "N total, distributed across nodes."

What the interviewer is testing: Practical knowledge of when each workload type is appropriate.

Common wrong answer: "I'd use a Deployment with high replica count instead of a DaemonSet." This doesn't guarantee one-per-node and wastes resources when you add nodes.

Ideal answer approach: Mention DaemonSet update strategies (RollingUpdate vs OnDelete), and that you can use nodeSelector or nodeAffinity to run a DaemonSet on only a subset of nodes (e.g., only GPU nodes).

Q13: What is a StatefulSet? How is it different from a Deployment?

Answer: A StatefulSet is designed for workloads that require stable, persistent identities — each pod gets a stable hostname (pod-0, pod-1), a stable DNS name (pod-0.svc-name.namespace.svc.cluster.local), and a PersistentVolumeClaim that stays bound to that pod identity even if the pod is rescheduled to a different node. StatefulSets deploy pods in order (pod-0, then pod-1, etc.) and scale down in reverse order. Rolling updates are also ordered. This is critical for distributed stateful systems like databases (Cassandra, MongoDB, PostgreSQL with Patroni), message queues (Kafka), and ZooKeeper. In contrast, Deployments treat all pods as interchangeable, use random names, and can delete/recreate pods in any order. The tradeoff is that StatefulSets are slower to scale and harder to operate than Deployments.

What the interviewer is testing: Whether you understand the stable identity guarantees and when they matter.

Common wrong answer: "StatefulSets are for databases, Deployments are for stateless apps." While roughly true, this doesn't explain why — what property of databases requires StatefulSet?

Ideal answer approach: Explain that the stable DNS name is critical for leader election in clustered databases (Kafka brokers reference each other by name, e.g., kafka-0:9092), and that the stable PVC means kafka-0 always gets its data back after a restart, regardless of which physical node it lands on.

Q14: Explain Pod QoS classes. Which class gets evicted first under memory pressure?

Answer: Kubernetes assigns one of three Quality of Service classes to every pod based on its resource requests and limits. Guaranteed: every container in the pod has both memory and CPU limits set, and limits equal requests. These pods are the last to be evicted. Burstable: at least one container has a memory or CPU request set, but not all containers have limits equal to requests.BestEffort: no containers have any resource requests or limits. Under node memory pressure, the kubelet evicts pods in this order: BestEffort first, then Burstable (starting with those most over their request), and Guaranteed last. The OOM killer in the Linux kernel uses a similar priority when evicting processes. This is why production services should always set resource requests and limits — to achieve at least Burstable class, ideally Guaranteed for critical services.

What the interviewer is testing: Whether you understand how resource settings affect scheduling and eviction behavior.

Common wrong answer: Not knowing the three classes or saying "memory limits don't affect eviction order."

Ideal answer approach: Explain that you can check a pod's QoS class with kubectl get pod mypod -o jsonpath='{.status.qosClass}', and that in practice most teams target Burstable for dev/staging and Guaranteed for production critical services.

Q15: What is the difference between a readinessProbe and a livenessProbe?

Answer: A readinessProbe controls whether a pod receives traffic from a Service. If the readinessProbe fails, the pod is removed from the Service's EndpointSlice — no new requests are routed to it, but the container keeps running. When the probe passes again, the pod is re-added to the endpoint list. A livenessProbe determines whether a container should be restarted. If the livenessProbe fails, the kubelet kills the container and restarts it according to the pod'srestartPolicy. The critical distinction: a failing readinessProbe keeps the container running but stops traffic; a failing livenessProbe kills and restarts the container. Misusing livenessProbe is one of the most common production mistakes — setting it too aggressively can cause restart loops during high load, making an overloaded service worse.

What the interviewer is testing: This is a fundamentals question with a common production pitfall, so the interviewer is testing both correctness and operational awareness.

Common wrong answer: Confusing the two, or saying both cause restarts. Or not knowing that readinessProbe affects Service endpoints.

Ideal answer approach: Give a concrete example: during startup, your app may not be ready to serve traffic yet (readinessProbe should fail), but it's not deadlocked (livenessProbe should pass). Mention that an overly aggressive livenessProbe on a CPU-heavy endpoint is a classic outage cause during traffic spikes.

Q16: What is a startupProbe and when should you use it?

Answer: A startupProbe disables the livenessProbe and readinessProbe until it succeeds, giving slow-starting applications time to initialize without triggering a restart loop. Before startupProbe was introduced (Kubernetes 1.16), engineers worked around slow startup by setting large initialDelaySeconds on the livenessProbe. The problem: if you set it to 120 seconds, a container that deadlocks at runtime waits 120 seconds before being restarted, slowing recovery. With startupProbe, you can set failureThreshold: 30 and periodSeconds: 10, giving the container up to 5 minutes to start while still detecting deadlocks quickly once startup completes. Use it for JVM applications, ML model servers, or any service that loads large data into memory at startup.

What the interviewer is testing: Whether you know the full probe toolset and when to reach for each tool.

Common wrong answer: "I just set a high initialDelaySeconds." This is the old approach and has the drawback described above.

Ideal answer approach: Explain the interaction: once startupProbe succeeds, it stops running, and then liveness and readiness take over. This is the correct three-probe setup for production services.

Q17: What is a preStop hook and why is it important for zero-downtime deployments?

Answer: A preStop hook is a command or HTTP call that Kubernetes executes inside a container before sending SIGTERM. The hook runs synchronously — SIGTERM is not sent until the hook completes or terminationGracePeriodSeconds expires. It matters for zero-downtime deployments because of a race condition in the endpoint removal flow: when a pod is deleted, two things happen in parallel — the EndpointSlice controller removes the pod from the endpoint list (which propagates to kube-proxy and then to iptables rules), and the pod receives SIGTERM. In a large cluster, iptables rule propagation can take several seconds. If the application exits immediately on SIGTERM, in-flight requests and new requests routed by still-stale rules will get connection refused. A preStop: exec: sleep 5 holds the container alive long enough for endpoint removal to propagate, draining traffic gracefully before the process receives SIGTERM.

What the interviewer is testing: Deep knowledge of the pod termination lifecycle and the real-world consequence of getting it wrong.

Common wrong answer: "I handle SIGTERM in my app, so I don't need preStop." SIGTERM handling is necessary but not sufficient — the endpoint propagation race is what preStop solves.

Ideal answer approach: Draw the timeline: pod deletion event → SIGTERM + endpoint removal in parallel → iptables update on all nodes (1-5 seconds) → preStop sleep covers this window → SIGTERM to process → process drains connections and exits → SIGKILL at terminationGracePeriodSeconds.

Q18: What happens when a Pod receives SIGTERM? Walk me through the termination sequence.

Answer: The full termination sequence is: (1) Pod is set to Terminating state; the pod is removed from Service endpoints in parallel. (2) If a preStop hook is defined, it runs. (3) SIGTERM is sent to the main process (PID 1) in each container. (4) The pod has terminationGracePeriodSeconds (default 30) to shut down cleanly. During this time, the application should stop accepting new connections, drain in-flight requests, flush buffers, and exit. (5) If the process is still running after the grace period, SIGKILL is sent and the container is force-killed. A common mistake is writing a preStop hook that takes longer than terminationGracePeriodSeconds; in that case, SIGKILL fires while the hook is still running. Always set terminationGracePeriodSeconds to accommodate your preStop hook duration plus your app's drain time.

What the interviewer is testing: Operational maturity around graceful shutdown, a critical production concern.

Common wrong answer: "The pod shuts down immediately." Or not knowing about the grace period and SIGKILL.

Ideal answer approach: Mention that PID 1 has special significance in Linux containers — if your main process is not PID 1 (e.g., it's started by a shell script), SIGTERM may not be forwarded to it. Use exec in the CMD to replace the shell process, or use tini as a proper init process.

Q19: What is an init container and what problems does it solve?

Answer: Init containers run to completion before any app containers start. They run sequentially — each must exit with code 0 before the next one begins. If an init container fails, Kubernetes restarts the pod according to its restart policy. They solve several bootstrapping problems: waiting for a dependency to be ready (e.g., until nc -z db-service 5432; do sleep 2; done), loading data into a shared volume before the app needs it, setting up configuration files that require tools not present in the main container image, and running database migrations before the application starts. Unlike sidecar containers (the new native sidecar feature in 1.29), init containers are not running concurrently with the main container — they are strictly sequential startup tasks.

What the interviewer is testing: Practical Kubernetes tool knowledge and awareness of startup dependency management.

Common wrong answer: Confusing init containers with sidecar containers, or not knowing that they run sequentially to completion.

Ideal answer approach: Distinguish between old-style sidecars (regular containers in the same pod that don't have defined startup/shutdown ordering), init containers (sequential, run to completion before app starts), and the new native sidecar containers (added in 1.29 via restartPolicy: Always on an init container) which start before app containers and remain running alongside them with proper lifecycle management.

Q20: What are PodDisruptionBudgets and why should every production service have one?

Answer: A PodDisruptionBudget (PDB) limits the number of pods that can be voluntarily disrupted simultaneously. "Voluntary disruption" includes node drains (during cluster upgrades), node deletions, and manual pod evictions. A PDB does not protect against involuntary disruptions like node crashes or OOM kills. You set either minAvailable (minimum number of pods that must remain running) or maxUnavailable (maximum number of pods that can be unavailable). During a node drain, the drain operation checks PDBs and will block if evicting a pod would violate the budget. Without a PDB, a kubectl drain on a node running all your pods will bring your service down. With a PDB of maxUnavailable: 1 on a 3-replica service, draining any single node will always leave at least 2 pods running. Every production service that cannot tolerate complete downtime should have one.

What the interviewer is testing: Operational maturity and understanding of the cluster upgrade impact on workloads.

Common wrong answer: "PDBs protect against any pod disruption." They only apply to voluntary disruptions — they cannot prevent the scheduler from evicting pods if a node becomes NotReady.

Ideal answer approach: Mention that a PDB of maxUnavailable: 0 with only 1 replica will block all node drains permanently — a common footgun. Always have at least 2 replicas if you want meaningful disruption protection.

PodDisruptionBudget Example

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-server-pdb
  namespace: production
spec:
  # Use minAvailable OR maxUnavailable, not both
  # minAvailable: 2 — always keep at least 2 pods running
  # maxUnavailable: 1 — allow only 1 pod to be unavailable at a time
  maxUnavailable: 1
  selector:
    matchLabels:
      app: api-server
# With 3 replicas and maxUnavailable:1, drain can only proceed when 2 pods are healthy.
# This prevents node drains from taking down your entire service during cluster upgrades.

Section 3: Networking (Q21–Q30)

Q21: Explain the difference between ClusterIP, NodePort, and LoadBalancer Service types.

Answer: ClusterIP (the default) creates a virtual IP address reachable only from within the cluster. kube-proxy programs iptables or IPVS rules on every node so that traffic to the ClusterIP is DNAT'd to one of the healthy pod endpoints. It has no external reachability. NodePort allocates a high port (30000-32767) on every cluster node and forwards traffic from that port to the Service's pods. It builds on top of ClusterIP — traffic hits the NodePort, gets forwarded to the ClusterIP, then to a pod. External clients must know a node's IP to use it, which is fragile for production. LoadBalancer provisions a cloud load balancer (via the cloud controller manager) and routes traffic from that load balancer to the NodePorts on worker nodes. It builds on top of NodePort, which builds on top of ClusterIP — all three types are layered. LoadBalancer is the standard for production external traffic in cloud environments.

What the interviewer is testing: Whether you understand that the three types are layered, not independent alternatives.

Common wrong answer: Describing them as three separate, independent implementations. Or not knowing that LoadBalancer creates a NodePort and a ClusterIP internally.

Ideal answer approach: Explain the layering and add that in modern clusters, NodePort for direct external access is rare — most external traffic goes through a LoadBalancer to an Ingress controller, which then routes to ClusterIP Services internally.

Q22: What is a headless Service and when would you use it?

Answer: A headless Service is created by setting clusterIP: None. Instead of allocating a virtual IP and doing load-balanced routing, it returns the individual pod IP addresses directly in DNS. A DNS query for a headless Service returns multiple A records — one per ready pod. This is critical for StatefulSets where the client needs to address specific pods by name (e.g., kafka-0.kafka-headless.default.svc.cluster.local) rather than a load-balanced endpoint. Databases that implement their own clustering (Cassandra, MongoDB, Kafka) use headless Services so clients can discover all cluster members and connect to specific ones. It's also used with StatefulSet DNS for stable pod identity: each pod in a StatefulSet gets a DNS record through the headless Service that persists across rescheduling.

What the interviewer is testing: Whether you understand the DNS vs VIP distinction and the use case for direct pod addressing.

Common wrong answer: "Headless Services are for Services without a type." Or not knowing that clusterIP: None changes the DNS behavior.

Ideal answer approach: Explain the DNS behavior: for a regular ClusterIP Service, DNS returns the single virtual IP. For a headless Service, DNS returns all pod IPs. Some clients (Cassandra drivers, MongoDB drivers) need to discover all nodes to perform consistent hash routing — this is why headless Services exist.

Q23: How does kube-proxy implement Service routing? Explain the iptables chain.

Answer: kube-proxy watches Services and EndpointSlices and programs iptables NAT rules on every node. When a packet destined for a Service's ClusterIP hits the node, it traverses the PREROUTING chain (for incoming packets) or the OUTPUT chain (for packets from local processes). Both chains jump to KUBE-SERVICES. For each Service, there is a KUBE-SVC-* chain that uses statistically random rule matching to distribute traffic. For 3 pods, the first rule matches with 33% probability, the second with 50% of remaining (33% overall), and the third catches all remaining traffic (33%). Each KUBE-SEP-* (Service Endpoint) chain performs a DNAT rule that rewrites the destination IP from the ClusterIP to a specific pod IP and port. The reply packets have source NAT applied in POSTROUTING to maintain connection tracking symmetry. This is why kube-proxy in iptables mode is O(n) in the number of Service endpoints — every packet must traverse rules linearly.

What the interviewer is testing: Deep networking knowledge that separates senior engineers from mid-level ones. Almost no one at the junior level can answer this fully.

Common wrong answer: "kube-proxy acts as a reverse proxy that forwards packets." In iptables mode, kube-proxy does NOT forward packets — it programs kernel netfilter rules and then stays out of the data path entirely.

Ideal answer approach: Clarify the userspace vs iptables vs IPVS distinction: in the old userspace mode, kube-proxy did sit in the data path. In iptables mode (default), it only programs kernel rules at control time, not data time. You can inspect these rules with iptables -t nat -L KUBE-SERVICES -n | grep <clusterIP>.

Q24: What is the difference between iptables mode and IPVS mode in kube-proxy?

Answer: In iptables mode, kube-proxy creates a linear chain of iptables NAT rules. Each rule is evaluated sequentially until a match is found. With many Services and endpoints, this becomes O(n) per packet for the rule traversal, causing measurable latency at scale. In practice, clusters with more than ~1,000 Services start seeing latency increases. IPVS mode uses the IP Virtual Server kernel module, which maintains backend endpoints in a hash table. Rule lookup is O(1) regardless of the number of Services. IPVS also supports more load-balancing algorithms: round-robin, least connections, destination hashing, source hashing. At large scale (>1,000 Services), IPVS mode has meaningfully better performance and is recommended. The tradeoff is that IPVS requires additional kernel modules (ip_vs, ip_vs_rr, etc.) to be loaded on every node.

What the interviewer is testing: Performance and scalability awareness — relevant for teams running at scale.

Common wrong answer: "IPVS is always better, everyone should use it." IPVS adds complexity and the improvement is only meaningful at scale. Below 500 Services, iptables mode is simpler and fine.

Ideal answer approach: Mention that some CNI plugins (like Cilium) bypass kube-proxy entirely using eBPF for Service routing, achieving even better performance than IPVS while also enabling network observability and policy enforcement at the same layer.

Q25: What is a CNI plugin? Name 3 CNI plugins and their key differences.

Answer: The Container Network Interface (CNI) is a specification for how container runtimes should invoke network plugins to configure pod networking. When a pod is created, the kubelet calls the configured CNI plugin, which sets up a network namespace, creates a virtual ethernet pair, assigns an IP address from the pod CIDR, and configures routing so the pod can communicate with other pods and nodes. Flannel is the simplest — it creates a flat overlay network using VXLAN encapsulation. Easy to configure, no NetworkPolicy support, limited to basic routing. Calico operates in either BGP mode (routes pod CIDRs natively through BGP, no overlay, low latency) or VXLAN mode. Calico has full NetworkPolicy support and performance-optimized eBPF dataplane. It is the most widely used CNI in production. Cilium is eBPF-native — it implements networking, NetworkPolicy, and load balancing entirely in the kernel using eBPF programs, bypassing iptables entirely. It provides the best observability (Hubble UI) and performance at scale, and is increasingly the default for new clusters.

What the interviewer is testing: Practical knowledge of the CNI ecosystem and the tradeoffs between plugins.

Common wrong answer: Listing CNI plugins without any differentiation, or not knowing that Flannel has no NetworkPolicy support.

Ideal answer approach: Frame the choice as: Flannel for simplicity in dev/test, Calico for production environments that need NetworkPolicy without eBPF complexity, Cilium for scale and observability requirements. Mention that EKS now supports Cilium as a CNI, and GKE Dataplane V2 is built on Cilium.

Q26: How do pods on different nodes communicate in Kubernetes?

Answer: Kubernetes requires that all pods can communicate with all other pods without NAT, regardless of which node they are on. This is called the flat networking model. How it is implemented depends on the CNI plugin. In overlay mode (Flannel VXLAN, Calico VXLAN): the CNI plugin wraps pod packets in a UDP envelope (VXLAN) with the destination node's IP as the outer IP. The encapsulated packet traverses the regular node network, and the receiving node's VTEP (Virtual Tunnel Endpoint) device decapsulates it and delivers it to the destination pod. In BGP mode (Calico BGP, on bare metal or cloud VPCs that support BGP peering): each node advertises its pod CIDR subnet via BGP. Routers learn that pod CIDR X is reachable via node Y, and route packets natively without encapsulation, achieving lower latency and higher throughput.

What the interviewer is testing: Whether you can explain cross-node communication at the packet level.

Common wrong answer: "The CNI handles it" without explaining how. Or not knowing about VXLAN encapsulation.

Ideal answer approach: Trace a single packet: pod A (10.244.1.5 on node-1) sends to pod B (10.244.2.8 on node-2). VXLAN: kernel on node-1 wraps it in UDP to node-2's IP, node-2 decapsulates, routes to pod B's veth pair. BGP: kernel routes 10.244.2.0/24 to node-2 directly via IP forwarding.

Q27: What is a NetworkPolicy? What is the default behavior without any NetworkPolicy?

Answer: A NetworkPolicy is a Kubernetes resource that specifies which pods can send and receive network traffic to/from other pods, namespaces, or external IP ranges. By default, if no NetworkPolicy selects a pod, that pod accepts all ingress and egress traffic from any source. NetworkPolicies are additive and allow-only: you cannot write a deny rule, and all allowed rules are unioned together. If a pod is selected by a NetworkPolicy, it switches to deny-all mode for the affected traffic direction, and only the explicitly allowed connections are permitted. NetworkPolicies are only enforced if your CNI plugin supports them — Flannel does not, Calico and Cilium do. A common production pattern is a default-deny NetworkPolicy in every namespace, followed by specific allow rules.

What the interviewer is testing: Security awareness and understanding of the default-open model.

Common wrong answer: "By default, pods can only talk to pods in the same namespace." This is wrong — by default, all pods across all namespaces can communicate.

Ideal answer approach: Emphasize the security risk of the default-allow model and explain the zero-trust pattern: apply a default-deny NetworkPolicy to every namespace, then explicitly allow only the required connections. Note that NetworkPolicies operate at L3/L4 only — for L7 (HTTP path, headers), you need a service mesh policy (Istio, Cilium L7).

Q28: How does DNS work in Kubernetes? What is the format of a Service DNS name?

Answer: Kubernetes runs a cluster DNS server, CoreDNS, deployed as a Deployment in thekube-system namespace and exposed as a ClusterIP Service at a well-known IP (typically 10.96.0.10). The kubelet configures each pod's /etc/resolv.conf with this DNS IP as the nameserver and adds search domains: default.svc.cluster.local, svc.cluster.local, cluster.local. The full DNS name for a Service is <service-name>.<namespace>.svc.cluster.local. Within the same namespace, you can use just <service-name> because of the search domain. For StatefulSet pods, each pod gets a DNS record: <pod-name>.<service-name>.<namespace>.svc.cluster.local. CoreDNS reads Service and EndpointSlice objects from the apiserver and serves A records (for regular Services) or multiple A records (for headless Services).

What the interviewer is testing: Knowledge of CoreDNS and the DNS resolution path.

Common wrong answer: Not knowing that CoreDNS is the DNS provider, or not knowing the search domain behavior that makes short names work.

Ideal answer approach: Mention that the search domain adds a small latency cost (each short name triggers multiple DNS lookups until one resolves), and that in high-request-rate microservices, using full DNS names or ndots:5 tuning can reduce DNS lookup overhead.

Q29: What is an Ingress resource? What is the difference between an Ingress and a Service?

Answer: An Ingress is a Kubernetes resource that defines HTTP/HTTPS routing rules — routing based on hostname and URL path to backend Services. A Service handles raw TCP/UDP load balancing at L4 with no awareness of HTTP paths or hostnames. An Ingress works at L7. The Ingress resource itself is just a configuration object — it does nothing without an Ingress controller (NGINX, Traefik, AWS ALB Ingress Controller, GKE Gateway) watching for Ingress objects and configuring its load-balancing layer accordingly. A typical setup: one LoadBalancer Service exposes the Ingress controller, and all application Services are ClusterIP. The Ingress controller handles TLS termination, routing rules, rate limiting, and authentication at the HTTP layer, then proxies to backend ClusterIP Services.

What the interviewer is testing: Understanding of the L4 vs L7 distinction and the Ingress controller architecture.

Common wrong answer: Treating Ingress as a standalone feature that works without a controller. Or not knowing that you need to deploy an Ingress controller separately.

Ideal answer approach: Mention the Gateway API (the successor to Ingress, generally available in 1.31) which provides HTTPRoute, GRPCRoute, and TCPRoute resources with better multi-team support and more expressive routing rules.

Q30: What are EndpointSlices and why were they introduced to replace Endpoints?

Answer: The original Endpoints resource had a scalability problem: it stored all pod IPs for a Service in a single object. For a Service with 1,000 pods, a single pod addition or deletion caused the entire 1,000-entry object to be re-sent to every node via the watch stream, creating O(n²) update costs. EndpointSlices (introduced in 1.17, default in 1.21) break endpoints into chunks of up to 100 entries each. When one pod changes, only the affected slice is updated and distributed, making updates O(1) per change in most cases. This dramatically reduces apiserver and kube-proxy memory and CPU usage in large clusters. EndpointSlices also add support for dual-stack addresses (IPv4 and IPv6) and additional topology hints for zone-aware routing, which the old Endpoints resource could not express.

What the interviewer is testing: Awareness of Kubernetes scaling challenges and its evolution.

Common wrong answer: Not knowing EndpointSlices exist, or thinking Endpoints and EndpointSlices are the same thing with a different name.

Ideal answer approach: Quantify the problem: with 10,000 pods, a single pod restart triggers one 1KB update (EndpointSlice) vs a 10,000-entry Endpoints object being rebroadcast to all nodes. At 1,000 nodes, that's a significant bandwidth and CPU difference.

Section 4: Storage (Q31–Q35)

Q31: What is the difference between a PersistentVolume and a PersistentVolumeClaim?

Answer: A PersistentVolume (PV) is a cluster-level resource that represents a piece of physical storage — an EBS volume, an NFS share, a local disk — provisioned either manually by an admin or dynamically by a StorageClass. A PersistentVolumeClaim (PVC) is a namespaced resource that represents a pod's request for storage — specifying the size, access mode, and optional StorageClass. Kubernetes binds a PVC to a PV that satisfies its requirements (size &geq; requested, access mode matches, StorageClass matches). Once bound, the PV-PVC binding is 1:1. The pod mounts the PVC, not the PV directly. This abstraction decouples workloads from the underlying storage infrastructure: a pod that requests a 20Gi PVC does not need to know whether the actual storage is EBS, GCS, or Ceph.

What the interviewer is testing: Understanding of the PV/PVC abstraction and the binding lifecycle.

Common wrong answer: "PVC is just another name for PV." Or not knowing the binding process.

Ideal answer approach: Explain the reclaim policies: Retain (PV stays after PVC deletion, data preserved, requires manual cleanup), Delete (PV and underlying storage are deleted when the PVC is deleted — default for dynamically provisioned volumes), and the deprecatedRecycle. A wrong reclaim policy is one of the most common data-loss scenarios in Kubernetes.

Q32: What are StorageClasses and what is dynamic provisioning?

Answer: A StorageClass defines a "class" of storage with a specific provisioner (e.g., ebs.csi.aws.com), parameters (volume type, IOPS, encryption), and reclaim policy. Dynamic provisioning means that when a PVC is created with a StorageClass, the CSI driver automatically creates the backing storage and a PV without any manual admin intervention. Before dynamic provisioning, admins had to pre-provision PVs manually. A cluster can have multiple StorageClasses (e.g., one for fast SSD, one for slow HDD, one for shared NFS). One can be designated as the default — PVCs that don't specify a StorageClass will use it. This is how EKS's gp3 StorageClass works: you create a PVC requesting 50Gi, and the EBS CSI driver calls the AWS API to create a 50Gi gp3 EBS volume, then binds the PVC to it automatically.

What the interviewer is testing: Operational knowledge of how storage actually gets provisioned in cloud environments.

Common wrong answer: Not knowing about the default StorageClass or that it can be changed, leading to "why is everything using gp2?" incidents on EKS.

Ideal answer approach: Mention volume binding mode: WaitForFirstConsumerdelays PV binding until a pod using the PVC is scheduled, which ensures the volume is created in the same AZ as the pod. The Immediate mode creates the volume before the pod schedules, which can cause a pod to be stuck in Pending if it gets scheduled to a different AZ.

Q33: What are the access modes for PersistentVolumes (RWO, ROX, RWX, RWOP)?

Answer: Access modes describe how many nodes can mount a volume simultaneously. ReadWriteOnce (RWO): the volume can be mounted as read-write by a single node. Most block storage (EBS, GCE PD) is RWO — an EBS volume can only be attached to one EC2 instance at a time. ReadOnlyMany (ROX): many nodes can mount the volume read-only simultaneously — used for distributing read-only data like model weights or static assets. ReadWriteMany (RWX): many nodes can mount read-write simultaneously — requires network filesystems like NFS, EFS, or Azure Files. Block storage cannot support RWX. ReadWriteOncePod (RWOP): introduced in 1.22, this is stricter than RWO — only a single pod (not just a single node) can mount the volume, preventing two pods on the same node from accessing it.

What the interviewer is testing: Whether you know the access mode semantics and their hardware constraints.

Common wrong answer: Thinking RWO means single pod. RWO is per-node, not per-pod.

Ideal answer approach: Give a concrete failure scenario: requesting RWX on an EBS-backed StorageClass will result in the PVC binding but the second pod failing to mount because EBS is single-attachment. This is a common mistake when migrating from NFS-based environments to EBS.

Q34: What is a CSI driver and why did Kubernetes move from in-tree to out-of-tree storage plugins?

Answer: The Container Storage Interface (CSI) is a standard API for container orchestrators to expose block and file storage to containerized workloads. CSI drivers are out-of-tree: they run as pods in the cluster and communicate with the kubelet via a Unix socket, rather than being compiled into the Kubernetes binary. Before CSI, storage drivers were in-tree — compiled directly into kube-controller-manager and the kubelet. The problem: every storage bug required a Kubernetes release to fix, and storage vendors had to wait for Kubernetes release cycles to ship improvements. With CSI, storage vendors ship their drivers independently (AWS EBS CSI, GCE PD CSI, OpenEBS, Rook/Ceph). In-tree storage plugins are deprecated and being removed from Kubernetes — as of 1.27, several in-tree drivers are removed and their CSI replacements are required.

What the interviewer is testing: Understanding of the extensibility model and why the migration matters operationally.

Common wrong answer: Not knowing that in-tree plugins existed or that migration is required — a common gap that causes upgrade problems.

Ideal answer approach: Mention that CSI drivers also enable volume snapshots and volume cloning, which the old in-tree drivers could not support. This is what enables backup solutions like Velero to take consistent volume snapshots.

Q35: How would you debug a pod stuck in ContainerCreating due to a PVC not binding?

Answer: Start with kubectl describe pod <name> and look at the Events section. You'll typically see a FailedMount or FailedAttachVolume event. Then check the PVC: kubectl describe pvc <pvc-name> — is it Pending or Bound? If Pending, check whether a matching PV exists (kubectl get pv), whether the StorageClass is correct, and whether the CSI driver is running (kubectl get pods -n kube-system | grep csi). For dynamic provisioning failures, check the CSI controller pod logs. Common causes: wrong StorageClass name (typo), ReclaimPolicy set to Retain with the old PV still bound to a deleted PVC, volume binding mode of WaitForFirstConsumer waiting for a pod to schedule, or the pod and the volume being in different AZs. Check node events with kubectl get events --field-selector involvedObject.name=<nodeName>for attachment failures.

What the interviewer is testing: Systematic debugging methodology, not just knowledge of what could go wrong.

Common wrong answer: "I'd delete the pod and try again." This shows no diagnostic process.

Ideal answer approach: Walk through the debugging tree systematically: pod events → PVC status → PV availability → StorageClass → CSI driver health → cloud provider limits. This structured approach is the mark of an experienced engineer.

Section 5: Security (Q36–Q40)

Q36: What is RBAC in Kubernetes? Explain Roles vs ClusterRoles.

Answer: Role-Based Access Control (RBAC) is the authorization mechanism that controls which principals (users, ServiceAccounts, groups) can perform which verbs (get, list, watch, create, update, patch, delete) on which resources in which namespaces. A Role is namespaced — it grants permissions to resources within a single namespace. A ClusterRole is cluster-scoped — it can grant permissions to namespaced resources across all namespaces, or to cluster-scoped resources (Nodes, PersistentVolumes, ClusterRoles themselves). A RoleBinding binds a Role or ClusterRole to subjects within a namespace. A ClusterRoleBinding binds a ClusterRole to subjects cluster-wide. A useful pattern: define a ClusterRole with common permissions (e.g., read ConfigMaps) and bind it with a RoleBinding in each namespace — this avoids duplicating Role definitions across namespaces.

What the interviewer is testing: Security fundamentals and whether you understand the namespaced vs cluster-scoped distinction.

Common wrong answer: Confusing RoleBinding and ClusterRoleBinding scope. A ClusterRole bound with a RoleBinding only applies within that namespace — not cluster-wide.

Ideal answer approach: Mention kubectl auth can-i for debugging RBAC:kubectl auth can-i get pods --as=system:serviceaccount:default:my-sa is the fastest way to verify whether a ServiceAccount has the expected permissions.

RBAC Example: ServiceAccount + Role + RoleBinding

# 1. ServiceAccount — the identity of the pod
apiVersion: v1
kind: ServiceAccount
metadata:
  name: api-reader
  namespace: production
  annotations:
    # If using IRSA (AWS) or Workload Identity (GCP), add cloud role here
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789:role/api-reader-role
automountServiceAccountToken: false   # mount explicitly per-pod, not globally
---
# 2. Role — namespaced permissions (use ClusterRole for cluster-wide resources)
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: configmap-reader
  namespace: production
rules:
  - apiGroups: [""]           # "" is the core API group
    resources: ["configmaps"]
    verbs: ["get", "list", "watch"]
    resourceNames: ["app-config"]   # restrict to specific ConfigMap names
---
# 3. RoleBinding — binds the Role to the ServiceAccount
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: api-reader-configmap-reader
  namespace: production
subjects:
  - kind: ServiceAccount
    name: api-reader
    namespace: production
roleRef:
  kind: Role
  name: configmap-reader
  apiGroup: rbac.authorization.k8s.io

Q37: What is a ServiceAccount? Why should most pods have automountServiceAccountToken: false?

Answer: A ServiceAccount is a Kubernetes identity for pods. When a pod runs, Kubernetes can automatically mount a ServiceAccount token as a file at /var/run/secrets/kubernetes.io/serviceaccount/token. This token is a valid credential for the Kubernetes API. The default ServiceAccount in every namespace has limited permissions, but if an attacker compromises a container and finds a mounted token, they can use it to enumerate the cluster, read ConfigMaps and Secrets, or (if the RBAC is misconfigured) escalate privileges. Setting automountServiceAccountToken: false prevents the token from being mounted. Only pods that actually need to call the Kubernetes API (Operators, CI/CD agents, monitoring tools) should have tokens mounted. This follows the principle of least privilege: most application pods have no business reason to call the Kubernetes API.

What the interviewer is testing: Practical security awareness, not just knowing what ServiceAccounts are.

Common wrong answer: "I'd use the default ServiceAccount and restrict its permissions." Better, but not best — you should not mount a token at all if it isn't needed.

Ideal answer approach: Mention projected service account tokens (introduced in 1.12), which are time-limited and audience-restricted, replacing the old long-lived static tokens. Also note Workload Identity (GKE) and IRSA (EKS) which bind pod identity to cloud IAM roles, making the Kubernetes token unnecessary for cloud API access.

Q38: Are Kubernetes Secrets secure? Why or why not?

Answer: By default, Kubernetes Secrets are not strongly secure. They are stored in etcd base64-encoded, not encrypted. Base64 is not encryption — it is trivially reversible. Anyone with read access to etcd has all Secrets in plaintext. To make Secrets secure, you need two things: encryption at rest (configure the apiserver with an EncryptionConfiguration using AES-GCM or a KMS provider like AWS KMS, GCP KMS, or Vault Transit) and tight RBAC (restrict who can get or list Secret objects). Even with encryption at rest, a pod with a mounted Secret has it in plaintext in memory. Many organizations use external secret management (Vault, AWS Secrets Manager, GCP Secret Manager) with the External Secrets Operator to sync secrets, providing centralized rotation, auditing, and versioning that native Secrets do not offer.

What the interviewer is testing: Security depth — a critical question for security-sensitive roles.

Common wrong answer: "Yes, Secrets are encrypted." They are only encrypted in transit (TLS to the apiserver), not encrypted at rest by default.

Ideal answer approach: Distinguish between the threat models: etcd compromise (solved by encryption at rest + etcd auth), API access (solved by RBAC), and container memory access (not solvable by Kubernetes alone — this is a process isolation concern).

Q39: What is Pod Security Admission (PSA) and what replaced PodSecurityPolicy?

Answer: PodSecurityPolicy (PSP) was deprecated in Kubernetes 1.21 and removed in 1.25. It was replaced by Pod Security Admission (PSA), a built-in admission controller that enforces security profiles defined by the Pod Security Standards at the namespace level. PSA defines three levels: privileged (no restrictions), baseline (prevents known privilege escalations — no hostNetwork, no hostPID, no privileged containers), and restricted(heavily restricted, requires non-root, no privilege escalation, restricted volume types). You configure PSA by labeling namespaces: pod-security.kubernetes.io/enforce: restricted. PSA can enforce (reject violating pods), audit (log violations), or warn (warn in API response). PSP was complex because it required ClusterRole binding to work, leading to common misconfigurations where PSP was deployed but not enforced.

What the interviewer is testing: Whether you know that PSP is gone and what replaced it.

Common wrong answer: Still talking about PodSecurityPolicy as the current standard. This is a 2022-era answer and signals the candidate hasn't kept up.

Ideal answer approach: Acknowledge PSA's limitations (namespace-level granularity only, no per-pod customization), and mention that for fine-grained policy enforcement many teams use OPA/Gatekeeper or Kyverno on top of PSA.

Q40: What security context fields should you set on every production container?

Answer: At minimum, every production container should have: runAsNonRoot: true (prevents running as root), runAsUser and runAsGroup (specific non-root UID/GID), allowPrivilegeEscalation: false (prevents a process from gaining more privileges than its parent — blocks setuid exploits), readOnlyRootFilesystem: true (prevents writing to the container filesystem, limiting the blast radius of an exploit), and capabilities: drop: ["ALL"] (removes all Linux capabilities — add back only what is specifically needed). At the pod level, set seccompProfile: type: RuntimeDefault (restricts syscalls to the default container runtime profile, blocking many kernel-level exploits). These settings together implement the principle of least privilege and significantly raise the cost of post-exploit lateral movement.

What the interviewer is testing: Whether you know the container security primitives beyond just "don't run as root."

Common wrong answer: Only mentioning runAsNonRoot. This is the most basic setting and missing the others suggests shallow security knowledge.

Ideal answer approach: Explain the threat model each setting addresses: readOnlyRootFilesystem limits attacker persistence (can't write backdoors), capability dropping limits exploit surface (can't bind privileged ports or modify network interfaces), seccomp limits kernel attack surface.

Section 6: Operations and Troubleshooting (Q41–Q50)

Q41: A pod is stuck in Pending. Walk me through your debugging process.

Answer: First, kubectl describe pod <name> and read the Events section. A Pending pod with no events means the scheduler hasn't tried yet, or it hasn't had time. Events will show the specific scheduler reason: Insufficient cpu or Insufficient memorymeans the node pool is undersized or requests are too high. 0/3 nodes are available: 3 node(s) had untolerated taint means there are node taints the pod doesn't tolerate. Check the cluster node status: kubectl get nodes — are all nodes Ready? Check node allocatable resources:kubectl describe nodes | grep -A5 Allocatable. If no nodes have capacity, check if the cluster autoscaler is configured and its logs (kubectl logs -n kube-system deployment/cluster-autoscaler). Also check for affinity rules that might be impossible to satisfy, and PVC binding issues (if the pod has volumes). The scheduler reason in the event message is usually diagnostic.

What the interviewer is testing: Systematic debugging approach and breadth of known causes.

Common wrong answer: Starting with logs before reading events. You cannot get logs from a pod that has never started.

Ideal answer approach: Describe the debugging as a decision tree: events present? Inspect reason. No events? Check scheduler health. Node exists? Check node capacity. Capacity exists? Check taints, affinity, topology constraints. PVC? Check PVC status separately.

Q42: A pod is in CrashLoopBackOff. What are the most common causes and how do you debug each?

Answer: CrashLoopBackOff means the container starts, crashes, and Kubernetes backs off before restarting it (1s, 2s, 4s... up to 5 minutes). Debug with kubectl logs <pod> --previous to see the last crash's logs. Common causes: (1) Application misconfiguration — missing environment variables or ConfigMap data; fix by checking kubectl describe pod for EnvFrom issues. (2) Secret not found or wrong key — check Secret exists and key name matches. (3) Liveness probe misconfigured — too aggressive thresholds killing a healthy container; check probe settings. (4) OOMKill — the container is hitting its memory limit; check kubectl describe pod for OOMKilled in Last State. (5) Application bug — panic or unhandled exception; logs will show the stack trace. (6) Missing startup dependency — app tries to connect to a database that's not ready; consider init containers or retry logic.

What the interviewer is testing: Systematic debugging and breadth of knowledge of failure modes.

Common wrong answer: Only mentioning application bugs. Most CrashLoopBackOff cases in production are configuration or resource issues, not code bugs.

Ideal answer approach: Always start with --previous logs before the current logs. If logs are empty (container crashed before writing anything), use kubectl get pod -o yaml to check lastState.terminated.exitCode — exit code 137 is OOMKill, 1 is application error, 139 is segfault.

Q43: Your service has 0 endpoints. What does that mean and how do you fix it?

Answer: Zero endpoints means the Service has no ready pods matching its selector. Run kubectl get endpoints <service-name> (or kubectl get endpointslices -l kubernetes.io/service-name=<service-name>) to confirm. Then check the Service's selector field with kubectl describe service <name> and compare it to the pod's labels with kubectl get pods --show-labels. The most common cause is a label mismatch — a typo in the selector (app: myapp vs App: myapp) or pods that have a different label version. Other causes: all pods failing readinessProbe (they exist but are not ready), pods stuck in Pending, or the Deployment selector doesn't match the pod template labels (this would also break the ReplicaSet).

What the interviewer is testing: Whether you understand the selector-based endpoint registration mechanism.

Common wrong answer: Restarting the pods without diagnosing the root cause. If the label is wrong, restarting changes nothing.

Ideal answer approach: Use kubectl get pods -l <selector> where selector is the Service's selector expression to directly test whether any pods match. Zero results means the selector is wrong. Results with pods not in Running/Ready state means a different problem.

Q44: How would you do a zero-downtime deployment in Kubernetes?

Answer: Zero-downtime deployment in Kubernetes requires: (1) A Deployment with strategy.rollingUpdate.maxUnavailable: 0 and maxSurge: 1 — this ensures old pods are not removed until new ones are ready. (2) A proper readinessProbe — new pods only receive traffic after the readinessProbe passes, ensuring they're fully initialized. (3) A preStop hook (sleep 5-10 seconds) to drain traffic before SIGTERM. (4) Adequate terminationGracePeriodSecondsfor your application to finish in-flight requests. (5) A PodDisruptionBudget so concurrent node drains can't take down the service. (6) Enough pod replicas (minimum 2, ideally 3+) so a rolling update doesn't leave zero pods running. If you have all of these, a Deployment update produces zero dropped requests.

What the interviewer is testing: Whether you know that zero-downtime requires multiple cooperating mechanisms, not just the rolling update strategy alone.

Common wrong answer: "Set the strategy to RollingUpdate." This is necessary but far from sufficient.

Ideal answer approach: Walk through the full checklist and explain what failure each item prevents. This shows operational depth and demonstrates that you've debugged zero-downtime issues in real systems.

Q45: What is the difference between kubectl cordon, drain, and delete node?

Answer: kubectl cordon <node> marks the node as unschedulable (adds the node.kubernetes.io/unschedulable taint) so no new pods are scheduled on it. Existing pods continue to run. This is the first step before maintenance. kubectl drain <node> cordons the node and then evicts all non-DaemonSet pods from it, respecting PodDisruptionBudgets. This moves all workloads off the node. DaemonSet pods are not evicted by default (pass --ignore-daemonsets to drain them). kubectl delete node <node> removes the node object from the cluster entirely. The node's pods are evicted and rescheduled immediately (no drain grace period). The actual machine is not affected — if the kubelet is still running, the node will re-register itself. Use cordon+drain for planned maintenance, delete for decommissioning a node.

What the interviewer is testing: Operational knowledge of the node lifecycle management.

Common wrong answer: Saying drain removes the node from the cluster. Drain evacuates pods but leaves the node object. Delete removes the node object.

Ideal answer approach: Describe a typical cluster upgrade workflow: for each node, cordon → drain → update OS/k8s components → uncordon. Knowing the workflow shows the commands in operational context.

Q46: Your cluster has pods with OOMKilled events. What caused this and how do you prevent it?

Answer: OOMKilled means the Linux OOM killer terminated the container process because it exceeded its memory limit. When a container's memory usage reaches its resources.limits.memoryvalue, the kernel invokes the OOM killer. The container exits with code 137 (128 + SIGKILL). Prevention requires accurate memory limit setting: use actual memory profiles from your application under realistic load, not guesses. Run with metrics (Prometheus + kube-state-metrics) to see actual usage vs limits. Tools like Vertical Pod Autoscaler (VPA) in recommendation mode will analyze historical usage and suggest appropriate requests and limits. Also look for memory leaks in your application — if memory steadily climbs until OOMKill, the problem is in the code, not the Kubernetes config. Setting limits too low relative to actual usage is the most common cause.

What the interviewer is testing: Resource management knowledge and the ability to distinguish configuration from application bugs.

Common wrong answer: "Just increase the memory limit." Sometimes correct, but first you need to verify whether the usage is expected or a leak.

Ideal answer approach: Describe a data-driven approach: use VPA in recommendation mode for 1-2 weeks to gather usage data, then set limits at p99 usage + 20% headroom. Monitor with a Prometheus alert on kube_pod_container_status_last_terminated_reason == OOMKilled.

Q47: How do you investigate CPU throttling in a Kubernetes container?

Answer: CPU throttling occurs when a container's CPU usage hits its limits.cpu value. Linux cgroups enforce the limit by pausing the process for the remainder of the scheduling window (100ms by default). The container is not killed, but it runs slower. You can measure throttling through the container_cpu_cfs_throttled_periods_total and container_cpu_cfs_periods_total metrics in Prometheus — the throttle ratio is throttled_periods / total_periods. If this ratio is above 25%, the container is meaningfully throttled. Fix options: increase the CPU limit, optimize the application to use less CPU, or consider setting no CPU limit (controversial: avoids throttling but allows noisy-neighbor effects). The --cpu-cfs-quotaflag controls the enforcement period. Note: CPU requests affect scheduling and node allocation, while CPU limits affect in-pod runtime behavior.

What the interviewer is testing: Deep knowledge of cgroup CPU scheduling, which is relevant for performance-sensitive roles.

Common wrong answer: Confusing CPU throttling with OOMKill, or not knowing about the CFS throttling metrics.

Ideal answer approach: Mention the debate about removing CPU limits: teams like Cloudflare and Netflix have published that removing CPU limits improves latency because of CFS scheduler bursting behavior, at the cost of needing accurate CPU requests instead. This shows you follow the community discourse on Kubernetes resource management.

Q48: What is the difference between kubectl apply, replace, and patch?

Answer: kubectl apply performs a three-way merge: it compares the desired manifest, the last-applied-configuration annotation, and the live state in the cluster, applying only the diff. It is idempotent and suitable for GitOps workflows. kubectl replace does a full replacement: it removes the existing resource and creates a new one. This is disruptive for running workloads and fails if the resource doesn't exist. Some immutable fields (like a pod's container image in a running pod) require replace. kubectl patch applies a partial update using a patch strategy (strategic merge patch, JSON merge patch, or JSON patch). It's useful for scripted, targeted changes: kubectl patch deployment my-app -p '{"spec":{"replicas":5}}'. For CI/CD pipelines, apply is almost always the right choice. Replace is for resources with immutable fields. Patch is for scripted, single-field changes.

What the interviewer is testing: Practical CLI knowledge and understanding of Kubernetes API semantics.

Common wrong answer: Using replace in a GitOps pipeline. This can cause downtime because replace re-creates the resource.

Ideal answer approach: Mention kubectl apply --server-side (server-side apply), which moves the merge logic to the apiserver, enabling better conflict detection in multi-owner scenarios (e.g., Helm and ArgoCD both managing the same resource).

Q49: How would you set up horizontal pod autoscaling? What metrics does it use?

Answer: The HorizontalPodAutoscaler (HPA) controller adjusts the replica count of a Deployment, ReplicaSet, or StatefulSet based on observed metrics. Basic setup requires the Metrics Server installed in the cluster (for CPU/memory). The HPA controller queries Metrics Server every 15 seconds and computes the desired replica count as: ceil(currentReplicas * currentMetricValue / desiredMetricValue). Built-in resource metrics (CPU and memory) use the autoscaling/v2 API. Custom metrics (HTTP request rate, queue depth, latency) require a custom metrics API server — either via the Prometheus Adapter or KEDA (Kubernetes Event-Driven Autoscaler). KEDA is the modern approach: it supports 50+ event sources (Kafka, SQS, Datadog, Prometheus) and can scale to zero, which the standard HPA cannot. Configure scale-down stabilization windows to prevent flapping.

What the interviewer is testing: Practical autoscaling knowledge including the modern tooling.

Common wrong answer: Only knowing CPU-based autoscaling and Metrics Server. Real production autoscaling almost always needs custom metrics.

Ideal answer approach: Mention the tradeoffs between HPA and KEDA, and the need for the Cluster Autoscaler to scale nodes when the HPA increases replicas beyond current node capacity. HPA + CA + PDB is the full autoscaling triangle for production.

HPA with Custom Metrics Example

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3
  maxReplicas: 20
  metrics:
    # CPU — classic metric, good for CPU-bound workloads
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60    # scale up before you hit 100%
    # Memory — useful but be careful: memory leaks will trigger false scale-outs
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 70
    # Custom metric from Prometheus via KEDA or custom-metrics-apiserver
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "1000"
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300   # wait 5 min before scaling down
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60             # scale down max 10% per minute
    scaleUp:
      stabilizationWindowSeconds: 0    # scale up immediately
      policies:
        - type: Percent
          value: 100
          periodSeconds: 30            # can double pod count every 30s

Q50: Describe how you would approach a production outage where kubectl hangs.

Answer: kubectl hangs when the apiserver is unreachable or unresponsive. Start with the network: can you reach the apiserver endpoint directly? curl -k https://<apiserver-ip>:6443/healthz. If that hangs, check whether the apiserver process is running on control plane nodes (SSH if accessible, or use cloud provider console). Check etcd health: etcd unresponsiveness (high disk latency, quorum loss) causes apiserver to hang on all writes and most reads. Check control plane node resources — a full disk on a control plane node is a common cause of silent failures. If apiserver is responding but slow, check for API priority and fairness (APF) exhaustion: a rogue controller flood-querying the API can exhaust request buckets. Meanwhile, the data plane (running pods) is likely unaffected — communicate this to stakeholders immediately. Check metrics and logs from your monitoring stack (which should be independent of the cluster apiserver) before trying to make changes.

What the interviewer is testing: Incident response methodology and understanding that the control plane and data plane are independent failure domains.

Common wrong answer: Immediately restarting the apiserver without diagnosing. If the cause is a full disk, a restart makes it worse.

Ideal answer approach: Lead with the communication aspect (service is probably still running, data plane is independent), then triage systematically from infrastructure outward. Mention that you should have out-of-band access to control plane nodes (SSH jump host, cloud console) before you need it — not when you're in the middle of an outage.

Production-Ready Deployment Example

Here is the complete YAML for a production-ready Deployment that incorporates all the best practices discussed above: probes, resources, security context, anti-affinity, preStop hook, and non-root execution.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server
  namespace: production
  labels:
    app: api-server
    version: "1.0.0"
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-server
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0      # zero-downtime: never kill before replacement is ready
  template:
    metadata:
      labels:
        app: api-server
    spec:
      # Spread pods across nodes — single-node failure cannot take down the service
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: api-server
      # Never run two replicas on the same node (belt-and-suspenders)
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app: api-server
                topologyKey: kubernetes.io/hostname
      automountServiceAccountToken: false   # only mount if you actually call the API
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: api
          image: myregistry/api-server:1.0.0
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          # ALWAYS set requests AND limits — no requests = wrong node placement,
          # no limits = noisy-neighbour OOM kills
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "1000m"
              memory: "512Mi"
          # startupProbe gives slow-starting apps time to boot without triggering
          # livenessProbe restarts
          startupProbe:
            httpGet:
              path: /healthz
              port: 8080
            failureThreshold: 30
            periodSeconds: 5            # up to 150 s to start
          # readinessProbe controls Service traffic — pod only gets traffic when ready
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 5
            failureThreshold: 3
          # livenessProbe restarts the container if it deadlocks
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 15
            periodSeconds: 10
            failureThreshold: 3
          lifecycle:
            # preStop gives in-flight requests time to drain before SIGTERM
            preStop:
              exec:
                command: ["/bin/sh", "-c", "sleep 5"]
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: cache
              mountPath: /app/cache
      volumes:
        - name: tmp
          emptyDir: {}
        - name: cache
          emptyDir: {}
      terminationGracePeriodSeconds: 60

How to Prepare for a Kubernetes Interview

The most effective preparation schedule for two weeks: spend the first week on the Kubernetes documentation for each component (not tutorials — the actual reference docs). Read the architecture page, the concepts section for each topic, and the API reference for the resources you use most. For the second week, set up a kind cluster and break things intentionally. Delete a DaemonSet pod and watch it come back. Apply a wrong NetworkPolicy and debug why your curl fails. Create a Service with a wrong selector and fix it. Create a Deployment with no readinessProbe and observe what happens during a rolling update.

Study resources worth your time: the official Kubernetes documentation (always current), the CNCF Kubernetes YouTube channel (kubecon talks on specific components), the "Kubernetes: Up and Running" book (O'Reilly), and for advanced topics, the Kubernetes source code itself. The kubernetes/kubernetes GitHub repository is surprisingly readable for learning how controllers work. For certifications, CKA (Certified Kubernetes Administrator) validates operational knowledge and is performance-based — it is taken with a real cluster, not multiple choice.

Common Interview Formats

Whiteboard / virtual screen share: You are asked to draw architecture diagrams or walk through request flows. Practice narrating your diagrams out loud. Interviewers are assessing how you think, not just whether you get the right answer. Talk through assumptions and tradeoffs.

Scenario-based / incident simulation: "Your users are reporting 50x errors. Here is the output of these commands. What's wrong?" These test systematic debugging. Start by reading all the provided output before jumping to conclusions. State your hypothesis, then describe what you would check to confirm or disprove it.

Take-home technical assessment: Deploy a multi-tier application, secure it, make it observable. Read the requirements twice before writing any YAML. Reviewers look for: naming conventions, resource requests/limits set, probes defined, non-root execution, comments explaining non-obvious choices. Submit a README explaining your design decisions.

Live coding / hands-on cluster: You are given a broken cluster or a task to complete with real kubectl access. Type carefully, use --dry-run=client -o yaml before applying, and narrate what you are doing.

Red Flags Interviewers Watch For

• Describing Kubernetes features without being able to explain the failure mode. "I use Deployments for my services" is worth nothing without knowing what happens when maxUnavailable is misconfigured.
• Knowing Docker well but not knowing the container runtime separation (CRI) in Kubernetes. This signals someone who has used containers but hasn't operated Kubernetes at depth.
• Using kubectl exec as the first debugging step. Experienced engineers read events and logs first. Executing into containers is a late-stage debugging tool, not a first response.
• Not knowing that NetworkPolicy does nothing without a CNI that supports it. This is a common and dangerous security misunderstanding.
• Answering questions about managed Kubernetes (EKS, GKE) only. Interviewers at companies running self-managed clusters will probe for whether you understand what managed services abstract away.
• Not having opinions. Senior engineers have opinions about tradeoffs: "I prefer Calico over Cilium for this use case because..." Candidates who describe every tool as "it depends" without reasoning signal lack of hands-on experience.

How to Answer "I Don't Know"

The best answer to a question you don't know is not silence or a wrong guess. Say: "I haven't worked with that specific feature directly, but based on how Kubernetes handles similar problems, I would expect it to work like..." This demonstrates reasoning ability and intellectual honesty — both more valuable than memorized answers. Interviewers hire people they want to work with. Saying "I don't know, but here's how I'd find out" is a strong answer. Guessing confidently and being wrong is the worst outcome.

For operational questions about unfamiliar tools, describe your methodology: "I'd start with the documentation, then check GitHub issues for similar problems, then add verbose logging and trace the request path." This shows that your debugging process transfers to any system, even unfamiliar ones.