Scaling Without Surprises: Requests, Autoscaling & Cost

In this module, you'll learn to:

• Explain why resource requests drive scheduling, autoscaling, and cost
• Combine pod autoscaling (HPA) with node autoscaling (Cluster Autoscaler or Karpenter)
• Protect workloads during scale-down with PodDisruptionBudgets

Before you start: you'll want metrics-server running in the cluster (Platform Add-ons covers installing it) and kubectl top working.

Requests are a promise, and the cluster plans around them

Every scaling decision in Kubernetes starts from resource requests. The scheduler places a pod on a node only if the node has room for what the pod requests, not what it actually uses. Node autoscalers add nodes when pods can't be placed. The Horizontal Pod Autoscaler measures CPU as a percentage of the request.

So when requests are wrong, everything built on top of them is wrong too. Set them too high and you pay for nodes that sit half-empty. Set them too low, or leave them out, and pods get packed onto nodes that can't actually carry them.

Pods, Deployments & Rollouts explains requests and limits in full. The short version: requests reserve capacity, a CPU limit throttles, and a memory limit kills.

Two layers of autoscaling

It helps to keep the two layers apart in your head:

• Pods scale out with the Horizontal Pod Autoscaler (HPA). It adds or removes replicas based on a metric, usually CPU or memory usage relative to the request.
• Nodes scale out with Cluster Autoscaler or Karpenter. They add capacity when pods are stuck in Pending because no node has room, and remove nodes that sit mostly idle.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70   # 70% of the CPU *request*, not of the node

The HPA needs metrics-server (or another metrics source) running in the cluster, and it needs requests set on the pods it watches. Without a CPU request there's nothing to calculate a percentage against, and it won't scale on CPU.

Here's how the two layers work together when traffic spikes:

Keeping scale-down from hurting you

Scaling in is where outages hide. When a node autoscaler removes a node, or a Spot instance is reclaimed (AWS gives a two-minute warning), the pods on it are evicted. A PodDisruptionBudget tells Kubernetes how many replicas must stay up during that kind of voluntary disruption.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: api

A PDB only protects against voluntary disruptions: drains, scale-downs, upgrades. It does nothing for a crash or a node that dies outright. For that you still need more than one replica, spread across Availability Zones.

Key terms

• HPA: the Horizontal Pod Autoscaler, which changes replica counts based on a metric.
• Cluster Autoscaler: grows and shrinks node groups when pods can't be placed or nodes sit idle.
• Karpenter: launches right-sized EC2 instances directly for waiting pods.
• PodDisruptionBudget (PDB): how many replicas must stay up during voluntary disruptions.
• Throttling: slowing down a container that goes over its CPU limit.

Pop quiz

Five questions: three on the ideas in this module, and two scenarios where you apply them. The order changes every time you take it, and 4 out of 5 passes.