Core concepts · Packaging

Packaging Apps with Helm and Kustomize

Real apps need the same manifests in several environments, with small differences, and most cluster software ships as a package. How Kustomize layers environment changes over a shared base, how Helm installs and upgrades packaged charts, and when to use each.

Module: 09 of 20 Level: Core concepts · Packaging Time: 11 min read Published:
All modules in this track (20)

In this module, you'll learn to:

  • Keep one app's manifests consistent across environments with Kustomize overlays
  • Install, configure, upgrade, and roll back packaged software with Helm
  • Decide which tool fits which job

Before you start: this builds on the declarative kubectl apply workflow from Reading Kubernetes YAML. Have the kind cluster running, and install Helm (brew install helm, winget install Helm.Helm, or the script on helm.sh).

Principle

One app, many environments, no copy-paste

So far each app has been a few YAML files. Two problems show up quickly in real work:

  • Environments differ a little. Dev runs one replica, Production runs four; the image tag, namespace, and resource sizes change. Copying the whole folder per environment works until someone fixes a bug in one copy and forgets the others.
  • Other people's software comes as a bundle. An ingress controller or a monitoring stack is dozens of objects with hundreds of settings. You want to install a version, change a few settings, and upgrade it later, without editing their YAML.

Kustomize solves the first problem. Helm solves both, in a different way. You'll meet each of them throughout the rest of this track.

Kustomize

A shared base, plus small patches per environment

Kustomize is built into kubectl. You keep one base with the full manifests, and an overlay per environment that lists only what's different. There's no templating language: every file stays valid Kubernetes YAML.

hello/
├── base/
│   ├── deployment.yaml        # the Deployment from module 03
│   ├── service.yaml
│   └── kustomization.yaml
└── overlays/
    ├── dev/
    │   └── kustomization.yaml
    └── prod/
        └── kustomization.yaml
Scroll sideways to see the full diagram →
# base/kustomization.yaml: which files make up the app
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml
Scroll sideways to see the full diagram →
# overlays/prod/kustomization.yaml: only what's different in Production
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../base
namespace: shop-prod
images:
  - name: nginx
    newTag: "1.27.2"           # pin an exact version in Production
patches:
  - target:
      kind: Deployment
      name: hello
    patch: |-
      - op: replace
        path: /spec/replicas
        value: 4
Scroll sideways to see the full diagram →
# overlays/dev/kustomization.yaml: the base as it is, in its own namespace
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../base
namespace: shop-dev
Scroll sideways to see the full diagram →
# Print the final YAML for an environment, without applying it
kubectl kustomize overlays/prod

# Apply an environment (-k instead of -f)
kubectl create namespace shop-prod
kubectl apply -k overlays/prod
Scroll sideways to see the full diagram →
Helm

A package manager for Kubernetes

Helm works like a package manager (apt, brew, npm) for Kubernetes. A chart is a package: a folder of templated manifests plus a values.yaml file of settings with sensible defaults. Installing a chart creates a release, which Helm tracks so you can upgrade it, roll it back, or uninstall it as one unit.

# Inside a chart: templates/deployment.yaml (an excerpt)
spec:
  replicas: {{ .Values.replicaCount }}
  template:
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
Scroll sideways to see the full diagram →
# my-values.yaml: your settings, overriding the chart's defaults
replicaCount: 2
Scroll sideways to see the full diagram →
# Add a chart repository and look at what it offers
helm repo add podinfo https://stefanprodan.github.io/podinfo
helm repo update
helm search repo podinfo --versions | head

# See every setting the chart supports
helm show values podinfo/podinfo

# Install a specific version, with your values
helm install my-podinfo podinfo/podinfo --version <chart-version> -f my-values.yaml

# Change a setting, see the release history, roll back, and uninstall
helm upgrade my-podinfo podinfo/podinfo --version <chart-version> --set replicaCount=3
helm history my-podinfo
helm rollback my-podinfo 1
helm uninstall my-podinfo

# Render the YAML without installing, to review what a chart will create
helm template my-podinfo podinfo/podinfo -f my-values.yaml
Scroll sideways to see the full diagram →
Choosing

Helm, Kustomize, or both

KustomizeHelm
How it worksPatches plain YAMLFills in templates from values
Installed withBuilt into kubectlA separate CLI
Best forYour own apps across environmentsInstalling and upgrading third-party software
Tracks releases and rollbacksNo; Git history does thatYes, per release
Learning curveLow: it's still just YAMLHigher: Go templating for chart authors

A common split, and the one used later in this track: Helm for third-party add-ons (load balancer controller, ExternalDNS, metrics) in The Platform Add-on Layer, and Kustomize overlays (or Helm values files) for your own apps, applied by Argo CD in GitOps with Argo CD. Argo CD understands both natively.

Try it

One app, two environments

# Build the hello/ folder above, reusing deployment.yaml and service.yaml
# from module 03, then compare the two environments
kubectl kustomize hello/overlays/dev
kubectl kustomize hello/overlays/prod

# Apply both side by side on kind
kubectl create namespace shop-dev
kubectl create namespace shop-prod
kubectl apply -k hello/overlays/dev
kubectl apply -k hello/overlays/prod
kubectl get deployments -A -l app=hello

# Clean up
kubectl delete -k hello/overlays/dev
kubectl delete -k hello/overlays/prod
Scroll sideways to see the full diagram →
Implementation notes
  • Always pin chart versions. helm install without --version installs whatever is newest today, so two environments installed a week apart can differ. The same lesson appears in The Platform Add-on Layer.
  • Keep your values files in Git. --set on the command line is fine in a lab, but the settings then exist only in Helm's release history.
  • Review what a chart creates before installing it. helm template shows every object, including cluster-wide roles and permissions some charts ask for.
  • Don't mix tools on the same objects. If Helm installed something, change it with Helm. A kubectl edit on a Helm-managed object isn't in the chart or the values, so nobody can see it was made, and a later upgrade or rollback may silently overwrite it.
Recap

Key terms

  • Kustomize: builds per-environment YAML from a shared base plus patches; built into kubectl.
  • Base and overlay: the shared manifests, and one environment's differences from them.
  • Helm chart: a package of templated manifests with default settings.
  • Values: the settings you pass to a chart to override its defaults.
  • Release: one installed copy of a chart, with a revision history you can roll back.
Check yourself

Pop quiz

Five questions: three on the ideas in this module, and two scenarios where you apply them. The order changes every time you take it, and 4 out of 5 passes.

5 questions. Decide on your answer to each, then check it.

  1. Scenario

    Dev and Production differ only in replica count, namespace, and image tag. What's the Kustomize way to handle that?

    1. Copy the whole folder for each environment
    2. One base, plus a small overlay per environment listing only those differences
    3. A separate Helm chart per environment
    4. One file with if-statements for each environment
    Show answer

    One base, plus a small overlay per environment listing only those differences

    Overlays keep the shared manifests in one place, so a fix in the base reaches every environment, and each overlay stays a small, readable diff.

  2. Concept

    What does helm install without --version install?

    1. The version you installed last time
    2. Whatever the newest chart version is right now
    3. The version pinned in the cluster
    4. Nothing; the flag is required
    Show answer

    Whatever the newest chart version is right now

    Without --version you get today's newest chart, so two environments installed a week apart can differ. Always pin chart versions.

  3. Concept

    Before installing a chart, you want to see every object it will create, including cluster-wide permissions. Which command?

    1. helm list
    2. helm history
    3. helm template
    4. helm rollback
    Show answer

    helm template

    helm template renders the chart to plain YAML without touching the cluster, so you can review it first.

  4. Scenario

    Someone fixes a Helm-managed Deployment with kubectl edit during an incident. What's the problem with leaving it that way?

    1. Nothing; Helm always keeps manual edits
    2. The fix isn't in the chart or values, so it's invisible, and a later upgrade or rollback may overwrite it
    3. Helm deletes the Deployment at the next upgrade
    4. The Deployment stops being managed by Helm
    Show answer

    The fix isn't in the chart or values, so it's invisible, and a later upgrade or rollback may overwrite it

    Helm merges each upgrade with the live state, so an edit can survive one upgrade and be lost on the next, or on a rollback. Put the fix into the values file, so it's recorded and reviewed.

  5. Concept

    Which split does this track use?

    1. Kustomize for everything
    2. Helm for your own apps, Kustomize for add-ons
    3. Helm for third-party add-ons; overlays or values files for your own apps
    4. Neither; plain kubectl apply only
    Show answer

    Helm for third-party add-ons; overlays or values files for your own apps

    Helm is good at installing and upgrading other people's software. Your own apps fit Kustomize overlays or per-environment Helm values, applied by Argo CD.

Questions about this learning path?

I'm happy to go into more detail on any of this: how the pieces fit together, ingress costs, RBAC design, or how I triage incidents.

anupojuprudhvi@gmail.com