Platform · Access

Multi-Environment Clusters & EKS Access Entries

Isolating Dev, QA, Staging, and Production as separate clusters, and granting people and pipelines cluster access with EKS access entries instead of a shared superuser.

Module: 11 of 20 Level: Platform · Access Time: 10 min read Updated:
All modules in this track (20)

In this module, you'll learn to:

  • Decide when environments need separate clusters rather than namespaces
  • Grant people and pipelines access with EKS access entries and access policies
  • Move off the deprecated aws-auth ConfigMap, and switch contexts safely

Before you start: you'll want the AWS CLI and kubectl, permission to manage EKS clusters, the RBAC basics from Namespaces, RBAC & Service Accounts, and how access entries work from Kubernetes on Amazon EKS. This module takes those ideas across several clusters.

Principle

Separate clusters, not separate namespaces, for hard environment boundaries

Namespaces are a good way to organize workloads inside one cluster, but they're a soft boundary. A misconfigured RBAC role or a cluster-wide resource (a CRD, a webhook, a node setting) can still reach across them. When environments carry very different risks, such as a development sandbox versus a production cluster serving live traffic, a real boundary means separate clusters, each with its own control plane, node groups, and IAM trust.

The deployment this track is based on ran one EKS cluster each for Development, QA, Staging, and Production. Engineers connected to whichever cluster their task needed, rather than to one shared, always-on context.

Mechanism

IAM identities become Kubernetes identities through access entries

EKS doesn't keep its own list of users. Signing in to a cluster is IAM authentication, and access entries decide what each IAM role can do once it's in. You create an access entry for a role, then attach an access policy to it, either for the whole cluster or for specific namespaces.

One identity provider, separate access in every cluster
Engineer or CI pipelinesigns in through SSO, or OIDC for pipelines
assumes an IAM role, such as platform-admin or ci-deploy
Dev cluster
Access entry: platform-admincluster admin
Access entry: ci-deployedit, in app namespaces
Production cluster
Access entry: platform-admincluster admin, for a smaller group
Access entry: ci-deployedit, in the orders namespace only
each cluster decides on its own; admin in Dev grants nothing in Production
# Let the platform team's SSO role administer the whole cluster
aws eks create-access-entry --cluster-name prod \
  --principal-arn arn:aws:iam::<your-account-number>:role/<platform-admin-role>

aws eks associate-access-policy --cluster-name prod \
  --principal-arn arn:aws:iam::<your-account-number>:role/<platform-admin-role> \
  --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy \
  --access-scope type=cluster

# Let the CI/CD deploy role manage workloads in one namespace only
aws eks create-access-entry --cluster-name prod \
  --principal-arn arn:aws:iam::<your-account-number>:role/<ci-deploy-role>

aws eks associate-access-policy --cluster-name prod \
  --principal-arn arn:aws:iam::<your-account-number>:role/<ci-deploy-role> \
  --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSEditPolicy \
  --access-scope type=namespace,namespaces=orders
Scroll sideways to see the full diagram →

Access entries are ordinary AWS API calls, so they show up in CloudTrail, can be managed in Terraform, and can't be broken by a typo in a YAML file.

What about aws-auth?

Older clusters, including the one this track is based on, used the aws-auth ConfigMap in kube-system to map IAM roles to Kubernetes groups. AWS has deprecated it in favour of access entries. If you still have it, switch the cluster's authentication mode to API_AND_CONFIG_MAP, recreate each mapping as an access entry, check everyone can still get in, and then move to API only.

aws eks update-cluster-config --name prod \
  --access-config authenticationMode=API_AND_CONFIG_MAP
Scroll sideways to see the full diagram →
Implementation notes
  • Grant access to roles, not IAM users. People should sign in through SSO and assume a role. That way, removing someone from the identity provider removes their cluster access too, and there are no long-lived access keys to leak.
  • Give cluster admin on purpose, per environment. Someone who is admin in Development shouldn't automatically be admin in Production. Each cluster has its own access entries, which is exactly the point of separate clusters.
  • Avoid system:masters. Membership in that group bypasses RBAC entirely and can't be taken away by an RBAC change. The access policies above do the same job and can be revoked.
  • Treat access changes like IAM policy changes. They decide who can reach the cluster at all, so review them the same way. A mistake can lock everyone out, or let the wrong people in.

Switching context safely between environments

# Point kubectl at a specific cluster
aws eks --region <region> update-kubeconfig --name <cluster-name>

# Check which cluster and namespace you're actually pointed at
kubectl config current-context
kubectl config set-context --current --namespace=<namespace>
Scroll sideways to see the full diagram →

Running kubectl config current-context before anything destructive is a cheap habit that prevents the most common multi-cluster mistake: running a command meant for QA against Production because a terminal tab was left on the wrong context earlier in the day.

Trade-offs

Separate clusters per environment cost more than namespaces on one shared cluster, since each has its own control plane and minimum set of nodes. When a mistake in Development must never be able to reach Production data or traffic, that's a deliberate trade, not waste. For lower-stakes workloads, namespaces on fewer clusters are a reasonable, cheaper option.

Recap

Key terms

  • Access policy: an EKS-managed permission set, such as cluster admin or edit, attached to an access entry.
  • Access scope: whether an access policy applies to the whole cluster or to named namespaces.
  • aws-auth: the older, deprecated ConfigMap that mapped IAM roles to Kubernetes groups.
  • Authentication mode: whether a cluster reads access entries (API), aws-auth, or both.
  • system:masters: a group that bypasses RBAC entirely; avoid it.
Check yourself

Pop quiz

Five questions: three on the ideas in this module, and two scenarios where you apply them. The order changes every time you take it, and 4 out of 5 passes.

5 questions. Decide on your answer to each, then check it.

  1. Concept

    Why give each environment its own cluster, rather than a namespace on one shared cluster?

    1. Namespaces can't run more than one app
    2. Cluster-wide resources and permissions reach across namespaces, so only separate clusters give a hard boundary
    3. Separate clusters are cheaper
    4. EKS allows only one namespace per cluster
    Show answer

    Cluster-wide resources and permissions reach across namespaces, so only separate clusters give a hard boundary

    CRDs, webhooks, node settings, and cluster-wide roles span every namespace. When a mistake in Dev must never reach Production, the boundary has to be a cluster.

  2. Scenario

    A CI pipeline should deploy only to the orders namespace in Production. What's the right access entry setup?

    1. Add the pipeline's role to system:masters
    2. Attach AmazonEKSClusterAdminPolicy with a cluster scope
    3. Attach AmazonEKSEditPolicy with a namespace scope of orders
    4. Share an engineer's admin credentials with the pipeline
    Show answer

    Attach AmazonEKSEditPolicy with a namespace scope of orders

    Scope access to exactly what the pipeline deploys. Admin rights for a pipeline turn a compromised build into a compromised cluster.

  3. Scenario

    An engineer is cluster admin in Dev. What access do they have in Production?

    1. The same, because they use the same IAM role
    2. Only what Production's own access entries grant them
    3. Read-only, automatically
    4. None, ever
    Show answer

    Only what Production's own access entries grant them

    Each cluster has its own access entries. Admin in one grants nothing in another, which is exactly the point of separate clusters.

  4. Concept

    What's the safe order for migrating from aws-auth to access entries?

    1. Switch to API mode, then recreate the mappings
    2. Switch to API_AND_CONFIG_MAP, recreate each mapping as an access entry, check access, then switch to API
    3. Delete aws-auth, then create access entries
    4. They can't be migrated; build a new cluster
    Show answer

    Switch to API_AND_CONFIG_MAP, recreate each mapping as an access entry, check access, then switch to API

    Running both modes first means nobody is locked out while you recreate the mappings. Drop aws-auth only once everyone can still get in.

  5. Concept

    What's the cheapest habit that prevents running a QA command against Production?

    1. Running kubectl config current-context before anything destructive
    2. Using the same context name for every cluster
    3. Giving everyone admin in every cluster
    4. Keeping a single kubeconfig entry
    Show answer

    Running kubectl config current-context before anything destructive

    A terminal left on the wrong context is the classic multi-cluster mistake, and checking takes a second.

Questions about this learning path?

I'm happy to go into more detail on any of this: how the pieces fit together, ingress costs, RBAC design, or how I triage incidents.

anupojuprudhvi@gmail.com