Structuring Per-Environment Configuration in Pulumi

Open Pulumi.dev.yaml and Pulumi.prod.yaml side by side on a project that has been running for a year and you will usually find them differing in twenty-odd keys, of which perhaps four represent a real decision. The rest is accumulated noise: a key copied to production and never removed, a value that drifted, a flag someone added to dev while debugging. This guide sits under Pulumi secrets and configuration within Pulumi patterns and provider management, and it is about file layout rather than accessors — deciding which line belongs in which file so that the diff between two environments is short enough to review and honest about what actually varies.

Context

Pulumi resolves a configuration key from at most three places, and the order is fixed. A --config flag on the command line beats everything for that one invocation. Failing that, the value in Pulumi.<stack>.yaml wins. Failing that, a default: declared in the config: block of Pulumi.yaml applies. There is no fourth layer, no Pulumi.base.yaml, and no mechanism by which one stack file inherits from another. Every apparent inheritance scheme you will see in the wild is one of those three layers used deliberately, or a Python data structure doing the work outside of Pulumi entirely.

That constraint drives the whole design. Because there is no inheritance between stack files, anything you write into one stack file must be written into all of them or it is an environment-specific decision by definition. The question for every key therefore becomes: is this genuinely per-environment, or is it a project-wide value that got copied five times?

Two categories of key survive that test. The first is identity: facts that describe where this deployment lives — the region, the AWS account, the VPC CIDR block, the DNS zone, the ciphertext of the environment's own database password. These have no sensible default and differ by definition. The second is tier: a single word — dev, staging, prod — that names the operational posture, from which every sizing decision can be derived in code.

Everything else is neither, and the most common failure of a Pulumi project is treating sizing as identity. Once instanceClass, minCapacity, maxCapacity, logRetentionDays, multiAz, deletionProtection and backupRetentionDays are all individually settable per stack, the stack files become the place where the environments silently diverge. Nobody reviews the fact that staging quietly lost multi-AZ eight months ago, because the key was always there and the value just changed.

The mirror-image mistake lives in the program: if pulumi.get_stack() == "prod": scattered through the resource code. That branching moves the environment differences into Python, which sounds better, except the differences are now spread across every module that needed one. Answering "what is different about production?" means reading the whole program, and adding a fourth environment means finding every branch.

Prerequisites

• A Pulumi project with at least two stacks initialised and pulumi>=3.0 in the virtualenv.
• jq on the workstation, used below to diff configuration between stacks.
• Write access to Pulumi.yaml and the discipline to put the project file under review — it is now a schema, not a scratchpad.
• A typed settings loader in the program, of the kind described in using Pulumi config and typed settings in Python. This guide decides what goes into the files; that one decides how the values arrive in Python.

# CLI: see how far apart two environments have drifted before changing anything
pulumi config --stack dev --json | jq -r 'keys[]' > /tmp/dev.keys
pulumi config --stack prod --json | jq -r 'keys[]' > /tmp/prod.keys
diff /tmp/dev.keys /tmp/prod.keys

Implementation

Step 1 — Make Pulumi.yaml the schema, not a dumping ground

The config: block of the project file declares every key the program understands, with a type, an optional default, and a secret flag. Declared keys are validated by the CLI before the language host starts, so a value of notanumber for an integer key fails at pulumi preview with error: stack configuration is invalid rather than deep inside your program. Anything absent from this block is undocumented by construction.

# Pulumi.yaml — the schema every stack file is checked against
# CLI: pulumi preview --stack dev   # validates types and applies defaults
name: platform
runtime: python
description: Application platform, one stack per environment

config:
  platform:tier:
    type: string          # dev | staging | prod — drives everything sized
  platform:vpcCidr:
    type: string          # identity: differs per environment, no default
  platform:dnsZone:
    type: string
  platform:adminCidrs:
    type: array
    items:
      type: string
    default: []
  platform:dbPassword:
    type: string
    secret: true
  platform:featureFlags:
    type: array
    items:
      type: string
    default: []

Note what is missing: there is no instanceClass, no multiAz, no logRetentionDays. Those are consequences of tier, and giving them keys would give someone the ability to set them independently.

Step 2 — Put only identity in each stack file

With the schema in place, a stack file collapses to the handful of lines that genuinely describe one environment. This is the single biggest change in reviewability: a nine-line Pulumi.prod.yaml can be read completely during a pull request, and a new line in it is conspicuous.

# Pulumi.dev.yaml — identity only, everything else derives from tier
# CLI: pulumi config set vpcCidr 10.10.0.0/16 --stack dev
config:
  aws:region: eu-west-1
  platform:tier: dev
  platform:vpcCidr: 10.10.0.0/16
  platform:dnsZone: dev.example.internal
  platform:adminCidrs:
    - 10.10.0.0/16
  platform:dbPassword:
    secure: AAABANlx3vQ0dEV0YnJhbmNoOnNlY3JldA==

# Pulumi.prod.yaml — the same shape; the diff against dev is five values
# CLI: pulumi config set vpcCidr 10.60.0.0/16 --stack prod
config:
  aws:region: eu-west-1
  platform:tier: prod
  platform:vpcCidr: 10.60.0.0/16
  platform:dnsZone: example.com
  platform:adminCidrs:
    - 10.60.0.0/16
    - 198.51.100.0/24
  platform:dbPassword:
    secure: AAABAKm2R7yTz0YtcHJvZDpzZWNyZXQ=

Both files carry the same key set. That is the invariant the verification step below enforces mechanically: a key present in one stack and absent from another is either a bug or a decision that needs writing down.

Step 3 — Derive sizing from the tier with one profile table

The environment differences that are not identity now live in exactly one Python module, as a mapping from tier name to a frozen profile. This is the object that replaces every if stack == "prod" branch, and its virtue is that the entire operational difference between environments fits on one screen and diffs cleanly when it changes.

# profiles.py — every sized difference between environments, in one table
# CLI: python -c "import profiles; print(profiles.for_tier('prod'))"
from __future__ import annotations

from dataclasses import dataclass
from typing import Mapping, Tuple


@dataclass(frozen=True)
class EnvironmentProfile:
    tier: str
    db_instance_class: str
    db_multi_az: bool
    db_backup_retention_days: int
    min_capacity: int
    max_capacity: int
    log_retention_days: int
    deletion_protection: bool
    single_nat_gateway: bool
    availability_zones: Tuple[str, ...]


_PROFILES: Mapping[str, EnvironmentProfile] = {
    "dev": EnvironmentProfile(
        tier="dev", db_instance_class="db.t4g.micro", db_multi_az=False,
        db_backup_retention_days=1, min_capacity=1, max_capacity=2,
        log_retention_days=7, deletion_protection=False,
        single_nat_gateway=True, availability_zones=("a",),
    ),
    "staging": EnvironmentProfile(
        tier="staging", db_instance_class="db.t4g.medium", db_multi_az=True,
        db_backup_retention_days=7, min_capacity=2, max_capacity=4,
        log_retention_days=30, deletion_protection=False,
        single_nat_gateway=True, availability_zones=("a", "b"),
    ),
    "prod": EnvironmentProfile(
        tier="prod", db_instance_class="db.r6g.large", db_multi_az=True,
        db_backup_retention_days=30, min_capacity=3, max_capacity=12,
        log_retention_days=365, deletion_protection=True,
        single_nat_gateway=False, availability_zones=("a", "b", "c"),
    ),
}


def for_tier(tier: str) -> EnvironmentProfile:
    try:
        return _PROFILES[tier]
    except KeyError:
        raise ValueError(
            f"platform:tier is '{tier}'; expected one of {', '.join(sorted(_PROFILES))}"
        ) from None

Resource code then reads from the profile and never inspects the stack name at all:

# __main__.py — no stack-name branching anywhere below this point
# CLI: pulumi up --stack prod
from __future__ import annotations

import pulumi
import pulumi_aws as aws

import profiles

cfg = pulumi.Config()
profile = profiles.for_tier(cfg.require("tier"))
region = pulumi.Config("aws").require("region")

vpc = aws.ec2.Vpc("platform", cidr_block=cfg.require("vpcCidr"), enable_dns_hostnames=True)

subnets = [
    aws.ec2.Subnet(
        f"platform-{az}",
        vpc_id=vpc.id,
        availability_zone=f"{region}{az}",
        cidr_block=f"10.{60 if profile.tier == 'prod' else 10}.{i}.0/24",
    )
    for i, az in enumerate(profile.availability_zones)
]
# State implication: shrinking availability_zones for a tier deletes subnets on the
# next update — change the profile table only with a preview in front of you.

db = aws.rds.Instance(
    "platform",
    engine="postgres",
    engine_version="16.3",
    instance_class=profile.db_instance_class,
    multi_az=profile.db_multi_az,
    backup_retention_period=profile.db_backup_retention_days,
    deletion_protection=profile.deletion_protection,
    allocated_storage=100,
    username="platform",
    password=cfg.require_secret("dbPassword"),
    skip_final_snapshot=not profile.deletion_protection,
)

The one remaining conditional — the CIDR octet in the subnet loop — is a signal that the value should have been an identity key. Moving it to platform:subnetOctet in each stack file removes the last branch.

Step 4 — Choose a deliberate mechanism for shared values

Three keys genuinely want the same value everywhere: the tagging owner, the log shipping endpoint, the organisation name. Repeating them across five stack files means five places to update. Pulumi offers three honest ways to avoid that, and picking one per project matters more than which one you pick.

A default: in Pulumi.yaml is the simplest and works for anything non-secret whose value is stable. pulumi config cp --dest <stack> copies a key from the current stack to another and is a one-time operation, useful when standing up a new environment from an existing one. Pulumi ESC imports a named environment into a stack file with a top-level environment: list, which is the right answer for shared credentials that several projects need. What does not work is a Pulumi.common.yaml — the CLI will not read it, and any tooling that merges it into the real stack files reintroduces the drift problem one layer up.

# CLI: stand up a new environment from an existing one, then trim it
pulumi stack init staging-eu --copy-config-from staging
pulumi config rm dbPassword --stack staging-eu   # never inherit a live secret
pulumi config set vpcCidr 10.40.0.0/16 --stack staging-eu
pulumi config set dnsZone eu.staging.example.internal --stack staging-eu

Verification

The property worth asserting is narrow and mechanical: two stacks must carry the same key set, and differ only in keys you have declared as identity. Encode it as a script that runs in CI on every change to a stack file.

# CLI: prove dev and prod differ only in the keys that are allowed to differ
IDENTITY='^(aws:region|platform:tier|platform:vpcCidr|platform:dnsZone|platform:adminCidrs|platform:dbPassword)$'

keyset() { pulumi config --stack "$1" --json | jq -r 'keys[]' | sort; }
diff <(keyset dev) <(keyset prod) && echo "key sets match"

pulumi config --stack dev  --json | jq -r 'to_entries[] | .key' | grep -vE "$IDENTITY" \
  | while read -r k; do
      a=$(pulumi config get "$k" --stack dev)
      b=$(pulumi config get "$k" --stack prod)
      [ "$a" = "$b" ] || echo "UNEXPECTED DIVERGENCE: $k  dev=$a prod=$b"
    done

The second check belongs in the test suite and guards the profile table rather than the files: production must never be able to acquire a development posture through a typo in one field.

# tests/test_profiles.py — invariants the profile table must satisfy
# CLI: pytest tests/test_profiles.py -q
from __future__ import annotations

import pytest

import profiles


def test_every_tier_is_loadable() -> None:
    for tier in ("dev", "staging", "prod"):
        assert profiles.for_tier(tier).tier == tier


def test_unknown_tier_names_the_valid_values() -> None:
    with pytest.raises(ValueError, match="expected one of dev, prod, staging"):
        profiles.for_tier("production")


def test_prod_posture_is_stricter_than_staging() -> None:
    prod, staging = profiles.for_tier("prod"), profiles.for_tier("staging")
    assert prod.deletion_protection and not staging.deletion_protection
    assert prod.db_backup_retention_days > staging.db_backup_retention_days
    assert prod.log_retention_days >= staging.log_retention_days
    assert len(prod.availability_zones) >= len(staging.availability_zones)
    assert not prod.single_nat_gateway

Finally, confirm the files themselves survive a round trip. Run pulumi config set on a scratch stack and inspect the resulting YAML — the CLI rewrites the whole file, and knowing exactly what it does to formatting saves an argument in code review later.

Gotchas & Edge Cases

pulumi config set destroys comments. The CLI parses the stack file, mutates the structure, and re-serialises it. Every comment explaining why production uses a wider admin range is gone after the next set. Put that reasoning in the project file, in the profile table, or in the commit message — anywhere the CLI does not rewrite.

The selected stack is sticky and invisible. pulumi config set adminCidrs ... with no --stack writes to whichever stack pulumi stack select last chose, which may have been production two days ago. Make --stack mandatory by habit, and treat any stack-file change touching more than one file in a commit as suspicious.

--copy-config-from copies secrets too. pulumi stack init dev2 --copy-config-from prod decrypts the production secrets and re-encrypts them into the new stack, which now holds live production credentials behind whatever access control a scratch stack has. Remove them immediately, and prefer copying from the lowest environment rather than the highest.

encryptionsalt churns the diff. Every stack file created with the passphrase provider carries an encryptionsalt: line, and re-keying rewrites it along with every ciphertext. A pull request that appears to change all six secrets may have changed none of them. Check whether the salt moved before reading anything into a wall of base64.

Ciphertext never diffs usefully. Two encryptions of the same plaintext differ, because each carries its own nonce. A reviewer cannot tell from the file whether a secret actually changed, which is why "the value changed" belongs in the pull request description and the rotation procedure belongs in a runbook.

Array keys are awkward from the CLI. pulumi config set adminCidrs '["10.0.0.0/8"]' stores a string, not a list. Use pulumi config set --path 'adminCidrs[0]' 10.0.0.0/8, which creates or extends a real array, and verify with pulumi config get adminCidrs.

A default in Pulumi.yaml is invisible in the stack file. Someone reading Pulumi.prod.yaml alone sees no featureFlags key and concludes none are set, while the project default supplies []. Neither view is complete; pulumi config --stack prod shows the resolved values and is the thing to paste into a review.

Operational Notes

Give the production stack file its own reviewers. A CODEOWNERS entry on Pulumi.prod.yaml costs nothing and means the four lines that define production cannot change without the people who carry the pager seeing it. The profile table deserves the same treatment, since it now holds the sizing decisions the stack file used to.

Promote configuration changes in the same direction as code. A new identity key is added to Pulumi.dev.yaml first, exercised by a real pulumi up, then added to staging and production in a change that also removes the temporary default from Pulumi.yaml if one was used to bridge the gap. Adding a required key to all stacks simultaneously means the first person to run an unrelated update in any environment discovers the problem.

Keep the tier vocabulary closed and small. Three tiers with clear postures beat seven with subtle differences, and a per-customer or per-region deployment is better expressed as more stacks sharing a tier than as more tiers. When a stack genuinely needs a one-off deviation — a load test environment sized like production but without deletion protection — add an explicit override field to the profile rather than a new tier, and make the override name what it is.

Watch for keys that exist only in one stack. They are almost always debris from an experiment, and they are the mechanism by which the stack files stop being comparable. The key-set diff in the verification section catches them the day they appear rather than a year later, which is the whole point of running it in CI rather than by hand.

If several projects share an environment's identity — the same account, region and VPC feed a network project and an application project — publish those facts as stack outputs and consume them with StackReference rather than duplicating them in both projects' stack files. The boundaries that make that safe are covered in structuring Pulumi stacks per environment.

FAQ

Should Pulumi.<stack>.yaml be committed to the repository?

Yes, including the production one. Secret values in it are ciphertext, and the plaintext values are precisely the environment description you want reviewed. A stack file kept out of version control means the only record of what production is configured with lives in the backend, where nobody reviews it.

Can one stack file inherit from another?

No, and building that yourself with a merge script is a trap — the file the CLI reads stops being the file people review. Use Pulumi.yaml defaults for shared static values, a Python profile table for derived sizing, and Pulumi ESC when several projects need the same shared credentials.

How many keys should a stack file have?

Roughly five to ten. If a stack file needs thirty keys, most of them are almost certainly sizing that should have been derived from a tier, or facts about a neighbouring stack that should have come through a StackReference.

What if staging genuinely needs a production-sized database for a week?

Add a nullable override field to the profile — db_instance_class_override: Optional[str] fed by an optional config key — and set it on the staging stack with an expiry noted in the change. That leaves one visible line in one stack file instead of a temporary edit to the shared table that someone forgets to revert.

How do I stop a value from being set on the wrong stack?

You cannot prevent it at the CLI, so catch it in the program and in CI. Validate at startup that the tier matches an expectation encoded in the stack name, and run the key-set diff on every pull request that touches a Pulumi.*.yaml file.

Does the project file's type: field actually enforce anything?

It does. The CLI validates stack values against the declared types before the language host starts, so an integer key set to "large" fails with a configuration error rather than surfacing as a Python ValueError mid-update. Declaring types costs one line per key and moves a whole class of error earlier.

Related

• Pulumi Secrets and Configuration in Python — the parent topic covering how configuration and secret encryption work end to end.
• Using Pulumi Config and Typed Settings in Python — how the program reads the files this guide lays out.
• Structuring Pulumi Stacks per Environment — the stack, state and blast-radius boundaries these files sit inside.
• Managing Multi-Account AWS Environments with Pulumi Python — when each environment's identity includes a separate cloud account.