Using Pulumi Config and Typed Settings in Python

Python puts no compiler between a misspelled configuration key and a deployment. pulumi.Config().get("instanceClas") returns None, the expression falls through to an or default that nobody remembers writing, and a staging database comes up two instance classes smaller than the one that was budgeted. This guide belongs to Pulumi secrets and configuration inside Pulumi patterns and provider management, and it replaces accessor calls sprinkled through a program with one frozen settings object that is constructed, validated and type-checked before the first resource is declared.

Context

pulumi.Config is a thin reader over a flat, string-keyed map that the CLI hands to the language host at startup. It is deliberately simple: there is no schema, no cross-field validation, and no notion of "this key is only meaningful when that other key is set". Everything beyond is this key present is your job, and the SDK gives you exactly two behaviours to build on — an accessor that returns Optional[T] and one that raises.

The trouble starts when those accessors are called where the value is used rather than where the program starts. A pulumi.Config() object is cheap to construct, so it is tempting to instantiate one inside every module and read keys inline. The result is a program whose configuration contract is discoverable only by grepping for require( and reading each call site to work out whether the key is optional, what its default is, and whether the default lives in Pulumi.yaml or in a Python or expression.

That sprawl has a specific and expensive failure mode: the key is read late. Pulumi builds the resource graph by executing your program top to bottom, and a require() call sitting inside a helper that constructs the twelfth resource does not run until the eleven before it have already been registered. Some of those registrations have already started work in the cloud provider. When the twelfth line raises ConfigMissingError, the update aborts with resources created and a state file that now contains a partial deployment you have to reconcile.

Worse, the misspelling case never raises at all. get() returns None for both "the operator did not set this" and "you typed the key wrong", and those two conditions deserve completely different responses. mypy cannot help, because the key is a string literal — from the type checker's perspective cfg.get("instanceClass") and cfg.get("instnaceClass") are the same well-typed expression returning Optional[str]. Move the same value onto a dataclass field and the picture inverts: settings.instnace_class is an attribute error that mypy --strict reports before the program is ever handed to the engine.

Prerequisites

• Python 3.9 or newer with pulumi>=3.0 in the project virtualenv, plus pulumi-aws>=6.0 for the resource examples below.
• An initialised stack with a Pulumi.yaml you can edit and at least one Pulumi.<stack>.yaml holding real values.
• mypy installed and pointed at the infrastructure package. Typed settings that are never type-checked buy you documentation and nothing else.
• Familiarity with Output semantics — a secret read through this pattern stays wrapped, and the rules for composing wrapped values are the ones described in typing Pulumi component inputs and outputs.

# CLI: confirm which keys the current stack actually carries before refactoring
pulumi config --stack staging
pulumi config get vpcCidr --stack staging
pulumi stack --show-name

Implementation

Step 1 — Know what each accessor does before you wrap it

The Python SDK exposes four families of accessor, and the differences between them are the whole design space. get* returns Optional[T] and never raises for absence. require* raises pulumi.ConfigMissingError with the message Missing required configuration variable '<key>' followed by the exact pulumi config set command to fix it. The *_object variants parse the stored value as structured data — YAML written in the stack file arrives as a dict or list. The *_secret* variants return Output[T] instead of a bare value, which is the mechanism that keeps the secret marking attached.

Namespacing is the other thing to settle now. pulumi.Config() with no argument uses the project name from Pulumi.yaml, so a key written as pulumi config set vpcCidr 10.0.0.0/16 is stored as myapp:vpcCidr. Provider settings live in their own namespace and need their own reader: pulumi.Config("aws").require("region") reads aws:region. Mixing the two through one object is the most common reason a value that is visibly present in pulumi config reads back as None.

# namespaces.py — one reader per namespace, constructed once
# CLI: pulumi preview --stack staging
from __future__ import annotations

import pulumi

app_cfg: pulumi.Config = pulumi.Config()            # namespace = project name, e.g. "myapp"
aws_cfg: pulumi.Config = pulumi.Config("aws")       # provider namespace, e.g. "aws:region"

region: str = aws_cfg.require("region")
# Provider note: reading aws:region does not configure the provider — the engine already
# did that. This read exists so the program can put the region into resource names/tags.

Step 2 — Build one frozen settings object and load it once

Model the program's entire configuration contract as a small tree of frozen dataclasses. Frozen matters: a settings object that can be mutated halfway down the program reintroduces the "where did this value come from?" question the refactor was meant to kill. Use tuples rather than lists so the freeze is real rather than decorative.

# settings.py — the complete configuration contract for this project
# CLI: python -c "import settings; print(settings.AppSettings.load())"
from __future__ import annotations

import ipaddress
from dataclasses import dataclass
from typing import Any, Mapping, Optional, Tuple

import pulumi

VALID_ENVIRONMENTS: Tuple[str, ...] = ("dev", "staging", "prod")


class SettingsError(Exception):
    """Raised at program start when the stack configuration is not usable."""


@dataclass(frozen=True)
class DatabaseSizing:
    instance_class: str
    allocated_storage_gb: int
    multi_az: bool
    backup_retention_days: int

    @staticmethod
    def from_object(raw: Mapping[str, Any]) -> "DatabaseSizing":
        try:
            return DatabaseSizing(
                instance_class=str(raw["instanceClass"]),
                allocated_storage_gb=int(raw["allocatedStorageGb"]),
                multi_az=bool(raw["multiAz"]),
                backup_retention_days=int(raw.get("backupRetentionDays", 7)),
            )
        except (KeyError, TypeError, ValueError) as exc:
            raise SettingsError(f"myapp:database is malformed: {exc}") from exc


@dataclass(frozen=True)
class AppSettings:
    environment: str
    region: str
    vpc_cidr: str
    database: DatabaseSizing
    admin_cidrs: Tuple[str, ...]
    enable_deletion_protection: bool
    db_password: pulumi.Output[str]

    @staticmethod
    def load() -> "AppSettings":
        cfg = pulumi.Config()
        return AppSettings(
            environment=cfg.require("environment"),
            region=pulumi.Config("aws").require("region"),
            vpc_cidr=cfg.require("vpcCidr"),
            database=DatabaseSizing.from_object(cfg.require_object("database")),
            admin_cidrs=tuple(cfg.require_object("adminCidrs")),
            enable_deletion_protection=cfg.get_bool("enableDeletionProtection") or False,
            # State implication: require_secret keeps the value inside an Output, so the
            # engine records it as secret in the checkpoint rather than as plaintext.
            db_password=cfg.require_secret("dbPassword"),
        )

The stack file this reads is ordinary YAML, and require_object is what lets a related group of settings travel together instead of being smeared across four flat keys:

# Pulumi.staging.yaml — structured values keep related settings adjacent
# CLI: pulumi config set --path database.instanceClass db.t4g.medium --stack staging
config:
  aws:region: eu-west-1
  myapp:environment: staging
  myapp:vpcCidr: 10.20.0.0/16
  myapp:adminCidrs:
    - 10.20.0.0/16
    - 192.0.2.0/24
  myapp:database:
    instanceClass: db.t4g.medium
    allocatedStorageGb: 100
    multiAz: false
  myapp:dbPassword:
    secure: AAABAGmH0K7pQx0xMkR0eXBlOnNlY3JldA==

Step 3 — Validate eagerly, on the first line of the program

Loading is not validating. require_object("adminCidrs") will happily hand back ["10.20.0.0/16", "not-a-cidr"], and the resulting aws.ec2.SecurityGroupRule fails during the update with a provider error rather than at parse time. Put the semantic checks in one function, call it before any resource is constructed, and raise a message that names the key and the fix.

# settings.py (continued) — semantic validation with actionable messages
# CLI: pulumi preview --stack staging   # fails in under a second when config is wrong
def validate(settings: AppSettings) -> None:
    problems: list[str] = []

    if settings.environment not in VALID_ENVIRONMENTS:
        problems.append(
            f"myapp:environment is '{settings.environment}'; "
            f"expected one of {', '.join(VALID_ENVIRONMENTS)}"
        )

    try:
        network = ipaddress.ip_network(settings.vpc_cidr, strict=True)
        if network.prefixlen > 20:
            problems.append(f"myapp:vpcCidr {settings.vpc_cidr} is too small to subnet")
    except ValueError as exc:
        problems.append(f"myapp:vpcCidr is not a valid network: {exc}")

    for cidr in settings.admin_cidrs:
        try:
            ipaddress.ip_network(cidr, strict=True)
        except ValueError as exc:
            problems.append(f"myapp:adminCidrs entry '{cidr}' is invalid: {exc}")

    if settings.environment == "prod":
        if not settings.database.multi_az:
            problems.append("myapp:database.multiAz must be true in prod")
        if settings.database.backup_retention_days < 14:
            problems.append("myapp:database.backupRetentionDays must be >= 14 in prod")
        if not settings.enable_deletion_protection:
            problems.append("myapp:enableDeletionProtection must be true in prod")

    if problems:
        raise SettingsError(
            "stack configuration rejected:\n  - " + "\n  - ".join(problems)
        )

The __main__.py that consumes it becomes short enough to read in one screen, and every resource below the validation call can treat its inputs as known-good:

# __main__.py — load, validate, then build
# CLI: pulumi up --stack staging
from __future__ import annotations

import pulumi
import pulumi_aws as aws

from settings import AppSettings, validate

settings = AppSettings.load()
validate(settings)          # nothing is registered with the engine before this returns

vpc = aws.ec2.Vpc(
    "app",
    cidr_block=settings.vpc_cidr,
    enable_dns_hostnames=True,
    tags={"Environment": settings.environment},
)

db = aws.rds.Instance(
    "app",
    engine="postgres",
    engine_version="16.3",
    instance_class=settings.database.instance_class,
    allocated_storage=settings.database.allocated_storage_gb,
    multi_az=settings.database.multi_az,
    backup_retention_period=settings.database.backup_retention_days,
    deletion_protection=settings.enable_deletion_protection,
    username="appadmin",
    password=settings.db_password,      # an Output[str], passed through unresolved
    skip_final_snapshot=settings.environment != "prod",
)
# State implication: because password arrived as a secret Output, the RDS input is
# stored encrypted in the checkpoint instead of appearing in plaintext state.

Step 4 — Keep the secret inside its Output all the way to the resource

require_secret returns Output[str], and that is not an inconvenience to be worked around — it is the taint marker. Any value derived through .apply() or Output.concat() keeps the marking; any value you extract into a plain str loses it permanently. The SDK actively blocks the easy mistake: interpolating an Output into an f-string raises Calling __str__ on an Output[T] is not supported.

# connection.py — deriving a connection string without unwrapping the secret
# CLI: pulumi stack output dsn --show-secrets --stack staging
from __future__ import annotations

import pulumi

from settings import AppSettings


def build_dsn(settings: AppSettings, host: pulumi.Output[str], port: pulumi.Output[int]) -> pulumi.Output[str]:
    return pulumi.Output.all(host, port, settings.db_password).apply(
        lambda parts: f"postgresql://appadmin:{parts[2]}@{parts[0]}:{parts[1]}/app"
    )
    # State implication: the result inherits the secret marking from db_password, so
    # exporting it writes ciphertext to the checkpoint.


dsn: pulumi.Output[str] = build_dsn(settings, db.address, db.port)
pulumi.export("dsn", dsn)

Verification

Three checks prove the refactor did what it claims. The first is that a wrong value is now rejected before anything is registered — time pulumi preview against a deliberately broken stack and confirm the failure arrives in about a second, not after a resource has been created.

# CLI: prove eager validation fires before resource registration
pulumi config set vpcCidr 10.20.0.0/28 --stack staging
pulumi preview --stack staging 2>&1 | head -5
# Diagnostics:
#   pulumi:pulumi:Stack (myapp-staging):
#     error: stack configuration rejected:
#       - myapp:vpcCidr 10.20.0.0/28 is too small to subnet
pulumi config set vpcCidr 10.20.0.0/16 --stack staging

The second is static: mypy --strict must now flag the class of typo it previously could not see.

# CLI: the typo that cfg.get() would have swallowed is now a type error
mypy --strict infra/
# infra/network.py:14: error: "AppSettings" has no attribute "vpc_cider";
#   maybe "vpc_cidr"?  [attr-defined]

The third is a unit test that constructs the settings object from an in-memory config map, with no CLI and no cloud account involved. pulumi.runtime.set_all_config seeds exactly what the language host would have received.

# tests/test_settings.py — settings validation as a fast unit test
# CLI: pytest tests/test_settings.py -q
from __future__ import annotations

import pytest
import pulumi

from settings import AppSettings, SettingsError, validate

BASE = {
    "myapp:environment": "prod",
    "myapp:vpcCidr": "10.30.0.0/16",
    "myapp:adminCidrs": '["10.30.0.0/16"]',
    "myapp:database": '{"instanceClass":"db.r6g.large","allocatedStorageGb":200,'
                      '"multiAz":true,"backupRetentionDays":30}',
    "myapp:enableDeletionProtection": "true",
    "myapp:dbPassword": "unit-test-only",
    "aws:region": "eu-west-1",
}


def test_prod_profile_is_accepted() -> None:
    pulumi.runtime.set_all_config(dict(BASE))
    validate(AppSettings.load())


def test_prod_rejects_single_az_database() -> None:
    broken = dict(BASE)
    broken["myapp:database"] = ('{"instanceClass":"db.t4g.micro","allocatedStorageGb":20,'
                                '"multiAz":false,"backupRetentionDays":1}')
    pulumi.runtime.set_all_config(broken)
    with pytest.raises(SettingsError, match="multiAz must be true in prod"):
        validate(AppSettings.load())

Gotchas & Edge Cases

get_bool exists for a reason. cfg.get("enableDeletionProtection") on a stack that stores false returns the four-character string "false", which is truthy in Python. Every deployment then runs with deletion protection on, and nobody notices until a teardown fails. Use get_bool/require_bool, which raise ConfigTypeError on anything other than a recognised boolean literal.

Two defaults are one too many. A key can carry a default in the config: block of Pulumi.yaml and another in an or expression in Python. When they disagree the YAML wins silently, because the engine substitutes it before your code runs. Pick one home for defaults — the project file if operators should be able to see them with pulumi config, the dataclass if they are implementation detail — and delete the other.

require_object does not validate shape. It parses and returns. A stack file where allocatedStorageGb was typed as "100gb" yields a string that int() rejects with invalid literal for int() with base 10: '100gb', which is why the conversion lives inside from_object where the error message can name the key.

Frozen does not mean deeply frozen. @dataclass(frozen=True) blocks attribute assignment, not mutation of a list or dict held in a field. A settings object holding admin_cidrs: List[str] can still be appended to from anywhere in the program. Tuples close that door.

A secret read with require is no longer a secret. Both require("dbPassword") and require_secret("dbPassword") decrypt the stored value; only the second keeps the marking. The first hands back a plain str that will be written into the checkpoint in cleartext the moment it lands on a resource input, and no warning is emitted.

Config values are strings, including numbers you set from a shell. pulumi config set replicas 03 stores "03"; require_int parses it to 3, but a require() plus manual comparison against "3" does not match. Convert once, in the loader.

Structured keys and flat keys collide. Setting myapp:database as an object and later running pulumi config set myapp:database somestring replaces the whole structure with a scalar. The next require_object raises rather than merging, which is correct but surprising if the setter was a --path-free copy-paste.

Operational Notes

Keep settings.py free of resource imports. It should depend on pulumi and the standard library only, so the module can be imported by tests, by a linting script, and by a small CLI that prints the resolved configuration without pulling in provider packages that need credentials to import cleanly.

Resist threading the whole settings object into component constructors. A component that accepts AppSettings can read anything, which makes its real dependencies invisible and its unit tests dependent on a full configuration map. Pass the two or three values it needs as explicit typed arguments; the settings object stays a top-level concern of __main__.py. The same reasoning applies when values cross a stack boundary rather than a function boundary, as described in handling Pulumi stack outputs and cross-stack references.

Print the non-secret settings at program start, once, as a structured log line. pulumi.log.info(...) attaches it to the stack resource in the update output, so every update carries a record of what the program believed its configuration to be. Never include an Output in that line — it will not render, and the attempt to force it into a string raises.

Version the contract deliberately. Adding a require-backed field to the settings dataclass breaks every stack that has not yet set the key, and it breaks them at the moment someone runs an unrelated update. Add new fields with get* and a default first, backfill the stacks with pulumi config set, then tighten to require in a second change. This two-step is the same discipline used when introducing new secret material, covered in securing Pulumi secrets with AWS KMS and HashiCorp Vault.

If the program is driven by the Automation API rather than the CLI, the same settings module works unchanged — the driver sets configuration through stack.set_config() before stack.up(), and the eager validation still runs inside the program where the error is attributable to a stack rather than to the caller.

FAQ

Should I use Pydantic instead of a dataclass?

Pydantic buys you coercion and field-level constraints for free, which is genuinely useful once the settings tree grows past a dozen fields. The cost is a dependency in the infrastructure program and slightly noisier error messages. A frozen dataclass with an explicit validate() function is enough for most projects and has no import cost; switch when you find yourself hand-writing the fifth range check.

Why does mypy not catch a wrong config key string?

Because the key is a runtime string and Config.get is annotated as returning Optional[str] for any input. Type checking can only help once the value has a name in your own type system, which is exactly what moving it onto a dataclass field does. The string literal appears once, in the loader, where a typo is visible next to the stack file.

How do I give a value a default without duplicating it?

Declare it in the config: block of Pulumi.yaml with a default:, then read it with require. The engine substitutes the default before your program starts, so require never raises for a key that has one, and pulumi config shows operators the effective value.

Can I log a secret temporarily while debugging?

You can — settings.db_password.apply(print) works — and it writes the plaintext into the update log where it is retained by CI. Treat any such line as a rotation trigger rather than a debugging convenience, and prefer asserting on a hash or a length instead.

What happens if two stacks need different shapes of the same key?

They cannot have different shapes; the loader is one piece of code for all stacks. Model the union in the dataclass with optional fields and enforce the per-environment rules in validate(), which is where "prod must be multi-AZ" belongs. Diverging shapes are a sign the two stacks want separate projects.

Does require_secret decrypt on every read?

The engine decrypts stack configuration once when it starts the language host, so repeated reads are free. The Output wrapper is bookkeeping in your process, not a call back to the secrets provider, which is why reading the same secret in three places costs nothing but still keeps all three markings intact.

Related

• Pulumi Secrets and Configuration in Python — the parent topic covering the configuration hierarchy and secret encryption end to end.
• Structuring Per-Environment Configuration in Pulumi — how to lay out the stack files this settings object reads from.
• Typing Pulumi Component Inputs and Outputs — the annotation rules for passing these values into reusable components.
• Structuring Pulumi Stacks per Environment — the stack and state boundaries that give each configuration file its scope.