Testing Python Infrastructure Code

Testing Python infrastructure code means proving that your resource definitions produce the configuration you intend before any cloud API is touched — and it is a primary reason engineers choose Python over a DSL. This page is part of the broader Python IaC fundamentals and strategy approach, and it lays out a testing pyramid that moves from fast unit checks up through snapshot, integration, and policy gates. Four guides sit underneath it, one per rung of that pyramid, and each is introduced in context below.

The IaC testing pyramid Four stacked layers: unit tests with mocks at the wide base, then snapshot tests, then integration tests, then policy checks at the narrow top. A side note shows cost and speed. Policy Integration LocalStack / moto Snapshot synthesized JSON Unit (mocks) fast, no network slow fast
The IaC testing pyramid: many fast unit tests at the base, fewer slow integration and policy checks at the top.

Problem Framing: Why Untested Infrastructure Code Breaks

Infrastructure expressed as Python is still code, and code that is never asserted against drifts from intent the moment it grows past a single file. A mistyped CIDR block, a security group rule opened to 0.0.0.0/0, or a missing parent on a child resource will all synthesize cleanly and only surface as an incident after deployment. The cost of catching these defects rises sharply the later you find them: a failed assertion in pytest costs seconds, a failed apply against production costs an outage.

Why Untested Infrastructure Code Breaks Why Untested Infrastructure Code Breaks: layered from parent down to HCL. parent apply Code Breaks Python HCL
Why Untested Infrastructure Code Breaks: the building blocks this section assembles.

The specific failure mode that makes infrastructure worse than application code is that the compiler cannot help you. pulumi.Output[str] is opaque: a bucket name, an ARN, and a KMS key alias are all the same type, so passing the wrong one type-checks perfectly and only fails when the provider rejects it. CDKTF is no better — a Terraform interpolation token such as ${aws_kms_key.logs.arn} is just a Python str until Terraform resolves it, so a wrong reference survives synthesis and dies during terraform apply, halfway through creating resources, leaving the deployment partially applied.

The second failure mode is invisible coupling. Infrastructure modules read configuration, compute names, derive CIDRs, and stitch outputs from one stack into inputs of another. None of that logic touches a cloud API, yet it is where most real bugs live: an off-by-one in a subnet calculation, a name that exceeds the 63-character limit for an Application Load Balancer target group, a tag map that silently loses a key because two helpers both build it. Those are ordinary Python defects, and ordinary Python tests catch them in milliseconds.

The third is regression by upgrade. Provider releases change defaults. When pulumi-aws moved bucket configuration onto separate BucketAclV2 and BucketServerSideEncryptionConfigurationV2 resources, stacks that relied on inline arguments kept synthesizing but stopped configuring what they used to. Only a suite that asserts the resulting configuration — not the code that produces it — notices that kind of silent semantic change on a dependency bump.

The Python ecosystem is what makes IaC testing tractable. You reuse pytest, fixtures, mocks, and CI runners you already operate for application code. That parity is the differentiator over HCL, where testing is bolted on through separate tooling. Each layer of the pyramid trades speed for fidelity, and a healthy suite weights heavily toward the fast base.

Prerequisites

Before writing the first test, get the toolchain and the directory layout in place. The layout matters because CI selects layers by path, and a test that lands in the wrong directory silently changes what runs on a push.

# CLI: run once per checkout to build the test toolchain
python -m venv .venv && . .venv/bin/activate
pip install "pulumi>=3.100" "pulumi-aws>=6.30" "cdktf>=0.20" "pytest>=8.0" \
            "moto[s3,ec2,ssm]>=5.0" "hypothesis>=6.100" "checkov>=3.2"
# Provider note: CDKTF additionally needs Node.js 20+ on PATH; run `cdktf get` once to
# generate the Python provider bindings into ./imports before any snapshot test imports them.

You also need:

  • Python 3.9 or newer, so built-in generic annotations such as dict[str, str] work without from __future__ import annotations.
  • A stack that can be imported as a module. A Pulumi __main__.py that builds resources at import time cannot be unit tested cleanly; move the resource construction into a function or a ComponentResource so tests can call it after mocks are installed.
  • Pinned provider versions. Snapshot tests compare generated configuration, and an unpinned provider turns every dependency update into a red build with a meaningless diff. Pin in requirements.txt for Pulumi and in cdktf.json for CDKTF.
  • No ambient cloud credentials in the test environment. Export dummy values (AWS_ACCESS_KEY_ID=testing, AWS_SECRET_ACCESS_KEY=testing, AWS_DEFAULT_REGION=eu-west-1) so a test that escapes its mock fails loudly instead of quietly mutating a real account.

The conventional tree is four directories under tests/, mirroring the four layers, plus a conftest.py per directory so fixtures never leak between layers:

# CLI: tree tests -L 2
tests/unit/conftest.py            # installs pulumi.runtime.set_mocks
tests/unit/test_naming.py
tests/snapshot/__snapshots__/     # golden Terraform JSON, committed
tests/snapshot/test_network.py
tests/integration/conftest.py     # starts ThreadedMotoServer, yields an endpoint URL
tests/policy/test_checkov_baseline.py

The Four Layers of the Testing Pyramid

The Four Layers The Four Layers: layered from Unit tests with mocks down to API. Unit tests with mocks Snapshot tests Integration tests Policy tests API
The Four Layers: the building blocks this section assembles.

Unit tests with mocks

Unit tests run your resource graph in-process with the cloud API replaced by a mock, so they execute in milliseconds and need no credentials. For Pulumi this means pulumi.runtime.set_mocks; the dedicated walkthrough lives in Unit Testing Pulumi Programs with Mocks. When your IaC helper code calls boto3 directly for a lookup, you mock the AWS SDK itself with moto, covered in Mocking AWS Services with moto in pytest.

# CLI: pytest tests/test_tags.py -v
# State implication: pure in-memory assertion, no provider call, no state mutation.
from dataclasses import dataclass

@dataclass(frozen=True)
class TagPolicy:
    cost_center: str
    environment: str

def required_tags(policy: TagPolicy) -> dict[str, str]:
    return {"CostCenter": policy.cost_center, "Environment": policy.environment}

def test_required_tags_complete() -> None:
    tags = required_tags(TagPolicy(cost_center="cc-42", environment="prod"))
    assert set(tags) == {"CostCenter", "Environment"}

The base of the pyramid also holds the layer most teams skip: generated-input testing. Example-based unit tests only cover the values you thought to write down, and naming or CIDR helpers fail on the inputs you did not — a hyphen at the end of a truncated name, a /28 subnet requested from a /24 VPC. Property-Based Testing for Python IaC with Hypothesis covers asserting invariants across thousands of generated inputs instead, which is the cheapest way to harden that pure logic.

Snapshot tests

Snapshot tests synthesize the stack to its final declarative form and compare it against a stored golden file, so any unintended change to the generated configuration fails the diff. This is the natural fit for CDKTF, whose synthesis step emits Terraform JSON you can assert against; the full pattern is in Snapshot Testing CDKTF Stacks with pytest.

The value of a snapshot is that it asserts on the output rather than the code path, which is exactly what catches provider-default changes and refactors that were supposed to be behaviour-preserving. The cost is review discipline: a golden file that engineers regenerate reflexively whenever a test goes red provides no protection at all. Treat a snapshot diff like a schema migration — it must be read line by line in the pull request, and the regeneration command must be a deliberate act, never part of the default test invocation.

Integration tests

Integration tests provision against a real or emulated backend — moto for AWS API surface, or LocalStack for a fuller emulation — and verify that resources actually come up. They are slower and need a clean teardown, so keep them few and run them on merge rather than on every keystroke.

Emulation fidelity is the thing to keep honest about. moto implements API shapes and a useful subset of behaviour, but it does not evaluate IAM policies the way AWS does, and it will happily accept a resource configuration that the real service rejects. That makes integration tests excellent for wiring questions ("does the stack actually create the queue, and does the Lambda event source mapping reference it?") and poor for authorization questions. Anything that depends on a real policy evaluation belongs in a preview against a sandbox account, not in an emulator.

Policy tests

Policy checks enforce organizational rules (no public buckets, mandatory encryption, approved instance types) across the synthesized output. They sit at the narrow top of the pyramid and act as the final gate before deployment.

Two shapes exist. Checkov scans the synthesized artefact — the Terraform JSON in cdktf.out, or a Pulumi preview exported to JSON — and is the right tool when you want the same rule set applied across HCL and Python projects. Pulumi CrossGuard runs inside the preview and can therefore see resource inputs before anything is emitted, including values a scanner would only see as unresolved tokens.

# CLI: pulumi preview --policy-pack ./policy
from pulumi_policy import (EnforcementLevel, PolicyPack, ReportViolation,
                           ResourceValidationArgs, ResourceValidationPolicy)

def _no_public_bucket(args: ResourceValidationArgs, report: ReportViolation) -> None:
    # Provider note: resource_type is the provider token, not the Python class name.
    if args.resource_type == "aws:s3/bucketV2:BucketV2" and args.props.get("acl") == "public-read":
        report(f"bucket {args.name} sets acl=public-read; buckets must stay private")

PolicyPack(
    name="baseline",
    enforcement_level=EnforcementLevel.MANDATORY,
    policies=[ResourceValidationPolicy(
        name="no-public-bucket",
        description="S3 buckets must not be world-readable.",
        validate=_no_public_bucket,
    )],
)

How set_mocks Intercepts the Resource Graph

Understanding the interception point is what separates tests that assert something from tests that pass vacuously. A normal Pulumi run is two processes: the language host runs your Python, and the engine (started by the pulumi CLI) answers registration calls over a gRPC resource monitor. Every aws.s3.BucketV2(...) you construct sends a RegisterResource request; the engine talks to the provider, gets real state back, and resolves the Output values your program is waiting on.

pulumi.runtime.set_mocks replaces that monitor with an in-process object. Your Mocks subclass answers two questions: new_resource returns the (id, state) pair for a registration, and call returns the result of a provider function invoke such as getCallerIdentity or getAmi. Nothing crosses a socket, no state file is opened, and the run is deterministic.

How set_mocks intercepts a resource registration How set_mocks intercepts a resource registration: pytest test → Pulumi runtime → Mocks subclass → Output resolver. pytest test Pulumi runtime Mocks subclass Output resolver import stack new_resource(args) (id, state) resolve Output assert on value
A mocked Pulumi run: registration is answered by your Mocks subclass, never by a cloud provider.

Two ordering rules follow directly from that design. First, set_mocks must run before the module that constructs resources is imported, because construction is what triggers registration — put the call in tests/unit/conftest.py, which pytest imports before collecting any test module. Second, assertions must run inside an apply callback and the resulting Output must be returned from the test, so the runtime can await it; @pulumi.runtime.test does exactly that.

# CLI: pytest tests/unit -q
# State implication: no state file is opened; the fake ids exist only for this process.
from typing import Any

import pulumi


class StackMocks(pulumi.runtime.Mocks):
    def new_resource(self, args: pulumi.runtime.MockResourceArgs) -> tuple[str, dict[str, Any]]:
        outputs: dict[str, Any] = dict(args.inputs)
        if args.typ == "aws:s3/bucketV2:BucketV2":
            outputs["arn"] = f"arn:aws:s3:::{args.name}"
        return f"{args.name}-id", outputs

    def call(self, args: pulumi.runtime.MockCallArgs) -> dict[str, Any]:
        if args.token == "aws:index/getCallerIdentity:getCallerIdentity":
            return {"accountId": "123456789012",
                    "arn": "arn:aws:iam::123456789012:root",
                    "userId": "AIDAEXAMPLE"}
        return {}


pulumi.runtime.set_mocks(StackMocks(), project="billing", stack="test", preview=False)

Note preview=False. Under preview=True the engine marks outputs of not-yet-created resources as unknown, and apply callbacks on unknown values are skipped entirely — a test written that way passes without ever executing its assertion. That single flag is responsible for a large share of green-but-worthless Pulumi suites.

# CLI: pytest tests/unit/test_artifacts.py -q
from typing import Any

import pulumi

import infra  # imported after conftest.py has installed the mocks


@pulumi.runtime.test
def test_artifact_bucket_is_tagged_and_private() -> pulumi.Output[None]:
    def check(values: list[Any]) -> None:
        acl, tags = values
        assert acl == "private", f"expected private acl, got {acl!r}"
        assert tags["CostCenter"] == "cc-42"

    # State implication: Output.all resolves through StackMocks, never through AWS.
    return pulumi.Output.all(infra.artifact_acl.acl, infra.artifacts.tags).apply(check)

Where Each Tool Fits

Where Each Tool Fits Where Each Tool Fits: Where Each Tool Fits with 4 facets. Where Each Tool Fits pulumi.runtime key element cdktf.Testing key element pulumi.runtime key element Where Each key element
Where Each Tool Fits: how pulumi.runtime, cdktf.Testing, pulumi.runtime relate in this pattern.
Layer Tool Needs cloud creds Speed What it catches
Unit pulumi.runtime mocks, moto No Milliseconds Wrong inputs, missing tags, graph shape
Snapshot cdktf.Testing.synth No Fast Drift in synthesized JSON
Integration moto / LocalStack Emulated Seconds–minutes Resources that fail to create
Policy Checkov, CrossGuard No Fast Compliance violations

Both major Python IaC tools support this stack. The Pulumi patterns and provider management section uses pulumi.runtime.Mocks for its component tests, and the CDKTF workflows and Terraform synthesis section leans on synthesized-JSON snapshots — same pyramid, different synthesis model.

The asymmetry is worth stating plainly. Pulumi has no stable declarative artefact to snapshot: the resource graph is built at runtime and the closest equivalent, a preview export, contains URNs and diff metadata that churn between releases. So Pulumi projects put their weight on mocked unit tests. CDKTF, conversely, produces cdktf.out/stacks/<name>/cdk.tf.json, a byte-stable document that is ideal for golden-file comparison but tells you nothing about the Python that produced it. CDKTF projects therefore pair snapshots with plain unit tests over their construct helper functions.

CDKTF also ships matcher helpers so you can assert on one resource instead of the whole document — Testing.to_have_resource_with_properties(synthesized, "aws_s3_bucket", {"bucket": "artifacts"}) returns a boolean, and Testing.to_be_valid_terraform(Testing.full_synth(stack)) shells out to terraform validate for a real syntax check. Use targeted matchers for behavioural assertions and reserve the full golden file for change detection; mixing the two produces snapshots nobody dares to update.

Choosing the Layer for a Given Defect

Every bug you have shipped is evidence about where a test was missing. The useful question during a post-incident review is not "should we add a test?" but "which is the cheapest layer that could have proven this wrong?" Pushing a check down the pyramid is almost always right: the same guarantee at a lower layer runs more often, fails faster, and does not need credentials.

Which test layer catches this defect? Which test layer catches this defect?: choose among 4 options. A defect reaches review inputs Unit test with mocks shape Snapshot vs goldenJSON real API Integration on moto rule Policy scan
Route each class of defect to the cheapest layer that can prove it wrong.

Concretely: a wrong retention period on a log group is an inputs defect and belongs in a mocked unit test. A refactor that accidentally dropped the depends_on edge between a NAT gateway and an internet gateway is a shape defect, visible only in the synthesized document, so it belongs in a snapshot. A Lambda that cannot read from its own event source because the helper built the wrong queue URL is a real API defect — moto will answer sqs:GetQueueUrl and expose the mismatch. And "someone can open a bucket to the world" is a rule, which belongs in a policy pack where it applies to every stack rather than the one you remembered to test.

The failure pattern to avoid is duplicating the same assertion at three layers. If a policy pack already forbids unencrypted volumes as a mandatory rule, do not also write per-stack snapshot assertions about encryption: you now have three places to update and no extra safety. Assert a fact exactly once, at the lowest layer that can see it.

Step-by-Step: Building the Suite from Zero

The order below gets a project from no tests to a full gate in four passes, each of which is independently useful. Do not attempt all four in one change — the base layer is where the value is, and it is also where the refactoring cost lands.

1. Make the stack importable

Move resource construction out of module scope into a function that takes typed configuration. This is the only step that touches production code, and it is what makes every later step possible.

# CLI: pulumi up --stack prod   (and: pytest tests/unit -q)
from dataclasses import dataclass

import pulumi
import pulumi_aws as aws


@dataclass(frozen=True)
class StackConfig:
    environment: str
    cost_center: str
    retention_days: int


def build(cfg: StackConfig) -> aws.s3.BucketV2:
    tags = {"Environment": cfg.environment, "CostCenter": cfg.cost_center}
    bucket = aws.s3.BucketV2(f"artifacts-{cfg.environment}", tags=tags)
    # State implication: each construction registers one resource in the stack's state.
    aws.s3.BucketLifecycleConfigurationV2(
        f"artifacts-lifecycle-{cfg.environment}",
        bucket=bucket.id,
        rules=[aws.s3.BucketLifecycleConfigurationV2RuleArgs(
            id="expire", status="Enabled",
            expiration=aws.s3.BucketLifecycleConfigurationV2RuleExpirationArgs(
                days=cfg.retention_days))],
    )
    return bucket

2. Install mocks and assert on inputs

Add tests/unit/conftest.py with the StackMocks class shown earlier, then write assertions against build(...). Start with the three facts that break most often: tags, encryption, and anything numeric that came from configuration.

3. Add a golden file for the synthesized output

For CDKTF, synthesize in-process and compare. Normalize the metadata block first — CDKTF writes its own version and construct stack traces under the // key, which change on every toolchain bump and would otherwise make the golden file unreadable.

# CLI: pytest tests/snapshot -q      (regenerate: SNAPSHOT_UPDATE=1 pytest tests/snapshot -q)
import json
import os
from pathlib import Path
from typing import Any

from cdktf import Testing

from stacks.network import NetworkStack

GOLDEN = Path(__file__).parent / "__snapshots__" / "network.json"


def _normalise(doc: dict[str, Any]) -> dict[str, Any]:
    doc.pop("//", None)  # cdktf metadata: version + construct stack traces
    return doc


def test_network_stack_matches_golden() -> None:
    # State implication: Testing.synth runs synthesis in memory and reads no tfstate.
    synthesized = _normalise(json.loads(Testing.synth(NetworkStack(Testing.app(), "network"))))
    if os.environ.get("SNAPSHOT_UPDATE") == "1":
        GOLDEN.write_text(json.dumps(synthesized, indent=2, sort_keys=True) + "\n")
    assert synthesized == _normalise(json.loads(GOLDEN.read_text()))

4. Add one integration test and the policy gate

Stand up ThreadedMotoServer in a fixture, point the provider at it, and cover a single end-to-end path. Then wire Checkov over the synthesized directory. Both belong in CI rather than the local loop.

# CLI: pytest tests/integration -q
from collections.abc import Iterator

import boto3
import pytest
from moto.server import ThreadedMotoServer


@pytest.fixture(scope="session")
def moto_endpoint() -> Iterator[str]:
    server = ThreadedMotoServer(port=0)
    server.start()
    host, port = server.get_host_and_port()
    # Provider note: pass this URL as the aws provider's endpoint so the real API is never hit.
    yield f"http://{host}:{port}"
    server.stop()


def test_artifacts_bucket_is_created(moto_endpoint: str) -> None:
    s3 = boto3.client("s3", region_name="eu-west-1", endpoint_url=moto_endpoint)
    s3.create_bucket(Bucket="artifacts-prod",
                     CreateBucketConfiguration={"LocationConstraint": "eu-west-1"})
    assert "artifacts-prod" in [b["Name"] for b in s3.list_buckets()["Buckets"]]
Wall-clock cost per stage, 40-resource stack Wall-clock cost per stage, 40-resource stack: Unit (set_mocks), Snapshot (Testing.synth), Policy (checkov -d), Integration (moto server), Preview against AWS. Unit (set_mocks) ~2 s Snapshot (Testing.synth) ~9 s Policy (checkov -d) ~14 s Integration (moto server) ~95 s Preview against AWS ~210 s
Measured on a 40-resource stack: each layer up the pyramid costs roughly an order of magnitude more feedback time.

Wiring It Into CI

Run the fast layers on every push and gate merges on the slow ones:

Wiring It Into CI Wiring It Into CI: Author then Preview then Apply then Verify Author Preview Apply Verify
Wiring It Into CI: the stages run left to right — Author, Preview, Apply, Verify.
# CLI: invoked from .github/workflows/test.yml
# Provider note: only the integration stage needs cloud or LocalStack credentials.
pytest tests/unit -q                 # base layer, runs on every push
pytest tests/snapshot -q             # golden-file comparison
pytest tests/integration -q          # gated behind merge to main
checkov -d cdktf.out --quiet         # policy gate before deploy

Keep the unit and snapshot stages under a few seconds total so engineers run them locally without friction. The integration and policy stages belong in the pipeline where their latency and credential needs are acceptable.

Two details make the difference between a pipeline people trust and one they route around. First, split the job so a failing policy scan is distinguishable from a failing unit test at a glance — Checkov exits 1 on any failed check, which is indistinguishable from a pytest failure if both run in the same step. Second, never use --soft-fail on the merge-gating run; it downgrades every violation to exit code 0, and a gate that cannot fail is documentation, not a gate.

# CLI: committed as .github/workflows/test.yml and run by GitHub Actions on every push
name: test
on: [push, pull_request]
jobs:
  fast:
    runs-on: ubuntu-latest
    env:
      AWS_ACCESS_KEY_ID: testing
      AWS_SECRET_ACCESS_KEY: testing
      AWS_DEFAULT_REGION: eu-west-1
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install -r requirements-dev.txt
      - run: pytest tests/unit tests/snapshot -q --durations=5
      - run: checkov -d cdktf.out --framework terraform --quiet --compact

Verification

A test suite is itself code, and the failure mode is silent uselessness rather than a crash. Verify the suite the same way you verify infrastructure — by proving it reacts to change.

# CLI: run from the repository root after wiring the suite
pytest tests/unit -q --durations=10          # confirm the base layer stays under ~2 s
pytest --collect-only -q | tail -1           # confirm every test module was actually imported
pytest tests/unit -q --cov=infra --cov-report=term-missing   # find unasserted construction paths

Then run three deliberate checks:

  1. Break something on purpose. Change retention_days=30 to retention_days=3 in StackConfig and re-run. If the suite still passes, the assertion you thought covered retention does not exist, or it lives inside an unreturned apply callback.
  2. Confirm the snapshot is load-bearing. Add a tag to the stack and run pytest tests/snapshot -q without SNAPSHOT_UPDATE. It must fail with a dictionary diff naming the new key. If it passes, your normalization is stripping too much.
  3. Confirm no test reaches the network. Run the fast layers with the dummy credentials above and no network route; anything that was quietly calling AWS surfaces immediately as a botocore endpoint error rather than passing.

For a stack of forty resources, expect the base layer in a couple of seconds, snapshots in under ten, and everything above that in CI only. If your unit stage exceeds ten seconds, something in it is doing real work — usually a provider function invoke that has no entry in Mocks.call and is falling through to a live lookup.

Troubleshooting

Tests fail with "Program run without the Pulumi engine available"

Problem. pytest tests/unit -q aborts with Exception: Program run without the Pulumi engine available; re-run using the 'pulumi' CLI.

Cause. A resource was constructed before pulumi.runtime.set_mocks installed the in-process monitor. This almost always means the stack module was imported at collection time — a top-level import infra in a test file that pytest imports before conftest.py, or resource construction at module scope inside infra itself.

Fix. Put set_mocks in tests/unit/conftest.py (pytest guarantees conftest import order) and move resource construction into a build() function so importing the module registers nothing.

A Pulumi test passes but the assertion never runs

Problem. pytest -q reports 1 passed, yet deliberately corrupting the value under test changes nothing.

Cause. The assertion lives inside an Output.apply callback whose Output was never returned, so the runtime had nothing to await and the callback never fired. The same symptom appears when set_mocks(..., preview=True) marks the value unknown, because apply is skipped entirely for unknown values during a preview.

Fix. Decorate with @pulumi.runtime.test, return the Output from the test function, and pass preview=False. Then verify by breaking the expected value on purpose.

moto import fails after upgrading to version 5

Problem. Collection dies with ImportError: cannot import name 'mock_s3' from 'moto'.

Cause. moto 5.0 collapsed every per-service decorator (mock_s3, mock_ec2, mock_iam, …) into a single mock_aws. Old test files written against moto 4 fail at import, so pytest reports a collection error rather than a test failure.

Fix. Replace the import with from moto import mock_aws and apply @mock_aws to the test or use it as a context manager. Create every boto3 client inside the mocked scope — a client built beforehand keeps the real endpoint and will raise botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the GetCallerIdentity operation.

Snapshot tests go red on every dependency bump

Problem. pytest tests/snapshot -q fails with a large AssertionError: assert {'//': {...}, 'terraform': {...}} == {...} after an unrelated upgrade.

Cause. The golden file captured toolchain-derived content: the // metadata block CDKTF writes (its own version plus construct stack traces) and the required_providers version constraint. Both change when the toolchain changes, not when your infrastructure does.

Fix. Strip the // key in a normalization helper, pin the provider version in cdktf.json so required_providers is stable, and keep regeneration behind an explicit SNAPSHOT_UPDATE=1 so the diff is always reviewed.

CDKTF tests cannot import the generated provider bindings

Problem. pytest tests/snapshot -q fails with ModuleNotFoundError: No module named 'imports.aws' on a fresh checkout or a fresh CI runner.

Cause. CDKTF generates Python provider bindings from cdktf.json at build time into imports/, and that directory is conventionally git-ignored. Nothing in pip install recreates it.

Fix. Run cdktf get before the test step and cache imports/ between CI runs keyed on the hash of cdktf.json. If the failure is instead jsii.errors.JavaScriptError: Error: Cannot find module 'cdktf', the Node.js toolchain is missing — cdktf.Testing drives the real CDKTF library through jsii and needs Node.js on PATH even in a pure-Python test run.

FAQ

Do I need real cloud credentials to test Python IaC?

No, for the base of the pyramid. Unit tests with pulumi.runtime.set_mocks and snapshot tests with cdktf.Testing.synth run entirely in-process. Only integration tests need credentials, and even those can target moto or LocalStack instead of a live account.

What is the difference between a snapshot test and an integration test?

A snapshot test compares the synthesized declarative output (Terraform JSON or a Pulumi resource graph) against a stored golden file without deploying anything. An integration test actually provisions resources against an emulated or real backend and checks they come up correctly.

How many integration tests should I write?

Few. They are the slowest and most brittle layer. Cover the critical paths that mocks cannot verify — actual resource creation, IAM evaluation, networking — and push everything else down to unit and snapshot tests.

Can I share one pytest suite across Pulumi and CDKTF projects?

You share the runner, fixtures, and CI conventions, but the assertion style differs: Pulumi tests resolve Output values through mocks, while CDKTF tests assert against synthesized JSON. Keep them in separate test modules under the same tests/ tree.

Why does my mocked test pass when the code is obviously wrong?

Almost always because the assertion sits in an apply callback that was never awaited, or because set_mocks was called with preview=True, which leaves outputs unknown and skips apply entirely. Prove the suite is load-bearing by breaking an expected value and confirming the test turns red.

Should the golden snapshot file be committed to version control?

Yes — it is the artefact under review. Commit it, regenerate it only through an explicit flag such as SNAPSHOT_UPDATE=1, and read the diff in the pull request. A snapshot that is regenerated automatically whenever tests fail provides no protection whatsoever.