How to Manage Python IaC Dependencies with Poetry and pip-tools

Reproducible Python IaC depends on locking the exact versions of your framework and provider SDKs so every developer and CI runner resolves an identical dependency graph. This guide shows how to pin Pulumi and CDKTF SDKs with both Poetry and pip-tools, separate development from runtime dependencies, and export pinned requirements for the runtimes that actually execute your stacks—a foundational step within Setting Up Dev Environments and the broader Python IaC Fundamentals & Strategy.

A minor, unpinned bump in a provider package can change default resource attributes and produce a destructive diff on the next pulumi up or cdktf deploy. Lockfiles convert that silent risk into a deliberate, reviewable change.

Context

The failure this guide prevents is specific. An unpinned pulumi-aws resolves to a newer minor release on the CI runner than on the laptop where the change was reviewed, that release adds a default value to a resource argument, and the preview a reviewer approved is not the plan that executes. Nothing errors. The infrastructure simply drifts from the reviewed intent, and the commit that caused it contains no dependency change at all because there was never a version to change.

Python IaC has a second complication that ordinary applications do not: there are two environments, not one. The environment you develop and test in is created by Poetry or pip-tools from your lockfile. The environment that actually evaluates your program is created by the IaC engine — Pulumi builds a virtualenv described in Pulumi.yaml, CDKTF installs into whatever interpreter its app command names — and neither engine reads poetry.lock. Bridging that gap is the whole reason the export step in this guide exists.

Prerequisites

Prerequisites Prerequisites: layered from cdktf down to SDK. cdktf Python Poetry SDK
Prerequisites: the building blocks this section assembles.
  • Python 3.9+ (examples assume 3.11; check with python3 --version).
  • One dependency manager installed: pipx install poetry (Poetry) or pip install pip-tools (pip-tools).
  • A target IaC framework: pulumi>=3.100.0,<4.0.0 or cdktf>=0.20.0,<1.0.0.
  • The matching provider package, for example pulumi-aws>=7.0.0,<8.0.0 or cdktf-cdktf-provider-aws>=20.0.0,<21.0.0.
  • A clean virtual environment per project to avoid host pollution.

The provider SDK version is tightly coupled to the framework version it was generated against, so pin both. CDKTF provider bindings in particular regenerate against a specific cdktf release.

Two Environments, One Source of Truth

From declared range to the binary that talks to the cloud From declared range to the binary that talks to the cloud: layered from pyproject.toml down to Provider plugin binary. pyproject.toml bounded ranges you wrote by hand poetry.lock one resolved version per package, transitive included requirements.txt the export the IaC runtime installs Runtime virtualenv created by the engine, not by your shell Provider plugin binary downloaded by the SDK, versioned separately
A lockfile pins Python packages; the provider plugin is a separate artefact one layer below.

Pulumi's Python runtime is configured in Pulumi.yaml. The virtualenv option names a directory the CLI creates and populates before it imports your program, and the toolchain option decides which installer it uses to do so — pip reads requirements.txt, while poetry and uv read their own manifests directly and remove the export step entirely.

# Pulumi.yaml — let the engine build the runtime environment itself
name: infra
runtime:
  name: python
  options:
    virtualenv: venv
    toolchain: pip
# CLI: build the runtime environment the engine will use, before previewing
pulumi install
pulumi preview --diff
# Provider note: `pulumi install` also fetches the provider plugin binaries the
# installed SDKs ask for, which is why it can succeed offline only from cache.

That last point is the layer people miss. pulumi-aws is a Python package, but the code that talks to AWS lives in a separate pulumi-resource-aws plugin binary the SDK downloads at first use. The Python package requests a specific plugin version, so pinning the package does pin the plugin — but the plugin is cached per machine under ~/.pulumi/plugins, not per project, and a runner with a cold cache and no network fails with error: Could not automatically download and install resource plugin 'pulumi-resource-aws'. Pre-warming that cache in the CI image is worth doing once.

# CLI: inspect and pre-seed the plugin cache so CI never downloads mid-deploy
pulumi plugin ls
pulumi plugin install resource aws 7.2.0
# State implication: a plugin version mismatch changes how inputs are diffed,
# so two runners with different cached plugins can produce different previews.

CDKTF splits the same way, with an extra runtime underneath. cdktf.json names the app command that synthesises the stack, and that command must run in the interpreter your lockfile built. Beneath that, the cdktf Python bindings are jsii wrappers that shell out to a bundled Node process, so a locked Python graph still sits on an unlocked Node version — pin the Node major in the same image that pins the interpreter, or expect import-time failures that mention jsii and nothing about your code.

Implementation

Step 1 — Declare typed, separated dependencies in pyproject.toml

Implementation Implementation: — Declare typed then — Generate and then — Export pinned — Declare typed — Generate and — Export pinned
Implementation: the stages run left to right — — Declare typed, — Generate and, — Export pinned.

Goal: keep runtime IaC dependencies (the framework and providers your stack imports) distinct from development tooling (pytest, mypy, ruff). Only the runtime group ships to the execution environment.

# CLI: initialize a project and add pinned runtime + dev dependencies
poetry init --no-interaction --name infra --python ">=3.11,<3.13"
poetry add "pulumi>=3.100.0,<4.0.0" "pulumi-aws>=7.0.0,<8.0.0"
poetry add --group dev "pytest>=8.0,<9.0" "mypy>=1.8,<2.0" "ruff>=0.4,<1.0"
# Provider note: pulumi-aws pulls the AWS provider plugin at version-compatible boundaries.

The resulting pyproject.toml keeps the two concerns explicit:

# pyproject.toml — runtime deps install in CI; dev deps stay local
[tool.poetry.dependencies]
python = ">=3.11,<3.13"
pulumi = ">=3.100.0,<4.0.0"
pulumi-aws = ">=7.0.0,<8.0.0"

[tool.poetry.group.dev.dependencies]
pytest = ">=8.0,<9.0"
mypy = ">=1.8,<2.0"
ruff = ">=0.4,<1.0"

If you prefer pip-tools, encode the same split in two input files and compile each to a hashed lockfile:

# CLI: compile separate locks for runtime and dev with hashes
pip-compile --generate-hashes -o requirements.txt requirements.in
pip-compile --generate-hashes -o requirements-dev.txt requirements-dev.in
# State implication: hashes make installs fail loudly if a published artifact changes.

Step 2 — Generate and commit the lockfile

Goal: capture the fully resolved transitive graph so installs are byte-for-byte reproducible.

# CLI: resolve the full graph (Poetry) and verify it is current
poetry lock
poetry install --sync
poetry check --lock   # fails if poetry.lock drifts from pyproject.toml

Commit poetry.lock (or requirements.txt / requirements-dev.txt) alongside source. Treat it as code: every change is reviewed in a pull request. The following typed helper validates that a checked-in lockfile exists before a pipeline proceeds, turning a missing lock into an early, clear failure rather than a nondeterministic install.

# lock_guard.py — run in CI before installing
# CLI: python lock_guard.py
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path

@dataclass(frozen=True)
class LockPolicy:
    """Which lockfile must be present for a reproducible install."""
    manager: str                 # "poetry" or "pip-tools"
    required_files: tuple[str, ...]

POLICIES: dict[str, LockPolicy] = {
    "poetry": LockPolicy("poetry", ("poetry.lock",)),
    "pip-tools": LockPolicy("pip-tools", ("requirements.txt",)),
}

def assert_lock_present(manager: str, root: Path = Path(".")) -> None:
    policy = POLICIES[manager]
    missing = [f for f in policy.required_files if not (root / f).is_file()]
    if missing:
        raise SystemExit(f"Missing lockfile(s) for {manager}: {', '.join(missing)}")

if __name__ == "__main__":
    # State implication: no resolved lock means CI could install a different
    # provider version than was reviewed, risking an unexpected resource diff.
    assert_lock_present("poetry")
    print("Lockfile present; safe to install.")

The two commands that look alike behave very differently on an existing environment, and picking the wrong one is why a stale package survives in CI long after the lockfile dropped it.

What each install command guarantees What each install command guarantees: comparison across Reads, Removes extras, Verifies hashes. Command Reads Removes extras Verifies hashes poetry install poetry.lock No Lock digest only poetry sync poetry.lock Yes Lock digest only pip install -r requirements.txt No Only with --require-hashes pip-sync requirements.txt Yes Yes, when compiled with hashes
Only the syncing variants make the environment match the lockfile exactly.

poetry install adds what is missing and leaves anything extra in place. poetry sync (poetry install --sync in older releases) removes packages that are not in the lockfile, which is what a build agent should do — otherwise a package removed from pyproject.toml lingers in a cached virtualenv and keeps a stale import working locally while it fails on a clean runner. The pip-tools equivalents are pip install -r, which only adds, and pip-sync, which reconciles in both directions.

Relocking is the other pairing worth getting right. poetry lock refreshes the entire resolved graph against the current ranges, so an unrelated transitive dependency can move in a pull request that was meant to bump one package. Scoping the change keeps the diff readable and the blame accurate.

# CLI: bump exactly one dependency and leave the rest of the graph alone
poetry update pulumi-aws                     # re-resolves only this package's subtree
pip-compile --upgrade-package pulumi-aws==7.2.0 -o requirements.txt requirements.in
git diff --stat poetry.lock requirements.txt  # expect a handful of lines, not hundreds

Step 3 — Export pinned requirements for the IaC runtime

Goal: Pulumi and CDKTF execute your program in their own runtime context, which installs from a requirements.txt, not from poetry.lock. Export a pinned, runtime-only requirements file so the engine installs exactly what you locked.

# CLI: export ONLY runtime deps, pinned, no dev tooling, no hashes for the runtime installer
poetry export --without dev --format requirements.txt --output requirements.txt
# Provider note: Pulumi's Python runtime reads requirements.txt at `pulumi up`;
# CDKTF reads it via the project's pip install step before `cdktf synth`.

For a CDKTF project, point the synthesis runtime at the same file so synthesis and deploy use identical pins:

# cdktf.json equivalent in code — app entrypoint imports come from the locked env
# CLI: pip install -r requirements.txt && cdktf synth
from __future__ import annotations
from dataclasses import dataclass

@dataclass(frozen=True)
class RuntimeDeps:
    """The runtime contract the synthesis step must satisfy."""
    requirements_file: str = "requirements.txt"
    python_min: tuple[int, int] = (3, 11)

# State implication: synthesizing with a different provider binding than was
# locked can emit HCL JSON whose plan differs from what reviewers approved.
RUNTIME = RuntimeDeps()

Upgrading a Provider SDK Deliberately

Pinning is only half the discipline; the other half is a repeatable way to move a pin. Provider SDK releases are generated from upstream API schemas, so a minor version can add attributes, change a default, or mark an argument deprecated — all of which show up as a diff against infrastructure you did not intend to change.

Upgrading one provider without a surprise diff Upgrading one provider without a surprise diff: Bump one package → Re-lock and export → Preview a canary stack → Read the diff → Merge or revert → repeat. Bump one package Re-lock andexport Preview a canarystack Read the diff Merge or revert
Change one dependency per pull request so a diff has exactly one candidate cause.

Run the upgrade as its own pull request, one package at a time, and gate it on a preview that must be empty. --expect-no-changes turns "the diff looked fine" into a build failure when it is not.

# CLI: prove a provider bump is behaviour-neutral before it reaches production
poetry update pulumi-aws
poetry export --without dev --format requirements.txt --output requirements.txt
pulumi preview --stack canary --diff --expect-no-changes
# State implication: a non-empty diff here means the new SDK changed a default;
# read the resource's changelog entry before deciding whether to accept it.

When the preview is not empty, the diff itself tells you which category you are in. A ~ update on an attribute you never set is a new default the provider now sends; usually safe, occasionally a replacement. A +- replace on a resource you did not touch means an input became force-new, and that is the one to stop and investigate rather than approve at the end of a Friday. An ImportError such as cannot import name 'ProviderAssumeRoleArgs' from 'pulumi_aws' means the SDK renamed or moved a class in a major bump, and the fix belongs in code, not in the lockfile.

For CDKTF the analogous rule is stricter: the provider bindings package and the cdktf library are generated as a pair, so bump them together and re-run cdktf get to regenerate any prebuilt bindings. A cdktf release ahead of its bindings typically fails at synthesis with a jsii type error rather than at import, which makes it look like a code bug when it is a version skew. The pinning specifics for that side are covered in pinning Terraform provider versions in CDKTF.

Verification

Confirm the locked environment resolves cleanly and matches the lockfile in CI:

Verification Verification: Test → Program → Mock/Cloud. Test Program Mock/Cloud invoke declare resolve assert
Verification: the test drives the program and asserts on resolved values.
# CLI: prove the install is reproducible and the lock is authoritative
poetry install --sync --no-root
poetry check --lock                 # exits non-zero on drift
pip install -r requirements.txt     # the exported runtime file used by Pulumi/CDKTF
python -c "import pulumi_aws; print(pulumi_aws.__version__)"

A passing poetry check --lock plus a successful import of the pinned provider at the expected version confirms the runtime will install precisely what you reviewed.

Gotchas & Edge Cases

Gotchas & Edge Cases Gotchas & Edge Cases: Where it breaks with 4 facets. Where it breaks pip watch this boundary poetry.lock watch this boundary cdktf watch this boundary pulumi watch this boundary
Gotchas & Edge Cases: the boundaries where things break and what to check.

poetry export excludes hashes the runtime installer rejects. If you pass --without-hashes for one tool and require hashes elsewhere, installs can diverge. Pick one policy per environment: hashed requirements for security-sensitive CI, plain pins for the Pulumi/CDKTF runtime if its installer does not support --require-hashes.

Mixing poetry add and manual pip install corrupts the lock. Installing a package with bare pip inside a Poetry-managed venv leaves poetry.lock stale, so the next poetry install --sync silently removes it. Always route additions through poetry add, then re-export.

Provider SDK and framework version skew. Upgrading cdktf without regenerating provider bindings (or bumping pulumi-aws past its compatible pulumi range) yields import errors or changed defaults. Pin both with bounded ranges and upgrade them together in one reviewed change.

poetry export is no longer part of Poetry itself. From Poetry 2.0 the export command lives in poetry-plugin-export, so a CI image built on a newer Poetry fails the export step with a command-not-found style error while the same repository still works on a developer's older install. Add poetry self add poetry-plugin-export to the image build, or move to a toolchain the engine understands natively and drop the export entirely.

A hash mismatch is a security signal, not a nuisance. pip install --require-hashes fails with THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE when the artefact on the index differs from the one you compiled against. That happens after a legitimate re-upload and after a compromised one, and the two look identical from the terminal. Re-compile deliberately and inspect what changed rather than dropping the flag.

Resolution failures name the wrong culprit. Poetry's SolverProblemError usually points at the last package it tried, not the constraint that made the graph unsatisfiable. The common cause in IaC projects is the python range: a provider package that requires >=3.9,<3.13 cannot coexist with a pyproject.toml declaring >=3.9, because the open upper bound admits interpreters the package excludes. Bound the interpreter range explicitly and most resolution failures disappear.

Wheels are platform-specific and your image may not be your laptop. Installing on an Apple Silicon machine and deploying into a linux/amd64 container resolves different wheels for anything with compiled extensions. Build the runtime environment inside the target image rather than copying a virtualenv into it, and keep the lockfile as the only artefact that crosses the boundary.

Operational Notes

Cache the environment on the lockfile's hash, not on a branch name. Keying a CI cache on poetry.lock means a dependency change invalidates it exactly once and every other build reuses it, while a branch-keyed cache quietly serves a stale environment to the one build that most needs a fresh one. Setting poetry config virtualenvs.in-project true puts the environment in .venv/ beside the code, which makes it trivial to cache and trivial to delete.

# CLI: deterministic install path for a build agent
poetry config virtualenvs.in-project true
poetry check --lock          # fails if poetry.lock no longer matches pyproject.toml
poetry sync --no-root        # environment matches the lock exactly, extras removed
# State implication: `poetry check --lock` failing in CI means someone edited
# pyproject.toml without re-locking, so the reviewed graph was never resolved.

Automate the bumps, but group them the way you review them. Renovate or Dependabot raising one pull request per provider SDK gives you the one-change-per-diff property the upgrade loop above depends on; a single weekly "update all dependencies" pull request destroys it, because a surprise diff then has a dozen candidate causes. Group development tooling together — pytest, mypy, ruff moving as one is fine — and keep every runtime package on its own.

Private indexes need declaring in both files, and this is where teams get a green local build and a red CI. poetry source add internal https://pypi.internal.example/simple records the source in pyproject.toml so the lockfile knows where each package came from; pip-tools users need the equivalent --index-url recorded in requirements.in rather than passed on the command line, or the compiled output silently omits it and the runtime install falls back to the public index.

Finally, decide how many lockfiles a repository holds. One shared lock across every stack is simplest and couples them: bumping a provider for one environment bumps it for all. A lock per stack decouples the upgrades at the cost of N environments to build and cache. Most teams are better served by one lock and disciplined canary previews, and should split only when two stacks genuinely need incompatible SDK majors during a migration — see how to structure Python IaC projects for scale for the wider layout question.

FAQ

Should I commit poetry.lock and the exported requirements.txt? Yes to both. The lockfile is the source of truth for resolution; the exported requirements.txt is the artifact the Pulumi or CDKTF runtime actually installs. Regenerate the export whenever the lock changes so they never drift apart.

Poetry or pip-tools for IaC—does it matter? Functionally both give you reproducible, pinned installs. Poetry manages the virtual environment and dev/runtime groups for you; pip-tools is lighter and composes well if you already build from requirements.in files. Teams running Pulumi or CDKTF often pick Poetry for the group separation, then poetry export to feed the engine's runtime.

How do I pin transitive dependencies, not just direct ones? Lockfiles do this automatically. poetry lock and pip-compile both resolve and freeze the entire transitive graph, including indirect provider dependencies, so a sub-dependency cannot float to a new version between installs.

Why separate dev and runtime dependencies at all? The execution runtime should never install pytest or mypy. Keeping them in a dev group shrinks the runtime install, reduces supply-chain surface, and ensures the environment that runs pulumi up contains only what the stack imports.

Does the lockfile pin the Pulumi provider plugin too? Indirectly. The Python SDK version requests a specific plugin version, so pinning pulumi-aws pins the plugin it asks for. The binary itself is cached per machine under ~/.pulumi/plugins, so a runner can still fail to fetch it — pre-install plugins in the CI image with pulumi plugin install.

Can I skip the export and let Pulumi use Poetry directly? Yes, on recent Pulumi releases. Setting toolchain: poetry in Pulumi.yaml makes the engine resolve from pyproject.toml and poetry.lock, which removes the export and the risk of the two files drifting. Keep the export only while some runner still installs with plain pip.