How to Manage Python IaC Dependencies with Poetry and pip-tools

Reproducible Python IaC depends on locking the exact versions of your framework and provider SDKs so every developer and CI runner resolves an identical dependency graph. This guide shows how to pin Pulumi and CDKTF SDKs with both Poetry and pip-tools, separate development from runtime dependencies, and export pinned requirements for the runtimes that actually execute your stacks—a foundational step within Setting Up Dev Environments and the broader Python IaC Fundamentals & Strategy.

A minor, unpinned bump in a provider package can change default resource attributes and produce a destructive diff on the next pulumi up or cdktf deploy. Lockfiles convert that silent risk into a deliberate, reviewable change.

Prerequisites

  • Python 3.9+ (examples assume 3.11; check with python3 --version).
  • One dependency manager installed: pipx install poetry (Poetry) or pip install pip-tools (pip-tools).
  • A target IaC framework: pulumi>=3.100.0,<4.0.0 or cdktf>=0.20.0,<1.0.0.
  • The matching provider package, for example pulumi-aws>=7.0.0,<8.0.0 or cdktf-cdktf-provider-aws>=20.0.0,<21.0.0.
  • A clean virtual environment per project to avoid host pollution.

The provider SDK version is tightly coupled to the framework version it was generated against, so pin both. CDKTF provider bindings in particular regenerate against a specific cdktf release.

Implementation

Step 1 — Declare typed, separated dependencies in pyproject.toml

Goal: keep runtime IaC dependencies (the framework and providers your stack imports) distinct from development tooling (pytest, mypy, ruff). Only the runtime group ships to the execution environment.

# CLI: initialize a project and add pinned runtime + dev dependencies
poetry init --no-interaction --name infra --python ">=3.11,<3.13"
poetry add "pulumi>=3.100.0,<4.0.0" "pulumi-aws>=7.0.0,<8.0.0"
poetry add --group dev "pytest>=8.0,<9.0" "mypy>=1.8,<2.0" "ruff>=0.4,<1.0"
# Provider note: pulumi-aws pulls the AWS provider plugin at version-compatible boundaries.

The resulting pyproject.toml keeps the two concerns explicit:

# pyproject.toml — runtime deps install in CI; dev deps stay local
[tool.poetry.dependencies]
python = ">=3.11,<3.13"
pulumi = ">=3.100.0,<4.0.0"
pulumi-aws = ">=7.0.0,<8.0.0"

[tool.poetry.group.dev.dependencies]
pytest = ">=8.0,<9.0"
mypy = ">=1.8,<2.0"
ruff = ">=0.4,<1.0"

If you prefer pip-tools, encode the same split in two input files and compile each to a hashed lockfile:

# CLI: compile separate locks for runtime and dev with hashes
pip-compile --generate-hashes -o requirements.txt requirements.in
pip-compile --generate-hashes -o requirements-dev.txt requirements-dev.in
# State implication: hashes make installs fail loudly if a published artifact changes.

Step 2 — Generate and commit the lockfile

Goal: capture the fully resolved transitive graph so installs are byte-for-byte reproducible.

# CLI: resolve the full graph (Poetry) and verify it is current
poetry lock
poetry install --sync
poetry check --lock   # fails if poetry.lock drifts from pyproject.toml

Commit poetry.lock (or requirements.txt / requirements-dev.txt) alongside source. Treat it as code: every change is reviewed in a pull request. The following typed helper validates that a checked-in lockfile exists before a pipeline proceeds, turning a missing lock into an early, clear failure rather than a nondeterministic install.

# lock_guard.py — run in CI before installing
# CLI: python lock_guard.py
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path

@dataclass(frozen=True)
class LockPolicy:
    """Which lockfile must be present for a reproducible install."""
    manager: str                 # "poetry" or "pip-tools"
    required_files: tuple[str, ...]

POLICIES: dict[str, LockPolicy] = {
    "poetry": LockPolicy("poetry", ("poetry.lock",)),
    "pip-tools": LockPolicy("pip-tools", ("requirements.txt",)),
}

def assert_lock_present(manager: str, root: Path = Path(".")) -> None:
    policy = POLICIES[manager]
    missing = [f for f in policy.required_files if not (root / f).is_file()]
    if missing:
        raise SystemExit(f"Missing lockfile(s) for {manager}: {', '.join(missing)}")

if __name__ == "__main__":
    # State implication: no resolved lock means CI could install a different
    # provider version than was reviewed, risking an unexpected resource diff.
    assert_lock_present("poetry")
    print("Lockfile present; safe to install.")

Step 3 — Export pinned requirements for the IaC runtime

Goal: Pulumi and CDKTF execute your program in their own runtime context, which installs from a requirements.txt, not from poetry.lock. Export a pinned, runtime-only requirements file so the engine installs exactly what you locked.

# CLI: export ONLY runtime deps, pinned, no dev tooling, no hashes for the runtime installer
poetry export --without dev --format requirements.txt --output requirements.txt
# Provider note: Pulumi's Python runtime reads requirements.txt at `pulumi up`;
# CDKTF reads it via the project's pip install step before `cdktf synth`.

For a CDKTF project, point the synthesis runtime at the same file so synthesis and deploy use identical pins:

# cdktf.json equivalent in code — app entrypoint imports come from the locked env
# CLI: pip install -r requirements.txt && cdktf synth
from __future__ import annotations
from dataclasses import dataclass

@dataclass(frozen=True)
class RuntimeDeps:
    """The runtime contract the synthesis step must satisfy."""
    requirements_file: str = "requirements.txt"
    python_min: tuple[int, int] = (3, 11)

# State implication: synthesizing with a different provider binding than was
# locked can emit HCL JSON whose plan differs from what reviewers approved.
RUNTIME = RuntimeDeps()

Verification

Confirm the locked environment resolves cleanly and matches the lockfile in CI:

# CLI: prove the install is reproducible and the lock is authoritative
poetry install --sync --no-root
poetry check --lock                 # exits non-zero on drift
pip install -r requirements.txt     # the exported runtime file used by Pulumi/CDKTF
python -c "import pulumi_aws; print(pulumi_aws.__version__)"

A passing poetry check --lock plus a successful import of the pinned provider at the expected version confirms the runtime will install precisely what you reviewed.

Gotchas & Edge Cases

poetry export excludes hashes the runtime installer rejects. If you pass --without-hashes for one tool and require hashes elsewhere, installs can diverge. Pick one policy per environment: hashed requirements for security-sensitive CI, plain pins for the Pulumi/CDKTF runtime if its installer does not support --require-hashes.

Mixing poetry add and manual pip install corrupts the lock. Installing a package with bare pip inside a Poetry-managed venv leaves poetry.lock stale, so the next poetry install --sync silently removes it. Always route additions through poetry add, then re-export.

Provider SDK and framework version skew. Upgrading cdktf without regenerating provider bindings (or bumping pulumi-aws past its compatible pulumi range) yields import errors or changed defaults. Pin both with bounded ranges and upgrade them together in one reviewed change.

Frequently Asked Questions

Should I commit poetry.lock and the exported requirements.txt? Yes to both. The lockfile is the source of truth for resolution; the exported requirements.txt is the artifact the Pulumi or CDKTF runtime actually installs. Regenerate the export whenever the lock changes so they never drift apart.

Poetry or pip-tools for IaC—does it matter? Functionally both give you reproducible, pinned installs. Poetry manages the virtual environment and dev/runtime groups for you; pip-tools is lighter and composes well if you already build from requirements.in files. Teams running Pulumi or CDKTF often pick Poetry for the group separation, then poetry export to feed the engine's runtime.

How do I pin transitive dependencies, not just direct ones? Lockfiles do this automatically. poetry lock and pip-compile both resolve and freeze the entire transitive graph, including indirect provider dependencies, so a sub-dependency cannot float to a new version between installs.

Why separate dev and runtime dependencies at all? The execution runtime should never install pytest or mypy. Keeping them in a dev group shrinks the runtime install, reduces supply-chain surface, and ensures the environment that runs pulumi up contains only what the stack imports.