How to Migrate IaC State Between Backends

Moving state between backends is the riskiest routine operation in Python IaC — a botched cutover orphans resources or duplicates them — so this guide walks the export/import and pull/push procedures for a verified, zero-downtime migration, part of Managing IaC State for Python Projects under Python IaC Fundamentals & Strategy.

You migrate state when you outgrow a local backend, consolidate onto a managed one, or change clouds. The cardinal rule is that the resources never change — only the ledger's location does — so every step is built around proving the new backend describes exactly the same infrastructure as the old one. Decide where you are migrating to first using Choosing a State Backend for Python IaC.

Context

A migration has three phases: freeze (stop all deploys so state cannot change mid-flight), transfer (export from the source, import to the destination), and verify (a no-op plan must show zero changes). Pulumi uses stack export/stack import; CDKTF delegates to Terraform's state pull/state push. Both keep a backup so you can roll back.

The same three phases in each toolchain The same three phases in each toolchain: comparison across Pulumi, CDKTF / Terraform. Phase Pulumi CDKTF / Terraform Freeze Disable deploy workflow Disable deploy workflow Snapshot stack export --file state pull > file Transfer login, stack init, stack import init -migrate-state Verify preview reports no changes plan -detailed-exitcode = 0
Both tools move a ledger, not infrastructure; only the command names differ.

What you are moving is a document, and it helps to know its shape before you move it. A Pulumi checkpoint is JSON with a version field and a deployment object holding an array of resources, each carrying a URN, a provider reference, its inputs and its outputs — plus a secrets_providers block recording how the sealed values were encrypted. A Terraform state file carries a version, a serial counter incremented on every write, a lineage UUID that identifies the state's ancestry, and a resources array of instances with their attributes.

Those bookkeeping fields are what make a transfer safe or unsafe. Terraform refuses to push a state whose lineage differs from the destination's, because that almost always means someone is about to overwrite an unrelated environment. Pulumi validates the resource graph on import and rejects a snapshot whose internal references do not resolve. Neither check knows anything about your cloud account — they are consistency checks on the document, which is why a clean import still has to be followed by a plan against real infrastructure.

Prerequisites

Conditions that must hold before the freeze starts Conditions that must hold before the freeze starts: layered from Destination chosen and provisioned down to Deploy freeze announced. Destination chosen and provisioned bucket, lock table, encryption, versioning Credentials for both backends read on the source, write on the destination Secrets provider decision made same key, or an explicit re-seal step Rehearsed on a non-production stack same commands, same tool versions Deploy freeze announced with an agreed window and an owner
Every condition above is cheaper to satisfy before the window than to discover inside it.
  • Python 3.9+ with the pulumi CLI or cdktf CLI plus the Terraform binary.
  • Read access to the source backend and write access to the destination backend.
  • A maintenance window or a deploy freeze announced to the team.
  • Versioning enabled on the destination store so a bad push can be rolled back.
  • A verified backup of the current state (the very first step below).
  • Identical tool versions on the machine doing the migration and in CI — a state file written by a newer Terraform is rejected by an older one, and the migration host is often someone's laptop.

Implementation

Who touches what during the transfer Who touches what during the transfer: Operator → Source backend → Destination → Cloud API. Operator Source backend Destination Cloud API export snapshot checkpoint JSON init + import plan / preview live resource state compare: zero diff
The cloud API is read but never written: a migration that calls a mutating API has gone wrong.

1. Freeze deploys and back up

Nothing may mutate state during the migration. Capture a backup before touching anything.

# Pulumi: snapshot the current stack state to a local file.
pulumi stack select prod
pulumi stack export --file prod-backup.json

# CDKTF/Terraform: pull current state from the source backend.
terraform -chdir=cdktf.out/stacks/prod state pull > prod-backup.tfstate
# State implication: these files are the rollback target — store them safely; they contain secrets.

The freeze has to be enforced, not requested. Disable the deploy workflow in CI, and — if the source backend supports it — revoke write access for the deploy role for the duration of the window. A colleague merging a pull request during the transfer is the single most common way a migration ends with two divergent ledgers.

# CLI: python -c "from migrate import StateBackup, Path; print(StateBackup(Path('prod-backup.json')).resource_count())"
from dataclasses import dataclass
from pathlib import Path
import json

@dataclass(frozen=True)
class StateBackup:
    path: Path

    def resource_count(self) -> int:
        # State implication: record this count now; it must match after migration.
        data = json.loads(self.path.read_text())
        # Pulumi export nests resources under deployment.resources
        return len(data.get("deployment", {}).get("resources", []))

    def urns(self) -> frozenset[str]:
        data = json.loads(self.path.read_text())
        resources = data.get("deployment", {}).get("resources", [])
        return frozenset(r["urn"] for r in resources)

Capture the URN set, not just the count — a migration that loses one resource and gains another keeps the count identical while quietly orphaning infrastructure. Comparing sets before and after is a two-line check that catches the failure a count never will.

2. Transfer to the new backend

Point the tool at the destination, then import the snapshot.

# Pulumi: log into the new backend, recreate the stack, import the snapshot.
pulumi login s3://my-new-iac-state
pulumi stack init prod
pulumi stack import --file prod-backup.json
# State implication: import writes the snapshot verbatim — no resources are created or destroyed.

If the destination uses a different secrets provider from the source, the sealed values in the snapshot cannot be read there, and the import produces a stack whose secrets are unusable. The reliable order is to export with --show-secrets, initialise the destination stack with its own provider, and import — the plaintext is re-sealed on the way in. Treat the intermediate file as a live credential: write it to a tmpfs path, and delete it in the same shell session.

# CLI: cdktf synth && terraform -chdir=cdktf.out/stacks/prod init -migrate-state
from constructs import Construct
from cdktf import TerraformStack

class MigratedStack(TerraformStack):
    def __init__(self, scope: Construct, ns: str) -> None:
        super().__init__(scope, ns)
        # Provider note: change ONLY the backend block; resources stay identical.
        self.add_override("terraform.backend", {"s3": {
            "bucket": "my-new-iac-state",
            "key": f"iac/{ns}/terraform.tfstate",
            "region": "us-east-1",
            "dynamodb_table": "iac-locks",
            "encrypt": True,
        }})

The key value deserves a moment's thought, because it is the one string that decides whether two stacks share a state object. Deriving it from the stack name, as above, guarantees a distinct path per stack; a hard-coded key copied between stacks is how two environments end up writing to the same object and destroying each other's resources on the next apply.

3. Re-initialize and push (CDKTF/Terraform)

Terraform can migrate state during init when the backend block changes.

# Terraform offers to copy existing state into the new backend on init.
cdktf synth
terraform -chdir=cdktf.out/stacks/prod init -migrate-state

# Or push an explicit backup if doing it manually:
terraform -chdir=cdktf.out/stacks/prod state push prod-backup.tfstate
# State implication: -migrate-state copies state; it does not modify real resources.

init -migrate-state prompts before copying; add -force-copy only in an automated runbook where you have already answered the question deliberately. The manual state push path is stricter: it compares lineage and serial with whatever is already at the destination and refuses a write that would overwrite an unrelated history. When the destination is genuinely empty, the push succeeds; when it is not, stop and find out what is there rather than reaching for -force.

Verification

A correct migration produces a plan with zero changes. This is the single most important check.

The post-import plan is not empty. Why? The post-import plan is not empty. Why?: choose among 4 options. Unexpected diff after import region Backend or providerregion differs secrets New provider cannotdecrypt values drift Real change happenedduring the freeze wrong stack Imported into thewrong stack name
Diagnose before applying: three of these four are fixed by configuration, not by a deploy.
# Pulumi: a no-op preview proves the new state matches reality.
pulumi preview --diff        # must report: no changes

# CDKTF/Terraform: plan must show 0 to add, 0 to change, 0 to destroy.
terraform -chdir=cdktf.out/stacks/prod plan
# CLI: python -c "from migrate import assert_no_drift; assert_no_drift('cdktf.out/stacks/prod')"
import subprocess

def assert_no_drift(stack_dir: str) -> None:
    # State implication: exit code 0 with -detailed-exitcode means zero drift; 2 means changes pending.
    result = subprocess.run(
        ["terraform", f"-chdir={stack_dir}", "plan", "-detailed-exitcode"],
        capture_output=True, text=True,
    )
    assert result.returncode == 0, f"Migration left drift: {result.stdout}"

Run three checks in order, and stop at the first failure. First the URN or address comparison against the backup, which proves nothing was dropped in transit. Then the no-op plan, which proves the ledger still describes reality. Finally a lock test — start a second plan while the first is running and confirm the destination reports a held lock, because a backend configured without its lock table looks perfectly healthy until two engineers apply simultaneously.

Only after a clean no-op plan should you lift the deploy freeze and decommission the old backend.

Gotchas & Edge Cases

The four ways a state transfer goes wrong The four ways a state transfer goes wrong: Failed cutover with 4 facets. Failed cutover Lineage mismatch state push into unrelated state history Integrity failure a resource references an unknown provider Split brain two backends both hold live state Lost secrets ciphertext sealed by a key you no longer use
The first two abort loudly; the second two are discovered days later.

A non-empty plan after import means the migration is wrong. If pulumi preview or terraform plan shows changes, do NOT apply. The most common cause is a region or provider config mismatch between source and destination. Re-check the backend block, fix it, and re-verify before unfreezing.

Pulumi secret provider changes break decryption. Importing into a backend with a different secrets provider can leave secrets unreadable. Migrate the secrets provider explicitly (pulumi stack change-secrets-provider) or keep the same provider during cutover.

Forgetting to delete the old state invites split-brain. If two backends both hold live state and someone runs against the old one, you get divergent, conflicting deploys. Only after verification, archive and then remove the source state object.

Error: Cannot import state with lineage ... over unrelated state. The destination already holds a state file from a different history. That is a safety net, not an obstacle to remove — confirm what is at that key before doing anything else, because -force here overwrites another environment's ledger.

error: snapshot integrity failure; refusing to use it. Pulumi found a resource in the imported snapshot whose parent or provider reference does not resolve, usually because the file was hand-edited. Restore the untouched backup and start again; editing a checkpoint by hand is a last resort that needs the URN graph kept consistent.

A stale lock blocks the first post-migration apply. If the source backend held a DynamoDB lock when you froze deploys, the record can outlive the migration. terraform force-unlock <lock-id> clears it, but verify first that no process is genuinely running — the lock exists precisely to stop the concurrent write that corrupts state.

The migration host has a newer CLI than CI. A state written by a newer Terraform is refused by an older binary with a version error, so the first CI run after the cutover fails even though the migration was correct. Pin the same version in both places before you start.

Operational Notes

State migration is the one operation where a mistake is unrecoverable, so the golden rule is: back up the current state file before touching anything, and never delete the source until the target proves itself. A migration is successful only when a preview against the new backend reports zero changes — that no-op is the evidence that every resource mapped across intact.

Safe migration Safe migration: back up state then configure target then migrate then no-op preview then cut over back up state configure target migrate no-op preview cut over
Every state migration ends in a no-op preview that proves nothing was lost.

Do the cutover behind unchanged resource names so no infrastructure is actually modified; you are moving the record of resources, not the resources themselves. Coordinate a lock or a change freeze during the window so no one applies against the old backend mid-migration, and rehearse the whole procedure in a non-production stack first, using the same drift discipline described in detecting and remediating state drift.

Sequence the environments so the cheapest failure comes first: a throwaway stack, then development, then staging, then production, with at least a day between the last rehearsal and the production window. Each pass is a real test of the runbook, and the mistakes it surfaces — a missing permission, a forgotten lock table, a secrets provider nobody had considered — are all cheaper to find in an environment nobody is paged for.

Keep the source state for a defined retention period rather than deleting it at cutover. Move the object to a read-only prefix or a separate archive bucket, keep it for a release cycle, and record its location in the change ticket. The point is not that you expect to need it — the no-op plan already proved you do not — but that the cost of keeping a few megabytes for a month is nothing against the cost of discovering, three weeks later, that one resource was never in the snapshot at all.

Finally, write down what changed operationally, not just what moved. A new backend usually means new credentials in CI, a new lock mechanism, a different failure mode when the store is unavailable, and often a different retention and versioning policy. Update the runbook, the CI secrets, and the on-call notes in the same change as the migration, or the next incident will be someone discovering that the documented recovery path points at a bucket that no longer holds anything.

FAQ

Is state migration zero-downtime? Yes for the running infrastructure — resources are never touched. The only "downtime" is a freeze on deploys during the transfer, which should last minutes.

How do I roll back a failed migration? Re-import the backup you captured in step 1 (pulumi stack import --file prod-backup.json or terraform state push prod-backup.tfstate) into the original backend, then verify with a no-op plan.

Can I migrate Pulumi state into a CDKTF/Terraform backend? No. The formats are incompatible. Moving between Pulumi and Terraform means re-importing resources with pulumi import or terraform import, not a state copy.

Do I need to stop the application during migration? No. Only IaC deploys are frozen; the workloads themselves keep serving traffic because no cloud resources are modified.

How long should the deploy freeze last? For a single stack of a few hundred resources, minutes: the export, import and verification are all fast, and the plan against the cloud API dominates. Budget an hour for the window anyway, because the time you need is not the happy path but the decision point when the first plan comes back non-empty.

Can I migrate several stacks at once? Migrate them one at a time, verifying each before starting the next, unless they share cross-stack references — in which case move the whole set inside one freeze, since a producer on the new backend and a consumer on the old one is exactly the split-brain state you are trying to avoid.