Generate CDKTF Python Code from Terraform State

There is a moment in every migration where somebody works out that the estate has three hundred resources and that transcribing them into Python by hand will take a month. The state file already contains a complete, provider-normalised description of every one of those resources, so the transcription is a text-processing job rather than a research job. This guide builds the small typed generator that does it, and is deliberate about the part people skip: which attributes must be thrown away, and why generated code is a first draft rather than a deliverable. It sits under importing existing infrastructure into CDKTF within CDKTF workflows and Terraform synthesis.

Fields of one entry in terraform show -json Fields of one entry in terraform show -json: layered from mode down to child_modules. mode managed or data — skip every data source type and name become the CDKTF class and the construct ID index int for count, string for for_each, absent for a single instance values the attribute map, including read-only fields sensitive_values flags which keys hold secrets — the values are not redacted child_modules the recursion the generator must not forget
Every decision the generator makes is a decision about one of these fields.

Context

Two different documents describe a managed resource, and picking the wrong one costs a day. The raw terraform.tfstate file is an internal serialisation format with a version field that HashiCorp is explicit about not treating as a public interface; its resources[].instances[].attributes map is the provider's own storage layout, and it carries index_key, schema_version, status and dependencies alongside the values. The output of terraform show -json is the documented, versioned view of the same data: a values.root_module.resources array where every entry has already been rendered through the provider schema.

Generate from show -json. It is stable across Terraform releases, it flattens nothing you need, and it exposes the module tree explicitly through child_modules, which matters the moment the estate you are adopting was built with modules.

The generator you are about to write is not a converter in the sense that cdktf convert is. That tool takes HCL and produces Python, so using it means running terraform plan -generate-config-out first, translating the result, and then discovering that the intermediate HCL discarded the attribute values you actually wanted. Going straight from state skips a lossy hop and gives you the real values the provider read from the cloud.

What it cannot do is recover intent. State stores vpc-04c9e1f7b3a25d6e0, not network.vpc.id. Every cross-resource reference in the original design has been collapsed into a literal string, and putting the references back is the human half of the job.

Prerequisites

  • Terraform 1.5 or newer, and a working directory that has been initialised against the state you intend to read
  • Read access to the state backend; the generator never writes state, but terraform show still acquires a read lock on some backends
  • Python 3.9 or newer with typing and dataclasses — no third-party dependencies are needed
  • The matching cdktf-cdktf-provider-aws (or equivalent) package installed, so you can check generated class names actually import
  • Somewhere private to write the output, because state values include secrets
# CLI: capture the two inputs the generator needs
terraform -chdir=infra/legacy show -json > /tmp/state.json
terraform -chdir=infra/legacy providers schema -json > /tmp/schema.json
jq '.values.root_module.resources | length' /tmp/state.json
jq -r '.values.root_module.resources[] | "\(.type).\(.name)"' /tmp/state.json | sort | head

Implementation

Step 1 — Learn the shape of one resource entry

Before writing any code, print a single entry and read it properly. Everything the generator does is a decision about one of these fields.

# CLI: inspect one entry in full, then just its keys
jq '.values.root_module.resources[] | select(.address == "aws_s3_bucket.reports")' /tmp/state.json
jq -r '.values.root_module.resources[0] | keys[]' /tmp/state.json
{
  "address": "aws_s3_bucket.reports",
  "mode": "managed",
  "type": "aws_s3_bucket",
  "name": "reports",
  "provider_name": "registry.terraform.io/hashicorp/aws",
  "schema_version": 0,
  "values": {
    "id": "acme-prod-reports",
    "arn": "arn:aws:s3:::acme-prod-reports",
    "bucket": "acme-prod-reports",
    "bucket_domain_name": "acme-prod-reports.s3.amazonaws.com",
    "force_destroy": false,
    "hosted_zone_id": "Z1BKCTXD74EZPE",
    "region": "eu-west-1",
    "tags": { "owner": "platform" },
    "tags_all": { "owner": "platform" }
  },
  "sensitive_values": { "tags": {}, "tags_all": {} }
}

mode separates managed resources from data sources — skip "data" entirely, since a data source has nothing to adopt. type and name give you the construct's identity. values is the whole payload, and nine of the keys above must never reach your Python.

Step 2 — Let the provider schema decide what to drop

The naive approach is a hand-maintained deny list containing id, arn, and whatever else bit you last week. It does not scale past one service, and it is unnecessary, because terraform providers schema -json states exactly which attributes are configurable. Each attribute carries some combination of required, optional and computed, and the rule is simple: emit an attribute only if it is required or optional.

Emit an attribute, or drop it Emit an attribute, or drop it: comparison across Example, Generator does. Schema flags Example Generator does required bucket, engine always emit optional force_destroy, tags emit when the value is set optional + computed availability_zone emit with a review comment computed only arn, id, hosted_zone_id drop — plan rejects it provider-owned tags_all drop — it mirrors another field
The provider schema answers this for every attribute, so no hand-maintained deny list is needed.

An attribute that is computed and nothing else is read-only. Setting one does not produce a diff — it produces a hard failure at plan time, and the message is unambiguous:

Error: Value for unconfigurable attribute

  with aws_s3_bucket.reports,
  on cdk.tf.json line 42, in resource.aws_s3_bucket.reports:
  42:         "arn": "arn:aws:s3:::acme-prod-reports",

Can't configure a value for "arn": its value will be decided automatically based on
the result of applying this configuration.

The genuinely awkward class is optional and computed together. tags_all, availability_zone, name_prefix and a great many similar fields accept a value but also fill themselves in when you omit them. Emitting them is legal, so the failure is quieter: the code compiles, the plan is clean today, and in six months a value that the provider would have derived is pinned to whatever it happened to be on the day you generated. Default to dropping the ones the provider is known to own — tags_all above all — and keep the rest with a review comment attached.

Step 3 — Write the generator

The whole thing is about a hundred and twenty lines. It loads both JSON documents, walks the module tree, filters each attribute map against the schema, and renders a construct call per resource.

# tools/state_to_cdktf.py — turn a Terraform state dump into draft CDKTF constructs
# CLI: python3 tools/state_to_cdktf.py --state /tmp/state.json --schema /tmp/schema.json --out draft_stack.py
from __future__ import annotations

import argparse
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, Iterator, List, Optional, Tuple

# Optional+computed fields the provider owns outright; keeping them pins a derived value.
PROVIDER_OWNED = {"tags_all", "arn", "id", "unique_id", "owner_id", "created_date"}


@dataclass(frozen=True)
class AttributeSpec:
    name: str
    required: bool
    optional: bool
    computed: bool

    @property
    def configurable(self) -> bool:
        return self.required or self.optional


ResourceSchemas = Dict[str, Dict[str, AttributeSpec]]


def load_schemas(path: Path) -> ResourceSchemas:
    doc: Dict[str, Any] = json.loads(path.read_text())
    out: ResourceSchemas = {}
    for provider in doc.get("provider_schemas", {}).values():
        for tf_type, entry in provider.get("resource_schemas", {}).items():
            block = entry.get("block", {})
            out[tf_type] = {
                name: AttributeSpec(
                    name=name,
                    required=bool(attr.get("required")),
                    optional=bool(attr.get("optional")),
                    computed=bool(attr.get("computed")),
                )
                for name, attr in block.get("attributes", {}).items()
            }
            # Nested blocks are configurable by definition; record them as optional.
            for name in block.get("block_types", {}):
                out[tf_type][name] = AttributeSpec(name, False, True, False)
    return out


def iter_managed(module: Dict[str, Any]) -> Iterator[Dict[str, Any]]:
    for res in module.get("resources", []):
        if res.get("mode") == "managed":
            yield res
    for child in module.get("child_modules", []):
        yield from iter_managed(child)


def python_binding(tf_type: str) -> Tuple[str, str]:
    """aws_s3_bucket -> ('cdktf_cdktf_provider_aws.s3_bucket', 'S3Bucket')."""
    provider, _, rest = tf_type.partition("_")
    module = f"cdktf_cdktf_provider_{provider}.{rest}"
    class_name = "".join(part.capitalize() for part in rest.split("_"))
    # Provider note: jsii appends an "A" when a resource name collides with a struct
    # name (S3BucketVersioningA). The generator cannot know; the reviewer must check.
    return module, class_name


def keep(spec: Optional[AttributeSpec], value: Any) -> bool:
    if spec is None or not spec.configurable:
        return False
    if spec.name in PROVIDER_OWNED:
        return False
    if value is None or value == [] or value == {}:
        return False
    return not (spec.computed and not spec.required and spec.name.endswith("_prefix"))

The rendering half is deliberately dumb. It emits repr() for scalars and containers, which is valid Python for every JSON value, and it leaves nested blocks as plain dictionaries with a marker comment rather than guessing a struct class name.

# tools/state_to_cdktf.py (continued) — rendering and the CLI entry point
# CLI: python3 tools/state_to_cdktf.py --state /tmp/state.json --schema /tmp/schema.json
def render_arguments(values: Dict[str, Any], specs: Dict[str, AttributeSpec]) -> List[str]:
    lines: List[str] = []
    for key in sorted(values):
        spec = specs.get(key)
        if not keep(spec, values[key]):
            continue
        rendered = repr(values[key])
        if isinstance(values[key], (list, dict)) and values[key]:
            lines.append(f"    {key}={rendered},  # TODO: replace with the typed struct class")
        else:
            lines.append(f"    {key}={rendered},")
    return lines


def render_resource(res: Dict[str, Any], schemas: ResourceSchemas) -> str:
    tf_type: str = res["type"]
    specs = schemas.get(tf_type, {})
    _, class_name = python_binding(tf_type)
    construct_id = res["name"] if res.get("index") is None else f"{res['name']}_{res['index']}"
    args = render_arguments(res.get("values", {}), specs)
    head = f'{class_name}(\n    self,\n    "{construct_id}",'
    return f"# from {res['address']}\n{head}\n" + "\n".join(args) + "\n)\n"


def main() -> int:
    ap = argparse.ArgumentParser(description="Draft CDKTF Python from a Terraform state dump.")
    ap.add_argument("--state", type=Path, required=True)
    ap.add_argument("--schema", type=Path, required=True)
    ap.add_argument("--out", type=Path)
    args = ap.parse_args()

    state: Dict[str, Any] = json.loads(args.state.read_text())
    schemas = load_schemas(args.schema)
    root = state.get("values", {}).get("root_module", {})

    imports = set()
    blocks: List[str] = []
    for res in sorted(iter_managed(root), key=lambda r: r["address"]):
        module, class_name = python_binding(res["type"])
        imports.add(f"from {module} import {class_name}")
        blocks.append(render_resource(res, schemas))

    body = "\n".join(sorted(imports)) + "\n\n\n" + "\n".join(blocks)
    # State implication: reading state never modifies it, but the output file WILL contain
    # every value the provider stored, including ones marked sensitive.
    if args.out:
        args.out.write_text(body)
        print(f"wrote {len(blocks)} constructs to {args.out}")
    else:
        print(body)
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Step 4 — Handle count and for_each expansions

A resource written once in HCL with count = 3 appears three times in the state dump, distinguished by an index field. An integer index came from count; a string index came from for_each. The raw state file calls the same field index_key, which is worth knowing when someone shares a terraform.tfstate excerpt with you instead of a show -json dump.

What the index field tells you What the index field tells you: index on a state entry with 4 facets. index on a state entry absent one declaration, one object — render a single construct call integer came from count — render a range loop over the varying arguments string came from for_each — render a loop over a dictionary literal mixed types a half-migrated resource; stop and read it by hand
Grouping instances by type and name before rendering recovers the original declaration.

Emitting three flat constructs is correct but ugly, and it loses the relationship that made them one declaration. Group the entries first, then decide.

# tools/expansions.py — collapse expanded instances back into one Python loop
# CLI: python3 -c "from tools.expansions import group_expansions; print(group_expansions([]))"
from __future__ import annotations

from collections import defaultdict
from typing import Any, Dict, List, Tuple


def group_expansions(resources: List[Dict[str, Any]]) -> Dict[Tuple[str, str], List[Dict[str, Any]]]:
    grouped: Dict[Tuple[str, str], List[Dict[str, Any]]] = defaultdict(list)
    for res in resources:
        grouped[(res["type"], res["name"])].append(res)
    return dict(grouped)


def expansion_kind(instances: List[Dict[str, Any]]) -> str:
    indexes = [inst.get("index") for inst in instances]
    if indexes == [None]:
        return "single"
    if all(isinstance(i, int) for i in indexes):
        return "count"
    if all(isinstance(i, str) for i in indexes):
        return "for_each"
    return "mixed"      # a partially migrated resource; always needs a human


def render_for_each(type_name: str, class_name: str, instances: List[Dict[str, Any]]) -> str:
    keys = ", ".join(repr(inst["index"]) for inst in sorted(instances, key=lambda r: str(r["index"])))
    return (
        f"# {type_name}: {len(instances)} instances keyed by {keys}\n"
        f"for key in [{keys}]:\n"
        f"    {class_name}(self, f\"{type_name}_\", **PER_KEY_ARGS[key])\n"
    )

CDKTF has no count or for_each; it has Python. A count expansion becomes for i in range(3), a for_each becomes a loop over a dictionary literal, and the per-instance differences become that dictionary's values. Deciding which attributes vary across the instances and which are constant is the one part of this that is genuinely worth a human's attention — and the payoff is real, because that dictionary is usually the configuration model the original author had in their head.

Verification

The generated file is not verified by reading it. Synthesize it and compare.

# CLI: does the draft produce the same resource blocks the state already describes?
cdktf synth
jq -S '.resource' cdktf.out/stacks/prod-data/cdk.tf.json > /tmp/generated.json
jq -S '[.values.root_module.resources[] | select(.mode=="managed") | .address] | sort' \
  /tmp/state.json > /tmp/expected.txt
python3 -c "import json;print(len(json.load(open('/tmp/generated.json'))))"

Then run the real check: point the generated stack at a copy of the original state and plan. An empty plan means the generator kept everything that mattered and dropped everything that did not.

# CLI: the acceptance test — a copy of state, never the original
aws s3 cp s3://acme-tfstate/legacy/terraform.tfstate /tmp/replica.tfstate
terraform -chdir=cdktf.out/stacks/prod-data plan -state=/tmp/replica.tfstate -detailed-exitcode
# 0 = the draft describes the estate exactly; 2 = attributes still missing or over-specified

Read the failures by direction. An update proposing to remove a value means the generator dropped an attribute it should have kept. An update proposing to add one means you kept something computed. A create means the address does not match, which is a construct-ID problem rather than an attribute problem.

Finally, run a type checker over the output before anyone reviews it. mypy will not know the provider bindings deeply, but it will catch every class name the generator got wrong — which, given the jsii suffix problem, is more of them than you expect.

Gotchas & Edge Cases

State contains secrets, and show -json does not redact them. The sensitive_values map flags which keys are sensitive; the values map still holds the plaintext. A database resource generates a construct call with password="..." in it. Add a redaction pass that replaces any key flagged in sensitive_values with a placeholder and a comment, and never write the generator's output into the repository working tree by default.

Nested blocks lose their types. A rule block arrives as a list of dictionaries. CDKTF wants a list of typed struct instances with names like S3BucketServerSideEncryptionConfigurationRuleA. There is no reliable mapping from the schema's block name to the jsii struct name, which is why the generator marks these with a TODO rather than guessing wrong and producing code that imports cleanly but is subtly incorrect.

Every reference is already resolved. Subnet IDs, security group IDs, role ARNs — all literals. Left alone, the generated stack is a hundred hard-coded identifiers with no dependency graph, so Terraform orders operations arbitrarily and a rebuild in a fresh account fails. Re-wiring literals into construct attributes is the main review task, not a polish step.

Data sources and provider blocks are absent. The generator skips mode == "data" by design, and provider_name tells you the source address but not which alias a resource used. If the estate uses aliased providers for multiple regions, you must reconstruct that from the plan JSON's provider_config_key, which the state view does not carry.

Tainted and deposed instances exist. The raw state file marks these with "status": "tainted" or a deposed key; the JSON view mostly hides them. Reconcile the resource count from terraform state list against what your generator emitted, and investigate any gap before assuming the dump was complete.

Schema versions drift. schema_version on a state entry reflects the provider version that wrote it. Generating against a much newer provider schema means attributes that have since been split into their own resource types appear in neither document properly. Upgrade the provider and run terraform refresh before generating, not after.

Operational Notes

Run the generator exactly once per resource. Its output is a scaffold: you review it, restructure it into real constructs with shared configuration, and commit the reviewed version. Re-running it over an evolving stack and diffing the result is a tempting workflow and a bad one, because it treats state as the source of truth for code that is supposed to be the source of truth for state.

Keep the generator in tools/ next to the stack it bootstrapped, pinned and unmaintained. It has one job, it does it during a migration window measured in weeks, and it should not accumulate features. If it needs a feature, that is usually a sign the resource in question deserves hand-written code.

Format aggressively before review. Run ruff format or black over the output, sort the imports, and split the file per service. A reviewer who is asked to check four hundred attribute assignments in one unformatted file will approve it without reading it, which defeats the point of calling it a draft.

Record the drop list in the commit message. When someone asks in a year why the code does not set hosted_zone_id, the answer — "it is computed, and the plan rejects it" — should be findable in version control rather than rediscovered by trying it. The same discipline described in Python typing for cloud resource definitions applies here: the generated dictionaries should become typed configuration objects before the migration closes, not after.

FAQ

Is it safe to run this against production state?

Reading is safe — terraform show -json opens no write lock and modifies nothing. The risk is entirely in the output file, which contains every secret the state holds. Generate into a temporary directory, redact, and only then move anything into a repository.

Why not use plan -generate-config-out and cdktf convert instead?

That path works and is the right call for a handful of resources, because it produces HCL that is guaranteed schema-valid. At scale it costs two lossy translations and gives you no control over which attributes survive. Going from state directly means the filtering rule is yours and is visible in code.

The generated code sets arn and the plan fails. What is the rule?

Emit an attribute only when the provider schema marks it required or optional. Anything that is only computed is read-only, and Terraform rejects it with Error: Value for unconfigurable attribute rather than silently ignoring it.

How do I handle resources that came from a Terraform module?

They live under values.root_module.child_modules[], each with its own address prefix like module.network.aws_subnet.private[0]. Recurse into them, and expect to restructure the result — a Terraform module usually maps onto a CDKTF construct class, not onto a flat list of resources in a stack.

Can the generator work out the right construct IDs?

Only partly. It uses the state's name, which reproduces the original HCL label, and that is a reasonable default. Where the estate was built by hand rather than by HCL, the names are whatever the earlier importer chose, and they are worth renaming deliberately before any of it is applied.

How much of the generated file typically survives review?

The attribute values almost entirely; the structure almost not at all. Expect to keep the numbers, strings and booleans, and to rewrite every reference, every repeated block and every module boundary by hand. That ratio is what makes the exercise worthwhile — the tedious part is automated and the judgement part is not.