Converting Existing Terraform HCL to CDKTF Python

Migrating legacy HCL to programmatic infrastructure requires strict state preservation. Converting to CDKTF Python eliminates configuration drift and enables deterministic CI/CD validation pipelines. This task sits within Terraform Provider Bridging, which translates Terraform provider schemas into the typed Python classes your converted code will target.

This guide details the exact workflow for safe translation. We prioritize state integrity and secure credential handling. Every step includes testable Python IaC patterns. The wider synthesis model — how a Python construct tree becomes cdktf.out/stacks/<name>/cdk.tf.json and then a Terraform plan — is covered in CDKTF Workflows & Terraform Synthesis.

The single fact that governs every decision below: conversion changes the code, not the infrastructure. Terraform tracks a resource by its address in state (aws_s3_bucket.data), and CDKTF derives that address from the construct path, not from your old HCL block labels. If those two strings disagree at the moment you run cdktf deploy, Terraform will not "notice the rename" — it will plan a create for the new address and a destroy for the old one. Every step below exists to make the addresses agree before a plan is ever applied.

Prerequisites

• Terraform CLI 1.5 or later on PATH. The CDKTF CLI shells out to it for provider schema resolution, and terraform plan -detailed-exitcode behaves consistently from 1.5 onward.
• Node.js 18+ and cdktf-cli installed globally (npm install --global cdktf-cli@latest). The Python bindings are generated by the CLI, so the CLI version and the cdktf PyPI package version must be in the same minor series.
• Python 3.9+ with a virtual environment holding cdktf, constructs, and the pre-built provider package cdktf-cdktf-provider-aws.
• Read access to the existing remote backend and permission to write a lock record — terraform state pull needs the former, terraform import needs both.
• The original HCL working directory, initialised (terraform init) and with a clean terraform plan. Do not begin a conversion against a configuration that already reports drift; you will not be able to tell conversion errors from pre-existing drift.

Pre-Migration State Audit & Backend Locking

Before modifying infrastructure code, verify remote state integrity. Enforce backend locking immediately. Immutable state snapshots prevent catastrophic resource recreation during translation.

Always export a full state backup before executing synthesis commands. Cross-reference provider configurations against your target environment.

CLI: State Audit & Backup

terraform state list
terraform state pull > state-backup.json

Validate the backend lock status. Confirm all resource addresses resolve correctly. For comprehensive backend compatibility and lock mechanisms, review CDKTF Workflows & Terraform Synthesis.

terraform state pull emits the full state document including the serial and lineage fields. Record both. lineage is a UUID minted when the state was first created and never changes; serial increments on every write. They are your proof of identity if you later have to push a restored snapshot — terraform state push refuses a document whose lineage differs from the remote one with Error: Invalid state file: Lineage does not match, and refuses an older serial unless you pass -force. Writing both numbers into the migration ticket before you touch anything turns a panicked rollback into a two-command operation.

With an S3 backend the lock is a DynamoDB item keyed on <bucket>/<key>-md5. If a colleague's plan is running, your terraform state pull still succeeds (reads are unlocked) but any write fails with:

Error: Error acquiring the state lock

Error message: operation error DynamoDB: PutItem, ConditionalCheckFailedException:
The conditional request failed
Lock Info:
  ID:        6f4c0f9a-3f2b-4d0f-9a1e-2b7e1a55c0d1
  Operation: OperationTypePlan
  Who:       ci-runner@build-4417

Do not reach for terraform force-unlock here. During a conversion the only safe response is to wait, because the holder of that lock may be mid-apply and force-unlocking lets two writers race the same state document. Announce a change freeze on the original HCL directory for the duration of the conversion instead — the whole migration is measured in hours, not days.

Validation Steps:

• Run terraform state list to verify resource inventory
• Export terraform state pull > state-backup.json for immutable rollback
• Confirm backend lock status via provider dashboard or CLI output

Automated Translation via cdktf convert

The cdktf convert utility generates baseline Python constructs from legacy HCL files. It does not produce production-ready code. You must explicitly pin provider versions to prevent silent breaking changes.

CLI: Automated HCL Translation

cdktf init --template="python" --local
# Convert a single HCL file to Python
cat main.tf | cdktf convert --language python --provider "hashicorp/aws@~> 6.0" > main.py

After generation, verify cdktf.json provider constraints match your requirements.txt. Replace dynamic HCL blocks with native Python iteration patterns. Run cdktf synth to validate the initial JSON payload.

cdktf convert reads from stdin and writes to stdout — it is a pure text transformer with no knowledge of your project. It parses the HCL into the same intermediate representation the CDKTF CLI uses, resolves attribute names against the provider schema named in --provider, and prints Python. That design has three consequences worth internalising before you trust the output:

• The provider argument selects the schema, so a wrong version silently renames arguments. Passing hashicorp/aws@~> 4.0 against HCL written for provider 6.x produces code that references attributes that no longer exist. The failure surfaces much later, at synth, as TypeError: __init__() got an unexpected keyword argument 'server_side_encryption_configuration'.
• module blocks do not become Python classes. A module "vpc" { source = "terraform-aws-modules/vpc/aws" } converts to a TerraformHclModule call that still points at the registry source. That is usually what you want for a first pass; see Adopting Terraform Modules into a CDKTF Python Stack for the decision about when to keep the HCL module and when to reimplement it as a construct.
• The terraform block is dropped. Your backend "s3" and required_providers stanzas do not survive the conversion. Backend configuration moves into Python as an S3Backend(self, bucket=..., key=..., dynamodb_table=...) call inside the stack constructor, and version constraints move into cdktf.json. Forgetting this is how a converted stack quietly writes a local terraform.tfstate next to cdk.tf.json and then reports every existing resource as needing creation.

Run cdktf get after editing cdktf.json so the generated bindings under .gen/ match the pinned constraint. If you skip it, cdktf synth fails at import time with ModuleNotFoundError: No module named 'imports.aws', which reads like a Python packaging bug and is actually a stale codegen directory. Pinning is covered in depth in Pinning Terraform Provider Versions in CDKTF.

Validation Steps:

• Audit cdktf.json for exact provider version constraints
• Run cdktf synth and inspect the output directory
• Compare synthesized JSON against the original terraform plan output for parity

Python 3.9+ Type Enforcement & Construct Refactoring

Automated translation strips type safety. Apply strict typing annotations to all construct parameters. Replace HCL count and for_each directives with Python list comprehensions or dictionary mappings.

# CLI: cdktf synth --app "python main.py"
from typing import Dict, List, Optional, Any
import os
from constructs import Construct
from cdktf import TerraformStack
from cdktf_cdktf_provider_aws.provider import AwsProvider
from cdktf_cdktf_provider_aws.s3_bucket import S3Bucket

class MigratedStack(TerraformStack):
    def __init__(
        self,
        scope: Construct,
        ns: str,
        config: Dict[str, Optional[Any]],
    ) -> None:
        super().__init__(scope, ns)

        # Prefer OIDC/IAM roles over static credentials in production
        AwsProvider(
            self,
            "aws",
            region=config.get("region", "us-east-1"),
        )

        bucket_name = config.get("bucket_name")
        if not bucket_name:
            raise ValueError("bucket_name is required for S3 provisioning")

        # State implication: the construct id "data" becomes the Terraform
        # address aws_s3_bucket.data — changing it renames the resource in
        # state and triggers a destroy/create pair.
        S3Bucket(
            self,
            "data",
            bucket=bucket_name,
            tags=config.get("tags", {}),
        )

Enforce mypy --strict or pyright checks before synthesis. This catches AttributeError exceptions caused by untyped nested configurations.

The interesting refactor is for_each. HCL's for_each = toset(var.environments) produces addresses of the form aws_s3_bucket.data["staging"] — a single resource block with instance keys. A Python loop produces aws_s3_bucket.data_staging — separate resource blocks. Those are different state addresses, so a loop is not a drop-in replacement unless you also plan the import addresses to match. The typed pattern below makes the mapping explicit rather than incidental:

# CLI: mypy --strict main.py && cdktf synth
from dataclasses import dataclass, field
from typing import Dict, List
from constructs import Construct
from cdktf import TerraformStack, S3Backend
from cdktf_cdktf_provider_aws.provider import AwsProvider
from cdktf_cdktf_provider_aws.s3_bucket import S3Bucket
from cdktf_cdktf_provider_aws.s3_bucket_versioning import (
    S3BucketVersioningA,
    S3BucketVersioningVersioningConfiguration,
)


@dataclass(frozen=True)
class BucketSpec:
    """One converted `for_each` element, with its old HCL key preserved."""
    hcl_key: str          # the map key from the original for_each
    bucket_name: str
    versioned: bool = True
    tags: Dict[str, str] = field(default_factory=dict)


class ConvertedBucketStack(TerraformStack):
    def __init__(self, scope: Construct, ns: str, specs: List[BucketSpec]) -> None:
        super().__init__(scope, ns)

        AwsProvider(self, "aws", region="eu-west-1")

        # Provider note: S3Backend must be declared in Python — the HCL
        # `backend "s3"` block is discarded by cdktf convert.
        S3Backend(
            self,
            bucket="acme-tfstate",
            key="converted/buckets.tfstate",
            region="eu-west-1",
            dynamodb_table="acme-tfstate-locks",
            encrypt=True,
        )

        for spec in specs:
            # State implication: construct id == the import address suffix.
            # Old address: aws_s3_bucket.data["logs"]
            # New address: aws_s3_bucket.data_logs
            bucket = S3Bucket(
                self,
                f"data_{spec.hcl_key}",
                bucket=spec.bucket_name,
                tags=spec.tags,
            )
            if spec.versioned:
                S3BucketVersioningA(
                    self,
                    f"data_{spec.hcl_key}_versioning",
                    bucket=bucket.id,
                    versioning_configuration=S3BucketVersioningVersioningConfiguration(
                        status="Enabled",
                    ),
                )

Two details in that snippet are easy to get wrong. S3BucketVersioningA carries the trailing A because the AWS provider exposes both a legacy aws_s3_bucket_versioning attribute and a top-level resource of the same name, and JSII disambiguates the collision by suffixing. And bucket=bucket.id rather than bucket=spec.bucket_name keeps the dependency edge in the graph — passing the literal string produces a valid plan with no ordering guarantee, so the versioning resource can be attempted before the bucket exists.

Validation Steps:

• Add explicit type hints to all __init__ parameters
• Replace HCL loops with Python comprehensions
• Execute mypy --strict main.py and resolve all warnings

State Reconciliation & Drift Detection

Legacy resource addresses rarely match CDKTF-generated identifiers. Map legacy IDs to the new construct tree safely. Use terraform import in the synthesized output directory to reconcile state files without triggering destructive replacements.

CLI: State Address Mapping & Import

# 1. Synthesize to generate the Terraform configuration
cdktf synth

# 2. Import existing resources into the synthesized Terraform state
terraform -chdir=cdktf.out/stacks/ import aws_s3_bucket.data 

# 3. Verify zero drift after import
terraform -chdir=cdktf.out/stacks/ plan -detailed-exitcode

Work out the new address before you run the import, not after. CDKTF joins the construct path with underscores: a resource created directly on the stack with id data synthesizes to aws_s3_bucket.data, while the same resource nested inside a component construct named storage synthesizes to aws_s3_bucket.storage_data. Read the address out of the synthesized JSON rather than predicting it — jq -r '.resource.aws_s3_bucket | keys[]' cdktf.out/stacks/<stack-name>/cdk.tf.json prints exactly the strings terraform import expects. If you need an address to match the legacy one verbatim (which avoids the import entirely when you are also migrating the state file), call override_logical_id("data") on the resource; it bypasses the construct-path derivation and pins the address.

Two failure signatures show up here constantly. Importing an address that Terraform already tracks gives Error: Resource already managed by Terraform — aws_s3_bucket.data is already managed via import, which means you ran the import twice or the state file was not as empty as you assumed. Importing an address with no matching block in cdk.tf.json gives Error: resource address "aws_s3_bucket.data" does not exist in the configuration — that is a synth you forgot to re-run after renaming a construct.

Note also that cdktf.out/ is regenerated on every synth. Any state Terraform wrote into that directory is disposable, which is precisely why the S3Backend declaration in the previous step is not optional: without it, your carefully imported state lives in a directory the next cdktf synth will overwrite.

When addressing provider-specific state mapping and ID translation, consult Terraform Provider Bridging. If your original HCL declared aliased or multi-region providers, replicate that topology with the patterns in Using Multiple Terraform Providers in One CDKTF Stack so imported resources resolve to the correct provider. A -detailed-exitcode of 0 from terraform plan confirms zero drift.

Validation Steps:

• Run cdktf synth to generate updated JSON payloads
• Execute terraform import for each legacy resource in the synthesized stack directory
• Validate terraform plan -detailed-exitcode returns 0 (no changes)

Production Rollback & CI/CD Pipeline Handoff

Safe deployment requires automated rollback triggers. Configure your CI/CD runner to execute plan validation before any production deployment. Implement state snapshot restoration on failure to maintain infrastructure consistency.

import sys
import subprocess

def validate_drift(stack_name: str) -> int:
    """Programmatic drift validation for CI/CD integration.

    Returns 0 if no changes detected, 1 if drift found or plan failed.
    Uses terraform plan --detailed-exitcode: 0=no changes, 2=changes pending, 1=error.
    """
    cdktf_out_dir = f"cdktf.out/stacks/{stack_name}"
    result = subprocess.run(
        ["terraform", "-chdir", cdktf_out_dir, "plan", "-detailed-exitcode"],
        capture_output=True,
        text=True,
        check=False,
    )
    if result.returncode == 0:
        print("State matches synthesized output: no changes pending.")
        return 0
    elif result.returncode == 2:
        print("Drift detected: changes are pending.", file=sys.stderr)
        return 1
    else:
        print(f"Plan failed:\n{result.stderr}", file=sys.stderr)
        return 1

if __name__ == "__main__":
    stack = sys.argv[1] if len(sys.argv) > 1 else "MigratedStack"
    sys.exit(validate_drift(stack))

Establish pre-deploy validation hooks. Schedule post-deploy drift monitoring. Define explicit failure boundaries to prevent partial deployments.

Validation Steps:

• Run cdktf synth then terraform plan before cdktf deploy
• Implement automated state snapshot restore on pipeline failure
• Schedule drift detection jobs for continuous compliance monitoring

Operational Notes

A conversion of any size is a sequencing problem, not a translation problem. The translation is mechanical and reviewable in an afternoon; the sequencing is what determines whether a mistake costs you a rerun or a restore. Order the work so that the cheapest-to-fix resource classes go first and the ones whose replacement is destructive go last, so that by the time you touch a database you have already validated the import mechanics on something disposable.

Run both configurations in parallel, briefly. Keep the HCL directory on disk and initialised, pointing at a copy of the state, until the converted stack has produced a zero-diff plan twice on separate days. The second plan catches provider-side normalisation: some attributes (an S3 bucket policy document, an IAM policy JSON blob) are reformatted by the API on write, so a plan that is clean immediately after import can become dirty once a nightly refresh pulls the canonical form back. Diffing the two is trivial when both directories still exist and impossible once you have deleted the HCL.

Give the converted stack its own state key, not the original one. Point S3Backend(key=...) at converted/<stack>.tfstate and import into that fresh document. The original state file then remains a byte-for-byte untouched rollback target rather than something you have mutated in place. Disk is free; a corrupted production state at 02:00 is not.

Budget for the review, not the conversion. The generated Python is unidiomatic — deeply nested dict literals where a dataclass belongs, positional construct ids that repeat the resource type, Token.as_string() calls the converter inserts defensively. Reviewers who see a 4,000-line machine-translated diff will approve it without reading it. Convert one HCL file per pull request, with the corresponding cdk.tf.json diff attached, so the reviewer is comparing synthesized Terraform against the original Terraform rather than reading Python and guessing.

Wire the drift job before the migration finishes, not after. The validate_drift helper above is worth running on a schedule from the moment the first resource is imported. During the conversion window it is the only signal that distinguishes "my Python is wrong" from "someone applied the old HCL behind my back", and those two look identical in a plan output.

Common Mistakes

• Assuming cdktf convert yields production-ready code without manual type annotation.
• Failing to pin provider versions in cdktf.json, causing silent synthesis drift.
• Deploying before executing terraform import for existing resources, causing state divergence.
• Ignoring Python typing boundaries, leading to runtime AttributeError on nested configs.
• Skipping cdktf synth validation in CI/CD, resulting in undetected JSON payload errors.

FAQ

How do I preserve existing Terraform state during CDKTF conversion? Lock the remote backend. Export a state snapshot with terraform state pull. Run cdktf synth. Use terraform import in the synthesized output directory to map legacy addresses before deploying.

Does cdktf convert handle dynamic blocks and for_each correctly? It generates baseline equivalents. Complex loops require manual refactoring into native Python comprehensions with strict type hints. Dynamic blocks typically become Python if statements or loops over construct IDs.

How do I enforce Python 3.9+ typing for complex infrastructure variables? Use typing.Dict[str, Any], typing.Optional, and typing.List for heterogeneous inputs. Validate at instantiation with pydantic or isinstance checks to prevent runtime synthesis failures.

What is the safest rollback procedure if cdktf deploy fails? Halt the pipeline immediately. Restore the pre-migration state snapshot via terraform state push. Revert to the legacy branch until drift is reconciled. Run terraform plan to verify the restored state before re-attempting the migration.

Can I convert a whole directory of .tf files in one command? Not directly — cdktf convert reads a single stream from stdin. Concatenating the directory (cat *.tf | cdktf convert --language python) works only if the files share one provider configuration and no duplicate local names; otherwise the converter emits Error: Duplicate variable declaration. Converting file by file into separate Python modules is slower but produces a diff a human can actually review.

Do I have to import resources one at a time, or can I move the existing state file instead? You can move it, and for large estates you should. Copy the original state to the new backend key, then use terraform state mv aws_s3_bucket.data aws_s3_bucket.data_logs for each address CDKTF derives differently — or avoid the renames entirely with override_logical_id. That path is faster than re-importing hundreds of resources and it preserves the lineage, but it mutates a real state document, so take the snapshot first.

Why does my converted stack plan a replacement even though nothing changed? Almost always a force_new attribute whose default differs between the HCL you wrote and the schema default the converter emitted explicitly. Run terraform plan and look for # forces replacement next to the offending attribute; the fix is to set the attribute in Python to the value already in state, not to accept the replacement.

Key Takeaways

The cdktf convert tool gets you 60-70% of the way through a migration—the rest is manual type annotation, construct refactoring, and state import. The most critical step is the terraform import phase: skipping it and deploying directly will either fail (if resources already exist and names conflict) or create duplicate resources (if CDKTF uses different naming). Budget time for this and validate zero drift before declaring the migration complete.

Related

• Terraform Provider Bridging — the parent guide on translating provider schemas into typed Python classes.
• Using Multiple Terraform Providers in One CDKTF Stack — replicate aliased and multi-region provider topologies from your original HCL.
• State Backend Configuration for CDKTF — lock and back up remote state before running conversion and import.