Migrating legacy bash scripts to Python IaC

Legacy shell scripts often accumulate implicit state, unvalidated environment variables, and fragile subprocess chains. Migrating to Python-based Infrastructure as Code (IaC) eliminates these failure modes.

You gain declarative state tracking, strict type enforcement, and reproducible execution graphs. This guide details the exact workflow for transitioning imperative bash logic into Pulumi or CDKTF resource definitions.

State integrity and secure credential handling remain the primary engineering objectives throughout this process.

Pre-Migration Audit and State Mapping

Begin by cataloging every external dependency your bash scripts invoke. Shell scripts frequently rely on implicit state stored in local files, hardcoded resource IDs, or transient environment variables.

Extract all environment references before writing a single line of Python. This prevents silent configuration drift during the transition.

🖥️ CLI: Extract and catalog environment variables

# List all exported variables referenced in legacy scripts
grep -oE '\$[A-Z_]+|${[A-Z_]+}' legacy_provision.sh | sort -u > env_inventory.txt

# Trace external API calls and CLI dependencies
grep -E '(curl|aws|gcloud|az|kubectl|terraform)' legacy_provision.sh | sort -u

Map each imperative step to a declarative IaC resource. Distinguish clearly between stateless operations (e.g., formatting disks) and stateful operations (e.g., provisioning VPCs).

Stateless steps belong in configuration management or CI pipelines. Stateful steps must map directly to provider resources in your Python stack.

Document every configuration input using strict Python 3.9+ type hints. Untyped configuration is the primary cause of runtime stack failures. Review foundational architectural patterns in Python IaC Fundamentals & Strategy to align your migration with declarative best practices.

Environment Isolation and Dependency Pinning

Python IaC requires a deterministic runtime. Global package installations introduce version conflicts that corrupt stack state.

Isolate your toolchain using virtual environments and strict dependency lockfiles. Pin both the IaC framework and cloud provider SDKs to exact versions.

🖥️ CLI: Initialize isolated workspace and install toolchains

# Create and activate a Python 3.9+ virtual environment
python3.9 -m venv .venv
source .venv/bin/activate

# Install Pulumi or CDKTF with exact version pinning
pip install "pulumi>=3.100.0,<3.101.0" "pulumi-aws>=6.20.0,<6.21.0"
# OR for CDKTF
pip install "cdktf>=0.20.0,<0.21.0" "cdktf-cdktf-provider-aws>=18.0.0,<18.1.0"

# Generate reproducible lockfile
pip freeze > requirements.lock

Validate your workspace configuration before proceeding. Ensure the CLI can authenticate using secure credential managers rather than plaintext exports.

Proper workspace isolation prevents cross-stack contamination. Follow standardized isolation patterns documented in Setting Up Dev Environments to guarantee consistent CI/CD execution.

Translating Bash Logic to Typed Python Constructs

Replace subprocess.run() calls with native provider resources. Wrapping legacy shell commands inside IaC breaks the declarative state graph and bypasses provider SDK validation.

Implement strict typing for all configuration inputs. Use dataclasses to enforce schema compliance at stack initialization.

from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Optional, Any

@dataclass(frozen=True)
class StackConfig:
 """Strictly typed replacement for bash environment expansion."""
 region: str
 vpc_cidr: str
 instance_type: str
 environment: str = "production"
 tags: Optional[dict[str, Any]] = None
 subnet_ids: list[str] | None = None

 def __post_init__(self) -> None:
 if not self.vpc_cidr.startswith("10.") and not self.vpc_cidr.startswith("172."):
 raise ValueError("Invalid CIDR block: must be RFC1918 compliant")
 if self.instance_type not in ("t3.micro", "t3.small", "m5.large"):
 raise ValueError(f"Unsupported instance type: {self.instance_type}")

def load_config_from_env() -> StackConfig:
 """Parse environment variables with explicit type coercion."""
 tags_raw = os.getenv("RESOURCE_TAGS")
 tags = eval(tags_raw) if tags_raw else None # Use json.loads in production
 
 return StackConfig(
 region=os.environ["AWS_DEFAULT_REGION"],
 vpc_cidr=os.environ["VPC_CIDR"],
 instance_type=os.environ.get("INSTANCE_TYPE", "t3.micro"),
 environment=os.environ.get("ENVIRONMENT", "production"),
 tags=tags,
 )

Convert iterative bash provisioning into declarative resource graphs. Bash loops execute sequentially and mask dependency failures. Python IaC constructs explicit dependency chains using Output.all() and list comprehensions.

import pulumi
import pulumi_aws as aws

def provision_worker_nodes(
 subnet_ids: pulumi.Output[list[str]],
 instance_type: str,
 count: int = 3
) -> list[aws.ec2.Instance]:
 """Replaces bash for-loops with declarative resource dependencies."""
 nodes = []
 
 # Pulumi resolves dependencies asynchronously during graph construction
 for idx in range(count):
 node = aws.ec2.Instance(
 f"worker-node-{idx}",
 instance_type=instance_type,
 ami="ami-0c55b159cbfafe1f0", # Replace with SSM parameter lookup
 subnet_id=subnet_ids[idx % len(subnet_ids)],
 tags={"role": "worker", "index": str(idx)},
 opts=pulumi.ResourceOptions(
 protect=True, # Prevent accidental deletion
 depends_on=[subnet_ids]
 )
 )
 nodes.append(node)
 
 return nodes

# Example usage within a Pulumi stack
config = load_config_from_env()
vpc = aws.ec2.Vpc("legacy-migration-vpc", cidr_block=config.vpc_cidr)
subnets = [aws.ec2.Subnet(f"subnet-{i}", vpc_id=vpc.id, cidr_block=f"10.0.{i}.0/24") for i in range(3)]
subnet_ids = pulumi.Output.all(*[s.id for s in subnets])

workers = provision_worker_nodes(subnet_ids, config.instance_type)
pulumi.export("worker_ids", pulumi.Output.all(*[w.id for w in workers]))

Handle legacy exit codes by mapping them to provider error handlers or custom dynamic resources. Never swallow shell errors in IaC.

State Import, Drift Detection, and Validation

Existing infrastructure must be synchronized with the new Python state file before any modifications occur. Blind imports cause resource duplication and orphaned cloud assets.

Execute explicit import commands using verified resource identifiers. Always validate the generated state against your typed configuration schema.

🖥️ CLI: State Recovery and Drift Detection Sequence

# 1. Export current state (if migrating from another tool)
pulumi stack export --file pre-migration-state.json

# 2. Import specific resources with explicit IDs
pulumi import aws:ec2/instance:Instance worker-1 i-0a1b2c3d4e5f6g7h8
cdktf import aws_instance.worker_1 i-0a1b2c3d4e5f6g7h8

# 3. Run non-destructive diff to verify mapping accuracy
pulumi preview --diff --stack dev
cdktf diff

# 4. Validate schema compliance before applying
python -m mypy stack_config.py --strict

Resolve import conflicts by comparing cloud resource metadata against your typed StackConfig definition. Partial state recovery requires manual reconciliation of missing attributes.

Run schema validation locally before committing state changes. Enforce strict mypy checks to catch type mismatches early.

Safe Rollback Strategies and Testing Boundaries

Failed deployments must not corrupt infrastructure state. Implement preview-only pipelines as the first line of defense.

Configure your CI/CD runner to halt execution on any non-zero exit code from pulumi preview or cdktf diff. Never allow unvalidated state mutations in production branches.

Maintain pre-migration state snapshots in version-controlled storage. If a deployment fails mid-execution, restore the previous state immediately.

🖥️ CLI: State Backup and Restore Workflow

# Backup current state before destructive operations
pulumi stack export > state-backup-$(date +%F).json

# Restore state on failure
pulumi stack import --file state-backup-2024-01-15.json

# Verify restoration integrity
pulumi stack export --show-secrets | jq '.version'

Define strict testing boundaries for your Python IaC code. Unit tests must never invoke live cloud APIs. Mock provider SDKs using unittest.mock or pytest-mock.

Validate resource graph construction by asserting Output dependencies and configuration values. Integration tests should run against isolated sandbox accounts only.

Common Pitfalls and Mitigations

Frequently Asked Questions

How do I safely import existing infrastructure managed by bash into Pulumi or CDKTF state? Use the provider's import CLI with explicit resource identifiers. Run a dry-run preview to verify mapping accuracy. Commit the generated state snapshot to version control before applying any configuration changes.

What is the recommended approach for handling bash environment variables in Python IaC? Replace shell variable expansion with typed configuration classes using Python 3.9+ dataclasses or pydantic models. Inject values via stack configuration files or secure secret managers. Never hardcode secrets in environment variables.

How do I implement safe rollback if a Python IaC migration fails mid-deployment? Maintain pre-migration state exports. Use stack-level snapshots and configure CI/CD pipelines to halt on non-zero exit codes. Restore state using the provider's import command and re-run preview to isolate the failure boundary.

Can I unit test Python IaC code that replaces complex bash provisioning logic? Yes. Isolate provider SDK calls using mocking frameworks. Validate resource graph construction with pytest. Enforce boundary tests that prevent actual cloud API calls during unit execution. Reserve live API calls for integration test suites only.