Security & Compliance Basics

A Python IaC program is the cheapest place a security control will ever be expressed. The same rule — "no S3 bucket is publicly readable", "every RDS instance is encrypted", "no IAM statement grants * on *" — costs a ValueError at construction time, a failed CI job at synthesis time, or a Sunday-night incident review after the fact. This topic, part of Python IaC fundamentals and strategy, covers how to write those rules as code, where to enforce them so they cannot be bypassed, how to keep credentials out of state, and how to prove afterwards that the controls actually held.

Problem Framing

The traditional compliance loop runs backwards. Infrastructure is provisioned, a scanner sweeps the account some days later, findings land in a ticket queue, and an engineer who has moved on to other work is asked to remediate a resource that now has production traffic on it. Every step in that loop adds cost, and the last step adds risk: changing a live security group, rotating a key that six services read, or enabling encryption on a database that requires a snapshot-and-restore are all changes with blast radius. The violation was free to prevent and expensive to fix, and nothing about the process reflects that asymmetry.

Python IaC inverts the loop because the resource graph exists as data before it exists as infrastructure. A Pulumi preview and a CDKTF synth both produce a complete, machine-readable description of every resource the merge will create — arguments, tags, policy documents and all. Any rule that can be written as a predicate over that description can be evaluated in under a second, in a process that has no cloud credentials attached and therefore cannot cause damage even if it is wrong.

The real work is not the enforcement mechanism; it is the translation. A control framework says "protect data at rest using cryptographic mechanisms". It does not say storage_encrypted=True on aws_db_instance, encrypted=True on aws_ebs_volume, kms_key_id set to a customer-managed key rather than the account default, and copy_tags_to_snapshot=True so the encrypted snapshot is still attributable. Every control expands into a set of resource-level predicates, and that expansion is engineering judgement that has to be written down once and then applied mechanically. Teams that skip the writing-down step end up with scanners that produce hundreds of findings nobody can map back to an obligation.

The second piece of framing is that exceptions are inevitable and must be first-class. A public bucket that serves a static site is a legitimate exception to "no public buckets". If the only way to ship it is to disable the check globally, the check dies within a quarter. A workable system suppresses a specific finding on a specific resource, records who approved it and why, and makes the suppression visible in review. Treat every blanket --soft-fail as a control that has been quietly switched off.

Finally, none of this replaces the provider-side guardrails. Policy that runs in your pipeline protects you from your pipeline. It does nothing about a console click, a rogue script, or a second pipeline nobody told you about. Enforcement has to exist at more than one layer, which is the subject of a later section on this page.

Prerequisites

• Python 3.9+ with pulumi>=3.0 or cdktf>=0.20 pinned in the same environment your pipeline uses — see setting up dev environments for the reproducibility baseline
• checkov>=3.2 installed in the CI image, pinned to an exact version so a new release cannot fail an unrelated merge
• A CI identity that can run a preview with read-only credentials, and a separate deploy identity assumed only after the gates pass
• A KMS key (or equivalent) available for state encryption, with a key policy that names the deploy role explicitly
• Knowledge of which control framework you are actually being audited against — CIS AWS Foundations, SOC 2 CC6, PCI DSS and ISO 27001 expand into overlapping but not identical resource predicates

# CLI: confirm the scanning identity cannot mutate anything and the toolchain is pinned
aws sts get-caller-identity --query 'Arn' --output text
checkov --version
pulumi version
# Provider note: run the scan under the READ-ONLY role. A scanner that holds
# write credentials is itself a finding in most audit programmes.

Defining Security Baselines in Python IaC

Before provisioning cloud resources, teams must establish a baseline security posture by integrating policy-as-code directly into deployment pipelines. Align initial architecture with Python IaC Fundamentals & Strategy to ensure compliance controls are baked into the resource graph from day one. Shift-left validation prevents non-compliant infrastructure from ever reaching production.

A baseline in this sense is a concrete artifact, not a posture statement: a Python module that maps each control identifier to a predicate over resource arguments, plus the severity that a violation carries. Writing it as typed Python rather than a spreadsheet has two consequences. The mapping becomes importable by the stack code, so the same definition drives construction-time validation and after-the-fact reporting; and it becomes reviewable, so tightening CKV_AWS_18 from warning to blocking is a pull request with a diff and an approver rather than a setting somebody changed in a console.

Translating CIS/NIST Controls into Pulumi and CDKTF Resource Constraints

Map regulatory requirements directly to resource constructors using typed validation layers. Enforce mandatory encryption flags and restrict public access during object instantiation. Run pulumi preview --diff to verify constraint propagation before state mutation. For CDKTF, run cdktf synth and then parse the generated JSON with a policy tool before executing cdktf deploy.

The mechanism that makes this reliable is ordinary Python: a frozen dataclass whose __post_init__ rejects any combination of arguments the baseline forbids, and a factory function that is the only supported way to create the resource. Because the dataclass is constructed during program evaluation — before Pulumi's engine has sent a single RPC to the provider — a violation surfaces as a stack trace in the developer's terminal, with the file and line of the offending call, and nothing at all happens in the cloud account.

# baseline.py — control register + typed constructor for encrypted, private buckets
# CLI: pulumi preview --stack dev
from dataclasses import dataclass

import pulumi
import pulumi_aws as aws

APPROVED_REGIONS = frozenset({"eu-west-1", "eu-central-1"})


@dataclass(frozen=True)
class BucketBaseline:
    """CIS AWS 2.1.1 (encryption at rest) + 2.1.5 (block public access)."""

    name: str
    kms_key_arn: str
    region: str
    versioning: bool = True

    def __post_init__(self) -> None:
        if self.region not in APPROVED_REGIONS:
            raise ValueError(
                f"{self.name}: region {self.region!r} is outside the approved set "
                f"{sorted(APPROVED_REGIONS)} (control DR-02)"
            )
        if not self.kms_key_arn.startswith("arn:aws:kms:"):
            raise ValueError(
                f"{self.name}: kms_key_arn must be a customer-managed key ARN, "
                f"got {self.kms_key_arn!r} (control CIS-2.1.1)"
            )


def compliant_bucket(spec: BucketBaseline) -> aws.s3.BucketV2:
    bucket = aws.s3.BucketV2(spec.name, bucket=spec.name)

    aws.s3.BucketServerSideEncryptionConfigurationV2(
        f"{spec.name}-sse",
        bucket=bucket.id,
        rules=[aws.s3.BucketServerSideEncryptionConfigurationV2RuleArgs(
            apply_server_side_encryption_by_default=aws.s3.BucketServerSideEncryptionConfigurationV2RuleApplyServerSideEncryptionByDefaultArgs(
                sse_algorithm="aws:kms",
                kms_master_key_id=spec.kms_key_arn,
            ),
            bucket_key_enabled=True,
        )],
    )
    # State implication: BucketPublicAccessBlock is a separate resource, so
    # deleting it from the program REMOVES the protection on the next update.
    # Keep it inside this factory so it can never be dropped independently.
    aws.s3.BucketPublicAccessBlock(
        f"{spec.name}-pab",
        bucket=bucket.id,
        block_public_acls=True,
        block_public_policy=True,
        ignore_public_acls=True,
        restrict_public_buckets=True,
    )
    return bucket


artifacts = compliant_bucket(BucketBaseline(
    name="acme-artifacts-prod",
    kms_key_arn="arn:aws:kms:eu-west-1:123456789012:key/8f1c...",
    region="eu-west-1",
))
pulumi.export("artifacts_bucket", artifacts.id)

CDKTF expresses the same idea with a construct class rather than a factory, and the equivalent validation runs inside __init__ before any TerraformResource is instantiated. The important property in both frameworks is that the raw resource class is never called directly from stack code; a lint rule or a code-review convention that forbids importing aws.s3.BucketV2 outside baseline.py is what keeps the escape hatch closed.

Implementing Pre-Flight Validation Hooks in Python IaC Workflows

Attach synchronous validation functions to stack initialization routines. Fail fast on missing compliance tags or unapproved instance families before any provider call executes. Run pytest -k preflight locally to block unsafe configurations before CI triggers.

Factories only cover resources you route through them. To catch everything — including resources created inside a third-party component — use the framework's whole-graph hook. Pulumi exposes pulumi.runtime.register_stack_transformation, which is invoked once per resource with its final argument dictionary and can rewrite or reject it. CDKTF exposes the same capability through aspects: Aspects.of(stack).add(TagAspect()) calls visit(node) on every construct in the tree during synthesis.

# preflight.py — reject unapproved instance families across the whole graph
# CLI: pulumi preview --stack prod
from typing import Optional

import pulumi

APPROVED_FAMILIES = ("t3.", "m6i.", "r6g.", "c6i.")


def enforce_instance_families(
    args: pulumi.ResourceTransformationArgs,
) -> Optional[pulumi.ResourceTransformationResult]:
    if args.type_ != "aws:ec2/instance:Instance":
        return None
    instance_type = args.props.get("instanceType", "")
    if not instance_type.startswith(APPROVED_FAMILIES):
        raise ValueError(
            f"{args.name}: instance_type {instance_type!r} is not in an approved "
            f"family {APPROVED_FAMILIES} (control FIN-04)"
        )
    props = dict(args.props)
    props.setdefault("metadataOptions", {"httpTokens": "required"})
    return pulumi.ResourceTransformationResult(props=props, opts=args.opts)


# State implication: transformations run before the diff is computed, so adding
# metadataOptions shows as an UPDATE on existing instances, not a replacement.
pulumi.runtime.register_stack_transformation(enforce_instance_families)

Forcing IMDSv2 in the same hook is deliberate. A transformation that only rejects is a gate; a transformation that also supplies the secure default is a paved road, and paved roads are adopted while gates are worked around. Reserve hard rejection for the cases where a safe default does not exist — an unapproved region, a wildcard IAM action, a database without a KMS key.

Structuring Compliance Metadata for Automated Auditing

Embed immutable audit trails using resource-level tags and custom metadata. Standardize naming conventions across environments to simplify log correlation. Query state exports via pulumi stack export or terraform state pull to generate compliance manifests.

Tags are the join key between the infrastructure graph and every downstream system that has to answer questions about it — the billing report, the vulnerability scanner, the incident-response runbook. Settle on a small, mandatory set (owner, data-classification, control-baseline, deployed-by) and apply it at the provider level with default_tags so no individual resource has to remember. The same standard drives cost attribution, which is covered from the spend side in cost awareness and governance.

# CLI: produce a compliance manifest from the recorded stack state, no cloud calls
pulumi stack export --stack prod \
  | jq -r '.deployment.resources[]
           | select(.type|startswith("aws:"))
           | [.urn, (.inputs.tags["data-classification"] // "MISSING")]
           | @tsv' \
  | grep MISSING
# State implication: this reads the checkpoint, not the live account — it tells
# you what the last successful deployment declared, which is exactly what an
# auditor asking "what did you intend" wants to see.

Where a Control Can Live

A rule can be enforced at four distinct points, and the choice determines who it protects, how fast it fails, and how easy it is to bypass. Most teams pick one, discover the gap the hard way, and then add the others in the wrong order.

Construction-time validation is the fastest and the weakest: it runs in the developer's own process, gives the best error message, and is trivially bypassed by not calling the factory. Synthesis-time scanning is the workhorse — it sees every resource regardless of how it was created, runs in CI where it cannot be skipped, and produces a machine-readable report. Provider-side controls such as AWS service control policies or Azure Policy are the only layer that survives someone bypassing the pipeline entirely, which is why the genuinely non-negotiable rules belong there. Post-deploy drift detection catches the rest: the console click, the emergency change, the resource an operator created by hand at 3 a.m.

The layers do not share a language, and that is the source of most of the duplication in real compliance stacks. Each tool reads a different representation of the same change.

The practical consequence is that "no unencrypted RDS instance" will be written three times: as a __post_init__ check on your typed spec, as a Checkov check identifier in the CI configuration, and as an SCP condition on rds:CreateDBInstance. That is acceptable if the control register is the single source of truth and the three expressions are generated or reviewed together. It is not acceptable when the three drift apart, which is what happens when the SCP is owned by a platform team who never sees the pipeline configuration. Keep the register in the repository, annotate each entry with the identifiers it maps to in every tool, and make a mismatch a review comment rather than an audit finding.

Custom rules are where the register earns its keep. Vendor checks cover the well-known baselines; nobody ships a check for your approved AMI list, your naming convention, or the requirement that every S3 bucket log to one specific audit account. Writing those as Python classes that plug into the same scan is covered step by step in writing custom Checkov policies in Python, and the Pulumi-native equivalent — mandatory and advisory policy packs evaluated inside pulumi up — is covered in Pulumi policy as code with CrossGuard.

Secure State Management & Secret Injection

State files often contain sensitive configuration data, making backend security a critical compliance requirement. Configure isolated, encrypted workspaces following the Setting Up Dev Environments protocol to prevent credential leakage during local synthesis and remote execution. Plaintext secrets in state files violate zero-trust mandates and trigger immediate audit failures.

The two frameworks behave differently here and the difference matters. Pulumi's checkpoint records every resource input and output, but values marked secret are stored as ciphertext produced by the stack's secrets provider — pulumi.Output.secret(...), a config value set with pulumi config set --secret, and any output derived from either are encrypted in the checkpoint and redacted in CLI output. Terraform, and therefore CDKTF, has no such marking: sensitive = true suppresses display in the plan and nothing more. The value is written to state in plaintext, including values pulled in by data sources. Reading a password out of Secrets Manager with a data source in order to pass it to a database resource puts that password in the state file, where it is only as protected as the bucket policy.

# secrets.py — resolve credentials at deploy time, never at author time
# CLI: pulumi up --stack prod
import pulumi
import pulumi_aws as aws
import boto3
from botocore.exceptions import ClientError

def resolve_runtime_secret(secret_name: str, region: str = "us-east-1") -> str:
    """Fetch credentials from AWS Secrets Manager at runtime.

    Ensures plaintext values never persist in state or version control.
    Returns the raw secret string; callers must wrap in pulumi.Output.secret()
    before passing to provider resources.
    """
    client = boto3.client("secretsmanager", region_name=region)
    try:
        response = client.get_secret_value(SecretId=secret_name)
        return response["SecretString"]
    except ClientError as e:
        raise RuntimeError(f"Secret resolution failed: {e}") from e

# Usage in a Pulumi stack: wrap in Output.secret() to prevent plaintext in state
# State implication: without Output.secret() the value is written to the
# checkpoint verbatim and is readable by anyone with `pulumi stack export`.
db_password = pulumi.Output.secret(resolve_runtime_secret("prod/db-credentials"))

The strongest version of this pattern removes the secret from your program entirely. On AWS, setting manage_master_user_password=True on aws.rds.Instance makes RDS generate and rotate the credential into Secrets Manager itself; the IaC program never sees the value, so no representation of it can reach state. Prefer that whenever the provider offers it, and fall back to the resolver above only for credentials that originate outside the cloud provider.

Enforcing Encryption at Rest and in Transit for Remote Backends

Mandate KMS-managed keys for all state storage providers. Configure backend TLS verification to reject unencrypted API calls. For Pulumi, run pulumi stack init --secrets-provider=awskms://alias/my-key?region=us-east-1 to bind state encryption to an organizational key policy.

Binding to a KMS alias rather than a passphrase has an operational payoff beyond encryption strength: access to secrets becomes an IAM decision that is logged. Every decrypt of a stack secret appears in CloudTrail as a kms:Decrypt call with the caller's principal, so "who read the production database password" is answerable without instrumenting anything. Revoking access is a key-policy edit, not a credential rotation across every developer laptop. Deeper treatment of the secrets provider and config layering lives in Pulumi secrets and configuration.

Replacing Static Environment Variables with Runtime Secret Resolvers

Eliminate .env files from IaC repositories to prevent accidental commits. Inject credentials dynamically during stack evaluation using the resolver pattern above. Validate resolution via pytest fixtures that mock boto3 and assert non-null outputs.

The resolver also needs a failure mode you can live with. A RuntimeError raised during preview aborts the whole program, which is correct in CI and hostile on a laptop where the engineer simply has not assumed a role yet. Catch ClientError and branch on e.response["Error"]["Code"]: ResourceNotFoundException means the secret name is wrong and should abort; ExpiredTokenException or AccessDeniedException should abort with a message telling the engineer which role to assume. Never fall back to a default value — a resolver that silently returns an empty string will happily provision a database with a blank password.

Implementing State Locking and Concurrency Controls

Enable distributed locking to prevent simultaneous state mutations. Configure DynamoDB lock tables or cloud-native equivalents before team collaboration begins. Monitor pulumi up exit codes to detect lock contention and retry safely.

Lock contention surfaces as error: the stack is currently locked by 1 lock(s) from Pulumi, or Error acquiring the state lock: ConditionalCheckFailedException from Terraform. Both are correct behaviour and neither should be resolved by force-unlocking as a reflex — a lock held by a job that is still running means an apply is in flight, and breaking it can interleave two writes to the same checkpoint. Set a lock timeout in CI, alert on it, and reserve pulumi cancel or terraform force-unlock for the case where you have confirmed the holding job is dead. The mechanics of backends and locking are covered in managing IaC state.

Automated Compliance Scanning in CI/CD Pipelines

Continuous compliance requires automated scanning at every merge and deployment stage. Pipeline gating must halt deployments on critical violations before state mutation occurs. Evaluate the trade-offs between native cloud SDKs and third-party policy engines in Python vs Terraform vs Ansible when selecting enforcement tools.

The cheapest scan of all is a unit test over the synthesized graph, because it needs no extra tooling and runs in the same pytest invocation as everything else. CDKTF's Testing.synth(app) returns the Terraform JSON as a string; parse it and assert directly on the resource map.

# tests/test_compliance.py — assert on the synthesized graph, no cloud calls
# CLI: pytest tests/test_compliance.py -q
import pytest
import json
from cdktf import Testing, App
from my_infra import VpcStack

@pytest.fixture
def synthesized_stack():
    """Synthesize CDKTF stack in-memory for isolated compliance validation."""
    app = Testing.app()
    VpcStack(app, "test-vpc")
    return json.loads(Testing.synth(app))

def test_security_group_no_open_ingress(synthesized_stack: dict) -> None:
    """Assert synthesized infrastructure has no 0.0.0.0/0 ingress rules."""
    sg_resources = synthesized_stack.get("resource", {}).get("aws_security_group", {})
    for sg_name, sg_config in sg_resources.items():
        ingress_rules = sg_config.get("ingress", [])
        for rule in ingress_rules:
            cidr_blocks = rule.get("cidr_blocks", [])
            assert "0.0.0.0/0" not in cidr_blocks, (
                f"CIS Violation: Unrestricted ingress detected in {sg_name}"
            )

def test_encryption_flags_across_resources(synthesized_stack: dict) -> None:
    """Verify storage_encrypted is enabled on all RDS instances."""
    rds_instances = synthesized_stack.get("resource", {}).get("aws_db_instance", {})
    for db_name, db_config in rds_instances.items():
        assert db_config.get("storage_encrypted") is True, (
            f"CIS Violation: RDS encryption disabled on {db_name}"
        )

Integrating Checkov and OPA Rego Policies into Python IaC Pipelines

Run checkov -d cdktf.out --framework terraform_json against synthesized JSON outputs from CDKTF. For Pulumi, export the resource graph via pulumi preview --json and pipe to OPA. Fail CI jobs immediately when policy evaluation returns FAIL. The full gating workflow — wiring Checkov against synthesized CDKTF JSON and Pulumi plans, handling suppressions, and reading sample findings — is covered in Scanning Python IaC with Checkov.

A finding is printed in a fixed shape, and knowing that shape is what makes the output actionable rather than noise:

# CLI: checkov -d cdktf.out/stacks/prod --framework terraform_json --compact
Check: CKV_AWS_18: "Ensure the S3 bucket has access logging enabled"
	FAILED for resource: aws_s3_bucket.artifacts
	File: /cdktf.out/stacks/prod/cdk.tf.json:1-1

Passed checks: 41, Failed checks: 2, Skipped checks: 1

The resource address in the FAILED for resource: line is the synthesized Terraform address, not your Python variable name. CDKTF derives it from the construct path, so a finding on aws_s3_bucket.artifacts maps back to the construct id you passed — keep those ids meaningful or every finding requires a grep. The :1-1 line range is expected: generated JSON is emitted on a single line, so Checkov cannot point at a source location and there is nothing to fix about it.

Configuring Pipeline Failure Thresholds for Critical vs. Warning Violations

Classify policy results by severity to prevent deployment paralysis. Block merges on CRITICAL findings while logging WARNING violations for remediation. Use pytest --strict-markers to enforce threshold boundaries in test suites.

Checkov implements this with --soft-fail-on, which downgrades named checks to a zero exit code while still printing them, and --hard-fail-on, which does the reverse. Drive both lists from the control register rather than hand-maintaining flags in a workflow file, and emit -o junitxml alongside the CLI output so the pipeline UI shows each failed check as a named test. Resist a blanket --soft-fail: it makes the job green forever and nobody notices for months.

Automating Drift Detection and Remediation Triggers

Schedule nightly pulumi refresh or cdktf diff (which runs terraform plan under the hood) executions against production state. Compare live configurations against synthesized baselines to detect unauthorized changes. Trigger automated rollback scripts when drift exceeds acceptable thresholds.

Run the nightly job with pulumi preview --refresh --diff --stack prod and treat a non-empty diff as an alert rather than an auto-remediation trigger, at least initially. Automatic reversion is the right end state, but the first weeks of drift reporting usually reveal legitimate out-of-band changes — a support engineer raising a connection limit, an autoscaling attribute the provider does not manage — and reverting those automatically before you have classified them turns a monitoring system into an outage generator.

Network & IAM Guardrails Implementation

Network isolation and identity management form the foundation of zero-trust infrastructure. Programmatically generating scoped IAM policies and enforcing strict VPC boundaries guarantees compliance without manual intervention. Explicit deny rules and resource-level conditions prevent privilege escalation across multi-account environments.

# iam.py — typed policy generator with an explicit deny fallback
# CLI: pulumi up --stack prod
import json
from typing import List, Dict, Any
import pulumi
import pulumi_aws as aws

class LeastPrivilegeIAMGenerator:
    """Constructs tightly scoped IAM policies with explicit deny fallbacks."""

    def __init__(self, service: str, actions: List[str], resource_arns: List[str]) -> None:
        self.service = service
        self.actions = actions
        self.resource_arns = resource_arns

    def build_policy(self) -> Dict[str, Any]:
        return {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Action": [f"{self.service}:{a}" for a in self.actions],
                    "Resource": self.resource_arns,
                    "Condition": {"StringEquals": {"aws:RequestedRegion": "us-east-1"}},
                },
                {
                    "Effect": "Deny",
                    "Action": ["*"],
                    "Resource": "*",
                    "Condition": {
                        "StringNotLike": {
                            "aws:PrincipalArn": "arn:aws:iam::*:role/ApprovedAdmin*"
                        }
                    },
                },
            ],
        }

# Pulumi integration
policy_gen = LeastPrivilegeIAMGenerator(
    "s3", ["GetObject", "PutObject"], ["arn:aws:s3:::secure-bucket/*"]
)
role = aws.iam.Role(
    "app-role",
    assume_role_policy=json.dumps({
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Principal": {"Service": "ec2.amazonaws.com"},
            "Action": "sts:AssumeRole",
        }],
    }),
)
# State implication: RolePolicy is an inline policy — deleting it from the
# program detaches the permission on the next update, unlike a managed policy
# attachment which leaves the policy object behind.
role_policy = aws.iam.RolePolicy(
    "scoped-s3",
    role=role.name,
    policy=json.dumps(policy_gen.build_policy()),
)

Programmatic Construction of Scoped IAM Roles and Policies

Generate policies dynamically using typed builders instead of static JSON files. Enforce explicit deny fallbacks to block wildcard permissions. Validate policy syntax via pytest before attaching to IAM roles—use the aws iam simulate-principal-policy CLI to verify effective permissions in integration environments. For a deeper treatment of typed policy builders, scoping resources, and testing for over-broad grants, see Enforcing IAM least privilege in Python IaC.

Two IAM mechanics are worth understanding before you rely on a generator. First, an explicit Deny always wins over any Allow, in any policy attached to the principal — which is what makes the deny fallback above meaningful rather than decorative. Second, a permissions boundary does not grant anything; it caps what an identity-based policy can grant. Attaching a boundary to every role your pipeline creates is the single highest-leverage IAM control available, because it bounds the damage of a badly written policy without requiring anyone to review that policy. Set it with permissions_boundary on aws.iam.Role and make it mandatory in the same factory that creates the role.

Standardizing Security Groups and NACLs for Zero-Trust Segmentation

Define reusable network constructs with default-deny ingress rules. Restrict lateral movement by isolating subnets per workload tier. Run pulumi preview or cdktf diff and validate the plan output before applying any network topology change.

Prefer the standalone rule resources — aws.vpc.SecurityGroupIngressRule and aws.vpc.SecurityGroupEgressRule — over inline ingress/egress blocks on the security group. Inline blocks are authoritative: the provider deletes any rule it does not find in the program, which is desirable for drift control but makes an incremental rule addition a full rewrite of the rule set, and mixing the two styles on one group produces a perpetual diff as each fights the other. One style per group, enforced in the factory.

Implementing Mandatory Tagging for Audit Trails and Cost Allocation

Attach compliance tags during resource instantiation to guarantee traceability. Enforce tag presence via pre-commit hooks and pipeline gates. Query state exports to verify tag coverage across all provisioned assets.

Remember that tags are reproduced verbatim into billing exports and are readable far more widely than the repository. Never put a personal email address, a ticket URL containing a token, or anything else sensitive in a tag value; use a team identifier and resolve it to people elsewhere.

Step-by-Step: Adding a Compliance Gate to an Existing Stack

The following sequence adds enforcement to a repository that already deploys successfully, without breaking the existing deployment on day one.

1. Write the control register

Start with five controls, not fifty. Pick the ones an auditor has already asked about, express each as a control identifier, a human sentence, a severity, and the tool identifiers it maps to.

# controls.py — the single source of truth for what is enforced and how hard
# CLI: python -c "import controls; print(len(controls.REGISTER))"
from dataclasses import dataclass, field
from typing import List, Literal

Severity = Literal["blocking", "warning"]


@dataclass(frozen=True)
class Control:
    control_id: str
    statement: str
    severity: Severity
    checkov_ids: List[str] = field(default_factory=list)


REGISTER: List[Control] = [
    Control("SEC-01", "Object storage is encrypted with a customer-managed key",
            "blocking", ["CKV_AWS_19", "CKV_AWS_145"]),
    Control("SEC-02", "No security group allows 0.0.0.0/0 on an administrative port",
            "blocking", ["CKV_AWS_24", "CKV_AWS_25"]),
    Control("SEC-03", "Relational databases are encrypted at rest",
            "blocking", ["CKV_AWS_16"]),
    Control("SEC-04", "Object storage has access logging enabled",
            "warning", ["CKV_AWS_18"]),
    Control("SEC-05", "IAM policies contain no wildcard action on a wildcard resource",
            "blocking", ["CKV_AWS_1", "CKV_AWS_63"]),
]

BLOCKING = [cid for c in REGISTER if c.severity == "blocking" for cid in c.checkov_ids]
WARNING = [cid for c in REGISTER if c.severity == "warning" for cid in c.checkov_ids]

2. Produce a scannable artifact

Both frameworks can emit a plan without touching the account, provided the identity can read existing state.

# CLI: emit the artifact the scanner will read
cdktf synth                       # writes cdktf.out/stacks/<stack>/cdk.tf.json
pulumi preview --stack prod --json > preview.json
# State implication: `preview` refreshes nothing by default. Add --refresh if
# the gate must judge the change against reality rather than the last checkpoint.

3. Run the scan in observe-only mode first

For the first week, run the scan with --soft-fail and publish the report. This produces the baseline finding count and, more importantly, exposes the false positives before anyone's merge is blocked by them.

# CLI: baseline run — reports everything, blocks nothing
checkov -d cdktf.out/stacks/prod --framework terraform_json \
        --soft-fail --compact -o cli -o junitxml --output-file-path console,report.xml

4. Turn on blocking for the register

Switch to hard failure only for the identifiers marked blocking, generated from the register so the two cannot drift.

# gate.py — build the checkov invocation from the control register
# CLI: python -m gate cdktf.out/stacks/prod
import subprocess
import sys
from typing import List

from controls import BLOCKING, WARNING


def run_gate(target_dir: str) -> int:
    cmd: List[str] = [
        "checkov", "-d", target_dir, "--framework", "terraform_json", "--compact",
        "--hard-fail-on", ",".join(BLOCKING),
        "--soft-fail-on", ",".join(WARNING),
    ]
    completed = subprocess.run(cmd, check=False)
    if completed.returncode != 0:
        print("gate: blocking control violated — see the FAILED lines above",
              file=sys.stderr)
    return completed.returncode


if __name__ == "__main__":
    raise SystemExit(run_gate(sys.argv[1]))

5. Wire the deploy behind the gate

The deploy step must depend on the gate step, and the deploy credentials must not exist in the gate job. With GitHub OIDC that is two different roles assumed by two different jobs.

# .github/workflows/deploy.yml
# CLI: gh workflow run deploy
name: deploy
on: [pull_request, push]
permissions:
  id-token: write
  contents: read
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements-ci.txt
      - run: cdktf synth
      - run: python -m gate cdktf.out/stacks/prod
  deploy:
    needs: gate
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements-ci.txt
      - run: cdktf deploy --auto-approve

Verification

Verification of a compliance gate means proving that it fails when it should, not that it passes when nothing is wrong. A gate nobody has ever seen fail is indistinguishable from a gate that is broken.

# CLI: negative test — deliberately break one control and confirm the exit code
git checkout -b verify-gate
python - <<'PY'
import pathlib, re
p = pathlib.Path("stacks/storage.py")
p.write_text(p.read_text().replace('sse_algorithm="aws:kms"', 'sse_algorithm="AES256"'))
PY
cdktf synth && python -m gate cdktf.out/stacks/prod ; echo "exit=$?"
# Expect: exit=1 with "Check: CKV_AWS_145 ... FAILED for resource: aws_s3_bucket.artifacts"
git checkout -- stacks/storage.py

Three further checks confirm the other layers are live. Confirm that state secrets are actually ciphertext: pulumi stack export --stack prod | jq '.deployment.resources[].outputs.password' should return an object with a ciphertext field, never a readable string. Confirm effective IAM permissions match intent with aws iam simulate-principal-policy --policy-source-arn arn:aws:iam::123456789012:role/app-role --action-names s3:DeleteBucket, which should report "EvalDecision": "implicitDeny" for anything outside the grant. And confirm the transformation hook covers resources it did not create, by adding a resource through a third-party component and checking that metadataOptions appears in the plan for it.

Record the results. An auditor asking "how do you know the control is enforced" is asking for exactly this artifact: a dated run showing the gate rejecting a known-bad change.

Troubleshooting

Checkov reports zero checks against a CDKTF build

Passed checks: 0, Failed checks: 0, Skipped checks: 0 with no findings at all. The cause is almost always the scan directory: cdktf synth writes cdk.tf.json into cdktf.out/stacks/<stack-name>/, and pointing -d at cdktf.out finds no parseable file at the top level. Point at the stack directory, or scan the file directly with checkov -f cdktf.out/stacks/prod/cdk.tf.json --framework terraform_json. If it still reports zero, check that --framework terraform_json is present — without it Checkov tries the terraform parser and silently skips JSON.

Pulumi refuses to decrypt the stack

error: getting secrets manager: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables. The stack was initialised with the passphrase secrets provider, usually by someone running pulumi stack init locally before the KMS key existed. Migrate it rather than distributing the passphrase: pulumi stack change-secrets-provider "awskms://alias/iac-state?region=eu-west-1" re-encrypts every secret config value and checkpoint secret under the key. Run it once, from an identity that can both decrypt with the old provider and encrypt with the new one, and commit the resulting Pulumi.<stack>.yaml.

A password appears in plaintext in Terraform state

terraform show -json | jq '.values.root_module.resources[].values.password' returns a readable string. The cause is a data source: aws_secretsmanager_secret_version returns the secret value into the graph, and Terraform persists every data source result in state regardless of sensitive. There is no flag that fixes this. Either let the provider own the credential — manage_master_user_password=True on aws.rds.Instance, which keeps the value in Secrets Manager and out of the graph — or pass only the secret ARN into the resource and let the consuming application resolve it at runtime.

The deploy fails with an explicit deny nobody wrote

An error occurred (AccessDenied) when calling the CreateBucket operation: User: arn:aws:sts::123456789012:assumed-role/deploy/GitHubActions is not authorized to perform: s3:CreateBucket on resource: ... with an explicit deny in a service control policy. The plan passed every pipeline check because the pipeline cannot see organisation-level policy. The fix is not to widen the role: read the SCP, confirm which condition it trips — most often aws:RequestedRegion or a resource-tag requirement — and encode the same constraint in the typed baseline so the next violation fails in the preview instead of halfway through an apply. A partially applied stack is the worst outcome of this failure mode; check the checkpoint before retrying.

The scan passes locally and fails in CI

The versions differ. Checkov adds checks in every minor release, so checkov==3.2.40 on a laptop and whatever pip install checkov resolved to in the CI image are running different rule sets. Pin the exact version in requirements-ci.txt, install the same file in both places, and print checkov --version at the top of the job so the report carries its own provenance. The same rule applies to provider plugin versions: a new provider major version can change a default that a check reads, turning a passing plan into a failing one with no code change.

A suppression comment has no effect in generated JSON

#checkov:skip=CKV_AWS_18:approved static site works in hand-written HCL and does nothing in CDKTF output, because the comment is not in the generated file and JSON has no comments. Suppress from configuration instead: a .checkov.yaml in the repository with a skip-check: list, or --skip-check CKV_AWS_18 scoped to the one stack that needs it. Record the justification in the control register next to the check identifier, so the suppression is reviewed on the same cadence as the control it weakens.

Key Takeaways

Security in Python IaC is a continuous discipline, not a deployment step. Enforce it at three layers: typed configuration objects that reject non-compliant inputs at construction time, policy gates that validate synthesized JSON before any cloud API call, and scheduled drift detection that surfaces unauthorized changes after deployment. Teams that implement all three layers spend significantly less time in security incident response.

FAQ

Where in the pipeline should compliance scanning run?

Run it twice. A fast typed-validation pass executes at construction time so non-compliant inputs fail before any provider call. Then run Checkov against the synthesized output as a CI gate before pulumi up or cdktf deploy, blocking the merge on critical findings.

How do I keep IAM policies least-privilege without breaking deployments?

Build policies with typed generators that emit explicit allow statements scoped to specific actions and resource ARNs, with a deny fallback for wildcards. Validate effective permissions with aws iam simulate-principal-policy in integration tests. See Enforcing IAM least privilege in Python IaC for the full builder and test pattern.

Should secrets ever appear in state files?

No. Resolve secrets at runtime from a managed store and wrap them in pulumi.Output.secret() so they are encrypted in state. Bind state encryption to a KMS key via pulumi stack init --secrets-provider=awskms://... and reject backends without encryption at rest. Remember that Terraform and CDKTF have no equivalent marking — there, the only safe answer is to keep the value out of the graph entirely.

How do I detect drift introduced outside the pipeline?

Schedule nightly pulumi refresh or cdktf diff against production state and compare live configuration to the synthesized baseline. Alert or trigger remediation when divergence exceeds your threshold.

Do the built-in checks cover enough, or do I need custom policies?

Built-ins cover published baselines such as CIS, and they are the right starting point. They cannot cover organisation-specific rules — approved AMIs, naming conventions, a mandatory audit-log destination — because no vendor knows them. Write those as Python check classes following writing custom Checkov policies in Python, and keep them in the same scan so there is one gate, not two.

How do I grant a legitimate exception without weakening the control everywhere?

Suppress the specific check on the specific resource, never globally. In CDKTF that means a skip-check entry scoped to the one stack, plus a line in the control register naming the resource, the approver and the review date. If the suppression list grows past a handful of entries, the control is wrong rather than the resources — rewrite the control.

Related

• Scanning Python IaC with Checkov — running Checkov against synthesized CDKTF JSON and Pulumi plans, CI gating, and suppressions.
• Enforcing IAM least privilege in Python IaC — typed policy builders, avoiding wildcard actions, scoping resources, and testing for over-broad grants.
• Write custom Checkov policies in Python — turning an organisation-specific rule into a Python check class that runs in the same scan as the built-ins.
• Python IaC Fundamentals & Strategy — the parent overview tying security and compliance to design principles and tooling choice.