Provisioning RDS PostgreSQL with Pulumi (Python)

Provisioning a managed PostgreSQL instance with Pulumi Python means wiring together a subnet group, a security group, and a secret-managed password into a single typed program — part of the broader AWS Provider Deep Dive workflow. The hard part is not the aws.rds.Instance resource itself; it is keeping the master password out of state in plaintext and exposing a connection string downstream without leaking it.

This guide builds a production-shaped RDS PostgreSQL instance: a DB subnet group spanning private subnets, a tightly scoped security group, an encrypted password sourced from a Pulumi config secret, and typed stack outputs for the endpoint and connection details.

Context

A database is the resource where a sloppy IaC pattern costs you the most. An over-broad security group exposes the instance to the internet; a hardcoded password lands in version control and the state file; a single-AZ subnet group blocks any later move to Multi-AZ. Doing it correctly the first time is cheaper than migrating a live database. The same secret-handling discipline applies when securing Pulumi secrets with AWS KMS and HashiCorp Vault, and the network primitives here are the ones an EKS cluster consumes from the same VPC.

What makes aws.rds.Instance unlike most resources in pulumi-aws is that its inputs fall into three behavioural classes that the type system cannot distinguish. Some inputs — allocated_storage, backup_retention_period, instance_class — are live modifications applied by a single ModifyDBInstance API call. Some are replacement inputs: change identifier, db_name, or engine, and Pulumi's diff engine plans a create-and-delete pair, which for a database means a new empty instance and a destroyed one holding your data. A third group is neither: values that live in a DB parameter group and only take effect after a reboot. A pulumi preview that reads as a harmless ~ update can therefore mean anything from "no customer impact" to "sixty-second outage" to "you lose the data". Reading the diff correctly is most of the skill.

The second structural issue is the password. Unlike a Kubernetes Secret or an SSM parameter, the RDS master password is a write-only attribute: AWS never returns it, so pulumi refresh cannot reconcile it and the value in your state file is the only record Pulumi has. That makes the secret marker on the config value load-bearing. If the password is read with Config().require() instead of require_secret(), it is written to the checkpoint as plaintext JSON and every engineer with read access to the state bucket has your database credentials. The rest of this guide treats that marker as non-negotiable.

Context Context: Context with 4 facets. Context Pulumi key element AWS KMS key element HashiCorp key element EKS key element
Context: how Pulumi, AWS KMS, HashiCorp relate in this pattern.

Prerequisites

Prerequisites Prerequisites: layered from mypy down to IAM. mypy Python VPC RDS IAM
Prerequisites: the building blocks this section assembles.
  • Python 3.9+ with pulumi>=3.0 and pulumi-aws>=6.0 installed in a virtualenv.
  • An existing VPC with at least two private subnets in different AZs (RDS subnet groups require multi-AZ coverage even for single-AZ instances).
  • IAM permissions for rds:*, ec2:*SecurityGroup*, and ec2:DescribeSubnets on the deployment role.
  • A master password staged as a Pulumi config secret: pulumi config set --secret dbPassword <value>.
  • mypy for static checking of the typed config object.
  • An application security group ID to reference as the ingress source. If the consuming workload does not exist yet, create an empty security group first and attach it later — an empty group is a valid ingress source and avoids a circular dependency between the database stack and the compute stack.

A note on ordering: create the subnet group and security group in the same Pulumi program as the instance, not by hand in the console. RDS validates the subnet group at instance-creation time, and a console-created group that Pulumi does not own will not be updated when you add an AZ, producing a confusing failure months later.

Implementation

1. Define a typed configuration object

Implementation Implementation: 1. Define a typed then 2. Create the then 3. Provision the then 4. Export typed 1. Define a typed 2. Create the 3. Provision the 4. Export typed
Implementation: the stages run left to right — 1. Define a typed, 2. Create the, 3. Provision the, 4. Export typed.

Model the instance parameters in a frozen dataclass so mypy --strict catches a misnamed engine version or instance class before a deploy attempt. Freezing matters here for a specific reason: the same config object is read by the subnet-group builder, the security-group builder, and the instance builder, and a mutation in one of them would silently change what the others provision. frozen=True turns that class of bug into a dataclasses.FrozenInstanceError at import time rather than a surprise replacement in pulumi preview.

# infra/rds_config.py
# CLI: mypy --strict infra/
from __future__ import annotations
from dataclasses import dataclass, field
from typing import List

@dataclass(frozen=True)
class RdsConfig:
    identifier: str
    db_name: str
    username: str
    instance_class: str = "db.t3.micro"
    engine_version: str = "16.3"
    allocated_storage: int = 20
    multi_az: bool = False
    subnet_ids: List[str] = field(default_factory=list)
    vpc_id: str = ""
    # State implication: changing `identifier` forces replacement of the
    # instance — Pulumi will destroy the old DB and create a new one.

2. Create the subnet group and security group

The subnet group pins the database to private subnets. The security group starts closed and admits PostgreSQL traffic only from a referenced application security group, never a CIDR like 0.0.0.0/0.

Referencing a security group instead of a CIDR is the difference between a rule that stays correct and one that rots. A CIDR rule encodes today's subnet layout; the moment someone adds a third private subnet the rule silently stops covering half the application. A source-security-group rule follows the workload wherever its ENIs are placed, and it survives a VPC re-address. Note the asymmetry in the code below: ingress is one narrow rule on TCP/5432, egress is wide open. That is deliberate — PostgreSQL initiates outbound connections for features such as extension downloads and CloudWatch log delivery, and clamping egress on a database usually produces a support ticket rather than a security win.

# infra/rds.py
# CLI: pulumi preview --diff
from __future__ import annotations
import pulumi
import pulumi_aws as aws
from infra.rds_config import RdsConfig

def build_network(cfg: RdsConfig, app_sg_id: pulumi.Input[str]) -> tuple[aws.rds.SubnetGroup, aws.ec2.SecurityGroup]:
    subnet_group = aws.rds.SubnetGroup(
        f"{cfg.identifier}-subnets",
        subnet_ids=cfg.subnet_ids,
        tags={"Name": f"{cfg.identifier}-subnets"},
    )

    sg = aws.ec2.SecurityGroup(
        f"{cfg.identifier}-sg",
        vpc_id=cfg.vpc_id,
        description=f"Postgres access for {cfg.identifier}",
        ingress=[aws.ec2.SecurityGroupIngressArgs(
            protocol="tcp",
            from_port=5432,
            to_port=5432,
            # Provider note: scope ingress to the app SG, not a CIDR block.
            security_groups=[app_sg_id],
        )],
        egress=[aws.ec2.SecurityGroupEgressArgs(
            protocol="-1", from_port=0, to_port=0, cidr_blocks=["0.0.0.0/0"],
        )],
    )
    return subnet_group, sg

3. Provision the instance with a secret password

Pull the password from pulumi.Config().require_secret(). Pulumi keeps secret config encrypted in state and marks the password input as a secret, so it never appears in plaintext in pulumi preview output or the state file. The secret marker is viral: any Output derived from that value through apply() inherits it, which is why the connection string assembled in step 4 stays masked without extra effort.

Several arguments below are worth setting explicitly even though they have defaults. storage_type="gp3" decouples IOPS from volume size — on gp2 the only way to buy IOPS is to over-provision storage. max_allocated_storage turns on storage autoscaling, which prevents the 3am page where PostgreSQL goes read-only at 100% disk; set it well above allocated_storage and let AWS grow the volume. deletion_protection=True is the cheapest insurance available, and auto_minor_version_upgrade=False keeps AWS from bumping the patch version out from under your state file. apply_immediately=False is the default and the right choice for production: it queues modifications to the maintenance_window instead of restarting the engine during business hours.

# infra/rds.py (continued)
# CLI: pulumi up
def build_instance(
    cfg: RdsConfig,
    subnet_group: aws.rds.SubnetGroup,
    sg: aws.ec2.SecurityGroup,
) -> aws.rds.Instance:
    config = pulumi.Config()
    password = config.require_secret("dbPassword")

    return aws.rds.Instance(
        cfg.identifier,
        identifier=cfg.identifier,
        engine="postgres",
        engine_version=cfg.engine_version,
        instance_class=cfg.instance_class,
        allocated_storage=cfg.allocated_storage,
        db_name=cfg.db_name,
        username=cfg.username,
        password=password,  # State implication: stored encrypted as a secret
        db_subnet_group_name=subnet_group.name,
        vpc_security_group_ids=[sg.id],
        multi_az=cfg.multi_az,
        storage_encrypted=True,
        storage_type="gp3",
        max_allocated_storage=cfg.allocated_storage * 5,
        skip_final_snapshot=False,
        final_snapshot_identifier=f"{cfg.identifier}-final",
        backup_retention_period=7,
        backup_window="03:00-04:00",
        maintenance_window="Mon:04:30-Mon:05:30",
        auto_minor_version_upgrade=False,
        deletion_protection=True,
        apply_immediately=False,
        enabled_cloudwatch_logs_exports=["postgresql", "upgrade"],
        # Provider note: omit `publicly_accessible` (defaults to False) to
        # keep the instance off public subnets.
        opts=pulumi.ResourceOptions(
            protect=True,
            # State implication: `protect` makes `pulumi destroy` refuse the
            # resource until you run `pulumi state unprotect <urn>`.
            additional_secret_outputs=["password"],
        ),
    )

additional_secret_outputs=["password"] is belt and braces: it forces the marker onto the output even if the input arrives from a non-secret source during a refactor. backup_window and maintenance_window must not overlap, and both are UTC — AWS rejects an overlap with InvalidParameterValue: The backup window and maintenance window must not overlap.

4. Export typed outputs

Export the endpoint and a composed connection string. Use Output.all().apply() to assemble the string so the secret password stays a secret in the resulting output.

# __main__.py
# CLI: pulumi stack output dbConnection --show-secrets
import pulumi
from infra.rds_config import RdsConfig
from infra.rds import build_network, build_instance

cfg = RdsConfig(
    identifier="orders-db",
    db_name="orders",
    username="app",
    vpc_id="vpc-0abc123",
    subnet_ids=["subnet-0aaa", "subnet-0bbb"],
)

subnet_group, sg = build_network(cfg, app_sg_id="sg-0app123")
instance = build_instance(cfg, subnet_group, sg)

pulumi.export("dbEndpoint", instance.endpoint)
conn = pulumi.Output.all(instance.address, instance.port).apply(
    lambda args: f"postgresql://{cfg.username}@{args[0]}:{args[1]}/{cfg.db_name}"
)
pulumi.export("dbConnection", conn)

Verification

Confirm the password input is treated as a secret and the instance is not publicly reachable.

Verification Verification: Test → Program → Mock/Cloud. Test Program Mock/Cloud invoke declare resolve assert
Verification: the test drives the program and asserts on resolved values.
# tests/test_rds.py
# CLI: pytest tests/test_rds.py
from __future__ import annotations
import pulumi
from typing import Any, Dict, Tuple

class Mocks(pulumi.runtime.Mocks):
    def new_resource(self, args: pulumi.runtime.MockResourceArgs) -> Tuple[str, Dict[str, Any]]:
        return (f"{args.name}-id", {**args.inputs, "endpoint": "db.local:5432", "address": "db.local", "port": 5432})
    def call(self, args: pulumi.runtime.MockCallArgs) -> Dict[str, Any]:
        return {}

pulumi.runtime.set_mocks(Mocks(), preview=False)

import importlib
infra_main = importlib.import_module("__main__")

@pulumi.runtime.test
def test_not_public() -> pulumi.Output:
    return infra_main.instance.publicly_accessible.apply(
        lambda v: pulumi.log.info("publicly_accessible") if v in (False, None) else (_ for _ in ()).throw(AssertionError("DB is public"))
    )

Out of band, confirm the live endpoint resolves to a private address:

# CLI: read the exported endpoint, then resolve it
aws rds describe-db-instances --db-instance-identifier orders-db \
  --query 'DBInstances[0].{Endpoint:Endpoint.Address,Public:PubliclyAccessible}'

Gotchas & Edge Cases

Gotchas & Edge Cases Gotchas & Edge Cases: Where it breaks with 4 facets. Where it breaks subnet_ids watch this boundary password watch this boundary ModifyDBInstan watch this boundary Edge Cases watch this boundary
Gotchas & Edge Cases: the boundaries where things break and what to check.

The subnet group needs two AZs even for a single-AZ instance. AWS rejects a DB subnet group whose subnets all sit in one AZ with DBSubnetGroupDoesNotCoverEnoughAZs. Supply at least two subnets in different AZs in subnet_ids even when multi_az=False; you can flip to Multi-AZ later without re-creating the subnet group.

Changing the password in config does not always rotate it. After pulumi config set --secret dbPassword, Pulumi updates the password input in place via ModifyDBInstance. But if you originally created the instance with manage_master_user_password=True (Secrets Manager integration), setting password directly conflicts. Pick one strategy and stay with it for the life of the instance.

skip_final_snapshot=False blocks pulumi destroy without a snapshot name. With a final snapshot required, you must also set final_snapshot_identifier. Omitting it makes pulumi destroy fail mid-operation, leaving the stack partially torn down. Keep both set together.

A reused final-snapshot identifier fails the second teardown. Snapshot identifiers are unique per account and region, and they outlive the instance. Tearing down and rebuilding orders-db twice with the same final_snapshot_identifier fails with DBSnapshotAlreadyExists: Cannot create the snapshot because a snapshot with the identifier orders-db-final already exists. Suffix the identifier with the stack name or a timestamp resolved at program start, not with a random value that changes on every preview.

A stopped instance restarts itself after seven days. RDS caps a manual stop at seven days and then starts the instance again. If someone stops a non-production database to save money, the next pulumi refresh may see it running or stopped depending on the day, and any drift check built on instance status will flap. Delete-and-restore-from-snapshot is the honest way to pause a database that Pulumi owns.

Operational Notes

The instance is the easy part; living with it for a year is where the design decisions land. Three of them recur.

The first is engine configuration. Anything beyond the handful of arguments on aws.rds.Instance — log_min_duration_statement, work_mem, shared_buffers — lives in a DB parameter group, and parameters divide into dynamic ones that take effect on the next ModifyDBInstance call and static ones that wait for a reboot. Pulumi reports both as applied the moment the API accepts them, so a static parameter shows as done in the update log while the running engine has not read it.

How RDS applies a changed input How RDS applies a changed input: choose among 3 options. Diff on aws.rds.Instance live ModifyDBInstance, norestart reboot Static parameter,waits for the window replace New empty instance,old one deleted
Reading the preview: the same tilde in pulumi preview can mean a no-impact modify, a deferred reboot, or the loss of the volume.
# infra/rds_params.py
# CLI: pulumi up --diff
from __future__ import annotations
import pulumi_aws as aws

params = aws.rds.ParameterGroup(
    "orders-db-pg16",
    family="postgres16",
    description="Query logging and connection limits for orders-db",
    parameters=[
        # Dynamic: takes effect as soon as the modify call returns.
        aws.rds.ParameterGroupParameterArgs(
            name="log_min_duration_statement", value="1000", apply_method="immediate",
        ),
        # Static: the engine only reads it after a reboot.
        # Provider note: apply_method="immediate" on a static parameter is rejected with
        # InvalidParameterCombination: cannot use immediate apply method for static parameter.
        aws.rds.ParameterGroupParameterArgs(
            name="shared_preload_libraries", value="pg_stat_statements", apply_method="pending-reboot",
        ),
    ],
)

Wire the group in with parameter_group_name=params.name. Note that family is tied to the major version: a postgres16 group cannot be attached to a PostgreSQL 17 instance, so a major upgrade means creating a new group and switching both attributes in the same update.

The second is version upgrades. Minor upgrades are routine because auto_minor_version_upgrade=False keeps them under your control. Major upgrades are not — bumping engine_version from 16.3 to 17.2 without also setting allow_major_version_upgrade=True fails with InvalidParameterCombination: The AllowMajorVersionUpgrade flag must be present when upgrading to a new major version. Set the flag, switch the parameter group family in the same change, take a manual snapshot first, and expect the instance to be unavailable for the duration of the upgrade rather than for a failover.

The third is storage autoscaling and the drift it creates. max_allocated_storage lets AWS grow the volume when free space runs low, which means the live allocated_storage will eventually be larger than the 20 in your dataclass. On the next pulumi refresh the real value lands in state, the program still asks for the original size, and the update tries to shrink a volume — which RDS refuses, because allocated storage only ever grows. Stop the fight explicitly:

# infra/rds.py (continued)
# CLI: pulumi up
# State implication: refresh still records the grown size; ignore_changes only stops
# Pulumi planning a modification back down to the value in code.
import pulumi

opts = pulumi.ResourceOptions(
    protect=True,
    ignore_changes=["allocated_storage"],
    additional_secret_outputs=["password"],
)

Enhanced Monitoring and Performance Insights are worth enabling on anything customer-facing, but monitoring_interval=60 is only valid alongside a monitoring_role_arn pointing at a role that trusts monitoring.rds.amazonaws.com; without it the create fails on InvalidParameterValue: RDS Enhanced Monitoring role ARN value is invalid. Create the role in the same program so the two can never drift apart, and remember that Performance Insights retention beyond the free seven days is a per-instance charge that shows up quietly on the bill.

FAQ

How do I keep the RDS master password out of the Pulumi state file in plaintext? Set it as a secret with pulumi config set --secret dbPassword and read it with Config().require_secret(). Pulumi encrypts secret config and propagates the secret marker to the password input, so the state file stores it encrypted and pulumi preview masks it.

Can I enable Multi-AZ without recreating the instance? Yes. Flip multi_az from False to True and run pulumi up. AWS performs an in-place modification (a brief failover) rather than a replacement, as long as the subnet group already covers multiple AZs.

Why does my subnet group fail with "does not cover enough AZs"? All the subnets you passed live in the same Availability Zone. RDS requires a subnet group to span at least two AZs. Add a subnet from a second AZ to subnet_ids.

How do I rotate the password after launch? Stage the new value with pulumi config set --secret dbPassword <new> and run pulumi up. Pulumi issues a ModifyDBInstance call to set the new password. Coordinate the change with application credential reloads to avoid a connection gap.

Should I let Pulumi manage the password or use Secrets Manager? For most stacks a Pulumi config secret is sufficient and keeps the value in one place. If you need automatic rotation and broad service access to the credential, use AWS-managed master passwords or a Vault integration instead — see the secrets page linked above.