Provision DynamoDB Tables with Pulumi Python

DynamoDB is deceptively simple to create and easy to misconfigure — wrong capacity mode, missing point-in-time recovery, unindexed access patterns. This guide, part of the AWS provider deep dive under Pulumi patterns and provider management, provisions a production-shaped table with a global secondary index in typed Python. The finished program declares one aws.dynamodb.Table, exports its name and ARN, and hands a caller an IAM policy narrow enough that a stray Scan is denied rather than merely expensive.

Context

A table's schema decisions are effectively permanent: the partition key and any global secondary index (GSI) are set at creation, and changing a key means a migration. Getting capacity mode, encryption, and recovery right up front — as with the other resources in the AWS provider deep dive — avoids expensive rework.

What makes DynamoDB different from a relational store in an IaC program is that the table definition is the query planner. There is no CREATE INDEX you can run later without consequence and no optimiser that will rescue a query you did not design for. The attributes, hash_key, range_key, and global_secondary_indexes arguments together enumerate every efficient read path the application will ever have. Anything outside that enumeration degrades to a Scan, which reads the whole table and bills for it.

The split in that diagram matters for how you review a pull request. The left half — keys and index structure — deserves the same scrutiny as a database migration, because Pulumi will happily replace the table to satisfy a one-character edit. The right half is reversible: billing mode, encryption, and recovery are UpdateTable calls that Pulumi applies in place, so a missing flag is an omission you can correct next sprint rather than an outage.

Prerequisites

• Python 3.9+ with pulumi>=3.0 and pulumi-aws>=6.0 pinned
• AWS credentials with dynamodb:CreateTable, UpdateTable, and tagging permissions
• A decided access pattern: the partition key and any GSI you will query by
• dynamodb:UpdateContinuousBackups and dynamodb:UpdateTimeToLive on the deploy role, since both are separate API calls the provider makes after CreateTable returns
• iam:CreatePolicy if the same stack publishes the consumer policy shown below

# CLI: confirm the AWS provider plugin is installed at the pinned version
pulumi plugin ls | grep aws

Implementation

1. Encode the access patterns before writing the resource

Write the queries down before writing any Pulumi code. Each row below is a read the service performs in production; the middle column is the key expression that satisfies it, and the right column names where it runs. A row you cannot fill in is a Scan waiting to appear in a latency graph six months from now.

Put that decision in a frozen dataclass rather than in scattered string literals. The declaration then has exactly one source for the key names, and the list of attribute definitions is derived instead of maintained by hand — which is where the most common create-time error comes from.

# spec.py — the access patterns, encoded once and reused by the table definition
# CLI: python -c "import spec; print(spec.ORDERS.key_attributes())"
from dataclasses import dataclass
from typing import List


@dataclass(frozen=True)
class TableSpec:
    name: str
    hash_key: str
    range_key: str
    index_hash_key: str
    index_range_key: str

    def key_attributes(self) -> List[str]:
        # DynamoDB wants ONLY attributes that appear in a key — never every field
        # the item happens to carry.
        return [self.hash_key, self.range_key,
                self.index_hash_key, self.index_range_key]


ORDERS = TableSpec(
    name="orders",
    hash_key="pk",
    range_key="sk",
    index_hash_key="gsi1pk",
    index_range_key="created_at",
)

2. Declare the table

Declare the table with on-demand billing, server-side encryption, and point-in-time recovery, plus one GSI for a secondary query path.

# table.py — a production-shaped DynamoDB table with a GSI
# CLI: pulumi up
import pulumi
import pulumi_aws as aws

from spec import ORDERS

table = aws.dynamodb.Table(
    "orders",
    name=ORDERS.name,
    billing_mode="PAY_PER_REQUEST",           # Provider note: no capacity to tune
    hash_key=ORDERS.hash_key,
    range_key=ORDERS.range_key,
    attributes=[
        aws.dynamodb.TableAttributeArgs(name=n, type="S")
        for n in ORDERS.key_attributes()
    ],
    global_secondary_indexes=[aws.dynamodb.TableGlobalSecondaryIndexArgs(
        name="gsi1",
        hash_key=ORDERS.index_hash_key,
        range_key=ORDERS.index_range_key,
        projection_type="INCLUDE",
        non_key_attributes=["status", "total_cents"],
    )],
    ttl=aws.dynamodb.TableTtlArgs(attribute_name="expires_at", enabled=True),
    point_in_time_recovery=aws.dynamodb.TablePointInTimeRecoveryArgs(enabled=True),
    server_side_encryption=aws.dynamodb.TableServerSideEncryptionArgs(enabled=True),
    deletion_protection_enabled=True,
    tags={"owner": "platform", "env": "prod"},
    opts=pulumi.ResourceOptions(protect=True),
)
# State implication: the table name and keys are recorded; changing a key replaces the table.

Three arguments there repay a closer look. projection_type="INCLUDE" copies only the two named attributes into the index instead of every field, so the index stores a fraction of the base item and writes cost proportionally less — ALL is convenient and is the single largest avoidable cost in most DynamoDB bills. ttl points at expires_at, which the application must write as epoch seconds in a Number attribute; it does not belong in attributes, because that list is only for key attributes. deletion_protection_enabled=True is the server-side guard and protect=True is the Pulumi-side guard — the first stops the console and the CLI, the second stops your own pulumi destroy, and you want both.

3. Export the outputs and scope the consumer policy

A table nobody can reach is not finished. Export the identifiers other stacks consume, then publish a policy whose Resource list covers the index ARNs as well as the table ARN.

# iam.py — a read/write policy scoped to the table and its index
# CLI: pulumi up
import json
from typing import Any, Dict

import pulumi
import pulumi_aws as aws

from table import table


def order_access_document(table_arn: str) -> str:
    doc: Dict[str, Any] = {
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Action": [
                "dynamodb:GetItem", "dynamodb:PutItem", "dynamodb:UpdateItem",
                "dynamodb:DeleteItem", "dynamodb:Query", "dynamodb:BatchGetItem",
            ],
            # Provider note: index ARNs are the table ARN plus /index/*. Omit them and
            # every Query against gsi1 fails with AccessDeniedException at runtime.
            "Resource": [table_arn, f"{table_arn}/index/*"],
        }],
    }
    return json.dumps(doc)


order_writer = aws.iam.Policy(
    "order-writer",
    policy=table.arn.apply(order_access_document),
)

pulumi.export("tableName", table.name)
pulumi.export("tableArn", table.arn)
# State implication: exported outputs land in the checkpoint and are readable by other stacks.

dynamodb:Scan is deliberately absent. Denying it turns an accidental full-table read into is not authorized to perform: dynamodb:Scan in a test run instead of a five-figure line item, which is the sort of enforcement the IAM least privilege guide argues for generally. Consumers pick the exports up through stack outputs and cross-stack references.

Capacity Modes and What They Cost

billing_mode is the argument most often chosen by copy-paste, and it is the one that decides whether the table degrades gracefully or throttles under a launch. The two modes are not just prices; they are different failure behaviours.

On-demand absorbs a traffic spike up to the previous peak instantly and beyond it within minutes; the bill follows the traffic. That open-ended bill is the reason on_demand_throughput exists — set max_read_request_units and max_write_request_units and a runaway loop throttles instead of billing. Provisioned mode is cheaper per unit at a flat baseline but requires an autoscaling target per dimension, and each of those targets is a resource Pulumi must own.

# capacity.py — provisioned reads with target-tracking autoscaling
# CLI: pulumi up --target-dependents
import pulumi
import pulumi_aws as aws

from table import table

read_target = aws.appautoscaling.Target(
    "orders-read-target",
    max_capacity=200,
    min_capacity=20,
    resource_id=table.name.apply(lambda n: f"table/{n}"),
    scalable_dimension="dynamodb:table:ReadCapacityUnits",
    service_namespace="dynamodb",
)

aws.appautoscaling.Policy(
    "orders-read-policy",
    policy_type="TargetTrackingScaling",
    resource_id=read_target.resource_id,
    scalable_dimension=read_target.scalable_dimension,
    service_namespace=read_target.service_namespace,
    target_tracking_scaling_policy_configuration=aws.appautoscaling.PolicyTargetTrackingScalingPolicyConfigurationArgs(
        target_value=70.0,
        predefined_metric_specification=aws.appautoscaling.PolicyTargetTrackingScalingPolicyConfigurationPredefinedMetricSpecificationArgs(
            predefined_metric_type="DynamoDBReadCapacityUtilization",
        ),
    ),
)
# State implication: autoscaling rewrites read_capacity outside Pulumi, so the table must
# carry pulumi.ResourceOptions(ignore_changes=["read_capacity", "write_capacity"]) or every
# preview reports a spurious diff.

That ignore_changes note is the practical cost of provisioned mode: two systems now write the same field, and only one of them is your program. The trade-off is the same one described in idempotency and drift detection.

Verification

Confirm the table is active with the GSI and recovery enabled.

# CLI: table status ACTIVE and PITR enabled
aws dynamodb describe-table --table-name $(pulumi stack output tableName) \
  --query 'Table.[TableStatus,GlobalSecondaryIndexes[0].IndexName]'
aws dynamodb describe-continuous-backups --table-name $(pulumi stack output tableName) \
  --query 'ContinuousBackupsDescription.PointInTimeRecoveryDescription.PointInTimeRecoveryStatus'

# CLI: TTL is a separate call — DescribeTable never shows it
aws dynamodb describe-time-to-live --table-name $(pulumi stack output tableName)

# CLI: the acceptance check — a preview right after an apply must report no changes
pulumi preview --diff --expect-no-changes

The last line is the one worth wiring into CI. --expect-no-changes exits non-zero if the provider sees any drift, which catches the two classic mistakes at once: an attribute list the provider reorders on read, and capacity fields being rewritten by autoscaling. For a check that runs without touching AWS at all, the same table can be asserted against a fake endpoint using the technique in mocking AWS services with moto.

Gotchas & Edge Cases

Adding a GSI later throttles. Backfilling an index on a large table consumes capacity; on provisioned tables this can throttle production. Prefer on-demand during index changes.

Attribute list must match keys. Every key and index attribute must appear in attributes, or the apply fails with all attributes must be indexed. The provider surfaces the API text verbatim: ValidationException: One or more parameter values were invalid: Some index key attributes are not defined in AttributeDefinitions. Keys: [gsi1pk], AttributeDefinitions: [pk, sk].

Replacement on key change. Changing hash_key or range_key replaces the table and drops data; treat key design as a migration, using the discipline from migrating IaC state.

The list must not be a superset either. Delete a GSI and forget to delete its attribute and the update fails with ValidationException: One or more parameter values were invalid: Number of attributes in KeySchema does not exactly match number of attributes defined in AttributeDefinitions. Deriving the list from TableSpec as above makes both directions impossible.

One index change per update. DynamoDB will not create and delete indexes in the same call: LimitExceededException: Subscriber limit exceeded: Only 1 online index can be created or deleted simultaneously per table. Renaming an index therefore takes two applies — remove it, apply, add the new one, apply — and Pulumi cannot batch them for you.

Turning encryption on changes the key. server_side_encryption with enabled=True and no kms_key_arn moves the table from the AWS-owned key to the AWS-managed alias/aws/dynamodb key. That is an improvement for auditability and it also starts billing KMS requests on every read and write, which is a real cost line on a high-throughput table.

TTL failures are silent. If the application writes expires_at as a string rather than a Number holding epoch seconds, DynamoDB never deletes those items and returns no error anywhere. The only signal is a table that grows forever, which is why the expiry attribute belongs in a schema test rather than in a code review checklist.

Deletion protection blocks teardown. With both guards set, a pulumi destroy on an ephemeral copy of the stack stops with ValidationException: Resource cannot be deleted as it is currently protected against deletion, or earlier with Pulumi's own refusal to delete a protected resource. Set both flags from config so preview environments can turn them off.

Operational Notes

Point-in-time recovery does not restore in place. A restore creates a new table from restore_source_name and restore_date_time, which means the recovery runbook is a Pulumi program change plus an application cutover, not a button. Write and rehearse that program before you need it, and keep the restored table's name derived from the timestamp so two restores never collide.

Streams are the other argument that changes the operational shape of the table. Setting stream_enabled=True with stream_view_type="NEW_AND_OLD_IMAGES" gives downstream consumers — usually a function from deploying AWS Lambda functions — an ordered change feed, at the price of a second ARN to grant on and a 24-hour retention window that turns a long consumer outage into permanent data loss.

Alarm on ThrottledRequests and UserErrors from the day the table is created, not after the first incident. UserErrors in particular catches ValidationException and ConditionalCheckFailedException volume that never reaches your error logs because the SDK counts them as normal responses. Pair those with the cost signals described in estimating infrastructure cost in Python IaC pipelines, since a DynamoDB regression usually shows up on the bill before it shows up on a latency graph.

FAQ

On-demand or provisioned capacity?

Start on-demand (PAY_PER_REQUEST) for unpredictable traffic; switch to provisioned with autoscaling only when steady-state load makes it cheaper.

Can I add a GSI without downtime?

Yes — DynamoDB adds a GSI online, but the backfill consumes capacity. Plan it during low traffic and watch for throttling.

How do I secure the table?

Enable server-side encryption (shown above) and scope IAM to specific actions, following IAM least privilege.

Why does pulumi preview show a diff on my table every single run?

Almost always autoscaling. Application Auto Scaling rewrites read_capacity and write_capacity outside Pulumi, so the program's declared numbers never match reality. Add those two fields to ignore_changes and the preview goes quiet.

Can I rename a DynamoDB table without losing the data?

Not in place — the name argument is part of the resource identity, so editing it destroys and recreates. Renaming means creating the new table, copying items with a stream or an export-to-S3 job, then cutting over. Only the Pulumi logical name can be changed safely, via pulumi.ResourceOptions(aliases=[...]).

How do I bring an existing table under Pulumi management?

Run pulumi import aws:dynamodb/table:Table orders orders, paste the generated code into your program, and reconcile it until pulumi preview is clean. Import writes the current live configuration into state, so any argument you leave out of the program will show up as a change on the next apply.

Related

• Provisioning RDS Postgres with Pulumi Python — a sibling data-store guide with its own durability concerns.
• AWS Provider Deep Dive — the parent topic covering core AWS services.
• Deploying AWS Lambda Functions with Pulumi Python — a common consumer of a DynamoDB table.
• Handling Pulumi Stack Outputs and Cross-Stack References — how other stacks pick up the table name and ARN exported here.