Programmatic Deployments with the Pulumi Automation API

This guide shows how to deploy infrastructure entirely from Python using auto.create_or_select_stack, an inline program, and up() — capturing typed outputs and handling failures without ever invoking the CLI. It extends The Pulumi Automation API overview, which is part of the broader Pulumi Patterns & Provider Management workflow. The goal is a single self-contained driver you can drop into a service, a test fixture, or a CI job.

Context

A programmatic deployment matters when infrastructure is triggered by something other than a human at a terminal: a web request that provisions a per-customer environment, an integration test that needs real cloud resources, or a pipeline step that computes configuration at runtime and then applies it. Doing this by shelling out to the pulumi binary forces you to parse stdout and lose typed access to outputs. The Automation API gives you the engine as a library, so a deployment is just a function call that returns structured results.

"As a library" is a slight simplification worth unpacking, because it explains most of the surprising behaviour later. auto.Stack is a typed wrapper that still spawns the pulumi binary as a child process for every operation. What changes is the direction of control: instead of the CLI finding a project directory and importing your __main__.py, the engine opens a gRPC language host and calls back into the Python process that started it, executing the callable you handed to program=. Your driver, the inline program, and the resource registrations all live in one interpreter; the engine, the state diff, and the provider plugins live in the subprocess and its children.

Three practical consequences follow directly from that layering. The pulumi binary must be on PATH even though you never type it. Environment variables the subprocess needs — credentials, PULUMI_CONFIG_PASSPHRASE — must be visible to the parent process or set explicitly on the workspace, because the child inherits from it. And an exception raised inside your inline program does not propagate as a normal Python traceback; it crosses the gRPC boundary, is reported to the engine as a failed resource registration, and comes back to your driver wrapped in a CommandError whose message contains the original traceback as text.

Prerequisites

• Python 3.9+ with pulumi>=3.0 installed: pip install pulumi.
• The Pulumi CLI binary on PATH (the engine the Automation API drives) and a configured backend — verify with pulumi whoami.
• AWS credentials in the environment (AWS_PROFILE or OIDC) with permission to create the demo resource (an S3 bucket): s3:CreateBucket, s3:DeleteBucket, s3:PutBucketTagging.
• A passphrase or secrets provider set (e.g. PULUMI_CONFIG_PASSPHRASE) so the stack can encrypt secret config.

The backend deserves a decision rather than a default. pulumi login --local writes checkpoints under ~/.pulumi, which is fine for a test fixture on one machine and actively dangerous for a service: two replicas each get their own state and will both try to create the same bucket. Any driver that runs in more than one place needs a shared backend — an S3 or Azure Blob bucket, or Pulumi Cloud — precisely so that the concurrency lock is shared too. The trade-offs are laid out in choosing a state backend for Python IaC.

Implementation

1. Define the inline program and select the stack

The inline program is a plain function. create_or_select_stack creates the stack if it does not exist or attaches to it if it does, which makes the driver idempotent across runs.

# CLI: python deploy.py
from typing import Callable
import pulumi
import pulumi_aws as aws
from pulumi import automation as auto

def make_program(env: str) -> Callable[[], None]:
    def program() -> None:
        # Provider note: executed by the engine, not at module import time.
        bucket = aws.s3.BucketV2(f"{env}-data", tags={"env": env})
        # State implication: this resource is recorded in the selected stack's checkpoint.
        pulumi.export("bucket_name", bucket.bucket)
    return program

stack = auto.create_or_select_stack(
    stack_name="dev",
    project_name="automation-demo",
    program=make_program("dev"),  # inline program: no project directory
)

The closure is doing real work here and is the main reason to prefer inline programs. make_program captures env from the caller, so a service handling a request for customer acme can build a program bound to that customer's parameters without templating a file or setting stack config first. Anything you can compute in Python before the call — a database lookup, a feature flag, a size derived from a plan tier — can be baked into the program object.

Two constraints on that closure are easy to violate. It must be safe to run more than once, because preview and up each execute it, and refresh may too; a program that appends to a module-level list or increments a counter produces different resources on the second call and the engine reports spurious diffs. It must also not perform blocking I/O of unbounded duration, because the engine is waiting on the language host while it runs, and a hung HTTP call inside the program looks exactly like a hung deployment from the outside.

create_or_select_stack accepts opts=auto.LocalWorkspaceOptions(...) when you need to control the environment the subprocess sees:

# CLI: python deploy.py
from pulumi import automation as auto

stack = auto.create_or_select_stack(
    stack_name="dev",
    project_name="automation-demo",
    program=make_program("dev"),
    opts=auto.LocalWorkspaceOptions(
        env_vars={
            "PULUMI_CONFIG_PASSPHRASE": passphrase,   # never hard-code this
            "AWS_REGION": "us-east-1",
        },
        # Provider note: work_dir is where the CLI writes Pulumi.yaml for the
        # synthetic project. Leave it unset and the API uses a temp directory
        # that is removed when the workspace is garbage collected.
        project_settings=auto.ProjectSettings(
            name="automation-demo",
            runtime="python",
            backend=auto.ProjectBackend(url="s3://acme-pulumi-state"),
        ),
    ),
)

Pinning backend in ProjectSettings rather than relying on an ambient pulumi login is the difference between a driver that behaves the same on a laptop and in a container, and one that quietly writes to ~/.pulumi in production.

2. Configure the workspace and stack

Install the provider plugin into the workspace, then set configuration through typed methods. Secret values are marked so the backend encrypts them.

# CLI: python deploy.py
stack.workspace.install_plugin("aws", "v6.0.0")
stack.set_config("aws:region", auto.ConfigValue(value="us-east-1"))
# State implication: secret=True encrypts the value with the stack's secrets provider.
stack.set_config("automation-demo:apiToken", auto.ConfigValue(value="tok_xyz", secret=True))

install_plugin is not optional in a container image built without a plugin cache. Skipping it produces error: no resource plugin 'aws' found in the workspace or on your $PATH, which appears at the first resource registration rather than at startup — so a driver that previews cleanly on a developer machine (where the plugin is already cached in ~/.pulumi/plugins) fails on the first real run in CI. Pin the version explicitly and treat it as part of the image build, not the request path: downloading a provider plugin adds seconds to a deployment that a user is waiting on.

Config set through set_config is persisted to the stack, not to the process. That means it survives across runs and is visible to pulumi config get from the CLI, which is usually what you want for auditability but is a leak if you pass per-request values that way. For values that should not outlive the run, close over them in the program instead. Secret handling is worth being deliberate about: secret=True encrypts at rest with the stack's secrets provider, and the pattern for rotating those values without a redeploy is covered in rotating Pulumi stack secrets without downtime.

3. Refresh, deploy, and capture outputs

Optionally refresh to reconcile state with reality, then up. The result object exposes outputs as typed values you return to the caller.

# CLI: python deploy.py
from typing import Dict

def deploy(stack: auto.Stack) -> Dict[str, str]:
    stack.refresh(on_output=print)  # State implication: reconciles drift before applying.
    result = stack.up(on_output=print)
    # outputs maps name -> OutputValue; .value is the resolved Python value.
    return {key: out.value for key, out in result.outputs.items()}

outputs = deploy(stack)
print("provisioned bucket:", outputs["bucket_name"])

refresh is a real cloud round-trip per resource and it is not free — on a stack of a few hundred resources it can dominate the wall-clock time of the whole operation. Run it when the resources may have been changed outside Pulumi (a human in the console, an autoscaler, another tool) and skip it when the stack is exclusively owned by this driver. Note also that refresh mutates the checkpoint: if reality has diverged, the refreshed state records reality, and the subsequent up plans from there. That is the correct behaviour but it means a refresh immediately before up can turn "no changes" into a destroy of something a human created by hand.

result.summary is the part most drivers under-use. It carries result ("succeeded", "failed", "in-progress"), resource_changes as a dict keyed by operation, and the start and end timestamps. Emitting those as metrics turns a deployment service into something you can operate:

# CLI: python deploy.py
from typing import Dict, Optional
from pulumi import automation as auto

def summarise(result: auto.UpResult) -> Dict[str, int]:
    """Normalise the change counts so a missing key is zero, not a KeyError."""
    changes: Optional[Dict[str, int]] = result.summary.resource_changes
    counts = changes or {}
    return {
        "created": counts.get("create", 0),
        "updated": counts.get("update", 0),
        "deleted": counts.get("delete", 0),
        "replaced": counts.get("replace", 0),
        "unchanged": counts.get("same", 0),
    }
# State implication: a non-zero "replace" count means a resource was destroyed
# and recreated — for a database or a bucket that is data loss, so gate on it.

A driver that refuses to proceed when replaced > 0 unless an explicit flag is passed catches an entire class of accident that no amount of code review does.

4. Wrap the run with structured error handling

Engine failures raise typed exceptions. Catch them so a service returns a clean error and a pipeline exits non-zero with a useful message instead of a stack trace.

# CLI: python deploy.py
from pulumi.automation.errors import (
    CommandError,
    ConcurrentUpdateError,
    StackAlreadyExistsError,
)

def safe_deploy(stack: auto.Stack) -> int:
    try:
        outputs = deploy(stack)
        print("ok:", outputs)
        return 0
    except ConcurrentUpdateError:
        # State implication: another operation holds the backend lock; do not retry blindly.
        print("stack is locked by another update; aborting")
        return 2
    except CommandError as exc:
        print(f"engine error: {exc}")
        return 1

if __name__ == "__main__":
    raise SystemExit(safe_deploy(stack))

The exception hierarchy is shallow and worth memorising because the recovery differs sharply per type. ConcurrentUpdateError means the backend already holds a lock for this stack — the message reads the stack is currently locked by 1 lock(s) and includes the lock holder and timestamp. Retrying immediately is wrong; either another replica is legitimately mid-update, or a previous run died and left a stale lock. Automatic cancel() is a footgun, because cancelling an update that is genuinely in flight leaves resources created in the cloud but absent from the checkpoint.

StackNotFoundError and StackAlreadyExistsError are both symptoms of using select_stack or create_stack where create_or_select_stack belongs. CommandError is the catch-all: the engine exited non-zero for a reason that is in its stdout, which is why on_output matters — without it the message you surface to a caller is code: 255 and nothing else. InvalidVersionError shows up when the installed pulumi binary is older than the pulumi Python package expects, a common failure in a container that pins the pip package but installs the CLI from a rolling script.

One further distinction: a failed deployment is not the same as a failed program. If your inline program raises before registering resources, the run fails with nothing created. If it raises after some resources are registered, the engine has already created them and records them in the checkpoint — the stack is now partially applied, and the next up will continue from there rather than starting clean. Drivers that treat any exception as "nothing happened" leak resources for exactly this reason.

5. Tear down and remove the stack

An ephemeral environment is only useful if it disappears. Destroy and removal are two separate operations, and skipping the second leaves an empty stack record in the backend for every environment you ever created.

# CLI: python deploy.py --teardown
from pulumi import automation as auto

def teardown(stack: auto.Stack) -> None:
    stack.destroy(on_output=print)
    # State implication: destroy empties the checkpoint but the stack still
    # exists in the backend; remove_stack deletes the record itself.
    stack.workspace.remove_stack(stack.name)

remove_stack refuses to delete a stack with resources still in its checkpoint, raising CommandError with error: refusing to remove stack 'dev' because it still has resources. That guard is correct — force-removing a populated stack orphans every resource it tracked, with no record anywhere of what to clean up. If a destroy partially fails, fix the failing resource and destroy again rather than reaching for --force.

Verification

Assert the operation succeeded and the expected output came back, then confirm the same stack is visible to the CLI on the shared backend.

# CLI: python -m pytest test_deploy.py
def test_deploy_returns_bucket_name() -> None:
    result = stack.up(on_output=print)
    assert result.summary.result == "succeeded"
    assert result.outputs["bucket_name"].value.startswith("dev-data")

# The driver and CLI share one backend, so the result is independently checkable.
pulumi stack output bucket_name --stack dev
aws s3 ls | grep dev-data

Gotchas & Edge Cases

Outputs you forgot to export are absent from result.outputs. The Automation API only returns what the program calls pulumi.export on. If result.outputs["bucket_name"] raises KeyError, the export is missing from the inline program, not lost by the API.

Secret outputs are redacted unless you opt in. A secret=True config value flows into a secret output. Reading .value gives you the plaintext in-process, but on_output logs will show [secret]. Never print secret outputs to shared CI logs.

create_or_select_stack is idempotent, but create_stack is not. If you call create_stack on a name that already exists you get StackAlreadyExistsError. Use create_or_select_stack in any driver that may run more than once, such as a retried CI job or a long-lived service.

on_output is a blocking callback on the engine's stream. Whatever you pass runs synchronously for every line the engine emits. A callback that writes to a slow sink — an HTTP log shipper, an unbuffered network socket — applies backpressure to the deployment itself. Append to an in-memory deque and ship asynchronously.

Two auto.Stack objects for the same stack name do not coordinate in-process. The lock lives in the backend, not in your program, so a local backend gives you no protection between threads or replicas. Even with a shared backend, the second caller gets ConcurrentUpdateError rather than queuing, so serialisation is your responsibility.

The inline program's exceptions arrive as text, not as objects. Because the traceback crosses a process boundary, except ValueError around stack.up() will never fire for a ValueError raised inside the program. You get a CommandError whose string contains the original traceback. Structured error handling therefore belongs inside the program, converting failures into something the engine can report cleanly.

Operational Notes

The single most important operational decision is inline versus local, and it is worth revisiting once the driver leaves the prototype stage.

Never run a deployment on the request thread. An up() against a real cloud takes tens of seconds at best and tens of minutes at worst, and it is not cancellable in a way that a dropped HTTP connection would trigger. Push the operation to a worker with a durable queue, return a job id immediately, and let the caller poll. The pattern is worked through end to end in embedding Pulumi in a FastAPI service with the Automation API.

Bound the concurrency per stack to exactly one, and per host to something small. Each concurrent operation spawns a pulumi process plus one provider plugin process per provider, and each of those holds its own connection pool. A worker that happily starts twenty deployments will exhaust file descriptors before it exhausts CPU. A semaphore keyed on stack name plus a global cap is enough.

Cache the plugins into the image. install_plugin downloads from the Pulumi CDN on a cold workspace. In a container that means every cold start pays a multi-megabyte download before the first resource is touched. Run the installs at build time so ~/.pulumi/plugins ships in the layer, and keep install_plugin in the code as a cheap no-op assertion.

Treat stack names as a namespace you are responsible for garbage collecting. A per-customer or per-pull-request stack is created programmatically and will be forgotten programmatically. Encode the owner and creation time in stack tags, and run a sweeper that destroys and removes stacks past a TTL. Without one, the backend accumulates thousands of stack records and list_stacks becomes the slowest call in the driver.

Log the summary, not the stream, to your permanent store. The on_output stream is verbose, contains resource URNs, and may include values you would rather not retain. Persist result.summary and the change counts for audit; keep the raw stream in short-retention logs for debugging. The audit-trail angle is covered in tracking IaC change ownership and audit trails.

FAQ

When should I use the Automation API over the CLI?

Reach for it when deployments must be triggered by events or embedded in a service — see embedding Pulumi in a FastAPI service. For interactive work, the CLI is simpler.

Does it share state with the CLI?

Yes — the Automation API drives the same engine and state backend, so a stack it creates is fully manageable from the CLI and vice versa.

How do I preview before applying?

Call stack.preview() and inspect the change summary; many self-service flows require a human to approve the preview before up() runs.

Can I run several stacks in parallel from one driver?

Yes, provided each thread or task owns a distinct stack. Different stacks take different backend locks, so they do not contend. Cap the concurrency well below what the machine appears to allow — every operation spawns a CLI process plus a provider plugin per provider.

How do I recover from ConcurrentUpdateError when the previous run crashed?

Confirm nothing is actually running, then stack.cancel() to clear the lock, then stack.refresh() before the next up. The refresh matters: a crashed update may have created resources the checkpoint does not know about, and refreshing reconciles the checkpoint with reality before you plan on top of it.

Does the Automation API work with a program that lives in a directory rather than a callable?

Yes — auto.create_or_select_stack(stack_name=..., work_dir="./infra") selects a local program with its own Pulumi.yaml. Everything in this guide about config, outputs, errors, and locking applies unchanged; only the source of the resource definitions differs.

Related

• The Pulumi Automation API — the parent overview of inline versus local programs and the execution model.
• Pulumi Stack Architecture — stack naming, config, and backend isolation this deployment relies on.
• Pulumi Patterns & Provider Management — the parent section covering Pulumi workflows and provider strategies.