The Pulumi Automation API

The Pulumi Automation API lets you drive preview, up, and destroy from a Python program instead of shelling out to the CLI, turning infrastructure provisioning into a library call you can embed in services, test suites, and self-service tooling. It is part of the broader Pulumi Patterns & Provider Management workflow, and it builds directly on the same stack and backend concepts covered in Pulumi Stack Architecture. This guide explains the execution model, the difference between inline and local programs, and how to operate stacks programmatically with proper error handling.

Automation API orchestration flow A driver program calls the Automation API, which creates or selects a stack and runs preview, up, and destroy against cloud providers and the state backend, without invoking the CLI. Driver program app.py / pytest Automation API create_or_select preview / up / destroy Cloud providers AWS / GCP / Azure State backend S3 / Pulumi Cloud no CLI
The driver program calls the Automation API, which orchestrates stack operations against cloud providers and the state backend with no CLI process involved.

Problem Framing

Without the Automation API, every Pulumi operation runs through the pulumi CLI: a human types a command, or a CI job invokes the binary and parses its text or JSON output. That works for hand-driven deployments, but it breaks down the moment you want infrastructure to be a feature of an application — a "create me an environment" button, a per-tenant provisioning service, or a test that spins up real resources and tears them down. Wrapping a subprocess to scrape stdout is brittle, and you lose typed access to outputs and structured error information. The Automation API solves this by exposing the same engine the CLI uses as a first-class Python library, so you get stacks, configuration, and results as objects.

Prerequisites

  • Python 3.9+ and the Pulumi CLI installed on the host (the Automation API still needs the engine binary on PATH, even though it never spawns the interactive CLI).
  • The pulumi Python package: pip install pulumi>=3.0.
  • A configured state backend and a passphrase or secrets provider. Verify with pulumi whoami and pulumi about.
  • Cloud credentials available in the environment (e.g. AWS_PROFILE or OIDC), exactly as a normal Pulumi program expects.
# Verify the engine and backend are reachable before driving them from Python.
pulumi version
pulumi whoami

Concept Explanation

Inline programs versus local programs

The Automation API supports two program sources. An inline program is a Python function passed directly to the stack — there is no __main__.py and no project directory; the function defines resources when the engine calls it. A local program points the Automation API at an existing on-disk Pulumi project, exactly the layout you already use with the CLI. Inline programs are ideal for embedding infrastructure in a service or test because everything lives in one process; local programs are best when you want to reuse a project that engineers also run by hand.

# Run with: python deploy.py
import pulumi_aws as aws
from pulumi import automation as auto

def pulumi_program() -> None:
    # Provider note: this runs inside the Pulumi engine, not at import time.
    bucket = aws.s3.BucketV2("data")
    # State implication: every resource declared here is tracked in the stack's state.
    import pulumi
    pulumi.export("bucket_name", bucket.bucket)

stack = auto.create_or_select_stack(
    stack_name="dev",
    project_name="automation-demo",
    program=pulumi_program,  # inline program: no project directory needed
)

Stacks and configuration as objects

create_or_select_stack returns a Stack object that owns its workspace. You set config and secrets through typed methods rather than editing Pulumi.<stack>.yaml by hand, which keeps secrets out of source files and lets a driver compute values at runtime.

# Part of deploy.py
from pulumi import automation as auto

stack.set_config("aws:region", auto.ConfigValue(value="us-east-1"))
# State implication: secrets are encrypted by the stack's secrets provider before storage.
stack.set_config("app:dbPassword", auto.ConfigValue(value="s3cr3t", secret=True))

Results are structured, not scraped

Operations like up() return a result object exposing summary, outputs, and per-resource change counts. You read outputs as Python values instead of parsing CLI text, which is what makes the Automation API safe to build services on. See Programmatic deployments with the Pulumi Automation API for a complete inline-program deployment with output capture and error handling.

Step-by-Step Implementation

1. Install plugins and select the stack

The first time a workspace runs, it must install the provider plugins the program needs.

# python deploy.py
from pulumi import automation as auto

stack = auto.create_or_select_stack(
    stack_name="dev",
    project_name="automation-demo",
    program=pulumi_program,
)
# Provider note: install the AWS plugin into the workspace before previewing.
stack.workspace.install_plugin("aws", "v6.0.0")
stack.set_config("aws:region", auto.ConfigValue(value="us-east-1"))

2. Preview, then apply

Run a preview to compute the diff, then up to apply it. Stream the engine's logs with on_output so a service or pipeline sees progress in real time.

# python deploy.py
preview = stack.preview(on_output=print)
print("planned changes:", preview.change_summary)

up_result = stack.up(on_output=print)
# State implication: up() writes the new checkpoint to the configured backend.
print("bucket:", up_result.outputs["bucket_name"].value)

3. Destroy when finished

For ephemeral environments — test fixtures, preview environments — tear everything down with destroy, optionally removing the stack record afterwards.

# python deploy.py
stack.destroy(on_output=print)
# State implication: remove_stack deletes the now-empty stack from the backend.
stack.workspace.remove_stack("dev")

Verification

Confirm the driver actually changed real state by reading the result counts and then cross-checking with the CLI against the same backend.

assert up_result.summary.kind == "update"
assert up_result.summary.result == "succeeded"
assert "bucket_name" in up_result.outputs
# The Automation API and CLI share one backend, so the stack is visible to both.
pulumi stack ls
pulumi stack output bucket_name --stack dev

Troubleshooting

automation.errors.CommandError mentioning "no Pulumi.yaml found". You used a local program but pointed it at a directory without a project file. Either pass work_dir to a real project root, or switch to an inline program with create_or_select_stack(..., program=...).

could not find plugin for provider aws. The workspace has not installed the provider. Call stack.workspace.install_plugin("aws", "v6.0.0") before preview/up, or ensure the plugin is present on the host.

Operation hangs or fails with "the stack is currently locked". A previous run or a concurrent CLI invocation holds the backend lock. Wait for it to finish, or run pulumi cancel --stack dev once you are certain no operation is in progress.

Frequently Asked Questions

Do I still need the Pulumi CLI installed if I use the Automation API? Yes. The Automation API drives the same engine binary the CLI uses, so the pulumi executable must be on PATH. What you avoid is invoking the interactive CLI yourself and parsing its text output — the engine is called as a library and returns structured results.

When should I use an inline program instead of a local one? Use inline programs when infrastructure is part of an application or test and you want everything in one process with no separate project directory. Use local programs when you are wrapping an existing Pulumi project that engineers also run by hand with the CLI, so both paths share the same code.

Is the Automation API safe to call concurrently for different stacks? Yes, as long as each stack is operated independently. The backend enforces a per-stack lock, so two drivers updating the same stack will conflict, but separate stacks can be provisioned in parallel. Build per-environment isolation as described in Pulumi Stack Architecture.

How do I get stack outputs back into my application? up() returns a result whose outputs dictionary maps output names to typed values, including secrets you can unwrap. Read them directly rather than calling pulumi stack output and parsing the CLI.