Authenticating Pulumi to Azure with Service Principals

A pipeline cannot run az login interactively, so CI needs a non-human identity. This guide, part of Azure provider configuration under Pulumi patterns and provider management, creates a scoped service principal and wires its credentials into Pulumi without writing a secret to disk.

Context

Azure authentication needs three values that fail independently: a tenant (which directory), a client (which identity), and a subscription (which billing scope). A principal that is valid in one tenant is meaningless in another, which is why SubscriptionNotFound is the usual symptom of a tenant mismatch rather than a missing subscription. Scoping the principal to exactly the resource groups it manages keeps a leaked credential from becoming a subscription-wide incident, applying the same reasoning as enforcing IAM least privilege.

Underneath, this is the OAuth 2.0 client credentials grant and nothing more exotic. The provider posts the client id and secret to https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token, asks for the scope https://management.azure.com/.default, and receives a bearer token valid for roughly an hour. Every subsequent ARM call carries that token in an Authorization header. Two consequences follow. First, the secret never leaves the token endpoint, so a network capture of an ARM request exposes a short-lived token rather than a long-lived credential. Second, authentication and authorization are separate steps with separate failure messages: a bad secret fails at the token endpoint with an AADSTS… code, while a valid token with insufficient rights fails at ARM with AuthorizationFailed. Reading which of the two you got tells you immediately whether to look at the app registration or at the role assignment.

It also helps to be precise about the objects involved, because the portal blurs them. An application registration is the global definition of the identity — it owns the client id and the credentials. A service principal is the per-tenant instance of that application, and it is the object role assignments actually point at, identified by its own object id rather than the client id. az ad sp create-for-rbac creates both in one call, which is why the distinction only surfaces later, usually when a role assignment made against the client id silently grants nothing.

Both Python providers read the same environment variables. pulumi-azure-native maps the Azure Resource Manager API surface directly and is the default choice for new work; pulumi-azure wraps the Terraform azurerm provider and is worth reaching for only when a resource has not been surfaced natively. Whichever you use, the credential resolution order is the same: explicit provider arguments beat ARM_* environment variables, which beat Pulumi config values under the azure-native: namespace, which beat the Azure CLI's cached login. That last fallback is what makes a local pulumi up work without any configuration and then fail in CI — the developer machine was quietly authenticating as a human.

Credential chain Credential chain: layered from Tenant down to Role assignment. Tenant which Entra ID directory Subscription which billing + scope boundary Service principal which identity, which secret Role assignment what it may do, and where
All four must line up; any one of them wrong produces an authorization error.

Prerequisites

  • Azure CLI 2.50+ and permission to create app registrations in the tenant
  • Owner or User Access Administrator on the target scope, so you can create role assignments
  • pulumi>=3.0 with pulumi-azure-native>=2.0 pinned
  • A secret store for the client secret — the pipeline's secret manager, never a committed file
# CLI: confirm you may create role assignments at the intended scope
az role assignment list --scope "/subscriptions/$SUB_ID/resourceGroups/platform-prod" -o table

Implementation

Step 1 — create a principal scoped to one resource group. Scoping at creation is easier than trimming permissions later.

Non-interactive login Non-interactive login: Pipeline → Entra ID → ARM API. Pipeline Entra ID ARM API client id + secret access token request with token resource created
The principal exchanges its secret for a token; ARM never sees the secret itself.
# CLI: create a Contributor principal limited to a single resource group
az ad sp create-for-rbac \
  --name pulumi-platform-prod \
  --role Contributor \
  --scopes "/subscriptions/$SUB_ID/resourceGroups/platform-prod"
# Emits appId (client), password (secret), and tenant — the password is shown once.

The resource group must already exist; --scopes is validated against a real ARM path and a typo produces Resource group 'platform-prod' could not be found. Note that Contributor is a convenience, not a requirement — it can create and delete anything in the scope but cannot grant access to it, which is usually what you want for a deploy identity. If the stack itself assigns roles (giving a function app access to a key vault, for example) the principal also needs Role Based Access Control Administrator or, more narrowly, User Access Administrator on the same scope; without it the deploy fails at the assignment step with AuthorizationFailed even though every other resource was created.

Step 2 — expose the values as environment variables. The Azure providers read these directly; nothing needs to be written into the Pulumi program.

# CLI: the four variables both Azure providers understand
export ARM_TENANT_ID="<tenant>"
export ARM_CLIENT_ID="<appId>"
export ARM_CLIENT_SECRET="<password>"   # from the pipeline secret store
export ARM_SUBSCRIPTION_ID="$SUB_ID"

ARM_SUBSCRIPTION_ID is the one people forget, because a local CLI login supplies it implicitly from the active account. With only the first three set, pulumi preview fails on the first resource with Error: no subscription ID was specified; ensure ARM_SUBSCRIPTION_ID is set rather than at provider initialisation, which makes it look like a resource problem. Set all four together and treat them as a unit.

When a single program targets more than one subscription — a shared network in one, workloads in another — do not rely on ambient variables at all. Instantiate an explicit provider per subscription and pass it to the resources that belong there:

# providers.py — explicit providers when one program spans subscriptions
# CLI: pulumi up --stack prod
import pulumi
import pulumi_azure_native as azure_native

cfg = pulumi.Config()

hub = azure_native.Provider(
    "hub",
    subscription_id=cfg.require("hubSubscriptionId"),
    # Provider note: tenant/client/secret still come from ARM_* — only the
    # subscription differs, so one principal must hold rights in both.
)

hub_rg = azure_native.resources.ResourceGroup(
    "hub-network",
    location="westeurope",
    opts=pulumi.ResourceOptions(provider=hub),
)
# State implication: the provider reference is recorded per resource. Removing
# it later re-targets the resource at the default provider and Pulumi plans a
# replacement, not an update.

Step 3 — read the identity back inside the program when a resource needs to grant the deploying principal access to something it creates.

# identity.py — resolve the running principal instead of hard-coding an object id
# CLI: pulumi up
import pulumi
import pulumi_azure_native.authorization as authz

current = authz.get_client_config()          # Provider note: reads the ambient credential
principal_id: pulumi.Output[str] = pulumi.Output.from_input(current.object_id)

pulumi.export("deployingPrincipal", principal_id)
# State implication: nothing is created here; the lookup runs on every preview,
# so a credential change is visible as a diff on any resource that depends on it.

get_client_config() returns object_id, client_id, tenant_id and subscription_id. The one you almost always want is object_id — the service principal object, not the application — because that is what RoleAssignment.principal_id expects. Passing the client id there produces a role assignment that ARM accepts and that grants nothing, a failure mode with no error message at all.

Step 4 — prefer a federated credential when the pipeline supports it. Workload identity federation replaces the client secret with a short-lived OIDC token that the CI platform mints for the specific repository and branch, so there is nothing left to rotate or leak.

Where does the deploy run? Where does the deploy run?: choose among 3 options. Choosing an Azure deployidentity on Azure Managed identity, nosecret exists GitHub OIDC Federated credentialper branch elsewhere Service principalwith a client secret
Reach for a client secret only when the runner cannot present a platform-issued token.
# CLI: trust one repository's main branch instead of issuing a secret
az ad app federated-credential create \
  --id "$APP_OBJECT_ID" \
  --parameters '{
    "name": "github-platform-main",
    "issuer": "https://token.actions.githubusercontent.com",
    "subject": "repo:acme/platform-infra:ref:refs/heads/main",
    "audiences": ["api://AzureADTokenExchange"]
  }'
# The subject is matched exactly — a deploy from a tag or a pull request
# has a different subject and is refused with AADSTS70021.

With the federated credential in place the pipeline exports ARM_TENANT_ID, ARM_CLIENT_ID, ARM_SUBSCRIPTION_ID and ARM_USE_OIDC=true, and omits ARM_CLIENT_SECRET entirely. Everything downstream of that — the Pulumi program, the role assignments, the scoping — is unchanged.

Verification

Prove the principal works and that it is scoped no wider than intended. A principal that can do everything will pass the first check and fail an audit.

# CLI: authenticate as the principal and confirm the scope of its rights
az login --service-principal -u "$ARM_CLIENT_ID" -p "$ARM_CLIENT_SECRET" -t "$ARM_TENANT_ID"
az role assignment list --assignee "$ARM_CLIENT_ID" --all -o table
pulumi preview --stack prod    # should plan without prompting for a login

The role assignment list should show exactly one row, scoped to the resource group — not the subscription. --all matters: without it the command only searches the current subscription, so an over-broad assignment made in a different subscription stays invisible.

Two further checks are worth building into the pipeline rather than running by hand. The first proves the token endpoint works independently of anything Pulumi does, which separates a credential problem from a program problem in one command:

# CLI: prove the principal can mint an ARM token at all
az account get-access-token --resource https://management.azure.com \
  --query '{tenant: tenant, expires: expiresOn}' -o json
# Failure here is authentication (AADSTS...). Success followed by a Pulumi
# failure is authorization (AuthorizationFailed) — a different fix entirely.

The second asserts the scope from inside the program, so a principal that is quietly widened later fails the deploy instead of passing unnoticed:

# guard.py — refuse to run if the deploy identity is subscription-wide
# CLI: pulumi preview --stack prod
import pulumi
import pulumi_azure_native.authorization as authz

cfg = pulumi.Config()
expected_scope: str = cfg.require("allowedScope")

assignments = authz.list_role_assignment_for_scope_output(
    scope=expected_scope,
)


def _reject_subscription_wide(result: authz.ListRoleAssignmentForScopeResult) -> int:
    # Provider note: this is a data lookup, so it runs during preview and adds
    # no resource to the stack — only a hard failure if the check does not hold.
    wide = [a for a in (result.value or []) if (a.scope or "").count("/") < 3]
    if wide:
        raise Exception(f"deploy identity holds {len(wide)} subscription-wide assignment(s)")
    return len(result.value or [])


pulumi.export("scopedAssignments", assignments.apply(_reject_subscription_wide))

Gotchas & Edge Cases

Client secrets expire. create-for-rbac issues a secret with a default lifetime; when it lapses, every deploy fails with AADSTS7000215: Invalid client secret provided. Track the expiry and rotate ahead of it, or use workload identity federation so there is no secret to expire.

Role assignments propagate slowly. A newly created assignment can take a minute to become effective, so a pipeline that creates a principal and immediately uses it fails intermittently. Separate identity bootstrap from the deploy that consumes it.

--scopes accepts a list. Granting one principal several resource groups is fine and still far narrower than subscription-wide Contributor. Widen deliberately, one scope at a time.

A deleted app registration leaves its role assignments behind. Removing the application does not remove the assignments that pointed at its service principal; they persist as orphaned entries showing Identity not found in the portal. They are harmless but they hide the real access picture, and a new principal created with the same display name does not inherit them. Clean up with az role assignment delete --assignee <object-id> before deleting the app.

The wrong id in principal_id. RoleAssignment wants the service principal's object id. Passing the application (client) id creates an assignment that ARM stores and that grants nothing — no error, no denial, just a resource that silently fails to work. Always source it from get_client_config().object_id or az ad sp show --id <appId> --query id -o tsv.

Local previews that pass and pipelines that fail. When ARM_CLIENT_SECRET is unset the provider silently falls through to the Azure CLI's cached token, so a developer running pulumi preview is testing their own rights, not the principal's. Reproduce CI locally by running az logout first, or by exporting ARM_USE_CLI=false so the fallback is disabled and the misconfiguration surfaces immediately.

Federated subjects are matched literally. A credential registered for ref:refs/heads/main rejects a run triggered by a tag or a pull request with AADSTS70021: No matching federated identity record found for presented assertion subject. Register one credential per subject you actually deploy from, and expect the environment-scoped form (repo:acme/platform-infra:environment:prod) to be the one you want for production.

Operational Notes

Treat each deploy identity as a named, owned asset with a lifecycle rather than a value in a secret store. The practical minimum is one principal per environment per subscription, each scoped to that environment's resource groups, each with a recorded owner and expiry.

Azure deploy credential types Azure deploy credential types: comparison across Lifetime, Rotation work. Credential Lifetime Rotation work Client secret months, fixed manual overlap rotation Federated (OIDC) minutes per run none, subject pinned Managed identity platform managed none, Azure only User CLI login hours not usable in CI
What each Azure credential type costs to operate over a year.

Rotation is the part that decays first. az ad sp create-for-rbac issues a secret with a default lifetime measured in months, and the failure when it lapses is total: every deploy in that environment stops. Add a second credential before the first expires rather than replacing it in place — an app registration can hold several passwords at once, so the sequence is add, update the pipeline secret, verify a deploy, then delete the old one. That ordering means the rollback is always available.

# CLI: overlap-rotate a client secret with no deploy outage
NEW=$(az ad app credential reset --id "$APP_OBJECT_ID" --append \
        --display-name "rotation-2026-08" --years 1 --query password -o tsv)
# --append is what makes this safe: without it, every existing secret is
# invalidated the moment the command returns.
az ad app credential list --id "$APP_OBJECT_ID" \
  --query '[].{name: displayName, expires: endDateTime}' -o table

Store the secret in the pipeline's own secret manager, not in Pulumi config. Putting it in the stack's encrypted config feels tidy but creates a bootstrap loop — the credential needed to decrypt the config lives in the config — and it puts the credential in the state file's provenance. The stack config is for values that describe the environment; the credential that authenticates the deploy belongs to the deploy platform. The wider version of that argument is in best practices for managing cloud credentials in Python, and the mechanics of rotating a secret a running stack depends on are covered in rotating Pulumi stack secrets without downtime.

Finally, make the identity observable. Entra ID sign-in logs record every token the principal requests, including the calling IP and whether the request succeeded, and service principal sign-ins are queryable separately from user sign-ins. An alert on a successful token request from outside the pipeline's egress range is a cheap and specific tripwire — far more actionable than a generic anomaly rule, because a deploy identity should only ever authenticate from one place.

FAQ

Should the client secret go into Pulumi config?

No. Keep it in the pipeline's secret store and expose it as ARM_CLIENT_SECRET. Pulumi config is for values that describe the stack, not for credentials the stack authenticates with.

What is the difference between a service principal and a managed identity?

A managed identity is a principal Azure creates and rotates for a resource that runs inside Azure, so there is no secret to store. Use one when the deploy runs on Azure compute; use a service principal when it runs elsewhere.

Why does SubscriptionNotFound appear when the subscription clearly exists?

The principal is authenticating against a different tenant. Compare az account show --query tenantId with the ARM_TENANT_ID the pipeline exports.

Can one principal serve every environment?

It can, and it should not. A shared principal makes a compromised development pipeline a production incident; issue one per environment and scope each to its own resource groups.

What does AuthorizationFailed mean if the credentials are correct?

It means authentication succeeded and authorization did not: the token is valid, but the principal has no role assignment granting the requested action at the requested scope. The message names both, so compare the action (Microsoft.Network/virtualNetworks/write, say) against az role assignment list --assignee "$ARM_CLIENT_ID" --all. A token problem would have failed earlier with an AADSTS code instead.

Do I need ARM_SUBSCRIPTION_ID if the principal only has one subscription?

Yes. The provider does not enumerate subscriptions to guess; without the variable it fails with no subscription ID was specified. A local run appears to work only because the Azure CLI fallback supplies the active account's subscription.