Creating and Securing GCS Buckets with Pulumi (Python)

Creating a Google Cloud Storage bucket with Pulumi Python is one resource; securing it is the rest of the work — uniform bucket-level access, scoped IAM bindings, object versioning, and customer-managed encryption, part of the broader GCP Provider Configuration workflow. A default bucket inherits legacy ACLs and project-wide IAM; this guide closes those gaps with a typed, repeatable pattern.

This guide provisions a bucket with uniform bucket-level access enabled, versioning on, a CMEK from Cloud KMS, and a single least-privilege IAM binding — then verifies that public access is impossible.

Context

A misconfigured GCS bucket is one of the most common cloud data leaks: legacy ACLs let an object be made public even when project IAM looks locked down. Uniform bucket-level access removes ACLs entirely, so IAM is the only access path — which makes the bucket auditable. Doing this at creation time costs nothing; retrofitting it onto a bucket already holding objects with per-object ACLs is a migration. The encryption and IAM discipline here is the same you apply when deploying a GKE cluster with Pulumi (Python) and pulls its credential routing from the parent provider configuration.

Prerequisites

Python 3.9+ with pulumi>=3.0 and pulumi-gcp>=7.0.
A GCP project with billing enabled and GCP_PROJECT / GCP_REGION set, or the provider configured per the parent guide.
IAM permissions for storage.buckets.create, storage.buckets.setIamPolicy, and (for CMEK) cloudkms.cryptoKeys.get plus the ability to grant the storage service agent encrypt/decrypt.
An existing Cloud KMS key ring and crypto key, or permission to create them.
mypy for static checking of the typed config.

Implementation

1. Define a typed bucket configuration

# infra/bucket_config.py
# CLI: mypy --strict infra/
from __future__ import annotations
from dataclasses import dataclass
from typing import Optional

@dataclass(frozen=True)
class BucketConfig:
    name: str
    location: str = "US"
    storage_class: str = "STANDARD"
    kms_key_id: Optional[str] = None
    reader_member: Optional[str] = None  # e.g. "serviceAccount:[email protected]"
    # State implication: changing `name` or `location` forces replacement.

2. Create the bucket with UBLA, versioning, and CMEK

uniform_bucket_level_access=True is the single most important field — it disables object ACLs. public_access_prevention="enforced" blocks any IAM grant to allUsers/allAuthenticatedUsers.

# infra/bucket.py
# CLI: pulumi preview --diff
from __future__ import annotations
import pulumi_gcp as gcp
from infra.bucket_config import BucketConfig

def build_bucket(cfg: BucketConfig) -> gcp.storage.Bucket:
    encryption = (
        gcp.storage.BucketEncryptionArgs(default_kms_key_name=cfg.kms_key_id)
        if cfg.kms_key_id else None
    )
    return gcp.storage.Bucket(
        cfg.name,
        name=cfg.name,
        location=cfg.location,
        storage_class=cfg.storage_class,
        # Provider note: UBLA removes ACLs so IAM is the only access path.
        uniform_bucket_level_access=True,
        public_access_prevention="enforced",
        versioning=gcp.storage.BucketVersioningArgs(enabled=True),
        encryption=encryption,
        force_destroy=False,  # State implication: blocks deleting a non-empty bucket
    )

3. Grant a single scoped IAM binding

Use BucketIAMMember (one principal, one role) rather than BucketIAMPolicy (authoritative, overwrites everything). Member is additive and safe; policy can lock you out if it omits your own access.

# infra/bucket.py (continued)
# CLI: pulumi up
import pulumi_gcp as gcp

def grant_reader(bucket: gcp.storage.Bucket, member: str) -> gcp.storage.BucketIAMMember:
    # Provider note: BucketIAMMember is additive; BucketIAMPolicy is
    # authoritative and will drop bindings it does not list.
    return gcp.storage.BucketIAMMember(
        f"{bucket._name}-reader",
        bucket=bucket.name,
        role="roles/storage.objectViewer",
        member=member,
    )

4. Wire the CMEK grant and export

Before the bucket can use a CMEK, the GCS service agent needs roles/cloudkms.cryptoKeyEncrypterDecrypter on the key. Grant it in the same program so the dependency is explicit.

# __main__.py
# CLI: pulumi up && pulumi stack output bucketUrl
import pulumi
import pulumi_gcp as gcp
from infra.bucket_config import BucketConfig
from infra.bucket import build_bucket, grant_reader

project = gcp.organizations.get_project()
sa = gcp.storage.get_project_service_account()

key_id = "projects/p/locations/us/keyRings/r/cryptoKeys/data"
gcp.kms.CryptoKeyIAMMember(
    "gcs-cmek-grant",
    crypto_key_id=key_id,
    role="roles/cloudkms.cryptoKeyEncrypterDecrypter",
    member=sa.email_address.apply(lambda e: f"serviceAccount:{e}"),
)

cfg = BucketConfig(name="acme-data-us", kms_key_id=key_id,
                   reader_member="serviceAccount:[email protected]")
bucket = build_bucket(cfg)
if cfg.reader_member:
    grant_reader(bucket, cfg.reader_member)

pulumi.export("bucketUrl", bucket.url)

Verification

Assert UBLA and public-access prevention are on, then confirm out of band that the bucket rejects a public grant.

# tests/test_bucket.py
# CLI: pytest tests/test_bucket.py
from __future__ import annotations
import pulumi
from typing import Any, Dict, Tuple

class Mocks(pulumi.runtime.Mocks):
    def new_resource(self, args: pulumi.runtime.MockResourceArgs) -> Tuple[str, Dict[str, Any]]:
        return (f"{args.name}-id", {**args.inputs, "url": f"gs://{args.inputs.get('name','')}"})
    def call(self, args: pulumi.runtime.MockCallArgs) -> Dict[str, Any]:
        return {"projectId": "p", "emailAddress": "[email protected]"}

pulumi.runtime.set_mocks(Mocks(), preview=False)

import importlib
main = importlib.import_module("__main__")

@pulumi.runtime.test
def test_ubla_enabled() -> pulumi.Output:
    return main.bucket.uniform_bucket_level_access.apply(
        lambda v: None if v else (_ for _ in ()).throw(AssertionError("UBLA not enabled"))
    )

# CLI: confirm enforcement on the live bucket; the public grant must fail
gcloud storage buckets describe gs://acme-data-us \
  --format='value(iamConfiguration.uniformBucketLevelAccess.enabled,iamConfiguration.publicAccessPrevention)'
gcloud storage buckets add-iam-policy-binding gs://acme-data-us \
  --member=allUsers --role=roles/storage.objectViewer   # expected: denied

Gotchas & Edge Cases

BucketIAMPolicy will silently remove your own access. The authoritative BucketIAMPolicy replaces the entire IAM policy with exactly what you declare. If your declaration omits the bindings GCP or you rely on, those are dropped on the next pulumi up. Prefer BucketIAMMember (or BucketIAMBinding per role) unless you genuinely intend to own the full policy.

CMEK fails until the service agent is granted the key. Bucket creation with default_kms_key_name returns Permission denied on Cloud KMS key if the GCS service agent lacks cryptoKeyEncrypterDecrypter. The service agent is created lazily; if get_project_service_account returns an agent that does not yet exist, the first deploy may need a retry after the agent is provisioned.

force_destroy=False blocks pulumi destroy on a non-empty bucket. This is intentional protection, but it means tearing down a populated bucket fails with bucket is not empty. Empty it first, or set force_destroy=True deliberately for ephemeral test buckets only.

How to Deploy a GKE Cluster with Pulumi (Python) — provisions the compute that reads and writes objects in this bucket via workload identity.
GCP Provider Configuration — the parent guide to credential routing, state backends, and provider setup for GCS resources.
Pulumi Patterns & Provider Management — cross-provider conventions for secrets, state, and IAM that this page follows.

Frequently Asked Questions

What is the difference between uniform bucket-level access and ACLs? ACLs grant access per object or per bucket outside of IAM, which is how buckets accidentally go public. Uniform bucket-level access disables ACLs so IAM is the only access mechanism, making permissions consistent and auditable. Enable it at creation to avoid a later migration.

How do I make sure the bucket can never be made public? Set public_access_prevention="enforced". GCP then rejects any IAM binding to allUsers or allAuthenticatedUsers at the bucket and the organization level can enforce it project-wide as well.

Should I use BucketIAMMember, BucketIAMBinding, or BucketIAMPolicy? Use BucketIAMMember for a single principal/role grant (additive, safest). Use BucketIAMBinding to own all members of one role. Use BucketIAMPolicy only when you intend to manage the entire bucket policy authoritatively, since it overwrites unlisted bindings.

Do I need to grant anything for customer-managed encryption keys? Yes. The GCS service agent for the project needs roles/cloudkms.cryptoKeyEncrypterDecrypter on the crypto key. Grant it with a CryptoKeyIAMMember in the same program so the bucket's CMEK dependency is explicit.

How does versioning interact with deletion? With versioning enabled, deleting an object creates a noncurrent version rather than removing the data. Add a lifecycle rule to expire noncurrent versions after N days if you need cost control, and remember force_destroy=False blocks destroying a bucket that still holds any versions.