Building a Cost Guardrail Check in Python

An estimate that nobody blocks on is a comment on a pull request that everyone learns to scroll past. A guardrail is the opposite: a small, boring program that reads the estimate, compares it against a written policy, and exits nonzero when the change crosses a line the team agreed to in advance. This guide sits in IaC cost and governance under Python IaC fundamentals and strategy, and it covers the part that is organisational as much as technical — picking a threshold that fires on the right changes, rolling it out without stopping every deploy on day one, and giving people a legitimate, recorded way through it.

Context

The guardrail's input is a cost delta produced upstream — the artifact described in estimating infrastructure cost in Python IaC pipelines. Everything here assumes that number already exists in a JSON file and treats it as opaque.

The first design decision is what the threshold is measured against, and the intuitive answer is wrong. An absolute monthly ceiling — "no stack over $10,000" — is unenforceable in a pipeline, because the gate sees a change, not a budget. It also punishes whichever unlucky change happens to cross the line while waving through the nine changes that walked the total up to it. Worse, a ceiling on the total means every routine deployment to a large workload fails from the moment the workload is large.

What works is a threshold on the change, expressed two ways at once. An absolute delta catches the single expensive resource: someone provisions a db.r6g.8xlarge and the gate fires regardless of how big the workload already is. A percentage delta catches the quiet multiplier: a desired count going from 4 to 40 on a small service may only be a few hundred dollars, but it is a 900% jump and almost certainly a typo. Requiring both a percentage and a minimum absolute floor stops the percentage rule from screaming every time a near-empty development stack gains its first load balancer.

Prerequisites

• A cost estimate written to a known path by an earlier pipeline step, containing at minimum a total delta and a baseline
• Python 3.9+ on the runner, with pytest if you take the test-suite route in step three
• Agreement from whoever owns the budget on the two numbers — the gate is a policy artifact, and inventing the numbers yourself is how it gets disabled
• A pipeline that actually honours nonzero exit codes; some hand-rolled shell steps swallow them

# CLI: sanity-check that the upstream step produced what the gate expects
python3 -c "import json,sys; d=json.load(open('estimate.json')); \
print(d['monthly_delta_usd'], d['baseline_monthly_usd'], d['unpriced_count'])"

Implementation

Step 1 — Write the policy as a dataclass, not as if-statements

Scattering magic numbers through a script guarantees that nobody can answer "what does the gate currently enforce?" without reading code. A frozen dataclass loaded from a checked-in file makes the policy reviewable on its own, and makes changing it a pull request with an approver.

# policy.py — the whole enforceable policy in one reviewable object
# CLI: python3 -c "from policy import CostPolicy; print(CostPolicy.from_file('cost-policy.json'))"
from __future__ import annotations

import json
from dataclasses import dataclass
from decimal import Decimal
from pathlib import Path
from typing import Optional


@dataclass(frozen=True)
class CostPolicy:
    """Thresholds are on the CHANGE, never on the running total."""
    max_delta_usd: Decimal          # absolute monthly increase that trips the gate
    max_delta_pct: Decimal          # percentage increase over the current baseline
    pct_floor_usd: Decimal          # ignore the percentage rule below this delta
    max_unpriced: int               # how many unpriced resources make the verdict meaningless
    enforcing: bool                 # False = advisory, print and pass
    owner: str                      # who to talk to about the numbers

    @staticmethod
    def from_file(path: Path | str) -> "CostPolicy":
        raw = json.loads(Path(path).read_text())
        return CostPolicy(
            max_delta_usd=Decimal(str(raw["max_delta_usd"])),
            max_delta_pct=Decimal(str(raw["max_delta_pct"])),
            pct_floor_usd=Decimal(str(raw.get("pct_floor_usd", "25"))),
            max_unpriced=int(raw.get("max_unpriced", 10)),
            enforcing=bool(raw.get("enforcing", False)),
            owner=str(raw["owner"]),
        )

The two thresholds that survive contact with a real team are usually smaller than people expect. Something in the region of a few hundred dollars absolute and twenty to thirty percent relative fires on perhaps one change in forty — often enough that the team remembers the gate exists, rarely enough that it is not treated as background noise.

Step 2 — Reduce the estimate and the policy to one verdict

Keep the decision logic in a pure function with no I/O. It becomes trivially testable, and the same function backs both the CLI and the pytest form without any duplicated branching.

# verdict.py — pure decision logic over an estimate and a policy
# CLI: python3 -m pytest tests/test_verdict.py -q
from __future__ import annotations

from dataclasses import dataclass
from decimal import Decimal
from enum import Enum
from typing import List

from policy import CostPolicy


class Outcome(str, Enum):
    PASS = "pass"
    BREACH = "breach"
    INDETERMINATE = "indeterminate"


@dataclass(frozen=True)
class Estimate:
    monthly_delta_usd: Decimal
    baseline_monthly_usd: Decimal
    unpriced_count: int


@dataclass(frozen=True)
class Verdict:
    outcome: Outcome
    reasons: List[str]
    delta_pct: Optional[Decimal] = None


def evaluate(est: Estimate, policy: CostPolicy) -> Verdict:
    if est.unpriced_count > policy.max_unpriced:
        return Verdict(Outcome.INDETERMINATE,
                       [f"{est.unpriced_count} resources had no price rule "
                        f"(limit {policy.max_unpriced}); the delta is not trustworthy"])

    reasons: List[str] = []
    if est.monthly_delta_usd >= policy.max_delta_usd:
        reasons.append(f"delta ${est.monthly_delta_usd}/mo >= "
                       f"${policy.max_delta_usd}/mo absolute limit")

    pct: Optional[Decimal] = None
    if est.baseline_monthly_usd > 0:
        pct = (est.monthly_delta_usd / est.baseline_monthly_usd * Decimal("100")).quantize(Decimal("0.1"))
        if est.monthly_delta_usd >= policy.pct_floor_usd and pct >= policy.max_delta_pct:
            reasons.append(f"delta {pct}% >= {policy.max_delta_pct}% relative limit "
                           f"on a ${est.baseline_monthly_usd}/mo baseline")

    return Verdict(Outcome.BREACH if reasons else Outcome.PASS, reasons, pct)

INDETERMINATE earns its place. If the estimator could not price a third of the change, a delta of +$0.00 is not a pass — it is the absence of information, and collapsing it into a pass is how a gate quietly stops protecting anything. Reporting it as its own outcome forces a human decision instead of a false green.

Step 3 — Wire it up with an explicit exit-code contract

The gate's contract with the pipeline is its exit code, and it should be written down where the pipeline author will read it. Three states are enough; a fourth for advisory breaches makes the rollout in step four visible in build logs without failing anything.

# cost_gate.py — the standalone entry point
# CLI: python3 cost_gate.py --estimate estimate.json --policy cost-policy.json
from __future__ import annotations

import argparse
import json
import sys
from decimal import Decimal, InvalidOperation
from pathlib import Path

from policy import CostPolicy
from verdict import Estimate, Outcome, evaluate

EXIT_OK, EXIT_BREACH, EXIT_INDETERMINATE, EXIT_USAGE = 0, 1, 2, 3


def load_estimate(path: Path) -> Estimate:
    raw = json.loads(path.read_text())
    return Estimate(
        monthly_delta_usd=Decimal(str(raw["monthly_delta_usd"])),
        baseline_monthly_usd=Decimal(str(raw["baseline_monthly_usd"])),
        unpriced_count=int(raw.get("unpriced_count", 0)),
    )


def main() -> int:
    parser = argparse.ArgumentParser(description="Fail the build on a runaway cost delta.")
    parser.add_argument("--estimate", type=Path, required=True)
    parser.add_argument("--policy", type=Path, required=True)
    args = parser.parse_args()

    try:
        policy = CostPolicy.from_file(args.policy)
        est = load_estimate(args.estimate)
    except (OSError, KeyError, ValueError, InvalidOperation) as exc:
        print(f"cost-gate: cannot evaluate policy: {exc}", file=sys.stderr)
        return EXIT_USAGE

    verdict = evaluate(est, policy)
    print(f"cost-gate: {verdict.outcome.value} "
          f"(delta ${est.monthly_delta_usd}/mo, {verdict.delta_pct}% of baseline)")
    for reason in verdict.reasons:
        print(f"  - {reason}")

    if verdict.outcome is Outcome.INDETERMINATE:
        return EXIT_INDETERMINATE
    if verdict.outcome is Outcome.BREACH:
        if not policy.enforcing:
            print(f"cost-gate: ADVISORY ONLY — would have blocked. Owner: {policy.owner}")
            return EXIT_OK
        print(f"cost-gate: blocking. Contact {policy.owner} or use the recorded override.")
        return EXIT_BREACH
    return EXIT_OK


if __name__ == "__main__":
    raise SystemExit(main())
# State implication: the gate reads two files and touches nothing else — it must
# never call the cloud, or a network blip becomes a failed deploy.

If the team already runs a test suite in the same job, the identical logic drops into pytest with no new machinery, which is often an easier sell than a new pipeline stage:

# tests/test_cost_gate.py — the guardrail as a test
# CLI: pytest tests/test_cost_gate.py -q
from decimal import Decimal
from pathlib import Path

import pytest

from cost_gate import load_estimate
from policy import CostPolicy
from verdict import Outcome, evaluate


@pytest.fixture(scope="module")
def policy() -> CostPolicy:
    return CostPolicy.from_file(Path("cost-policy.json"))


def test_planned_change_is_within_cost_policy(policy: CostPolicy) -> None:
    est = load_estimate(Path("estimate.json"))
    verdict = evaluate(est, policy)
    assert verdict.outcome is not Outcome.INDETERMINATE, verdict.reasons
    if policy.enforcing:
        assert verdict.outcome is Outcome.PASS, "; ".join(verdict.reasons)

Step 4 — Roll it out advisory first, then flip it

A gate switched on at full strength on a Monday blocks four unrelated deployments by Wednesday and is disabled by Friday. The enforcing flag exists so the same code can run in both modes with no branch in the pipeline.

Run advisory for two to four weeks. During that window the gate prints what it would have blocked and exits zero. Collect those lines. Almost always the first pass shows the thresholds are wrong in one direction or the other: either it would have blocked nothing at all, or it would have blocked the weekly autoscaling adjustment that everybody considers routine. Tune the numbers against real changes, not against a guess made in a planning meeting, then flip enforcing to true in a pull request that names the two thresholds in its title so the change is legible in the repository history.

Step 5 — Give large changes a legitimate way through

Every guardrail needs a door, because sometimes the $4,000/month increase is exactly what the migration requires. The question is only whether the door is recorded. An override that is an environment variable someone sets in a pipeline definition is invisible six months later; an override that is a file in the change itself is part of the diff and part of the review.

# override.py — a time-boxed, reviewable exception carried in the change itself
# CLI: python3 cost_gate.py --estimate estimate.json --policy cost-policy.json --override cost-override.json
from __future__ import annotations

import datetime as dt
import json
from dataclasses import dataclass
from decimal import Decimal
from pathlib import Path
from typing import Optional


@dataclass(frozen=True)
class Override:
    reason: str
    approver: str
    ticket: str
    expires: dt.date
    max_delta_usd: Decimal      # the override has its own ceiling; it is not a blank cheque

    @staticmethod
    def from_file(path: Path) -> "Override":
        raw = json.loads(path.read_text())
        return Override(
            reason=raw["reason"], approver=raw["approver"], ticket=raw["ticket"],
            expires=dt.date.fromisoformat(raw["expires"]),
            max_delta_usd=Decimal(str(raw["max_delta_usd"])),
        )

    def applies(self, delta_usd: Decimal, today: Optional[dt.date] = None) -> bool:
        today = today or dt.date.today()
        return today <= self.expires and delta_usd <= self.max_delta_usd

Four properties make an override auditable rather than a loophole: it names a human approver, it references a ticket, it expires on a date, and it carries its own ceiling so it cannot absorb an unrelated change that lands in the same branch. Print all four to the build log when one is used — that log line is the audit record, and it is what makes the difference between a governed exception and a bypass.

Verification

Test the gate the way you would test any decision function: with fixtures at the boundaries, not with a live plan.

# CLI: exercise every exit code deliberately
python3 cost_gate.py --estimate fixtures/small.json --policy cost-policy.json; echo "exit=$?"   # 0
python3 cost_gate.py --estimate fixtures/runaway.json --policy cost-policy.json; echo "exit=$?" # 1
python3 cost_gate.py --estimate fixtures/unpriced.json --policy cost-policy.json; echo "exit=$?"# 2
python3 cost_gate.py --estimate fixtures/missing.json --policy cost-policy.json; echo "exit=$?" # 3

Then verify the pipeline itself actually stops. Craft a branch with a deliberately oversized resource, push it, and confirm the apply step never runs. A guardrail that returns 1 into a shell step ending in || true — or into a task whose continueOnError is set — has been tested only in your terminal. The most common way a cost gate fails in production is that it works perfectly and nothing downstream is listening.

Finally, assert on the exact boundary. A delta equal to max_delta_usd should breach if the comparison is >=, and the fixture should encode that choice so a later refactor to > fails a test rather than silently widening the policy by one dollar.

Gotchas & Edge Cases

A threshold that never fires is worse than no gate. If six months of build logs contain no breach, the team has learned that the step is decorative. Instrument it: count breaches per month, and if the number is zero, the thresholds are wrong, not the changes.

A threshold that is too tight gets routed around. People do not fight a gate; they split one change into five, or add the override file by reflex, or copy the pipeline into a job that skips the stage. All three are visible in the repository if you look, and all three mean the numbers need raising.

Baselines of zero break the percentage rule. A brand-new stack has a baseline of 0, so the percentage is undefined. The code above guards it explicitly; a version that does not will raise decimal.DivisionByZero on exactly the changes people are most excited about shipping.

Deletes are deltas too. A change that removes $3,000/month of capacity produces a large negative number. Comparing an absolute value would block it, which is absurd. The comparisons above are directional on purpose — only increases breach.

Currency and units must match. If the estimator emits monthly figures and the policy is written in what someone assumed were hourly rates, the gate is off by a factor of 730 and will either never fire or always fire. Name the unit in the field itself, as max_delta_usd does.

Operational Notes

Keep the policy file next to the infrastructure code it governs, not in a central governance repository. Reviewers who understand the workload are the ones who should approve a threshold change, and the same review discipline described in enforcing IAM least privilege in Python IaC applies to a number that controls spend.

Different stacks deserve different numbers. A development environment might enforce a $50 delta while production tolerates $1,000, because the failure mode in development is a forgotten g5.12xlarge left running over a weekend, and in production it is a legitimate capacity increase. One policy file per stack, loaded by name, keeps that explicit rather than folding it into environment-variable overrides nobody can see.

Publish the verdict as a machine-readable artifact as well as human text. A one-line JSON summary — outcome, delta, percentage, thresholds in force, override used — collected across builds gives you the breach-rate metric that tells you whether the thresholds are still calibrated. Without it, the only signal is complaints.

Treat the gate's own dependencies as production code. It should import nothing beyond the standard library and, at most, your estimator's data classes. A guardrail that fails to run because of an unrelated package upgrade fails open in most pipeline configurations, which is the one behaviour you cannot afford. If you must fail, fail closed with exit 2 and a clear message, and make that a conscious, documented choice rather than an accident of how the step was written.

FAQ

Should the gate block the apply, or just the merge?

Blocking the merge is friendlier and catches the change earlier, but it only works if the estimate can be produced from the branch. Blocking the apply is the stronger guarantee, because that is the step that actually spends money. Teams with a long-lived deployment pipeline usually end up doing both, with the merge check advisory and the apply check enforcing.

What number should we start with?

Pull the last three months of merged changes, run the estimator over them offline, and pick thresholds that would have flagged the two or three you actually regret. That is a ten-minute exercise and it produces defensible numbers, unlike round figures chosen in a meeting.

How do we stop people abusing the override file?

You mostly do not, and you should not try to make it impossible — you make it visible. Require the approver field to be someone other than the change's author, expire it, and review override usage in whatever forum already looks at spend. Abuse that is easy to see tends to stop.

Does this work for CDKTF as well as Pulumi?

Yes. The gate never sees a plan document — it consumes the estimate's JSON output, so the upstream toolchain is irrelevant. Only the estimator has to know the difference.

What if the estimate step itself fails?

Fail the build. An estimator that crashed and a change that costs nothing produce the same missing file, and the gate cannot tell them apart. Exit code 3 exists for precisely this case, and the pipeline should treat it as a defect to fix, not a condition to retry past.

Related

• Estimating Infrastructure Cost in Python IaC Pipelines — produces the delta this gate consumes.
• IaC Cost and Governance — the parent topic covering spend controls and policy ownership.
• Enforcing Tagging Policies with Pulumi CrossGuard — the in-engine equivalent for rules that must run before the plan is even complete.