Cost Explorer, Budgets and cost allocation tags

Lorraine runs Cascade Ridge's finances by clinic. Each of the 11 clinics and the emergency hospital has its own budget, and she wants to know what each one costs to run. "When the cloud bill comes," she asks on November 10, "can you tell me how much of it is the Fort Collins clinic?" Gwen answers with three tools that work together: cost allocation tags to label resources, AWS Cost Explorer to analyze spending, and AWS Budgets to set limits and raise alarms before money is spent.

Cost Explorer

AWS Cost Explorer is the main tool for looking at costs over time. It shows charts and tables of spending by day or month, and lets you filter and group by service, account, Region, usage type, purchase option, tag and other dimensions. It includes about a year of past data by default and can forecast spending for the coming months from your history. Cost Explorer is free to use in the console; requests through its API carry a small charge.

Mateo opens Cost Explorer in the sandbox and groups October by service. The chart confirms what the bill said: EC2 is the tallest bar. He changes the grouping to usage type and the EC2 bar splits into instance hours, EBS volume storage and NAT gateway charges. Then he switches to daily granularity and sees the step on October 7 when the test server started, and a smaller one on October 12 when the NAT gateway appeared.

Cost allocation tags

A tag is a label attached to an AWS resource, made of a key and a value, such as clinic = fort-collins or environment = production. Tags help people find, secure and automate resources, and they make it possible to divide a bill. A cost allocation tag is a tag key that has been activated in the Billing and Cost Management console, so that it appears as a dimension in Cost Explorer, Budgets and the Cost and Usage Report.

There are two kinds of cost allocation tags. User-defined tags are the ones you create, such as clinic or application. AWS-generated tags are created by AWS, for example a tag that records which identity created a resource. Both must be activated before they appear in billing tools, and activation is done in the management account of an organization.

Two details catch many teams. First, tags are not retroactive for billing: after a tag key is activated, it appears in cost data from that point on, so costs from before activation, or from resources tagged later, show as untagged. Second, a resource must be tagged to be counted: an untagged server is invisible to a filter for clinic = fort-collins. Some shared costs, such as the NAT gateway or support charges, cannot sensibly be tagged to one clinic at all.

Cascade Ridge's tagging plan

Gwen and Lorraine agree on a short list of required tags. Short lists are followed; long lists are not.

Tagging by hand is unreliable, so the plan uses automation. Infrastructure is created from templates that apply the tags; an AWS Organizations tag policy defines the allowed keys and values; and a weekly report lists untagged resources. Shared costs are divided by a rule Lorraine chooses, for example by each clinic's number of appointments, using AWS Cost Categories, which group costs by rules on accounts, tags and services.

# Illustrative: allocate a month's costs to clinics from tagged and shared lines.
tagged = {"er-hospital": 2900, "clinic-a": 310, "clinic-b": 295, "clinic-c": 280}
shared = 3100                       # NAT, monitoring, backup and so on
appointments = {"er-hospital": 5200, "clinic-a": 1400, "clinic-b": 1300, "clinic-c": 1250}
total_appts = sum(appointments.values())
for clinic, direct in tagged.items():
    share = shared * appointments[clinic] / total_appts
    print(f"{clinic:11} direct {direct:6,d}  shared {share:8.2f}  total {direct + share:9.2f}")

er-hospital direct  2,900  shared  1761.75  total   4661.75
clinic-a    direct    310  shared   474.32  total    784.32
clinic-b    direct    295  shared   440.44  total    735.44
clinic-c    direct    280  shared   423.50  total    703.50

Only four sites appear in this illustrative output, but the method scales: tag what belongs to one place, then split what is truly shared by a fair, agreed rule. Lorraine will be able to answer her own question about Fort Collins in the first month of production.

AWS Budgets

Cost Explorer looks backward and forecasts; AWS Budgets watches spending and acts. A budget sets an amount for a period (monthly, quarterly or yearly) and sends alerts when actual or forecast spending crosses thresholds you choose. Alerts go by email or to an Amazon SNS topic, which can forward them to chat tools or trigger automation.

Budgets can also take budget actions when a threshold is crossed: apply an IAM policy or a service control policy that blocks new resources, or stop specific EC2 or RDS instances. Actions can run automatically or wait for someone to approve them. For the sandbox, Gwen sets a monthly cost budget of 150 US dollars with three alerts: when the forecast passes 150, when actual spend passes 100, and when actual spend passes 150. At 150 actual, a budget action applies a policy that stops anyone from launching new instances in the sandbox until the team reviews it.

Budgets do not cap spending by themselves. A budget alert is a message, not a limit; only a budget action, or a person reading the alert, stops anything. Gwen does not put automatic stop actions on production. "I would rather pay an extra hundred dollars than switch off the emergency hospital's records at 3 a.m.," she tells Lorraine. Production gets alerts to people, not automatic stops.

Cost Anomaly Detection

AWS Cost Anomaly Detection uses machine learning to learn each service's normal spending pattern and flags unusual jumps, such as a sudden rise in data transfer or a new service appearing. You create monitors (for example per account, per service or per cost category) and subscriptions that send alerts, and each anomaly comes with a likely root cause such as the account, service and usage type involved. It complements budgets: a budget catches spending that passes a fixed amount, anomaly detection catches spending that is unusual even if it is still under budget.

How the three fit together

• Tags label resources so costs can be divided.
• Cost Explorer shows where the money went and what it is likely to be, by any dimension including tags.
• Budgets set expectations and alert, or act, when spending crosses them.
• Cost Anomaly Detection spots unusual changes automatically.
• The Cost and Usage Report keeps the most detailed record for deep analysis.

Saved reports and sharing views

Cost Explorer becomes much more useful when the questions people ask often are saved as reports. Mateo creates four for production: monthly cost by service, monthly cost by the clinic tag, daily cost for the last 30 days by usage type (to spot sudden steps), and Savings Plans coverage. Each report keeps its filters, grouping and time range, so anyone with access sees the same view. Kwame gets a report filtered to application = bookvet, which answers his question about BookVet's cost without giving him access to the whole bill.

One budget per purpose

Gwen's rule for budgets is that each one should have an owner and a purpose. A budget nobody reads is noise. The plan for the organization has a small set:

Keeping tags clean

Tags decay. People create resources by hand in a hurry, scripts forget a key, values drift from "production" to "prod" to "Production". Three habits keep them clean: a tag policy that defines allowed keys and values; templates that apply tags automatically; and a weekly report of resources missing required tags, sent to their owners. A short program run against an inventory export is enough to start.

# Illustrative: find resources missing required tags in an inventory list.
required = {"clinic", "environment", "application", "owner", "costcode"}
inventory = [
    {"id": "i-01", "tags": {"clinic": "shared", "environment": "production", "application": "vetledger", "owner": "mateo.ruiz", "costcode": "IT-100"}},
    {"id": "vol-07", "tags": {"environment": "production"}},
    {"id": "i-09", "tags": {"clinic": "shared", "environment": "development", "application": "bookvet", "owner": "kwame.asante"}},
]
for r in inventory:
    missing = sorted(required - set(r["tags"]))
    if missing:
        print(r["id"], "is missing:", ", ".join(missing))

vol-07 is missing: application, clinic, costcode, owner
i-09 is missing: costcode

The volume with only one tag is exactly the kind of resource that ends up as an unexplained line on a bill. Finding it in week one is far cheaper than finding it in month six.

Budgets and forecasts together

A forecast in Cost Explorer and a forecast alert in Budgets use the same idea: extend this month's trend to the end of the month. Early in a month the forecast is rough, because a few days of data can mislead, so Gwen sets forecast alerts slightly above the budget amount and actual alerts at the amount itself.