GCP Cost — Status

As of 17 August 2026. All figures are net of credits and drawn from the BigQuery billing export. Expressed as relative movements rather than absolute spend.

Headline

August is tracking about 4% above July. We are drifting away from the target rather than toward it.

We are roughly 20% above where we want to land. The daily run rate is extremely stable — under 3% variance across the first fifteen days — so projections are reliable and any change we make will show up clearly.

What changed since July

Comparing the first fifteen days of each month:

Decommissioning a redundant managed-Airflow environment last month is still paying out and is the single biggest saving we have banked. It is being more than cancelled out by Compute Engine, and almost all of that increase is network egress.

Egress: we were paying to serve free storage

Two rows carry identical volume at a 39% price difference. Our first reading was that traffic had been misrouted onto a more expensive path and that a CDN would move it back. That was wrong twice over, and both corrections are worth more than the original theory.

First correction: the audience changed, not the route

Those two prices are not two roads for the same traffic. Which one you pay is decided by the destination network. Traffic to consumer ISPs — networks that peer directly with the cloud provider — bills at the cheaper rate. Traffic to other datacenters and public clouds, which do not peer, bills at the standard internet rate. Nothing was misrouted.

Attribution made this obvious. One host was the largest single egress source on the cluster, moving 1.36 TiB in seven days across only 70,000 requests — about 20 MB per request. Within that, a single client on a public-cloud network accounted for 990 GiB in seven days: roughly 18% of all egress, and about 44% of the expensive bucket, by itself. It sits in a public cloud, which is exactly why its bytes land on the $114 rate rather than the $82 one.

What it does: re-downloads two full-history CSV exports of about 100 MiB each, roughly eight times an hour, continuously — close to 1,400 fetches of each file per week, of objects that only ever gain a row per day.

Second correction: those files were never supposed to cost anything

The obvious next move was compression and a CDN. Before doing either, we checked where the files actually live.

They are in a zero-egress object store. Cloudflare R2 charges nothing to serve data out. The bill should have been zero.

It was not, because of one configuration line. Our CKAN storage extension supports a DOWNLOAD_PROXY setting, which rewrites the storage URL to point at a proxy service we run ourselves. With it set, the download path becomes:

1. Client asks the portal for a file
2. The portal issues a storage URL, then swaps the hostname for our proxy
3. The client hits the proxy, which runs on cloud compute
4. The proxy pulls 100 MiB from object storage — free, and free inbound
5. The proxy streams 100 MiB to the client — billed egress, at the standard internet rate

We bought storage specifically because it has no egress charge, and then paid egress by streaming it through a virtual machine.

We audited all 40 portal namespaces:

Everyone else hands the client a direct storage URL and the bytes never touch our cloud account. This single tenant is the reason that proxy is the largest egress source we have. It amounts to roughly 5.8 TiB/month, about 4–5% of the total bill, and unlike most cost work it needs no cooperation from anyone outside the team.

The proxy is not pointless — it enforces a domain allowlist so only the client's own sites can serve the files. That requirement is real and has to be preserved. But it belongs at the CDN edge, not in a streaming proxy on billed compute.

The fix, in order

1. Attach a custom domain to the storage bucket and serve directly from the edge. Zero egress, caching included, and the domain restriction moves to an edge rule — which is what the proxy is hand-rolling today, done properly and for free. This also makes the eight-fetches-per-hour problem disappear, because repeat pulls are served from cache and never reach an origin.
2. Interim, if the above needs scheduling: have the proxy return a redirect instead of streaming the body. It keeps the allowlist check and the payload bypasses billed egress entirely. A small change.
3. Talk to the operator of the bulk client. Conditional requests, or fetching the daily increment instead of the full history, removes most of the remaining volume.
4. Enable compression on whatever traffic is left. A numeric CSV of that size compresses by roughly 85–90%.

Note that compression and a CDN — the two things we nearly did first — were both second-order. We would have been optimising the delivery of bytes that should never have been billed at all.

One caution learned elsewhere in our estate: putting a proxy in front of datacenter clients can get them bot-blocked, turning a cost fix into an outage for exactly the consumers generating the traffic. Sequence matters.

The reserved-capacity trap

Our BigQuery slot reservation runs at roughly 2% utilisation. On paper that is indefensible waste, and it was the first thing we went after.

It was the wrong target. The comparison that matters is not slots billed versus slots used — it is reservation cost versus what the same queries would cost on demand:

The reservation is saving roughly twenty-four times what it costs. Low utilisation is real, but it is not recoverable money — and any change that quietly moved these workloads onto on-demand pricing would multiply that line by more than twenty.

There is still work to do here, but it is risk work, not savings work. Scan volume on our largest analytics workload has grown roughly tenfold since it was last measured, which makes the reservation something we now depend on structurally rather than something we could unwind.

Where the savings actually are

The target is reachable. It runs through the database platform and how we serve files, not through analytics.

The database upgrade is the biggest single item

Two managed Postgres instances are running a major version that has passed end of standard support, which means we pay a per-vCPU surcharge every month for the privilege. That surcharge alone is about 7% of the bill; right-sizing an oversized replica and removing an unused instance takes it to roughly 8%.

This stalled on an assumption rather than a technical obstacle. The team believed upgrading meant a migration with a client-visible cutover, which nobody wanted to schedule. It does not. An in-place major-version upgrade preserves the instance identity and its connection details, so the application is never reconfigured. What it needs is a maintenance window, not a migration project.

Reframing it that way is what unblocked it. There is preparatory work to do first, but none of it requires client involvement or a change freeze.

Decisions needed

1. Approve the database upgrade window. Biggest single item, and the preparation can start immediately.
2. Confirm the file-download domain restriction is still required, then move file serving to the storage edge. This needs one answer from one stakeholder and removes 4–5% of the bill.
3. Lower the analytics autoscale ceiling. A five-minute configuration change that cuts worst-case exposure by 40% with no impact at observed usage.

---

Postscript: three findings, one shape

Everything we found this month looked the same from a distance. A metric flagged an obvious inefficiency, and the obvious remedy would have been useless or harmful:

• A 39% price gap on identical volume looked like misrouting. It was a change in who was downloading.
• A reservation at 2% utilisation looked like waste. It was saving twenty-four times its cost.
• A large egress bill looked like a compression and CDN problem. The files were in storage that charges nothing to serve, and one configuration line was routing them through billed compute on the way out.

In all three cases the number was accurate and the story we attached to it was not. The cheap step we kept skipping was the same one every time: find out which bytes, whose queries, and what the alternative actually costs — before optimising anything.