I spent part of this week chasing race conditions in a webhook flow, and the fix turned out to be sending less, not more. That same idea kept showing up everywhere I looked in the data world this week: do the work once, reuse it, and stop paying to recompute things that already exist.
Rebuilding what didn't change is a tax you volunteered for
I've watched pipelines run on a schedule for years, and the dirty secret is how much of that work is pure waste. The clock hits the hour, every model rebuilds, and most of the source data hasn't moved an inch. You're paying warehouse compute to arrive at the exact same answer you had last hour.
dbt shipped State to GA this week, and the idea is refreshingly simple. Instead of rebuilding on a schedule, it checks warehouse metadata to figure out what actually changed, then rebuilds only that. Early adopters are reporting 15 to 30 percent compute savings, and more at scale. The logic is a three-step reuse: if the schema's the same, reuse it; if it exists in another schema, clone it; only rebuild when there's nothing to pull from.
What I like is that it moves the decision off the calendar and onto the data. I ran into a version of this problem years back, except it was disk. We had a volume set to auto-scale up at 90 percent, and it did its job the one time we hammered it. Then it sat at that size forever. You pay for headroom you touched once, permanently, unless you go in and force it back down. Scheduled rebuilds are the same shape. You provisioned the work for the worst-hour case and then kept running it every hour whether the work existed or not.
The pattern shows up on the AI side too
There was a piece this week about an agent burning 240k tokens on a single query because it was pulling raw transcript data straight from a transactional API. The fix wasn't a bigger context window. It was modeling the data first, compressing it in the warehouse, and handing the agent something pre-shaped. Twenty-x compression, same answer.
Same lesson wearing a different hat. Whether it's compute cycles or tokens, you don't want the system redoing work it could have cached, cloned, or compressed. My webhook fix this week was the smallest version of it: send a record ID and an event type, let the downstream step re-fetch current state when it needs it. Two race conditions I'd been chasing for a week just disappeared.
If you run dbt, turning State on is close to free and you'll see it on the bill. If you're feeding agents, look at whether you're shoving raw API payloads into context when a modeled table would do. The common move is to stop asking the machine to redo what already exists.
3 things worth your time
1. Context engineering lives in your warehouse
The skills analytics engineers already have, modeling and governance, are the same ones that feed agents good context. Vectors and unstructured data are warehouse-native now, so you don't have to leave the tools you know.
2. Dedicated read replicas for Postgres
Per-replica sizing lets you put pgvector, analytics, and OLTP on separate nodes so one long-running query doesn't take down production. A clean answer to workload isolation without standing up separate infra.
3. MongoDB to DuckDB with Java table functions
A real before/after: Excel exports dropping from two-plus minutes to seconds. The bottleneck analysis on those giant VARCHAR fields is worth reading even if you never touch DuckDB.
Curious what's still rebuilding on a schedule in your stack that could be running on change instead. Hit reply and tell me what you find.