Every impressive AI demo you have seen sits on an enormous, expensive, and very unglamorous foundation of data. On this week's Unscripted, I sat down with Keith Pijanowski, who spent over a decade at Microsoft, built data pipelines at BNY Mellon, and now specializes in the storage infrastructure AI actually runs on. His short answer to the question everyone is asking: no, the data center build-out is not a bubble, efficiency gains will make demand go up rather than down, and the thing we should actually worry about is not the machines. It is people who stop understanding what the machines produce.

Training and inference are two completely different workloads

When people say "AI," they are really talking about two workloads. Training a frontier model means clusters of hundreds of GPUs hammering storage over a high-speed network for days, sometimes months. That is a workload the software industry has simply never seen before.

Inference was supposed to be the easy part. It is not anymore. Modern language models generate key-value pairs as they make predictions, and caching those KV pairs in the right place, whether GPU memory, system memory, local storage, or network storage, dramatically changes performance. As Keith put it, you are not just putting a simple microservice into production. Storage decisions now have to be made alongside compute decisions, which is why his standing advice to companies is to figure out where you will train and run inference first, then buy storage from that provider, and start small.

Power is becoming the new performance rating

Keith told me about a data center architect who looked at the power requirements for Nvidia's latest superpod and was visibly shocked. His prediction: as AI gets more power hungry, GPU power ratings will become just as important as performance ratings.

That has consequences beyond the rack. Our grid was not designed for this, and parts of the country are already power starved. Keith's position is that it has to be a shared responsibility. Hyperscalers and neoclouds cannot just build wherever they want, plug in, and expect everything to work. If there is not enough power, the companies building the data centers should help fund the upgrades so the cost does not land on the local community. And if you are wondering about the exotic fixes: he considers data centers in the ocean genuinely interesting because of free cooling, and data centers in space science fiction.

The Jevons paradox: why efficiency makes demand bigger

Here is the counterintuitive part. If models get more efficient and GPUs get more efficient, do we eventually end up with more data centers than we need? Keith's answer is that history has never seen that happen, and there is a name for why: the Jevons paradox, first observed in the coal era.

When a resource gets more efficient, three things happen. The companies already using it use more of it, because they have a backlog of things they want to do. The players who were priced out come in. And then somebody invents an entirely new use for it. Fuel-efficient cars are the perfect non-tech example: when cars got more efficient, gas consumption went up, because people drove more and more people could afford to drive. Expect exactly the same curve for AI infrastructure.

Is the build-out rational or a bubble?

Keith spent years at Microsoft on the team evangelizing Azure before it was publicly released, so he has seen how these decisions actually get made. His view: the people who build data centers do not do it on a whim. That kind of money is researched, and there has to be legitimate demand behind it. Combine what enterprises are pushing their employees to do with AI, what consumers are doing, and the usage data the hyperscalers see, and the build-out looks rational.

The place he is watching for trouble is different. GPUs have gotten 13 to 25 times faster in about five years, with memory bandwidth up 11 times. Network and storage have to keep up, and if a GPU can only be kept 60 percent utilized because the rest of the stack lags, you are wasting your money. You would have been better off buying the previous generation.

The human in the middle

The most passionate part of our conversation had nothing to do with hardware. Keith said generative AI undermined a belief he held his whole career: that every software innovation was unconditionally good for mankind. He has watched marketers publish content they do not understand and engineering teams grind to a halt because they vibe coded the start of a sprint, hit production, and had nobody left who understood the logic well enough to fix the bugs.

His rule is simple. There should always be a human in the middle who fully understands what gets published or put into production. AI is a productivity tool, and he uses it daily. But as he put it, you have to make sure it does not turn you into an operator. That is the real risk in the AI era, and it applies to marketing teams every bit as much as engineering teams.

FAQ

Why does AI need so much storage infrastructure? Training runs on clusters of hundreds of GPUs pulling data from storage continuously for weeks or months, and modern inference caches key-value pairs across GPU memory, system memory, and network storage. Both workloads demand storage performance the software industry has never needed before.

Will more efficient AI models reduce data center demand? No. The Jevons paradox says that when a resource gets more efficient, total consumption rises: existing users do more, new users enter, and new use cases emerge. Fuel-efficient cars increased total gas consumption for the same reason.

Is the AI data center build-out a bubble? Keith Pijanowski argues it is rational. Hyperscalers research demand exhaustively before committing billions, and enterprise and consumer usage keeps climbing. The bigger financial risk is buying GPUs your network and storage cannot keep fed.

Are data centers in space or the ocean realistic? Ocean data centers are plausible because the surrounding water provides efficient cooling. Space data centers are, in his words, science fiction, because the costs are far too high.

What is the biggest risk of vibe coding? Code nobody on the team understands. When AI-generated code hits production and breaks, teams stall because no human can fix logic they never reviewed. The fix is treating every AI output to a real code review with a human in the middle.