AI Makes Marginal Cost Real Again
The internet era was built on the assumption of zero marginal cost: once the infrastructure is up, serving one more user costs almost nothing. AI shatters this assumption, reintroducing real, significant costs for every additional unit of usage.
"The era of aggregation theory is behind us, and AI is again making technology expensive. This relation of increased cost from increased consumption is anti-internet era thinking." Doug O'Laughlin, "2025 AI & Semiconductor Outlook"
For two decades, technology companies thrived on a simple model: high fixed costs to build the platform, near-zero marginal costs to serve users, and infinite returns to scale once users arrived. Google serving one more search, Facebook showing one more feed item, Netflix streaming one more video each incremental user cost essentially nothing. This made possible the aggregation theory business model where the platform that attracted the most users captured the most value.
AI breaks this model at its foundation. Every LLM inference query consumes GPU compute, memory bandwidth, and electricity in proportion to the number of tokens processed. Reasoning models like o1 deliberately use more inference-time compute to achieve better results, multiplying the per-query cost. Hidden costs compound: system prompts consume hundreds of tokens before the user says a word, agent frameworks make background API calls, and RAG pipelines run multiple queries per user request. Companies that carried tech debt could previously absorb it as an inconvenience; now AI "dramatically widens the gap in velocity" between clean and messy codebases, making tech debt directly more expensive.
This is the most significant structural change in the technology landscape since the rise of cloud computing. Hyperscalers must now contend with extremely high fixed costs and high marginal costs simultaneously. The businesses, pricing models, and architectural decisions that worked in the zero-marginal-cost era will not survive the transition unchanged.
Takeaway: AI ends the free lunch of zero marginal cost and the companies that thrive will be those that treat inference economics as a first-class engineering and business constraint.
See also: Inference Cost Dominates Training at Scale | Cloud Economics Are Not What They Seem | Skin In The Game Aligns Incentives