Technology companies are accumulating AI infrastructure far faster than they are putting it to work. A Cast AI report covering tens of thousands of Kubernetes clusters across AWS, Azure, and GCP found average GPU utilization of only 5%.
That leaves organizations paying for roughly 20 times the GPU capacity their workloads use at any given moment. Although many teams employ sophisticated AI tools to manage applications, they are not applying those tools to infrastructure optimization.
Resource efficiency continues to decline
The findings draw on direct measurements of production clusters and millions of compute resources before optimization. According to Cast AI co-founder and President Laurent Gil, results worsened in the report's third year: CPU utilization declined from 10% to 8%, while memory utilization fell from 23% to 20%.
The report also tracks overprovisioning—the difference between the resources workloads need and those allocated to them. CPU overprovisioning increased from 40% to 69% year over year, and memory overprovisioning reached 79%.
Organizations consequently reserve nearly twice the CPU resources and four times the memory their workloads consume. They are paying for infrastructure their workloads do not even request, with the imbalance growing rather than shrinking.
Idle GPUs carry a higher cost
An unused CPU core costs cents per hour; an unused GPU costs dollars. Meanwhile, GPU prices are rising for the first time since EC2 launched in 2006.
In January 2026, AWS increased H200 Capacity Block prices by 15%, citing supply and demand and breaking a two-decade precedent.
The report argues that 5% utilization makes the economics untenable. Long lead times help explain why companies hoard capacity, but stockpiling also reinforces the scarcity that pushes prices higher.
Automation separates the more efficient clusters
The average does not describe every deployment. One organization achieved 49% utilization on H200s and 30% on H100s, substantially exceeding the overall 5% figure.
The report attributes that difference to automation, rather than luck or superior hardware. Available approaches include automated rightsizing, GPU sharing or time slicing, and Spot management.
Most teams still favor overprovisioning because it feels safer than risking a capacity shortage. That reassurance is expensive. Teams that reduced the gap made resource efficiency a continuous, automated process instead of a manual, one-off exercise. Cast AI's data suggests most companies continue to absorb the cost rather than change their practices.
