Data Center Powerplay: The Chips Have to Go Somewhere

The following is a freely accessible summary of a published Citi Research report.

We explore the implications of the growing power demand from Generative Artificial Intelligence (GenAI) workloads in a recent report led by our communications infrastructure analyst Michael Rollins and drawing on the work of a global team spanning multiple sectors. Our report offers a forecast for the global data-center market, measured by both information-technology (IT) load and potential utility energy consumption in gigawatts (GW).

Demand for data-center infrastructure has exploded, with digital-transformation initiatives continuing to drive demand for core IT workloads. That demand is expected to accelerate due to AI, whose workloads are seen driving data-center demand for at least the first few years of this adoption cycle. We don’t expect existing general computing workloads to be cannibalized by the accelerated deployment of IT infrastructure, which is more likely to be focused on GenAI services’ expanding array of learning and inference models.

Data-center supply and demand is difficult to estimate, as capacity is fragmented across continents, countries, companies and use cases. To craft our estimates and outlook, we pulled together different data sources, tracking top-down demand for data-center power requirements and supply through the end of 2023.

That process allowed us to build a global model to track data-center power demand and supply; the model identifies power demand of about 33 GW of IT load. Our analysis has limitations: It employs a simple estimate for power managed directly by North America-based hyperscalers — operators of large data centers that can provide tremendous computing power and storage to enterprises — and is missing some key markets in Asia, such as Japan, Korea and China. Our plan is to further expand and refine our analysis as we build upon our initial research.

Our top-down forecasts consider quarterly absorption (net bookings demand for power) by region to establish forecasts for our model. We’ve also leveraged the outlook for AI chips crafted by our technology sector analysts to consider the potential power demand from AI workloads. By combining these two approaches, we can infer the power left for core and cloud workloads, which we see growing more slowly in the future.

GenAI chips

In creating a framework for considering GenAI workloads’ rising power demand, we started with NVIDIA’s anticipated market share, leveraging our NVIDIA forecasts by chip and estimated peak power requirements. We decelerate these estimated requirements over time, as we believe inference share is likely to rise and NVIDIA may incorporate further performance efficiencies.

For example, for 2023 we estimate that the IT Load requirement for NVIDIA chips that could be used for AI workloads was 1.3 GW. We forecast aggregate chip sales for potential AI workloads growing at a compound annual growth rate (CAGR) of 18% through 2030, but forecast the IT Load requirements from the chips will grow at a CAGR of 32% through 2030, a product of those chips’ rising power densities.

Next, we incorporate a declining share of NVIDIA chips within the GenAI market, given the likelihood that hyperscalers will use alternative, less-dense chips for inference in future years, with competition for chips expanding over time.

We estimate total GenAI chips will expand from about 3.1 million in 2023 at a CAGR of 22% through 2030. We then incorporate peak power expectations for these chips to model the total power peak from AI chips, which we see expanding from 1.4 GW in 2023 to 10.4 GW by 2030.

The final step in the process? It’s to convert peak power from AI chips into demand for data-center power from these workloads. After incorporating adjustments for factors such as storage and chip replacements, we estimate power demand for AI chips (excluding China) will rise from 1.7 GW in 2023 to 8.8 GW by 2030.

Data-center forecasts

In parallel, we established data-center power forecasts for the global markets we were able to track, kicking off a process we expect to be the most iterative part of our analysis as we incorporate additional markets and fine-tune our outlook.

The market for data-center capacity is segmented into two primary categories: leased data-center capacity from third parties (including public and private data-center firms), which is known as colocation, and capacity built by hyperscalers. Estimates are that 50 to 100 firms globally operate with such large amounts of data-center capacity that they are considered hyperscale and could build at least some of their own capacity.

For the markets we track, we estimate the current market size for colocation, measured by IT Load, at approximately 19 GW, with another ~14 GW from North American hyperscalers, yielding a total of 33 GW. Demand has accelerated substantially in North America, with more than 3.5 GW of supply added and absorbed during 2023.

We haven’t found a single barometer of demand that can explain the pace at which data-center demand has expanded, either in the U.S. or globally. In the absence of such a metric, we’ve looked at overall internet traffic growth and the growth of cloud services for potential indicators.

Our forecast for data-center demand anticipates a deceleration for core cloud and other workloads in future years, especially given the possibility that GenAI workloads begin to cannibalize some of the core workloads over time.

Starting with our year-end 2023 estimate that the size of the data-center market is 33 GW in IT Load, we see GenAI workloads growing from a year-end 2023 baseline of 4.3 GW. Our forecasts see AI workloads expanding at a CAGR of 43% through 2030, while core, cloud and other workloads are seen growing at a CAGR of 8% through 2030. That yields an estimate that total data-center power demand will grow at a seven-year CAGR at 17% to 100 GW by 2030. The chart below tracks our forecasts by region.

Tracked Global Data Center Forecasts by Region for 2030 Colocation

We forecast globally tracked data center colocation demand (ex-hyperscale) to be 65.6 GW by 2030 (19% CAGR)

 

© 2024 Citigroup Inc. No redistribution without Citigroup’s written permission.

*Note: 1 GW = 1,000 MW

Source: CBRE, Data Center Knowledge, datacenterHawk (DCH), FactSet, Mercury Compute, Company Reports, Citi Research

 

Utility demand

We next looked to forecast North America utility demand, with our data-center forecasts serving as a starting point. We estimate that the U.S. market will need to supply data centers with annual peak power of 5 GW to 8 GW per year through 2030 and average utilized power of 3 GW to 5 GW. That adds roughly 6% to the average annual utility market in the U.S.

Those estimates incorporate our expectations that electric utilities will provision for average utilization rate, which we understand to be 50% to 70%. We also expect book-to-bill and bill-to-fill lags to drive down average power utilization for the utilities in 2024 and 2025, followed by a recovery to an average of 65%. While power supply is likely to be enhanced by slower shutdowns of legacy coal, gas and nuclear plants, we warn that establishing reliable power delivery for the growing number of new data centers will take time.

Key insights

In looking at our key takeaways, we remain bullish on the demand drivers for data-center capacity given what needs to be deployed for the GenAI workloads anticipated; these drivers seem to be backed by improving pricing performance.

The pushback on robust demand for data centers is likely to come in two forms.

First, is there a risk of a glut if the capacity built for AI workloads isn’t absorbed by clouds, corporates and governments? We remain encouraged by companies’ comments about AI workloads to identify new revenue opportunities as well as promoting greater operating efficiencies. The use of higher-density graphics processing units (GPUs) and new software can substantially expand the inputs and data used in a model to create better outcomes, significantly reduce the processing time needed to create insights and answers, and expand potential use cases.

Second, can utilities deliver the power needed on a timely basis? Power delivery remains complicated and subject to significantly longer lead time to deliver the requested loads. Supply constraints could end up becoming a larger-for-longer bottleneck to delivering AI capacity over the next several years, and this may be contributing to the reasons data-center absorption has been accelerating across key markets at very favorable pricing compared to the past few years. See our Must C report Overcoming Gridlock: Powering Our Future and accompanying podcast for more on this critical topic.

We expect GenAI workloads will likely be more than half of the demand for data-center power by 2030, representing a CAGR of 43% from 2023. Core and cloud workloads, meanwhile, are likely to slow to a CAGR of 8% through 2030. While we anticipate GenAI workloads will be additive over the next few years, core and cloud workloads could eventually be cannibalized by GenAI workloads and migration of workloads from central processing units (CPUs) to GPUs. In addition, core cloud growth should eventually slow as digitalization and cloudification rates mature.

Absorption, which represents the net growth in data-center IT Load demand, is expected to be 9 GW to 11 GW per year through 2030 in the global markets we track, with roughly 70% coming from the colocation market and about 30% coming from hyperscalers. We see supply remaining generally tight for the next few years, given the expectation for favorable demand.

 

Globally Tracked Data Center Demand - Annual

© 2024 Citigroup Inc. No redistribution without Citigroup’s written permission.

Source: datacenterHawk (DCH), FactSet, Cogent Communications, Citi Research

 

Absorption for Globally Tracked Colo/Hyperscale

© 2024 Citigroup Inc. No redistribution without Citigroup’s written permission.

Source: datacenterHawk (DCH), FactSet, Company Reports, Citi Research