Energy use of AI: how electricity demand for AI is estimated
Estimating AI electricity demand requires measuring the energy used for model training and inference. Methods vary in scope and rely on hardware specifications, compute time, and data center efficiency. The IEA, academic researchers, and national laboratories provide approaches and highlight the need for transparency.
The challenge of measuring AI energy use
There is no AI without electricity, specifically for data centres, as the IEA's Energy and AI report states. Policy makers and stakeholders often lack the tools to analyze AI's energy impact because of a lack of comprehensive data.
The IEA data centres page tracks the electricity consumption of data centres and data transmission networks, providing a baseline for understanding the broader computing infrastructure that AI relies on.
Academic studies estimate energy use by measuring training runs. For example, the arXiv NLP energy paper quantified the financial and environmental costs of training recent neural network models.
Training and inference estimates
Training a large neural network consumes substantial electricity over many hours or days. The arXiv carbon emissions study calculates the energy use and carbon footprint of models such as T5 and GPT-3, and refines earlier estimates for neural architecture search.
Inference, the use of a trained model for predictions, adds energy use per query. While each query may be small, the total from millions of users can be large. The same study emphasizes that the choice of model architecture and hardware can reduce energy use and carbon footprint by a factor of one hundred to one thousand.
Factors that change the estimate
Several factors affect the accuracy of an energy estimate. The paper on carbon emissions notes that geographic location matters because the fraction of carbon-free energy varies five to ten times within the same country, and cloud data centers can be about 1.4 to 2 times more energy efficient than typical data centers.
The type of processor also matters. Machine learning accelerators can be two to five times more effective than off-the-shelf systems. The LBNL data center expertise center works with industry and utilities to accelerate AI infrastructure and develop efficient energy systems.
Transparency and public data sources
The paper on carbon emissions recommends that machine learning papers requiring large computational resources report energy consumption and CO2e. The authors collaborate with MLPerf developers to include energy usage in industry standard benchmarks.
Public data sources help researchers and policy makers. The Stanford HAI AI Index provides rigorously vetted data on AI, including its economic and societal impact, while the IEA's report includes projections for AI electricity consumption over the next decade.
| Component | Inputs | Influence |
|---|---|---|
| Training compute | GPU hours, model size, number of steps | Determines total energy for training |
| Hardware power draw | Watts per processor, utilization | Sets instantaneous consumption |
| Data center efficiency | PUE, cooling overhead | Multiplies server energy use |
| Grid carbon intensity | gCO2/kWh, renewable share | Converts energy to emissions |
| Workload type | Training or inference | Changes calculation approach |
Frequently asked questions
How is AI energy use measured?
It is estimated from compute time and hardware power, then adjusted for data center efficiency and grid carbon intensity.
Why do estimates vary so much?
Differences in hardware, data center efficiency, and geographic grid carbon intensity can change results by large factors.
What is the difference between training and inference energy?
Training uses large compute for one or few runs; inference is per query but can scale to many users.
Are there standards for reporting AI energy use?
The Patterson et al. paper suggests including energy and CO2e in ML papers and collaborating with MLPerf on energy benchmarks.
What public resources exist?
The IEA, LBNL, and Stanford AI Index provide data, tools, and reports.