What it is
“Compute” is the computing power needed to train and run AI models. Modern models do billions of simple calculations in parallel, which is why they run on chips built for exactly that: graphics processors and special AI accelerators.
Training and inference
| Training | Inference | |
|---|---|---|
| What happens | The model learns from data | The finished model answers requests |
| How often | Once per model version | Every time someone uses it |
| Cost profile | Very high, one-off | Small per request, large in total |
The chips
- GPUs (graphics processing units) were built for video games and turned out to suit AI; they dominate training.
- Special AI chips such as Google’s TPUs are designed only for AI calculations.3
- Memory close to the chip limits how large a model can run quickly.
Energy and data centres
According to the International Energy Agency, data centres used about 415 terawatt hours of electricity in 2024, around 1.5% of the world’s consumption, and demand could roughly double by 2030, driven largely by AI.1
Why it matters for a company
- Model prices follow compute costs; they have fallen quickly for older model generations.
- Availability decides how fast providers can offer new models.
- Running open models in-house requires own or rented GPUs.
Key terms
- GPU
- Graphics processing unit; the standard chip for AI training.
- Inference
- Using a trained model to produce answers.
- FLOP
- One floating-point operation; training compute is measured in these.2
Latest research
New papers and reports that mention this term, found by our daily source scan. One line per source, quoted as published.
- heise.deAus dem aktualisierten Börsenprospekt geht auch hervor, dass die Firma für die kommenden Jahre bereits Investitionen von 895 Milliarden Dollar für Kapazität in Rechenzentren und den Ausbau von Infrastruktur veranschlagte.2026-11-02
- arxiv.orgAbstract: Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers.2026-11-02
- techcrunch.comUS data centers use 17 to 19 billion gallons of water to cool their hot-running chips each year — and that number is expected to to triple in the next five years, according to the Water Foundation.2026-11-02
- heise.deVielmehr handelt es sich um ein separates Programm auf Basis des UI-Frameworks GPUI, das auf dem neuen Hintergrunddienst herdr-core (Kommandozeilenbefehl hc ) aufsetzt.2026-11-02
- theverge.comOpenAI pitches data center power plants as a way to protect nearby areas.2026-11-02