EN
Currency:
EUR – €
Choose a currency
  • Euro EUR – €
  • United States dollar USD – $
VAT:
OT 0%
Choose your country (VAT)
  • OT All others 0%

17.11.2026

Why AI Needs So Much Memory and What It Means for DDR5 Prices

server one
HOSTKEY

A 70-billion-parameter model at 16-bit precision takes up about 140 gigabytes of memory before it generates a single word, since each weight is stored in two bytes. In theory, 4-bit quantization shrinks the weights to 35 gigabytes, a fourfold reduction. In practice, the exact footprint depends on the compression format, and space for the KV cache, where the model stores intermediate results for the context it has already processed, still has to be allocated on top of the weights.

Memory capacity solves only half the problem. A model can fit entirely in the accelerator's memory and still respond slowly, because the GPU re-reads the weights it needs for every token it generates. When reading those weights takes longer than the math itself, adding more compute does not make the answer come out faster. This bandwidth requirement is why accelerators use HBM, where memory dies are stacked vertically in the same package as the accelerator and connected through a very wide interface.

NVIDIA and AMD GPU Servers
Dedicated and virtual servers with latest and previous generation GPUs on hourly billing

Assembling and testing those stacks takes considerably more work than producing conventional memory chips, and the competition for factory capacity starts before assembly, at the die production stage. Micron estimates that producing a given number of HBM3E bits takes about three times the wafer capacity of producing the same number of DDR5 bits on a comparable process node. When a manufacturer shifts more wafers to HBM, it has less room to grow output of other memory from that capacity. Servers for AI workloads also need ordinary system memory, so deploying more accelerators both raises demand for standard server RAM and makes it harder for suppliers to ramp up production of that memory.

Expanding cleanroom space and bringing new packaging and testing lines up to volume takes far longer than placing a server order, so the memory market can stay tight even as total output grows. Micron's estimate is about manufacturing capacity and does not translate directly into the price of a specific DDR5 module next month, and accelerator demand alone cannot explain every change in memory pricing.

In the eight-minute video below, we walk through the whole chain, from the arithmetic of model weights and the KV cache to factory capacity. At the end we show how measurements on a real request can tell you whether the bottleneck is memory capacity, single-user response speed, or throughput under dozens of concurrent requests.

NVIDIA and AMD GPU Servers
Dedicated and virtual servers with latest and previous generation GPUs on hourly billing

Other articles

13.11.2026

How Amazon S3 Grew from Simple Storage into the Cloud Storage Standard

In 2006, Amazon S3 fit into fifteen racks and could perform only two operations, but today its interface is replicated by storage providers that have nothing to do with Amazon. Drawing on engineers’ recollections and AWS documents, we trace how the service changed, where it broke, and which twenty-year-old design decisions became standard for all S3-compatible storage.

30.10.2026

OpenWebUI Pipes: Why Write Code When "Everything Is Already in the Settings"?

Configuring a custom model with tools and RAG in OpenWebUI is easy. Making it behave predictably in production is the hard part. Our support bot powered by Gemma 4 initially provided plausible answers, but they lacked actual system data. We analyzed why this happens and transformed a "single model in a UI" into a deterministic pipeline using Pipes.

30.10.2026

Which LLM Gateway to Install on Your Server: LiteLLM or OmniRoute

We compared LiteLLM and OmniRoute under load: latency, requests per second, memory and CPU usage, and the settings worth changing right after installation.

20.10.2026

GPU VPS vs GPU Dedicated Servers: Which to Choose?

The same RTX 4090 with a €150 price gap: what are you really paying for? Learn how to tell a whole GPU from a slice of one, when a GPU VPS is enough, and when it is time to move to a dedicated GPU server.

20.10.2026

How to Run Language Models (DeepSeek, Gemma, Phi) on Your Own Server: A Complete 2026 Guide

Step-by-step guide to running DeepSeek, Gemma, and Phi on your own server. Ollama, vLLM, Docker, GPU configuration, and code examples.

Upload