The short version
- Weights: parameters × bits per weight ÷ 8. With MLX's 4-bit format that is about 0.56 GB per billion parameters; at 8-bit about 1.06 GB, at 16-bit 2 GB.
- So an 8B model needs about 4.5 GB at 4-bit, a 32B model about 18 GB and a 70B model about 39 GB, before the context.
- The context (KV cache) comes on top and grows with every token: for Qwen3 8B, a 32,000-token context takes about 4.8 GB, more than its 4-bit weights.
- On a Mac, the GPU can use about 75% of unified memory by default, so a 16 GB Mac has about 12 GB for a model. On a PC, the model has to fit in the graphics card's own VRAM to run at full speed.
1The rule: parameters × bits
A model is mostly its weights: one number per parameter. Stored at 16 bits, each takes 2 bytes, so a model needs 2 GB per billion parameters. Quantization stores each weight in fewer bits, which shrinks the model almost in proportion; at 4 bits most models stay close to their original quality.
Quantized formats also store a little extra to rebuild the weights. Apple's MLX keeps a scale and a bias for every group of 64 weights by default, in the model's own 16-bit type (MLX), which adds half a bit per weight. That gives these sizes:
| Precision | Bits per weight | GB per billion parameters |
|---|---|---|
| 16-bit (original) | 16 | 2.0 |
| 8-bit | 8.5 | about 1.06 |
| 6-bit | 6.5 | about 0.81 |
| 4-bit | 4.5 | about 0.56 |
| 3-bit | 3.5 | about 0.44 |
| 2-bit | 2.5 | about 0.31 |
Below 4 bits, quality drops faster. A good default is the highest precision that fits with room to spare.
2Memory for common model sizes
| Model size | 16-bit | 8-bit | 4-bit | 3-bit |
|---|---|---|---|---|
| 1B | 2 GB | 1.1 GB | 0.6 GB | 0.4 GB |
| 3B | 6 GB | 3.2 GB | 1.7 GB | 1.3 GB |
| 8B | 16 GB | 8.5 GB | 4.5 GB | 3.5 GB |
| 14B | 28 GB | 14.9 GB | 7.9 GB | 6.1 GB |
| 32B | 64 GB | 34 GB | 18 GB | 14 GB |
| 70B | 140 GB | 74.4 GB | 39.4 GB | 30.6 GB |
| 120B | 240 GB | 127.5 GB | 67.5 GB | 52.5 GB |
| 235B | 470 GB | 249.7 GB | 132.2 GB | 102.8 GB |
| 405B | 810 GB | 430.3 GB | 227.8 GB | 177.2 GB |
| 671B | 1,342 GB | 712.9 GB | 377.4 GB | 293.6 GB |
Some models ship in a low precision of their own. OpenAI's gpt-oss-120b has 117 billion parameters and keeps its mixture-of-experts weights in a 4-bit format, MXFP4, so it fits on a single 80 GB GPU (OpenAI). For a specific model, open its page: every open model and the Mac it needs, for example gpt-oss-120b or Llama 3.3 70B.
3The context takes memory too
While a model reads and writes, it keeps a key and a value for every token of the conversation in each layer: the KV cache. Its size is 2 × layers × KV heads × head size × bytes per value × tokens.
Qwen3 8B, for example, has 36 layers, 8 KV heads and a head size of 128, stored in 16-bit (config). That is 147,456 bytes per token:
| Context | KV cache for Qwen3 8B (16-bit) |
|---|---|
| 8,192 tokens | about 1.2 GB |
| 32,768 tokens | about 4.8 GB |
| 40,960 tokens (its full context) | about 6.0 GB |
Models with grouped-query attention, like this one, keep fewer KV heads than attention heads (8 against 32), which already cuts the cache by four. You can save more by quantizing the cache: mlx-lm's server takes --kv-bits 4 (mlx-lm). For chat, a few thousand tokens of context is plenty; for agents, long documents or code, budget several gigabytes.
4How big a model fits on a Mac
A Mac's memory is shared by the CPU and the GPU, and macOS lets the GPU use only part of it by default. Our Compare page counts about 75%; mlx-lm's README shows how to raise the limit on macOS 15 or later with sudo sysctl iogpu.wired_limit_mb=N (mlx-lm), but macOS and your other apps still need room.
| Mac memory | For the GPU (about 75%) | Biggest model at 4-bit | Biggest at 8-bit |
|---|---|---|---|
| 8 GB | 6 GB | about 10B | about 5B |
| 16 GB | 12 GB | about 21B | about 11B |
| 24 GB | 18 GB | about 32B | about 16B |
| 32 GB | 24 GB | about 42B | about 22B |
| 36 GB | 27 GB | about 48B | about 25B |
| 48 GB | 36 GB | about 64B | about 33B |
| 64 GB | 48 GB | about 85B | about 45B |
| 96 GB | 72 GB | about 128B | about 67B |
| 128 GB | 96 GB | about 170B | about 90B |
| 256 GB | 192 GB | about 341B | about 180B |
| 512 GB | 384 GB | about 682B | about 361B |
Memory decides what fits; memory bandwidth decides how fast it writes. The best Mac for local LLMs explains the second half, and every Mac, ranked puts both together.
5RAM, VRAM and unified memory
On a PC with a graphics card, a model runs at full speed only if it fits in the card's own memory, its VRAM: 32 GB on an RTX 5090 (NVIDIA). Layers that don't fit can run from the PC's system RAM instead, but that memory is much slower to read, so the model slows down a lot.
A Mac has no separate VRAM: the GPU reads the same unified memory as the CPU, which is why a Mac with 64 GB or 128 GB can hold models no single consumer graphics card can. A few PCs now work the same way, such as NVIDIA's DGX Spark and AMD's Ryzen AI Max machines with 128 GB shared between CPU and GPU: see M5 Max vs DGX Spark and M5 Max vs RTX 5090.
6Mixture-of-experts models
A mixture-of-experts model holds all its parameters in memory but uses only a few of them for each token. Memory follows the total; speed follows the active part. gpt-oss-20b has 21 billion parameters with 3.6 billion active (OpenAI): it needs memory like a 21B model and writes about as fast as a 4B one.
That makes mixture-of-experts models a good fit for Macs, which have more memory than bandwidth compared with graphics cards. Compare shows both numbers for every popular model.
7Questions
- How much RAM do I need to run a 70B model?
- About 39 GB for the weights at 4-bit, plus the context: in practice a Mac with 64 GB or more, or several graphics cards. At 8-bit it needs about 74 GB, so 128 GB.
- Is 16 GB of RAM enough to run an LLM?
- Yes, for models up to about 14B parameters at 4-bit: a 16 GB Mac gives the GPU about 12 GB by default. 8B models run comfortably with room for context. See the best LLMs for a 16 GB Mac.
- How much VRAM does an 8B model need?
- About 4.5 GB at 4-bit, 8.5 GB at 8-bit and 16 GB at 16-bit, plus about 1.2 GB for an 8,000-token context in Qwen3 8B's case.
- Does context length use memory?
- Yes. The KV cache grows with every token in the conversation. For Qwen3 8B it is about 147 KB per token at 16-bit: about 4.8 GB at 32,000 tokens.
- Can I run a model that is bigger than my RAM?
- Not usefully. If the weights don't fit, the system swaps or reads from slower memory, and output slows to a crawl. Use a lower precision, a smaller model or a mixture-of-experts model, or spread it over several machines: see Mac AI clusters.