1The specs that matter for AI
| Unified memory | 512 GB |
|---|---|
| Memory bandwidth | 1.2 TB/s |
| Found in | Mac Studio (2026) |
| Dedicated seat for it | Ultra Latest seat: mines up to 480 MLX a day |
2Which models fit
At 4 bits per weight a model needs a little over half a gigabyte per billion parameters, plus room for its context and macOS. With 512 GB, the M5 Ultra comfortably holds models up to about 682 billion parameters.
| Model at 4-bit | Memory it needs | Fits on the M5 Ultra |
|---|---|---|
| 8B model | 4.5 GB | Yes |
| 14B model | 7.9 GB | Yes |
| 32B model | 18 GB | Yes |
| 70B model | 39.4 GB | Yes |
| gpt-oss-120b, mixture-of-experts | 65 GB | Yes |
| Qwen3.5-397B-A17B, mixture-of-experts | 223.3 GB | Yes |
| DeepSeek-V4.1-Flash, mixture-of-experts | 316 GB | Yes |
3How fast it runs
Writing text is limited by memory bandwidth: every new token reads the model's active weights once. Dividing the M5 Ultra's 1.2 TB/s by the bytes read per token gives the ceiling below. Real runs usually reach about 45% to 80% of it (llama.cpp benchmarks).
| Model at 4-bit | Read per token | Ceiling for one chat |
|---|---|---|
| 8B model | 4.5 GB | 267 tokens/s |
| 14B model | 7.9 GB | 152 tokens/s |
| 32B model | 18 GB | 67 tokens/s |
| 70B model | 39.4 GB | 30 tokens/s |
| gpt-oss-120b, mixture-of-experts | 2.7 GB | 444 tokens/s |
| Qwen3.5-397B-A17B, mixture-of-experts | 9.6 GB | 125 tokens/s |
| DeepSeek-V4.1-Flash, mixture-of-experts | 9 GB | 133 tokens/s |
Measured: on the M5 Ultra (80-core GPU), llama.cpp generates 179 tokens a second with Llama-2-7B (Q4_0) in its public benchmarks.
M5-generation chips add neural accelerators to the GPU, so they read prompts several times faster than M4: Apple measured the first token arriving 3.3 to 4.1 times sooner with MLX (Apple).
5Compare with other chips
6Questions
- Can the M5 Ultra run a 70B model?
- Yes. A 70B model at 4-bit needs about 40 GB, and the M5 Ultra has up to 512 GB. Expect a ceiling of about 30 tokens a second for one chat.
- How many tokens per second does the M5 Ultra get?
- For an 8B model at 4-bit, the ceiling is about 267 tokens a second, and real runs reach roughly half to four fifths of it. Bigger models are proportionally slower.
- Is the M5 Ultra good for AI?
- For running models locally, memory and bandwidth matter most. The M5 Ultra has up to 512 GB at 1.2 TB/s, enough for models up to about 682 billion parameters. It can also earn by running AI work for MLX Network with MLX Node.
Sources
- Apple: M6 and M5 Ultraapple.com
- llama.cpp: Apple Silicon benchmarksgithub.com