Log inCreate wallet

The M2 Ultra for local AI.

The M2 Ultra has up to 192 GB of unified memory, read at up to 800 GB/s. Here's what that means for running language models on a Mac with it, worked out from Apple's own figures.

1The specs that matter for AI

Unified memory192 GB
Memory bandwidth800 GB/s
Found inMac Studio (2023), Mac Pro (2023)
Dedicated seat for itUltra seat: mines up to 320 MLX a day

2Which models fit

At 4 bits per weight a model needs a little over half a gigabyte per billion parameters, plus room for its context and macOS. With 192 GB, the M2 Ultra comfortably holds models up to about 256 billion parameters.

Model at 4-bitMemory it needsFits on the M2 Ultra
8B model4.5 GBYes
14B model7.9 GBYes
32B model18 GBYes
70B model39.4 GBYes
gpt-oss-120b, mixture-of-experts65 GBYes
Qwen3.5-397B-A17B, mixture-of-experts223.3 GBNo
DeepSeek-V4.1-Flash, mixture-of-experts316 GBNo

3How fast it runs

Writing text is limited by memory bandwidth: every new token reads the model's active weights once. Dividing the M2 Ultra's 800 GB/s by the bytes read per token gives the ceiling below. Real runs usually reach about 45% to 80% of it (llama.cpp benchmarks).

Model at 4-bitRead per tokenCeiling for one chat
8B model4.5 GB178 tokens/s
14B model7.9 GB101 tokens/s
32B model18 GB44 tokens/s
70B model39.4 GB20 tokens/s
gpt-oss-120b, mixture-of-experts2.7 GB296 tokens/s

5Compare with other chips

6Questions

Can the M2 Ultra run a 70B model?
Yes. A 70B model at 4-bit needs about 40 GB, and the M2 Ultra has up to 192 GB. Expect a ceiling of about 20 tokens a second for one chat.
How many tokens per second does the M2 Ultra get?
For an 8B model at 4-bit, the ceiling is about 178 tokens a second, and real runs reach roughly half to four fifths of it. Bigger models are proportionally slower.
Is the M2 Ultra good for AI?
For running models locally, memory and bandwidth matter most. The M2 Ultra has up to 192 GB at 800 GB/s, enough for models up to about 256 billion parameters. It can also earn by running AI work for MLX Network with MLX Node.

Sources

  1. Apple: M2 Ultraapple.com
  2. llama.cpp: Apple Silicon benchmarksgithub.com
M2 Ultra for local LLMs: memory, speed and what fits | MLX