Log inCreate wallet

The M3 Ultra for local AI.

The M3 Ultra has up to 512 GB of unified memory, read at up to 819 GB/s. Here's what that means for running language models on a Mac with it, worked out from Apple's own figures.

1The specs that matter for AI

Unified memory512 GB
Memory bandwidth819 GB/s
Found inMac Studio (2025)
Dedicated seat for itUltra seat: mines up to 320 MLX a day

2Which models fit

At 4 bits per weight a model needs a little over half a gigabyte per billion parameters, plus room for its context and macOS. With 512 GB, the M3 Ultra comfortably holds models up to about 682 billion parameters.

Model at 4-bitMemory it needsFits on the M3 Ultra
8B model4.5 GBYes
14B model7.9 GBYes
32B model18 GBYes
70B model39.4 GBYes
gpt-oss-120b, mixture-of-experts65 GBYes
Qwen3.5-397B-A17B, mixture-of-experts223.3 GBYes
DeepSeek-V4.1-Flash, mixture-of-experts316 GBYes

3How fast it runs

Writing text is limited by memory bandwidth: every new token reads the model's active weights once. Dividing the M3 Ultra's 819 GB/s by the bytes read per token gives the ceiling below. Real runs usually reach about 45% to 80% of it (llama.cpp benchmarks).

Model at 4-bitRead per tokenCeiling for one chat
8B model4.5 GB182 tokens/s
14B model7.9 GB104 tokens/s
32B model18 GB46 tokens/s
70B model39.4 GB21 tokens/s
gpt-oss-120b, mixture-of-experts2.7 GB303 tokens/s
Qwen3.5-397B-A17B, mixture-of-experts9.6 GB85 tokens/s
DeepSeek-V4.1-Flash, mixture-of-experts9 GB91 tokens/s

Measured: on the M3 Ultra (80-core GPU), llama.cpp generates 92 tokens a second with Llama-2-7B (Q4_0) in its public benchmarks.

5Compare with other chips

6Questions

Can the M3 Ultra run a 70B model?
Yes. A 70B model at 4-bit needs about 40 GB, and the M3 Ultra has up to 512 GB. Expect a ceiling of about 21 tokens a second for one chat.
How many tokens per second does the M3 Ultra get?
For an 8B model at 4-bit, the ceiling is about 182 tokens a second, and real runs reach roughly half to four fifths of it. Bigger models are proportionally slower.
Is the M3 Ultra good for AI?
For running models locally, memory and bandwidth matter most. The M3 Ultra has up to 512 GB at 819 GB/s, enough for models up to about 682 billion parameters. It can also earn by running AI work for MLX Network with MLX Node.

Sources

  1. Apple: Mac Studio (2025) specssupport.apple.com
  2. llama.cpp: Apple Silicon benchmarksgithub.com
M3 Ultra for local LLMs: memory, speed and what fits | MLX