How do I know if an LLM will run on my Mac?

Compare the model's size at the precision you want with your Mac's unified memory. By default macOS lets the GPU use about 75% of it, and the model also needs room for its context. A 4-bit model needs a little over half a gigabyte per billion parameters.

How many tokens per second will I get?

Writing is limited by memory bandwidth: each token reads the model's active weights once. Bandwidth divided by the bytes read per token is the ceiling for one conversation; real runs usually reach about 45% to 80% of it.

Why do mixture-of-experts models run faster?

They hold many experts in memory but use only a few for each token, so each token reads far fewer bytes. They need memory for the whole model, but run at the speed of a much smaller one.

Is 4-bit good enough?

For most models, 4-bit keeps quality close to the original. Below 4 bits quality drops faster, which is why versions under 4 bits are marked.

What can your Mac run?

Pick your Mac, or any other machine, and see which open AI models fit its memory and how fast it can run them. Worked out from each model’s card and each chip’s published memory and bandwidth.

Your machine

Apple M5 with 16 GB: about 12 GB for models by default, 153 GB/s memory bandwidth. 20 of 43 popular models run on it. Its page Measured: 32 tokens/s with Llama 2 7B at 4 bits in llama.cpp, M5 with 10-core GPU.

MLX HarnessRun any model that fits on your Mac: chat, coding agents, images and a local API. Free, open source, and a download away.

Runs well Fits with room for context, at 10 or more tokens a second.

Gemma 4 E4BGoogle DeepMind · 2026-03 · 128K context · imagesIntelligence 8.9Model card MLX build8B8-bit8.5 GB Fits14–26 tok/s

Qwen3.5 4BQwen (Alibaba) · 2026-02 · 256K context · imagesModel card MLX build4B8-bit4.3 GB Fits16–28 tok/s

Llama 3.2 3B InstructMeta · 2024-09 · 128K contextModel card MLX build3.21B8-bit3.4 GB Fits20–36 tok/s

SmolLM3 3BHugging Face · 2025-07 · 64K contextModel card MLX build3B8-bit3.2 GB Fits22–38 tok/s

Qwen3.5 0.8BQwen (Alibaba) · 2026-02 · 256K context · imagesModel card MLX build0.8B8-bit0.9 GB Fits77–136 tok/s

Nemotron 3.5 Lightning 30B A3BNVIDIA · 2026-08 · 256K contextIntelligence 12.9Model card MLX build30B · 3B active2-bit lower quality9.4 GB Fits77–136 tok/s

gpt-oss-20bOpenAI · 2025-08 · 128K contextIntelligence 9Model card MLX build21B · 3.6B active3-bit lower quality9.2 GB Fits43–77 tok/s

Ships in a native low-bit format: about 14 GB of weights.

Qwen3 Coder 30B A3B InstructQwen (Alibaba) · 2025-07 · 256K contextModel card MLX build30.5B · 3.3B active2-bit lower quality9.5 GB Fits69–122 tok/s

GLM-4.7 FlashZ.ai (Zhipu) · 2026-01 · 198K contextModel card MLX build30B · 3B active2-bit lower quality9.4 GB Fits77–136 tok/s

Gemma 4 26B A4BGoogle DeepMind · 2026-03 · 256K context · imagesModel card MLX build25.2B · 3.8B active2-bit lower quality7.9 GB Fits57–102 tok/s

LFM2.5 24B A2BLiquid AI · 2026-10 · 128K contextModel card MLX build24B · 2B active3-bit lower quality10.5 GB Fits77–136 tok/s

Runs, but slowly Fits, but under 10 tokens a second: fine for batch work, slow for chat.

Qwen3.5 9BQwen (Alibaba) · 2026-02 · 256K context · imagesIntelligence 11.2Model card MLX build9B8-bit9.6 GB Fits7–13 tok/s

Granite 4.2 8BIBM · 2026-08 · 128K contextIntelligence 11.1Model card8B8-bit8.5 GB Fits8–14 tok/s

Phi-4 (14B)Microsoft · 2024-12 · 16K contextModel card MLX build14B4-bit7.9 GB Fits9–15 tok/s

Gemma 4 12B UnifiedGoogle DeepMind · 2026-05 · 256K context · imagesModel card MLX build12B6-bit9.7 GB Fits7–13 tok/s

Llama 3.1 8B InstructMeta · 2024-07 · 128K contextModel card MLX build8B8-bit8.5 GB Fits8–14 tok/s

Qwen3.8 27BQwen (Alibaba) · 2026-08 · 256K context · imagesIntelligence 33.7Model card MLX build27B2-bit lower quality8.4 GB Fits8–15 tok/s

Muse Glimmer 30BMeta (Meta Superintelligence Lab) · 2026-08 · 128K context · imagesIntelligence 17.5Model card MLX build29.6B2-bit lower quality9.3 GB Fits7–13 tok/s

Gemma 4 31BGoogle DeepMind · 2026-03 · 256K context · imagesIntelligence 14.7Model card MLX build30.7B2-bit lower quality9.6 GB Fits7–13 tok/s

Devstral Small 2 (24B)Mistral AI · 2025-11 · 256K context · imagesModel card MLX build24B3-bit lower quality10.5 GB Fits7–12 tok/s

Only just fits Needs almost all memory: raise the GPU memory limit, keep the context short.

Qwen3.6 35B A3BQwen (Alibaba) · 2026-04 · 256K context · imagesIntelligence 18.2Model card MLX build35B · 3B active2-bit lower quality10.9 GB Only just77–136 tok/s

Intelligence: Artificial Analysis Intelligence Index v4.3.2, where measured. Model names link to their official cards.

How it’s worked out

Memory. Each model is shown at the highest precision that fits: 8, 6, 4, 3 or 2 bits per weight, plus MLX’s group scales (half a bit per weight), or its native format when it ships in one. On a Mac the GPU can use about 75% of unified memory by default; we keep room for the context on top.

Speed. Writing reads every active weight once per token, so memory bandwidth divided by those bytes is the ceiling for one conversation. In llama.cpp’s public Apple Silicon benchmarks real runs reach about 45% to 80% of it, which is the range shown. Reading a long prompt is limited by compute instead, and isn’t shown.

Sources. Parameters, context and licences come from each model’s official card on Hugging Face (linked from its name); memory and bandwidth from Apple’s and the makers’ specifications. Checked 2026-10-19. Measure your own Mac on MLX Bench.

Browse. Every Mac, from an 8 GB M1 to a 512 GB M5 Ultra, with every model it runs, and every model with the smallest Mac it needs.

Run them. Step-by-step: download and run LLMs on a Mac, how to use MLX Harness, MLX vs Ollama vs llama.cpp and the best Mac for local LLMs. While your Mac is idle, it can earn MLX Tokens.

How do I know if an LLM will run on my Mac?: Compare the model's size at the precision you want with your Mac's unified memory. By default macOS lets the GPU use about 75% of it, and the model also needs room for its context. A 4-bit model needs a little over half a gigabyte per billion parameters.
How many tokens per second will I get?: Writing is limited by memory bandwidth: each token reads the model's active weights once. Bandwidth divided by the bytes read per token is the ceiling for one conversation; real runs usually reach about 45% to 80% of it.
Why do mixture-of-experts models run faster?: They hold many experts in memory but use only a few for each token, so each token reads far fewer bytes. They need memory for the whole model, but run at the speed of a much smaller one.
Is 4-bit good enough?: For most models, 4-bit keeps quality close to the original. Below 4 bits quality drops faster, which is why versions under 4 bits are marked.