Memory at each precision
| Version | Weights | With context | Smallest Mac |
|---|---|---|---|
| 8-bit | 25.5 GB | 27.5 GB | 48 GB |
| 6-bit | 19.5 GB | 21.1 GB | 32 GB |
| 4-bit | 13.5 GB | 15 GB | 24 GB |
| 3-bit lower quality | 10.5 GB | 12 GB | 16 GB |
| 2-bit lower quality | 7.5 GB | 9 GB | 16 GB |
On every Apple chip
The smallest memory size of each chip that runs Mistral Small 3.2 (24B), the best version that fits it, and the writing speed you can expect.
| Chip | Needs | Version | Tokens/s |
|---|---|---|---|
| M6 | 24 GB | 4-bit Slow | 6–10 |
| M5 Ultra | 96 GB | 8-bit Runs well | 21–38 |
| M5 Max | 36 GB | 6-bit Runs well | 11–19 |
| M5 Pro | 24 GB | 4-bit Runs well | 10–18 |
| M5 | 24 GB | 4-bit Slow | 5–9 |
| M4 Max | 36 GB | 4-bit Runs well | 14–24 |
| M4 Pro | 24 GB | 4-bit Slow | 9–16 |
| M4 | 24 GB | 4-bit Slow | 4–7 |
| M3 Ultra | 96 GB | 8-bit Runs well | 14–26 |
| M3 Max | 36 GB | 4-bit Runs well | 10–18 |
| M3 Pro | 36 GB | 6-bit Slow | 3–6 |
| M3 | 24 GB | 4-bit Slow | 3–6 |
| M2 Ultra | 64 GB | 8-bit Runs well | 14–25 |
| M2 Max | 32 GB | 4-bit Runs well | 13–24 |
| M2 Pro | 32 GB | 6-bit Slow | 5–8 |
| M2 | 24 GB | 4-bit Slow | 3–6 |
| M1 Ultra | 64 GB | 8-bit Runs well | 14–25 |
| M1 Max | 32 GB | 4-bit Runs well | 13–24 |
| M1 Pro | 32 GB | 6-bit Slow | 5–8 |
| M1 | 16 GB | 3-bit Slow | 3–5 |
How to run it
- The easy way: download MLX Harness, free and open source. Search for Mistral Small 3.2 (24B), download the version that fits your Mac, and chat. It also serves the model to your other apps over an OpenAI-compatible API.Download MLX HarnessFree · Apple Silicon · macOS 14.5 or later
- In Terminal with Apple’s MLX:
pip install mlx-lmmlx_lm.chat --model mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bitBuild: mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bit on Hugging Face. Other precisions sit next to it.
Step by step: download and run LLMs on a Mac · how to use MLX Harness · MLX vs Ollama vs llama.cpp
About Mistral Small 3.2 (24B)
| Made by | Mistral AI |
|---|---|
| Parameters | 24B (dense) |
| Context | 128K tokens |
| Licence | Apache-2.0 |
| Released | 2025-06 |
| Input | Text and images |
| Weights | mistralai/Mistral-Small-3.2-24B-Instruct-2506 |
Questions
- How much memory does Mistral Small 3.2 (24B) need?
- About 13.5 GB of weights at 4-bit, plus room for the context: 15 GB in all. On a Mac that means 24 GB of unified memory or more, since macOS lets the GPU use about 75% of it by default.
- How fast is Mistral Small 3.2 (24B) on a Mac?
- Writing speed depends on memory bandwidth. The fastest Mac for it here is an M5 Ultra with 96 GB at about 21–38 tokens a second (8-bit). The smallest Mac that runs it, with 24 GB, manages about 10–18 on an M5 Pro.
- Does quantization hurt Mistral Small 3.2 (24B)?
- At 4 bits most models stay close to the original quality; below 4 bits quality drops faster. Pick the highest precision that fits your Mac with room to spare.
Similar models
For your model card or README
Shows the smallest Mac that runs Mistral Small 3.2 (24B) well and links here.
[](https://mlx.app/run/mistral-small-3.2)How it’s worked out
Memory: the model’s parameters at 8, 6, 4, 3 or 2 bits per weight plus MLX’s group scales (half a bit per weight), or its native format, with room for the context on top. By default macOS lets the GPU use about 75% of unified memory. Speed: memory bandwidth divided by the bytes read per token is the ceiling for one conversation; in llama.cpp’s public Apple Silicon benchmarks real runs reach about 45% to 80% of it, which is the range shown. These are estimates, not measurements: measure your own Mac on MLX Bench. Model figures from each official model card, chip figures from Apple’s Tech Specs. Checked 2026-10-19. Same data as mlx.app/compare.