Mac AI clusters.

1What changed: RDMA over Thunderbolt 5

Since macOS 26.2, Macs can exchange memory directly over Thunderbolt 5 using RDMA, a feature Apple names for distributed AI inference with MLX (Apple). MLX exposes it through its JACCL backend, next to MPI and TCP rings (MLX docs).

The gain is latency: RDMA lowers memory-access latency from about 300 µs to under 50 µs, according to MLX's developers. In Jeff Geerling's tests, four M3 Ultra Mac Studios ran the 1-trillion-parameter Kimi K2 Thinking at about 30 tokens a second (Jeff Geerling).

2Two ways to split a model

Pipeline parallelism gives each Mac a run of consecutive layers. It lets a model bigger than any single Mac run at all, and long prompts can be processed in parallel chunks, but generation is about as fast as on one Mac.

Tensor parallelism splits every layer across the Macs. It speeds up generation too, but the Macs exchange data many times per token, so it needs RDMA's low latency. Apple has published measurements of both on Mac Studio clusters (Apple Machine Learning Research), and states that four Mac Studios run inference up to three times faster than one (Apple).

3Building a cluster

1. Choose Macs with Thunderbolt 5 for RDMA: M4 Pro, M4 Max, M3 Ultra and later Pro, Max and Ultra chips. Matching Macs make the split simplest.
2. Connect them with Thunderbolt 5 cables, each pair directly. Two cables between a pair double the link's bandwidth.
3. Update every Mac to macOS 26.2 or later and enable RDMA as Apple's documentation describes.
4. Run the model with MLX distributed (mlx.launch) or with exo, which discovers the Macs and splits the model for you.

exo reports tensor-parallel speedups of up to 1.8 times on two devices and 3.2 times on four. Measure your own setup: the gains depend heavily on the model.

4From a cluster to a network

A cluster joins a few Macs on one desk with cables. MLX Network joins Macs everywhere over the internet. Because internet latency is far too high to split one model across strangers' Macs, the network places each job whole on a single Mac, or on a pod of Macs that are already cabled together, by memory and bandwidth.

If you own a Mac Studio or a stack of Mac minis, MLX Node can lend their spare time to the network and earn MLX Tokens, and an Ultra dedicated seat running on an Ultra Mac earns the most. See every Mac on the network on MLX World.

5Questions

Can I cluster Mac minis for AI?

Yes. Mac minis with an M4 Pro or M5 Pro chip have Thunderbolt 5 and, with macOS 26.2 or later, can use RDMA; the M4 and M6 models have Thunderbolt 4. MLX distributed and exo both split a model across them.

Does a Mac cluster make a model faster?

It depends on how the model is split. Pipeline parallelism mainly lets bigger models run; tensor parallelism over RDMA also speeds up generation. Apple states up to three times faster on four Mac Studios.

Do I need Thunderbolt 5?

For RDMA, yes. Older Macs can still be joined over Thunderbolt or Ethernet with TCP, but latency is much higher, so pipeline splits work better than tensor splits.