Mac Studio for local LLMs: what fits in 36GB and 96GB

For a Mac Studio running local LLMs, memory decides almost everything. This guide shows how to estimate what fits in 36GB and 96GB, and how to get the most out of each.

Why unified memory changes the maths

On Apple silicon, the CPU and GPU share one pool of unified memory. A model loaded for GPU inference sits in the same memory as macOS and your apps. There is no separate graphics card with its own fixed VRAM, so the useful question is simply: how much of that pool can the GPU use, and how much does the model need?

By default macOS lets the GPU wire roughly two-thirds to three-quarters of unified memory. On a 36GB Mac Studio that is somewhere around 24–27GB. On a 96GB machine it is around 64–72GB. Plan against those figures, not the total on the spec sheet.

Step 1: estimate the weights

A model's weight memory is roughly parameters multiplied by bytes per parameter. A useful rule of thumb for 4-bit quantised models is 0.5–0.7GB per billion parameters, which includes some overhead for quantisation scales and runtime buffers.

Model size	4-bit (approx.)	8-bit (approx.)
8B	4–6GB	8–9GB
14B	7–10GB	15–16GB
32B	16–22GB	34–36GB
70B	35–49GB	70–75GB

These are planning figures. Exact sizes vary by quantisation format, so check the file size of the actual model you plan to load.

Step 2: add the context (KV cache)

The second cost is the key-value cache, which grows with context length. For a standard transformer it is approximately:

KV bytes per token = 2 x layers x kv_heads x head_dim x bytes_per_value

Take Llama 3.1 70B: 80 layers, 8 KV heads, head dimension 128, stored at 16-bit (2 bytes). That is 2 x 80 x 8 x 128 x 2 = 327,680 bytes, about 320KB per token.

• 8K tokens of context: about 2.5GB
• 32K tokens: about 10GB
• 128K tokens: about 40GB

A smaller model such as Llama 3.1 8B (32 layers, same KV layout) needs about 128KB per token, so 32K tokens costs about 4GB. The lesson: a long context can cost as much memory as a mid-sized model. Many runtimes can quantise the KV cache to 8-bit, which roughly halves this cost with a small quality trade-off.

What fits in 36GB (Mac Studio M5 Max)

With roughly 24–27GB available to the GPU by default:

• 7–14B models fit easily, even at 8-bit, with plenty of room for long context or several models loaded at once.
• ~30B-class models at 4-bit are the sweet spot. A 32B dense model needs roughly 16–22GB for weights, leaving room for a moderate context, typically 8K to 32K tokens depending on the model's KV layout.
• Mixture-of-experts models around 30B total parameters fit in the same memory and generate faster than dense models of that size, because only a fraction of parameters is active per token.
• 70B-class models do not fit comfortably. Even at aggressive 3-bit quantisation they leave little room for context and the system.

What fits in 96GB (Mac Studio M5 Ultra)

With roughly 64–72GB available by default:

• 70B-class models at 4-bit fit with room for a long context. At around 40GB of weights plus 10GB for 32K tokens, you stay well inside the default limit.
• 30B-class models at 8-bit fit easily, for when quality matters more than speed.
• Two or three mid-sized models can stay loaded at the same time, for example a coding model, a chat model and an embedding model behind one API.
• Larger mixture-of-experts models of around 100B total parameters at 4-bit may fit, tightly. Check the actual file size and leave headroom.
• 70B at 8-bit sits at or beyond the default limit and is not a comfortable everyday setup.

Raising the GPU memory limit

Recent macOS versions expose a setting that changes how much memory the GPU may wire:

# 36GB machine: allow about 28GB for the GPU
sudo sysctl iogpu.wired_limit_mb=28672

# 96GB machine: allow about 80GB for the GPU
sudo sysctl iogpu.wired_limit_mb=81920

# return to the default
sudo sysctl iogpu.wired_limit_mb=0

Use it carefully. The value resets at reboot, which is a useful safety net. Always leave several gigabytes for macOS, SSH, your remote desktop tool and any services. If you push it too far, the system starts swapping heavily or becomes unresponsive, which on a remote machine can mean losing access until it recovers. Raise it in steps, load your model, and watch memory_pressure and Activity Monitor before settling on a value.

MLX vs llama.cpp vs Ollama

All three run well on Apple silicon. They suit different jobs.

Runtime	Best for	Notes
MLX / mlx-lm	Apple-native performance, Python workflows, fine-tuning with LoRA	Uses MLX-format models; a large community library of pre-converted quantised models
llama.cpp	Fine control: context size, KV cache type, batching, GGUF files	Metal backend; llama-server provides an OpenAI-compatible API
Ollama	The quickest path to a running API	Built on llama.cpp; simple model management and sensible defaults

Getting started with each:

# MLX
pip install mlx-lm
mlx_lm.generate --model mlx-community/Qwen2.5-32B-Instruct-4bit --prompt "Hello"
mlx_lm.server --model mlx-community/Qwen2.5-32B-Instruct-4bit --port 8080

# llama.cpp
brew install llama.cpp
llama-server -m ./model-Q4_K_M.gguf -c 16384 --port 8080

# Ollama
ollama run qwen2.5:32b

See MLX on GitHub and llama.cpp on GitHub for current options. For a step-by-step Ollama service setup, read run Ollama on a Mac mini server; the same steps apply to a Mac Studio.

Throughput: what to expect

Exact speeds depend on the chip, model, quantisation, runtime version and context length, so measure on your own workload. The patterns are consistent, though:

• Generation speed falls as model size grows. Producing each token means reading the active weights from memory, so a 70B dense model generates noticeably slower than a 30B one.
• Mixture-of-experts models feel faster than dense models of the same total size, because fewer weights are read per token.
• Prompt processing is a separate cost. Long prompts take time to ingest before the first token appears. This matters for retrieval and coding tools that send large contexts.
• Lower-bit quantisation is usually faster, since there is less data to read.
• Interactive chat on 30B-class models is comfortable on Mac Studio hardware; 70B-class models are usable for chat and batch work, at a slower pace.

Choosing between 36GB and 96GB

Pick the Mac Studio M5 Max with 36GB if your target is 30B-class models or several smaller ones. Pick the Mac Studio M5 Ultra with 96GB if you need 70B-class models, long contexts, or several models resident at once. Both are dedicated physical machines at Hatmac on 6- or 12-month terms: the M5 Max is $549/mo for 6 months or $529/mo for 12, the M5 Ultra $1,199/mo or $999/mo. More memory is available on request. See Mac Studio LLM hosting, or our deeper guide to LLM memory on Apple silicon.

FAQ

Can a Mac Studio with 36GB run a 70B model?

Not comfortably. A 4-bit 70B model needs roughly 35–49GB for weights alone, which exceeds the GPU's default share of 36GB.

Is a Mac Studio good for local LLMs compared with a Mac mini?

Yes for larger models. A Mac mini with 16–24GB suits 7–14B models; a Mac Studio with 36GB or 96GB opens up 30B and 70B-class models.

Is it safe to change iogpu.wired_limit_mb?

It is a supported setting, but leave headroom for macOS. The value resets on reboot, so test in steps.