VRAM budgeting on Apple Silicon, where 'fits' means something different

On a machine with a discrete graphics card, the model must fit in the card's own memory. If it does not, it does not run, and the error is immediate and clear.

Apple Silicon shares one pool between the processor and the graphics unit. A large model can load where a mid-range card would refuse — which is the advantage, and it is substantial.

The complication is that the same pool holds your operating system, your browser and everything else. There is no dedicated allocation, so the question is not whether the model fits in graphics memory but whether it fits alongside everything else you are doing.

The system reserves a portion, and a normal working environment — browser, editor, communication apps — consumes a meaningful amount before you start.

On a machine with 48GB, planning around 32GB for model work is realistic. On 32GB, around 20GB. Exceeding it does not produce a clean error; it produces swapping.

The practical consequence is that closing a browser with many tabs genuinely changes what will run, which surprises people coming from dedicated-card machines where the two pools were separate.

During generation, memory holds the weights plus the text encoder, the decoder, intermediate tensors and whatever the sampler is working on.

A rough planning figure for image models is the file size plus about a third. A 20GB checkpoint wants around 27GB available; the same model quantised to 12GB wants around 16GB.

Resolution scales the working set independently of the weights. Generating at twice the linear dimensions is four times the pixels, and the intermediate buffers grow accordingly — which is why a model that runs comfortably at one size fails at another.

When memory runs out, the system pages to disk rather than refusing. Generation continues and becomes many times slower, with no error and no obvious cause.

The tell is a job that took three minutes yesterday taking twenty today with no change to the settings. That is almost always memory pressure, not a model problem.

Watch the memory pressure indicator rather than the raw usage figure. Usage near capacity is normal on this architecture; pressure in the warning range is the signal that swapping has begun.

• Unload models between tasks. A model held in memory from an earlier job is competing with the current one for no benefit.
• Do not run two runtimes at once unless you have planned for both. A vision model and an image model loaded simultaneously is often the cause of an unexplained slowdown.
• Close the browser for large jobs. It is frequently the second largest consumer on the machine.
• Drop quantisation before dropping resolution. A quantised model at full resolution generally beats a full-precision model at reduced size.

Take an afternoon and find your actual limits: the largest model that runs comfortably, the resolution at which it starts to struggle, and what happens when you also open your normal applications.

Write the numbers down. Afterwards, deciding whether something will run is a comparison rather than an experiment, and you stop losing sessions to jobs that were never going to work.

Re-test after a major system update. The reserved portion changes occasionally, and a workflow that fitted before can stop fitting without any change on your side.