How to choose a checkpoint, and decide in an hour instead of a week

Published sample images are the best output from many attempts, produced by the person who trained or tuned the model. They tell you what is possible, not what is typical, and comparing two models by their showcases compares two selection processes rather than two models.

The only reliable evidence is your own prompts, on your own machine, held constant.

The good news is that a genuinely fair comparison takes about an hour and settles the question for months.

Build a small fixed test set of five or six prompts covering what you actually make: a portrait, a product on a plain ground, a wide environment, something with difficult material, something with a specific lighting requirement.

Run all of them through each candidate at identical settings and seeds, and lay the results out side by side rather than viewing them in sequence.

Keep the test set forever. Its value grows: after a year you can compare a new model against every previous one on identical material, which no published benchmark gives you.

Look at the four generations per prompt as a group. A model producing four usable images is more valuable than one producing one excellent and three broken, even if the excellent one is the best image in the test.

Then examine the failures specifically. Every model fails; what matters is whether it fails in ways you can work around. A model that gets anatomy wrong occasionally is manageable if you shoot wide; one that cannot hold a material consistently is not, if materials are your work.

Judge at the size you will use. Differences that dominate at full magnification frequently disappear at delivery size.

Note generation time and whether the machine remains usable during a run. A model three times slower needs to be substantially better to be worth choosing, because the slowness changes how you work — you stop iterating and start committing.

Note memory too. A model that only fits with everything else closed is a model you will use less than you expect.

Availability of the right format matters as well. An excellent model published only in a format your hardware handles badly is, practically, not available to you.

Some models expect natural sentences; others respond better to comma-separated terms. Testing both with the style tuned for one is an unfair comparison that produces a confident wrong conclusion.

Run your test set in the phrasing each model's own documentation suggests, then also in your habitual style. If a model only performs well in a style you dislike writing, that is a legitimate reason to reject it — but know that is the reason.

Note the preferred style alongside the model in your records. Six months later you will not remember, and prompting a model in the wrong register looks like the model degrading.

Once the comparison is done, commit. The compounding value comes from knowing a model deeply — its vocabulary, its failure modes, the settings that suit your work — and that knowledge does not transfer.

Re-test when something significantly new appears, using the same set. Most releases will not beat what you have on your material, and having the test set means finding that out in an hour rather than losing a week to curiosity.

Keep a second model you know well as a fallback for the cases where the first one struggles. Two understood models beat six half-explored ones.