# Stable Diffusion 3 Medium: What the Open Weights Give You

> Published 2026-08-26 · https://www.promptzone.com/pietro_lefevre/stable-diffusion-3-medium-what-the-open-weights-give-you-e7j

Downloading a model's weights and calling a company's hosted endpoint for the same model are two different experiences, and Stable Diffusion 3 is the clearest illustration of that gap. This article covers what the SD3 Medium checkpoint actually contains, how to run it on consumer hardware, where its output falls apart, and why prompting it well means unlearning your [SDXL](/jaroslav/how-to-install-and-run-sdxl-models-in-comfyui-a-complete-guide-2nk2) habits.

## The API model and the downloadable model were never the same thing

Stability AI opened API access to Stable Diffusion 3 well before publishing any weights. When the open release arrived in June 2024, it was SD3 Medium — a roughly 2-billion-parameter member of the family, not whatever size was answering API calls. Plenty of people formed an opinion from hosted samples, ran the local checkpoint, and concluded the release had underdelivered. They were comparing two different models.

The pattern repeats across the field, so it is worth internalising rather than treating as a one-off grievance. A hosted endpoint can run a larger checkpoint, chain a prompt-rewriting language model in front of the image model, apply an automatic upscale pass, and quietly discard failed generations before you ever see them. A raw checkpoint on your own GPU does none of that. Benchmark like against like: local checkpoint versus local checkpoint, same sampler, same seed.

![A dense wall of desert sand rolling toward a city skyline](https://cdn.stocksnap.io/img-thumbs/960w/R1TEJ3C4DN.jpg)

## What is actually inside the checkpoint

SD3 is built on MMDiT, a multimodal diffusion transformer that gives image tokens and text tokens their own weights and lets the two streams attend to each other, rather than injecting text into a UNet through cross-attention alone. Training uses rectified flow, which learns a straighter path between noise and image than the older noise-prediction schedules and is a large part of why the model stays coherent at modest step counts.

The more practical detail is the text side. SD3 ships with three text encoders, and you do not have to load all of them.

| Encoder | What it contributes | Can you drop it |
| --- | --- | --- |
| CLIP-L | Core concept and style grounding | No |
| CLIP-G | Extra CLIP capacity, better subject fidelity | Not advisable |
| T5-XXL | Long natural-language prompts, in-image text, spatial relations | Yes, at a quality cost |

T5-XXL is by far the heaviest of the three. Stability published checkpoint files both with and without it precisely because dropping it is the single largest VRAM saving on offer, and the tradeoff is legible: short tag-style prompts barely change, while long descriptive prompts and any text rendered inside the image degrade noticeably.

## Getting it running

1. Pull the weights from the [official Hugging Face repository](https://huggingface.co/stabilityai/stable-diffusion-3-medium) and accept the model card terms — the download is gated.
2. Pick your variant. Take the file that bundles the text encoders if you have VRAM headroom; take the version without T5-XXL on 8–12 GB cards.
3. Put the checkpoint where your UI expects it. In [ComfyUI](/tara_suzuki/best-comfyui-custom-nodes-in-2026-the-ones-actually-worth-installing-75d) that is `models/checkpoints`, with any separately shipped text encoders in `models/clip`.
4. Start from a reference SD3 workflow rather than adapting an SDXL one. A graph built for a UNet model will mis-wire the triple encoder without throwing an error.
5. Set CFG lower than your SDXL reflex. The 4–5 range is a sane starting point; pushing to 7 or 8 tends to burn contrast and stiffen composition.
6. Generate at the resolution the model was trained for — 1024 on the long side and its aspect-ratio siblings — then upscale as a separate pass.

ComfyUI, which now has an [official organisation and blog](https://blog.comfy.org/) behind it, is the path of least resistance for this model, because a node graph makes a three-encoder text path visible instead of hiding it behind one prompt box.

![Abstract artwork of interconnected glowing nodes and lines](https://cdn.stocksnap.io/img-thumbs/960w/QO7AHBYWQE.jpg)

## Where it visibly fails

Two weaknesses surfaced immediately after release and are worth knowing before you burn an evening on them.

Human anatomy in non-standard poses is the notorious one. Figures lying on the ground, limbs at unusual angles, or bodies partially occluded come out mangled far more often than the same prompts on a mature SDXL fine-tune. Standing portraits are generally fine; anything reclining is a coin flip.

The second is fine detail under crowding. Hands gripping objects, dense crowds, and small background faces degrade faster than the overall image quality would lead you to expect.

Neither is fixable by wording. The workable responses are to compose around the weakness, run a dedicated inpainting or detailer pass over faces and hands, or use a fine-tune whose training data covers what the base model missed.

## Prompting it is not SDXL prompting

SDXL rewards comma-separated tag stacks because its CLIP encoders were trained on that style of caption. With T5-XXL in the loop, SD3 reads sentences. Describe the scene in ordinary prose, put the subject first, and state spatial relationships explicitly — behind, to the left of, reflected in — because those are precisely the tokens a T5 encoder can represent and a CLIP-only pipeline discards.

The older tag style still transfers for atmosphere and quality modifiers, though. This prompt, written in the classic SDXL register, is a good stress test for large-scale composition and runs on both model families:

```plaintext
A giant sandstorm monster enveloping a city, cinematic, epic realism, highly detailed, spooky atmosphere
```

On an SDXL checkpoint you get exactly the tag-soup behaviour it was written for. On SD3 the difference shows up in the scale relationship between the monster and the skyline, which holds together more consistently, while the trailing quality tags do much less work. Rewriting it as a sentence — a colossal figure of swirling sand engulfing a city skyline at dusk, seen from street level — is what SD3 actually wants, and the gap between the two phrasings is the fastest way to feel what the T5 encoder is doing.

![Close-up of a graphics card installed inside a desktop computer](https://cdn.stocksnap.io/img-thumbs/960w/XEAHGYTSIG.jpg)

## Check the licence before you build on it

SD3 Medium is not published under a plain permissive licence. Stability has revised its community licensing terms since the model first appeared, and the conditions attached to commercial use, fine-tuning, and redistribution are whatever the model card says at the time you download. Read that page directly rather than a forum summary — licensing is the part of an open-weights release most likely to have changed since any article about it was written, this one included.

## Practical takeaways

- Judge an open-weights release by the checkpoint you can download, never by hosted demos of a same-named model.
- Load T5-XXL when VRAM allows; drop it only if your prompts are short and you need no text rendered in the image.
- Start at CFG 4–5 and at native training resolution, then upscale separately.
- Plan on fixing anatomy in a second pass rather than in the prompt.
- Write prose for SD3 and keep tag stacks for the CLIP-only models.

## Related reading

- [How Open Model Licenses Shape the Image AI Ecosystem](/arif_lefevre/how-open-model-licenses-shape-the-image-ai-ecosystem-3c9j)
- [Few-Step Diffusion Models: Hyper-SD, Lightning, and Turbo](/qian_hansen/few-step-diffusion-models-hyper-sd-lightning-and-turbo-je5)
- [How to Evaluate an Image Model That Ships API-First](/eamon_nguyen/how-to-evaluate-an-image-model-that-ships-api-first-25kd)