Generating one good character is easy. Generating the same character forty times, across different poses and scenes, is the actual job, and nothing in a sampler is designed to help you do it. This lays out four approaches to holding identity steady, what each one costs, and how to diagnose which kind of drift you are looking at.
Why identity does not persist by default
A text-to-image model has no memory between calls. Each generation samples from a distribution conditioned on your prompt, and "a woman with red hair" describes a region of that space containing millions of distinguishable people. The model is not being inconsistent. You never specified an identity, only a category. Every technique below is a different way of narrowing that region until roughly one person is left inside it.
Instruction-editing models add a second failure mode on top. They can carry an identity across from a reference image, but every re-encode nudges it, so identity decays over a chain of edits instead of being resampled fresh each time.
Four approaches, in order of effort
| Approach | Setup cost | How well it holds | Best for |
|---|---|---|---|
| Fixed seed and prompt | None | Only while nothing changes | Variations on a single shot |
| Written character spec | Low | Family resemblance | Short series, varied scenes |
| Canonical reference image | Medium | Good over a few passes | Editors and multimodal models |
| Trained adapter | High | Strong, across any prompt | Long projects, many images |
Fixed seed is not character consistency
Same seed plus same prompt plus same settings gives you the same image. Change one word of the prompt and the seed no longer preserves the face; it preserves the noise layout, which is not the same thing. Seeds are for isolating a single variable, not for building a series.
Write the character spec once, then never paraphrase it
The cheapest real technique is a fixed block of identity text that you paste verbatim into every prompt. Four rules make it work:
- Use countable visual attributes. Hair length and colour, eye colour, face shape, build, skin tone, and one distinguishing mark. A scar, a gap tooth, a specific pair of glasses. The distinguishing mark does more work than everything else combined, because it is rare in the training data and therefore highly identifying.
- Drop subjective adjectives. "Beautiful", "striking" and "handsome" carry no identity information. They only shift the aesthetic prior, which is why every character described that way converges on the same generic face.
- Freeze the exact wording. Paraphrasing changes the conditioning even when the meaning is identical. Copy and paste; do not retype from memory.
- Keep wardrobe in a separate block. Identity block first, then clothing, then scene. That way you can change the outfit without touching the face.
A spec gets you a family resemblance rather than the same individual. For storyboards or small comic panels, where the reader tracks the character by hair and clothing, that is usually enough.
Build one canonical reference, and stop chaining outputs
For anything more demanding, generate a character sheet once: front view, three-quarter view, profile, neutral expression, plain background. Iterate until one is genuinely right, then treat that file as the reference for every image afterwards.
The mistake worth naming is using the previous output as the reference for the next image. Errors compound multiplicatively. Ten images in, you have a different person. Always re-reference the original sheet.
With a multimodal editor this is the entire technique: attach the sheet, describe the new scene in a sentence, and add an explicit clause telling it to keep the face and build unchanged. The reference supplies identity, the sentence supplies everything else, and there is no chain to accumulate drift.
Train it in when the project is long enough
Past a few hundred images, a LoRA or a textual-inversion embedding pays for its setup. It moves identity out of the prompt and into the weights, which means it survives prompt changes that a text spec cannot.
You need a dataset first, and the two previous steps are how you build one: the spec generates candidates, the canonical sheet anchors them, and you expand outward into varied poses, lighting and framing. Curate hard. A dataset containing three slightly different faces teaches the adapter to produce three slightly different faces. The diffusers LoRA training guide covers the mechanics. Note that the resulting adapter is bound to one base model: switch checkpoints and you retrain.
Style consistency is a separate problem
You can hold the character perfectly and still lose the series, because the rendering style drifted. Fix it the same way you fix identity: name it, then spell it out.
A small, kawaii-styled anime girl with pastel pink hair and large, expressive eyes stands on a gentle grassy slope, looking up at a colossal, friendly dragon head. The dragon's skin is soft green with smooth textures, and its eye is a sparkling, oversized blue. The background is a bright, ethereal sky filled with puffy, vibrant peach and yellow clouds. Use a Moe anime style, characterized by smooth digital shading, fine line art, and a bright, pastel color palette
This was written for ByteDance's Seedream 4.0 and transfers to other prompt-following models. The reusable part is the closing sentence: use a [named style], characterized by [shading technique], [line quality], and [palette]. A style name on its own is ambiguous across models. Naming the three concrete attributes gives the model something to act on regardless. Everything before that sentence is scene description and can be swapped freely without disturbing the look.
Reading the drift
- Face changes, clothes stay put. The identity block is too weak. Add a distinguishing mark and cut the subjective adjectives.
- Everything shifts gradually across a set. You are chaining outputs. Go back to referencing the canonical sheet.
- Character right, look wrong. The style spec is missing or buried mid-prompt. Move it to the end as its own sentence.
- Right in close-up, wrong in wide shots. There are not enough pixels on the face for the model to resolve it. Frame tighter, or generate the wide shot and fix the face in a second pass.
Practical takeaway
Write the identity spec once and paste it verbatim, with wardrobe in a separate block. Generate one canonical character sheet and reference that file every single time rather than the last image you made. Move to a trained adapter only when the image count justifies the dataset work. And pin the style with a named-style-plus-attributes sentence at the end of the prompt, because character consistency and style consistency fail independently and each needs its own fix.
Top comments (0)