Inpainting lets you fix one part of an image without regenerating the whole thing: mask a region, say what belongs there, and the model repaints inside the mask while matching the pixels around it. This covers what the model is really doing, the settings that decide whether the result blends or shows a seam, and the failure modes worth recognizing on sight.
What happens inside a mask
Adobe's Generative Fill made the idea mainstream, but masked regeneration long predates it in diffusion pipelines. The mechanics are short:
- The VAE encodes your image into latent space.
- Noise is added to those latents — how much is set by the denoising strength.
- The model denoises step by step, but at each step the unmasked region is re-composited from the original latents, so only the masked area is free to change.
- The final latents are decoded back to pixels.
Two things fall out of that. The model always sees the surrounding image as context, so your prompt negotiates with what is already there rather than working from nothing. And everything makes a VAE round-trip, so even untouched pixels come back slightly different — the source of the faint color drift you notice after several successive inpaint passes on one file.
Dedicated inpainting checkpoints go further: their UNet takes extra input channels carrying the masked image and the mask itself, so the mask is part of the conditioning rather than a compositing trick on top. They hold together much better at high denoise and on large masks. For gentle touch-ups, any general checkpoint is fine.
Denoising strength is the whole game
One slider decides most of the outcome: how much of the original content survives. Useful starting bands:
| Denoise | What it is for | What you get |
|---|---|---|
| 0.2 – 0.4 | Blemishes, texture, small artifacts | Composition and color preserved, subtle change |
| 0.4 – 0.6 | Modifying an object that stays roughly where it is | Silhouette survives, surface and detail change |
| 0.6 – 0.8 | Replacing an object | Prompt starts to outweigh the underlying pixels |
| 0.8 – 1.0 | Inventing new content in the region | Original is essentially discarded |
Start lower than feels right and climb. Pushing past roughly 0.7 on a general checkpoint is where "it produced an unrelated object floating in a rectangle" comes from: barely any original signal is left to blend with, and only an inpainting-conditioned model handles that cleanly.
The settings that decide whether the seam shows
Mask blur. A hard-edged mask produces a hard edge. A few pixels of blur feathers the transition; too much and the edit bleeds into pixels you wanted untouched. Small mask, small blur.
Inpaint area: whole picture or only masked. "Whole picture" generates at the image's resolution, so a small mask gets a small share of the model's attention and comes back soft. "Only masked" crops the mask plus a padding margin, generates at full resolution, and scales it back in — far more detail on faces, hands and small objects. Padding matters: too little and the model has no context to match, too much and you are generating the whole scene again.
Masked content / fill mode. What the masked pixels start as before denoising. Keeping the original suits edits that follow the existing shape. Filling with the surrounding average or with latent noise is what you want when removing something, because leaving the original there biases the model into regenerating it.
Write the prompt for the mask, not for the image
This is the mistake that costs the most time. The prompt applies to what is generated inside the mask, so it should describe that region and just enough of its surroundings to keep the style consistent. Masking a hand and pasting in the full scene description gives the model reason to draw a whole new scene into a hand-sized hole.
A short prompt naming content, material and lighting usually beats a long one. Keep style tokens if the surrounding image has a strong look; drop everything about composition, which the mask has already fixed.
Here is a character-style SDXL prompt that circulated as a fill-in-the-blanks template, usable for an initial generation or for repainting a masked figure into a scene you already have:
[Subject], Lego minifigure, realistic illustration, confident emotion with clean and streamlined design, [Setting], charming characters, modular, overview, perspective view, studio photo
ugly, saturated colors
Substitute your own subject and setting — "a cowboy" and "a ghost town" — with the second block as the negative prompt. Note how little of it is composition: that is the shape a prompt should have when you feed it into a mask.
Where inpainting goes wrong
- Visible seam. Almost always mask blur too low, or denoise high enough that the new region no longer shares the original's lighting. Fix the light first, the blur second.
- Blurry patch. You inpainted a small mask with "whole picture" selected. Switch to "only masked" with modest padding.
- Color drift across passes. Each pass is another VAE round-trip. Use as few passes as you can, and composite in an image editor when you need many.
- The removed object comes back. Fill mode is keeping the original pixels as the starting point. Switch to a noise or fill initialization and raise denoise.
- The mask is too tight. A mask hugging an outline leaves no room to redraw the edge, and you get a halo. Grow it a few pixels past the boundary.
Outpainting is the same machinery with the mask outside the original frame: extend the canvas, mask the empty area, inpaint it. The padding and context rules apply unchanged.
Tool notes
AUTOMATIC1111's WebUI exposes all of these settings by name in the img2img inpaint tab, which makes it the best place to learn what each does. Fooocus wraps the same operations behind intent-based options and applies its own inpainting patch, so you pick "improve detail" or "modify content" instead of tuning numbers. In ComfyUI the equivalent is an encode-for-inpainting node with a grow-mask parameter, plus the dedicated conditioning node for inpainting checkpoints. In Python, the diffusers inpainting guide covers the pipeline arguments.
Takeaways
Treat denoising strength as the primary control and everything else as trim. Use a dedicated inpainting checkpoint whenever the mask is large or the denoise is high, a general checkpoint for gentle repairs. Turn on "only masked" for anything small, give it enough padding to see context, and blur the mask in proportion to its size. Write the prompt about the masked region alone. And keep the pass count down: every VAE round-trip costs a little fidelity across the whole image, not just the part you edited.
Top comments (0)