Pixel-Identical: Making A Local Image Model Leave Your Holiday Photo Alone
Every image model redraws what you asked it to keep, so we enforced pixel protection with a mask and a composite.
The keep-mask in action: whatever it covers stays exactly as the camera left it.
The request was reasonable: a photo of the kids on holiday, and the snack pack on the table behind them becomes a cooler bag. What was not part of the deal was any change to the kids' faces.
Every image model we tried happily makes the edit you asked for and then redraws everything else as a bonus. Qwen-Image-2.1 did the cooler bag beautifully and delivered brand new children with it, with the cheerful confidence of a museum restorer handed a marker. There is a way to make a generative model leave pixels alone, and it comes down to arithmetic: let it render whatever it likes, then paste the protected pixels back from the original file.
The model understood the edit and refused the contract
Setup first: Qwen-Image-2.1 on ComfyUI's Flux Unchained fork, on an AMD 7900 XTX inside a Proxmox LXC. Yes, ROCm. Yes, it works. The results speak for themselves, in the dockets.
The catch is architectural. Qwen is a reference-edit model: you give it a photo and a prompt, and it re-renders the whole scene from both. Ask for a cooler bag and you get an excellent cooler bag, plus brand new children, because "preserve" is simply absent from its instructions.
The numbers from our first attempts: a family photo came back with the kids' faces moved by a mean pixel difference of 44 to 94 out of 255. For scale, 255 is "different colour of sky" territory, and we were at "identical stranger" territory. An earlier attempt turned a cartoon into what can only be described as a cave photo. The edit landed. The contract never existed.
What a model's "keep everything else" looks like when nobody checks.
The fix: paste the originals back
One idea, applied without mercy. Let the model render the whole scene freely, joyfully, with full creative licence over pixels it will never get to keep. Then throw away everything it wasn't allowed to touch and paste the originals back:
- Qwen full-scene edit → render.
- Invert the keep-mask (white means keep, black means editable).
ImageCompositeMaskedwithdestination= the original photo,source= the render, mask = the keep region.
The pipeline's bouncer: sub-50 KB output gets turned away at the door.
The protected regions of the output are now the original file's own bytes, bit for bit, the way the camera left them. The model can hallucinate a lizard where the sofa was, as far as the output is concerned; the kids are contractual.
The numbers, from the two demo edits:
| Subject | Plain re-render (face diff) | Protected edit (face diff) | Frame byte-identical |
|---|---|---|---|
| Holiday photo (snack pack → cooler bag) | 62–66/255 | 1.9/255 | ~55% |
| Lion statue → real lion, kids protected | 44–94/255 | ~1.0/255 | ~70% |
A mean difference of about 1 part in 255 is pixel-identical for anything a human can see. The full-frame diff still shows the model changed plenty, because the model's job is to change things. The mask's job is deciding what it is allowed to change. Two different jobs, finally staffed properly.
The three-day archaeology
Getting there taught us things the documentation does not contain, which is another way of saying the documentation contains them wrong.
The obvious wiring, VAEEncode into a KSampler, decodes to pure black on this fork. Mean RGB 0/0/0, verified. The working path is the edit encoder's own latent output, slot 2, at the photo's dimensions. Different latent space, different universe, same menu names.
The resolution parameter on the edit encoder is a total pixel budget, not width and height. Passing 0 sounds like "native" and mostly is, and it resizes a 1536×2048 photo into a colossal latent that takes eight minutes to sample, which looks exactly like a hang. Passing 1024 gets you about two minutes. The workflow that seemed stuck was never stuck, just slow in a way the parameter page does not mention.
The API's image input must use dotted top-level keys ("images.image_1"). Send a neat nested dict and the reference is silently ignored: no error, no warning, the run proceeds as pure text-to-image and paints a lone lion in a field where two children should be. A confidently wrong picture teaches a better lesson than a loud failure, and this fork trains hard.
And sometimes the failure is the server, not you. We got intermittent pure-white PNGs of about 4 KB, delivered with a cheerful "success" status. Restarting the service fixed it. The pipeline now ships with a guard: output under 50 KB means flake, requeue. Most of engineering is knowing when to trust the "success".
Tuning, in detail
Once the pipeline worked, the polish came from the official Qwen guidance and from scars:
- Rewrite edit prompts as a delta naming the protected subjects (
<image1>). Full scene descriptions drift; one sentence saying "keep these two children unchanged, change only the statue" behaves. - Around 40–50 steps with euler. At cfg 1 the negative prompt is mathematically near-irrelevant, so ours is empty, like our regrets.
- Hard mask edges resample at full denoise, so the editable region gets inset about 10 pixels. Without the inset, the render bleeds a halo into protected pixels at the boundary, and boundary halos are how protected pixels get their revenge.
- Composite the render onto the full-resolution original, never onto the downscaled render. Upscale the render to the original's size first, then mask-composite, and the protected pixels stay native no matter what the model daydreamed at lower resolution.
- AMD cameos, because the expedition wouldn't be complete: Q4_K_M quantisation for tall renders because Q8 OOMs at full height, and a GPU whose VRAM refuses to defragment itself after an OOM, cured by the oldest technique known to our people: turning it off and on again.
Version 3 results: kids' faces 0.8–1.0/255, the lion actually a lion, roughly 70% of the frame byte-identical to the original photo.
What this is good for
The pattern generalises past holiday snapshots. Portraits where faces are contractual. Product swaps in real room photos, where the room must stay the customer's actual room and not the model's cozy interpretation of one. Whenever a generator must provably not touch something, the working answer is a mask, a composite, and a diff that proves it worked. A better prompt just moves the guessing somewhere more expensive.
Every ambitious image-editing pipeline eventually rediscovers compositing. Ours had the decency to charge the GPU for the lesson.