Diffusion models can memorise a training image outright

The process wall

Two failure modes sit at opposite ends of an iterative generation loop, and both are documented rather than anecdotal. At one end, a paper on extracting training data from diffusion models, submitted 30 January 2023, states plainly that “diffusion models memorize individual images from their training data and emit them at generation time,” and reports extracting “over 1,000 training examples” from systems including DALL-E 2, Imagen and Stable Diffusion, ranging “from photographs of individual people to trademarked company logos.” At the other end sits a different failure: a system trained repeatedly on its own or another model's output rather than on original data.

What the documents show

That second failure is described in a Nature paper published 24 July 2024, which states that “indiscriminately learning from data produced by other models causes 'model collapse' — a degenerative process whereby, over time, models forget the true underlying data distribution, even in the absence of a shift in the distribution.” The paper attributes this to compounding error across generations of training, starting with the loss of rare, unusual cases and narrowing toward generic, low-variance output. Read together, the two papers describe opposite risks from repeated passes through a model: holding onto one training image too exactly, or losing an entire distribution's variety once a model's own outputs become the next round's training material.

Authorship and limitation

Neither paper is describing a maker's own iterative work of generating, picking, and regenerating from a chosen result — that is a much smaller, human-supervised loop, not the large-scale retraining the collapse paper studies. But the underlying mechanism transfers: a model's behaviour on repeated or narrow input can drift or over-fit in ways the maker does not directly observe from the interface. ComfyUI's documentation describes the alternative to trusting the model's memory: a workflow exported as a JSON file, holding the checkpoint, the node structure and the settings, independent of whatever the model itself retains or forgets between runs.

What survives

This is an editorial reading connecting two training-time papers to a maker's own iteration, not a claim that one causes the other. What a maker's process should keep fixed across an iterative loop is external to the model: a saved seed, a saved workflow file, and a kept copy of an early, less-refined result, so a distinctive but unusual output is not quietly averaged away toward whatever the model finds generic. A paper cannot verify a maker's own workflow discipline; only a kept file can.

• Keep the workflow file, seed, and checkpoint version alongside any image kept from an iterative session, not only the final export.
• Compare an early, rougher result against a heavily iterated one to check whether unusual detail has been smoothed toward something generic.
• Ask whether a tool trains or fine-tunes on submitted images, since that is a separate question from what the model was trained on originally.

A generation loop can fail by remembering too specifically or forgetting too broadly; keeping an external, fixed record of what was actually run is the documented way to tell which is happening.