Papers named why generators drew bad text and broken hands

The process wall

By the time Stability AI published the SDXL technical report on 4 July 2023, two failures were familiar to anyone generating images for a living: a sign in the background with scrambled letters, a hand with too few or too many fingers. The report does not treat these as folklore. Its own appendix of failure cases shows a hand with leaves growing from it and a generated sign, and states plainly that 'hands are not yet always correctly generated' and that the model 'still encounters difficulties when rendering long, legible text.' That is the process wall: a maker composing a hand-heavy or text-bearing scene had to plan around a documented, named limitation rather than an occasional glitch.

What the documents show

The report gives its own explanation for each failure rather than leaving it as an observation. For hands, it proposes that 'hands and similar objects appear with very high variance in photographs', making it hard for the model 'to extract the knowledge of the real 3D shape and physical limitations'. For text, it points to a separate cause and fix: a Google paper submitted seven months earlier had already argued that text-to-image models fail at spelling because their text encoders are 'character-blind', trained on subword tokens carrying no information about the letters making up a word. That paper's character-aware encoders produced '30+ point accuracy gains over competitors on rare words' on its own benchmark, and the SDXL report cites this work directly as one path to improving its own text rendering.

Authorship and limitation

Naming the mechanism separates two different kinds of fix. Hands are a data and geometry problem: photographs of hands vary enormously in pose and occlusion, and a 2D diffusion model has no explicit model of the 3D structure underneath, so the report frames further scaling and training technique as the likely route forward, not a quick patch. Text is an architectural bottleneck: the encoder discards character-level detail before the image model sees the prompt, so no amount of extra image training can recover letters the encoder never represented. A maker's practical decision, inpainting a hand or setting text as a separate typographic layer, responds to which of these two documented causes is in play.

What survives

What a maker keeps is the diagnosis, useful past any one model's headline accuracy: both reports attribute failure to a specific, named component, an encoder's tokenisation, a model's implicit geometry, rather than to the technology as a category. A later model's claim of 'fixed text' can then be checked against whether it actually changed the text encoder, as the cited paper did, or only scaled training.

• Read the SDXL report's own failure-case figures and captions on hands and text.
• Compare a model's stated text encoder against the character-aware approach the Google paper describes.
• Ask whether a newer model's improved hands are attributed to more training data, a different sampler, or a changed architecture.

Both documents describe failures with a named cause rather than an unexplained flaw, which is what makes it possible for a maker to plan around them rather than simply avoid them.