An audit counted unlawful images indexed inside a training dataset

The process wall

LAION released its five-billion-pair dataset on 31 March 2022, describing it as a set of URLs and captions rather than hosted images, built by filtering Common Crawl links through CLIP similarity and releasing 'punsafe' and 'pwatermark' detection scores alongside each entry. LAION's own release notes state the dataset is 'uncurated' and recommend it for research purposes, adding that they 'do not recommend using it for creating ready-to-go industrial products'. Just under two years later, on 20 December 2023, Stanford Internet Observatory researcher David Thiel published an audit of that dataset's indexed links using hash-matching and image-embedding search.

What the documents show

The audit evaluated 32,138,129 items scoring above its chosen safety cutoff, applying PhotoDNA and MD5 hash matching plus nearest-neighbour and classifier-based search, and reports its own total of 3,226 suspected dataset entries, 'much of which' it states 'was confirmed as CSAM by third parties'. The audit's background section states that the most widely used open model, Stable Diffusion 1.5, 'were ultimately trained on a wide array of content' from LAION-5B, both explicit and otherwise, and that Stability AI's subsequent 2.0 release filtered training images with an 'unsafe' score above 0.1, which the report says substantially reduced explicit material while leaving 1.5 as the version still used for that purpose. A separately cited 2021 audit of the earlier LAION-400M is described in the report as having found 'a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes.'

Authorship and limitation

The distinction the audit draws is between hosting and indexing: LAION-5B stores links and captions, not images, so removal is described as a multi-step problem spanning source URLs, the public metadata files, internal reference copies, researchers' own downloads and, hardest of all, the weights of models already trained. The report states plainly that it is 'unknown' whether removing matched embeddings from an already-trained model would meaningfully change its ability to reproduce this material. That limitation belongs to the documents, not to speculation: the audit does not claim which images any specific downstream model generated, only what its scan of the dataset's links found.

What survives

What the record supports is narrow. The dataset's own documentation already flagged itself as uncurated and unsuitable for production use; the audit adds a specific, dated count of what its screening found among the links. Editorially, a maker relying on any LAION-derived dataset should treat the safety scores LAION published as filters with a stated margin, not a guarantee.

• Read the audit's own methodology for how PhotoDNA, MD5 and nearest-neighbour detections were combined and validated.
• Check which safety threshold, if any, a downstream model's own documentation says was applied to its LAION-derived training subset.
• Ask a dataset host what happens to embeddings, not just source links, once a match is confirmed.

The audit is a dated, methodical count against a stated cutoff, not a claim about every image any model has produced; its value to a maker is in showing how an open, link-based dataset can carry unlawful material without anyone hosting it directly.