Data Poisoning & LLM Backdoors

Injecting a small number of crafted documents into a model’s training data to implant hidden behavior (a “backdoor”) triggered by a specific input.

Definition

Data poisoning is a training-time attack: an adversary contributes crafted documents so the model learns an unintended association. The Anthropic / UK AI Security Institute / Alan Turing Institute study (A small number of samples can poison LLMs of any size) showed that a near-constant ~250 documents can implant a denial-of-service backdoor — the model emits gibberish when it sees a trigger token (<SUDO>) — regardless of model size (tested 600M–13B params). The striking result is that the threshold is an absolute count, not a percentage of training data (250 docs ≈ 0.00016% at 13B).

The finding itself is established (a peer-reviewed-style empirical result). Its scope is narrow: a gibberish trigger, not complex/harmful behavior, and the researchers directly controlled the training set. This page exists partly as the primary corrective to the SEO corpus’s 250 Authority Protocol, which misapplies the “250” figure to brand recommendation via public content — a different outcome, a different data-access model, and diverse (not identical) documents. See that page for the four over-reaches.

Key Properties

• Absolute count matters, not percentage — ~250 docs backdoored models across a >20× size range.
• Narrow tested effect: DoS gibberish on a trigger token, not steering opinions/recommendations.
• Direct training-set control in the experiment — not “publish and hope it’s crawled.”
• Caveats (per authors): unclear at frontier scale or for harmful behaviors; “less useful for attackers.”
• Straddles web-dev/AI-security and SEO (via the GEO misapplication) → pillar: both.

Examples from Sources

In the Sources

Related

Concepts: 250 Authority Protocol (the SEO misapplication this corrects), GEO / AEO (Getting Recommended by AI) Entities: Anthropic, UK AI Security Institute, Alan Turing Institute, Claude