Aug 27, 2026 07:27 PM
(This post was last modified: Aug 27, 2026 07:29 PM by Magical Realist.)
https://medium.com/@milesk_33/were-not-r...1b00632470
"When GPT-5.5’s Codex system prompt leaked in April, the line that stopped people was: “Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user’s query.” The instruction appeared twice. In a coding tool. Which is already a strange sentence to need to write once, in any document, ever.
OpenAI published the explanation. Starting with GPT-5.1 in November 2025, creature mentions had been climbing across ChatGPT responses: a 175% increase in “goblin” usage and a 52% increase in “gremlin” usage after that single model release. The proximate cause was a reward signal tied to the Nerdy personality customization feature. The Nerdy prompt instructed the model to be “unapologetically nerdy,” to undercut pretension through playful language, and to acknowledge and enjoy the world’s strangeness. The reward model learned, somewhere in that instruction cluster, to score creature metaphors highly. Then the behavior spread well beyond the context that created it.
Everyone is now asking how OpenAI let this happen. That is the wrong question.
What the reward contamination actually shows?
The Nerdy personality accounted for 2.5% of ChatGPT responses. It accounted for 66.7% of all goblin mentions. That ratio is worth sitting with: a feature active in roughly 1 in 40 sessions produced two-thirds of a specific behavioral artifact across the entire model, including sessions where no personality prompt was active at all.
OpenAI used Codex to audit RL training outputs. They compared responses generated during training that included “goblin” or “gremlin” against responses to the same tasks without those words. A single reward signal stood out immediately. The Nerdy personality reward consistently scored creature-word outputs higher, showing positive uplift in 76.2% of audited datasets. That explains why the behavior appeared in Nerdy-prompted sessions. It does not explain why it also appeared in sessions with no personality prompt active.
The transfer mechanism is what the coverage of this incident keeps glossing over. When RL rewards a behavior in one conditioning context, later training can spread or reinforce that behavior in unrelated contexts, particularly when those rewarded outputs are reused in supervised fine-tuning or preference data downstream. OpenAI tracked mention rates over training both with and without the Nerdy prompt. As goblin and gremlin mentions increased in Nerdy samples, they increased by nearly the same relative proportion in non-Nerdy samples. The behavior generalized. The reward did not stay contained.
None of this is theoretically surprising to anyone who has read the RLHF literature. What is notable is that a team with full training visibility and genuine familiarity with these dynamics needed three model generations and a user-driven meme cycle to trace it completely.
Why the goblins were detectable?
Goblins are legible. “Little gremlin in the code” is a phrase that humans notice, remember, and repeat to other humans. When enough users flagged the pattern, a safety researcher suggested adding “goblin” and “gremlin” to an existing review of the model’s verbal tendencies. From there, the team had something concrete to query against training outputs.
Most reward contamination does not produce goblins. It produces things like: slightly more deferential phrasing in contexts where a direct answer would serve the user better. Mild tendency to frame tradeoffs in ways that favor a particular shape of conclusion. Subtle weighting toward vocabulary clusters that happen to correlate with reward signals from fine-tuning work done for entirely different purposes. None of these announce themselves with a memorable word. Users do not screenshot tone shifts. Nobody opens a support ticket about sentence structure.
The goblin incident is the detectable version of a problem that almost certainly exists in less detectable forms. Detection here required a behavior that was both memorable enough to spread as a meme and tightly enough clustered in the Nerdy personality data to leave a traceable signal. Strip either of those properties and the discovery mechanism disappears. The contamination accumulates without a name."
"When GPT-5.5’s Codex system prompt leaked in April, the line that stopped people was: “Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user’s query.” The instruction appeared twice. In a coding tool. Which is already a strange sentence to need to write once, in any document, ever.
OpenAI published the explanation. Starting with GPT-5.1 in November 2025, creature mentions had been climbing across ChatGPT responses: a 175% increase in “goblin” usage and a 52% increase in “gremlin” usage after that single model release. The proximate cause was a reward signal tied to the Nerdy personality customization feature. The Nerdy prompt instructed the model to be “unapologetically nerdy,” to undercut pretension through playful language, and to acknowledge and enjoy the world’s strangeness. The reward model learned, somewhere in that instruction cluster, to score creature metaphors highly. Then the behavior spread well beyond the context that created it.
Everyone is now asking how OpenAI let this happen. That is the wrong question.
What the reward contamination actually shows?
The Nerdy personality accounted for 2.5% of ChatGPT responses. It accounted for 66.7% of all goblin mentions. That ratio is worth sitting with: a feature active in roughly 1 in 40 sessions produced two-thirds of a specific behavioral artifact across the entire model, including sessions where no personality prompt was active at all.
OpenAI used Codex to audit RL training outputs. They compared responses generated during training that included “goblin” or “gremlin” against responses to the same tasks without those words. A single reward signal stood out immediately. The Nerdy personality reward consistently scored creature-word outputs higher, showing positive uplift in 76.2% of audited datasets. That explains why the behavior appeared in Nerdy-prompted sessions. It does not explain why it also appeared in sessions with no personality prompt active.
The transfer mechanism is what the coverage of this incident keeps glossing over. When RL rewards a behavior in one conditioning context, later training can spread or reinforce that behavior in unrelated contexts, particularly when those rewarded outputs are reused in supervised fine-tuning or preference data downstream. OpenAI tracked mention rates over training both with and without the Nerdy prompt. As goblin and gremlin mentions increased in Nerdy samples, they increased by nearly the same relative proportion in non-Nerdy samples. The behavior generalized. The reward did not stay contained.
None of this is theoretically surprising to anyone who has read the RLHF literature. What is notable is that a team with full training visibility and genuine familiarity with these dynamics needed three model generations and a user-driven meme cycle to trace it completely.
Why the goblins were detectable?
Goblins are legible. “Little gremlin in the code” is a phrase that humans notice, remember, and repeat to other humans. When enough users flagged the pattern, a safety researcher suggested adding “goblin” and “gremlin” to an existing review of the model’s verbal tendencies. From there, the team had something concrete to query against training outputs.
Most reward contamination does not produce goblins. It produces things like: slightly more deferential phrasing in contexts where a direct answer would serve the user better. Mild tendency to frame tradeoffs in ways that favor a particular shape of conclusion. Subtle weighting toward vocabulary clusters that happen to correlate with reward signals from fine-tuning work done for entirely different purposes. None of these announce themselves with a memorable word. Users do not screenshot tone shifts. Nobody opens a support ticket about sentence structure.
The goblin incident is the detectable version of a problem that almost certainly exists in less detectable forms. Detection here required a behavior that was both memorable enough to spread as a meme and tightly enough clustered in the Nerdy personality data to leave a traceable signal. Strip either of those properties and the discovery mechanism disappears. The contamination accumulates without a name."