The Invented Citation

One invented citation, traced from request to remedy, shows confabulation whole — mechanism, misreadings, and the audit that catches it.

Ask a language model for sources on a niche topic and it may produce a convincing article that does not exist — author-shaped names, a journal-shaped title, a plausible year. Trace the mechanism with owned tools. The request creates a position where citation-text typically follows, and the model generates the text that typically follows it. Where the residue holds a real citation, prediction often reproduces it; where the residue is thin, prediction continues anyway — because producing likely text is the entire mechanism, and no native channel marks which productions are true. The output carries the full cadence of scholarship precisely because cadence is what descent rewarded. Now the two misreadings. "The machine lied" fails: lying requires knowing better, and no knowing is present. "The machine glitched" fails too: every knob operated as trained; the failure is regional, not mechanical. Unit 6 supplies the reading frame — treat the model as an unreliable channel, and read its words the way the screening count read a positive test: the claim arrives with some chance of being wrong, and base rates plus stakes set how far a confident sentence should move you. One caveat earns its place: systems that bolt live document search onto the model can produce real citations — a different machine wrapped around this one, and the wrapper needs its own audit, starting with whether the fetched document says what the summary claims. The remedy is a stakes-priced audit, Unit 7's arithmetic in plain clothes. For a restaurant recommendation: one plausibility check, low stakes, done. For a medication interaction: verify against an authoritative source, weigh the lopsided cost of a miss, and treat fluent confidence as untested calibration. The difference between the audits is expected cost of error — never the model's tone, which is identical in both.