The “A” in “AI” Stands For Amnesia

My last article was blogging off Jeremey’s article which blogged off Chris’ article and, after publishing, a reader tipped me off to the Gell-Mann amnesia effect which sounds an awful lot like Chris’ “Jeopardy Phenomenon”. Here’s Wikipedia:

The Gell-Mann amnesia effect is a cognitive bias describing the tendency of individuals to critically assess media reports in a domain they are knowledgeable about, yet continue to trust reporting in other areas despite recognizing similar potential inaccuracies.

According to Wikipedia, the concept was named by Michael Crichton because of conversation he once had with physicist Murray Gell-Mann (humorously, he said by associating a famous name to the concept he could imply greater importance to it — and himself — than otherwise possible).

Here’s Crichton:

you read with exasperation or amusement the multiple errors in a story—and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know.

He argues that this effect doesn’t seem to translate to other aspects of our lives. The courts, for example, have a related concept of “false in one thing, false in everything”.

Even in ordinary life, Crichton says, “if somebody consistently exaggerates or lies to you, you soon discount everything they say”.

In other words: if your credibility takes a hit in one area, it’s gonna take a hit across the board.

At least, that’s his line of reasoning.

It’s kind of fascinating to think about this in our current moment of AI. Allow me to re-phrase Crichton.

You read with exasperation the multiple errors in AI’s “answer”, then start a new chat and read with renewed interest and faith as if the next “answer” is somehow more accurate than the last. You start a new prompt and forget what you know.

If a friend, acquaintance, or family member were to consistently exaggerate or lie to you, you’d quickly adopt a posture of discounting everything they say. But with AI — which even comes with a surgeon general’s warning, e.g. “AI can make mistakes. Check important info.” — we forgive and forget.

Forget. Maybe that’s the keyword for our behavior. It is for Crichton:

The only possible explanation for our behavior is amnesia.