I try using an LLM to write something about information security, a subject I know very well. The result is generic and wrong in some places. I conclude that it is easier to throw it away and write it from scratch.

Later the same day, I open one of the LLMs and ask it about some legal topic I’m wondering about. I read the answer and accept it as mostly correct. I might even make important decisions based on it.

I have somehow selectively forgotten how badly it performed earlier. I call this Gell-Mann AI-mnesia, after the better-known Gell-Mann Amnesia, which, if you aren’t already familiar with it, describes roughly the same realisation applied to newspapers.

You read an article on a subject you know well and notice that

Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the “wet streets cause rain” stories.

Then you turn the page and accept the next story as somehow more accurate.

The term was coined by Michael Crichton in a 2002 speech[archive.org].

Remember that LLMs will usually sound perfectly confident. Unless told not to, in which case they will comply and sound less confident. It does not have a real understanding of the difference, it only does what it is told.