Hofstadter on LLMs

https://www.theatlantic.com/ideas/archive/2023/07/godel-escher-bach-geb-ai/674589/
(free link) https://archive.ph/2VtiC

Hofstadter weighs on in the current batch of LLMs after a reader writes in with the output from one, supposedly in his voice. He's not a fan.

Before I go on, let me explain that I am profoundly troubled by today’s large language models, such as GPT-4. I find them repellent and threatening to humanity, partly because they are inundating the world with fakery, as is exemplified by the piece of text produced by the ersatz Hofstadter.

Includes some meat on his motivations and the history of GEB:

I typed up that first manuscript, roughly doubling its length, and one happy day, inspired by Lewis Carroll’s droll but deep dialogue called “What the Tortoise Said to Achilles” (it was reprinted in DeLong’s book), I tried my own hand at writing a couple of dialogues between those two amusing characters. My second Achilles-Tortoise dialogue wound up having an unusual structure, and so, on a random whim, I called it “FUGUE.” It wasn’t a fugue at all, but suddenly I had the epiphany that I might attempt to write further dialogues that genuinely possessed contrapuntal forms, and thus did J. S. Bach slip in through the back door of my budding book.

After which he compares it to the output from GPT-4, finishing with

In short, the piece that GPT-4 composed using the pronoun “I” has zero authenticity, it has no resemblance to my manner of expressing myself, and the artificiality of its creation runs against all the pillars of my lifelong belief system.

Worth reading in whole.