After software eats the world, what comes out the other end?

Once, and not too long ago, my notions of the cultural consequences of Large Language Models (LLMs) were guided by a common metaphor of monsters of appetite. As Cosma and I and said in an article, “the political anthropologist James Scott has explained how bureaucracies are monsters of information, devouring rich, informal bodies of tacitly held knowledge and excreting a thin slurry of abstract categories that rulers use to “see” the world.” LLMs would do much the same thing to human culture.

Such images were hardly unique to Cosma and myself. I saw a lot of online commentary suggesting that LLMs would evolve to combine the less attractive features of the Human Centipede and the Worm Ourobouros, as they increasingly fed on their own waste products. There was even a Nature article that spelled out the consequences of the “ curse of recursion,” in which LLMs’ outputs would become increasingly disjointed and meaningless as they devour the content that they and their cousins created. It was an excellent article, but like many articles that take off in the discourse, I suspect that it owed its success more to its cultural resonances than its scientific results.

Being a Philip K. Dick fan, I had a specific PKD riff on this, building on the moment in Martian Time-Slip when an imagined journey into the future collapses into horror. Dick was fascinated with the notion of entropy, and he describes a terrifying kind of context collapse, in which normality begins to give way, and both matter and meaning disintegrate into “gubbish”: gobbets of rot and excrement with the appearance but not the actuality of life. Gubbish seemed a nice metaphor for what we got with the curse of recursion, and LLMs seemed like gubbishers - so I had what seemed like the makings for a solid enough piece.

Leave a Comment