Large Language Models are data-hungry beasts and at the same time they are able to generate huge amounts of data that is almost indistinguishable from

Don’t feed meat to the model

submited by

Style Pass

2024-10-11 14:30:05

Large Language Models are data-hungry beasts and at the same time they are able to generate huge amounts of data that is almost indistinguishable from human generated data.

That’s like giving a cow a diet of beef — welcome to the AI version of BSE. Just like cows weren’t designed for a carnivorous feedback loop, LLMs start to lose the plot when they’re forced to binge on their own words. It’s a recipe for machine insanity — a digital echo chamber where each round of learning makes the output a bit weirder.

This is a very abstract problem, and it’s hard to get an intuitive understanding of what’s really happening “under the hood”. To stick with the BSE cow metaphor for a little longer– this is how the FDA explains BSE and what happened back then:

The parts of a cow that are not eaten by people are cooked, dried, and ground into a powder. The powder is then used for a variety of purposes, including as an ingredient in animal feed. A cow gets BSE by eating feed contaminated with parts that came from another cow that was sick with BSE.