Artificial General Intelligence is a Red Herring
If you are into AI or futurism, it’s pretty hard to avoid conversations about AGI. When will we achieve AGI? What will AGI be like? Will AGI end the human race? This is a little frustrating, because I think a lot of the energy being put into thinking about and working towards AGI is being wasted. To be clear, I’m not an AI doomer, I’m very much on the “AI will change the world” bandwagon, I just think people are approaching the problem the wrong way.
People throw the term AGI around, but it lacks a clear definition that we can rally around. It could range from “smarter than the average human at most things” all the way to “smarter than the smartest humans at everything we can measure.” If we use the weak definition, our current language models are arguably AGI. If we use the strongest definition, we might never achieve AGI since the smartest humans will be able to use the AI as a tool and stand on its shoulders for better results than the AI could achieve alone in most cases. Additionally, I think the intelligence of the model creators biases the intelligence of the training corpus in an unavoidable and limiting way.
To be clear, I’m not saying that model creators can’t create data sets that are more intelligent than they are, but it is unavoidable that creating such a data set will be a trial and error process as the model creators lack the ability to accurately assess superhuman intelligence a priori. That means that as AI approaches peak human intelligence we will have to resort to large scale data generation and a global optimization algorithm to select the best data set over time in order to continue progressing. That will probably keep things moving slowly, however I expect the progress drop off to be brutal as using optimization to explore the space of superintelligent models is definitely much slower than bootstrapping from human abilities which are themselves the result of genetic algorithms over geological time scales. I imagine that some of the cost of running these data set experiments could be mitigated by the right architecture and training techniques, so if you want to get ahead of the curve consider how you could reduce the cost of trying and comparing various different combinations of data during training.
