Do AI companies work? - by Benn Stancil - benn.substack

Large language models cost a fortune to build. OpenAI, which is reportedly in the process of raising $6.5 billion, needs $6.5 billion dollars, because, “by some estimates, it’s burning through $7 billion a year to fund research and new A.I. services and hire more employees.” Anthropic is expected to spend $2.7 billion this year. Facebook is spending billions more. 

It probably won’t get cheaper. Chips might get better; compute costs might go down; Moore's law; etc, etc, etc. But as models get better, pushing the frontier further out will likely get more difficult. The research gets more harder, and the absolute amount of compute required to train a new model goes up. It’s climbing Mount Everest: The higher you go, the thinner the air, and the tougher each step gets. 1 Even if it gets cheaper to do the math required to build new models, that math has diminishing returns. To build a better model in 2024, you have to do more and harder math than you had to do in 2023.

Despite these costs, people will probably keep building new models. People believe that LLMs are the next technological gold rush, and the companies that build the best ones will make their employees and investors a fortune. They are trying to build artificial general intelligence. Human nature compels us to make everything faster, higher, and stronger. 

Leave a Comment