The Plummeting Cost of Intelligence - by Tanay Jaipuria
This is a weekly newsletter about the business of the technology industry. To receive Tanay’s Newsletter in your inbox, subscribe here for free:
While foundation models are often perceived as costly and inference bills are rising, the reality is that the cost of intelligence is in free fall—and this trend shows no signs of slowing. Elad Gil shared this chart on the cost of GPT-4 equivalent intelligence from Open AI, which has fallen 240x in the last 18 months, from $180/million tokens to less than $1.
There are a number of factors driving this continued decline in the cost of intelligence, which if played out fully, can see a world where for all practical purposes, the cost of intelligence via models (for the vast majority of use cases) will tend to zero.
As more models, including open source ones from Meta, have caught up in performance to GPT-4, the resulting competition, both between various model developers and inference providers, has pushed down costs.
