Satya Nadella and the three stages of scientific truth
In early March 2022 I wrote an essay for Nautilus that went viral but also almost killed my career, engendering more hate than anything I have ever written. I conjectured that deep learning, at least as we knew it, instantiated by LLMs, would hit a wall, and that new approaches would be needed sooner or later.
the so-called [LLM] scaling laws aren’t universal laws like gravity but rather mere observations that might not hold forever, much like Moore’s law, a trend in computer chip production that held for decades but arguably began to slow a decade ago…. we may already [be] approaching a point of diminishing returns… [The question becomes] What else might we need?
Opposition has continued throughout. As recently as earlier this week Dario Amodei, CEO of Anthropic, said in an interview “ nothing I've seen... leads me to expect that things will slow down.”
In recent weeks, the third stage has finally arrived. For example, Microsoft CEO Satya Nadella, yesterday, November 2024, said this:
**%20is%20an%20essential%20concept%20in%20linear%20algebra.%20It%20serves%20as%20a%20standardized%20way%20to%20simp&textTailwind=mt-4%20text-3xl%20text-white&textFontFamily=Poppins&logoTailwind=h-8%20bg-transparent&bgTailwind=bg-black&footer=The Daily Chronicle&footerTailwind=text-3xl%20font-bold%20text-orange-400%20bg-black&bgUrl=https%3A%2F%2Fimages.pexels.com%2Fphotos%2F4004376%2Fpexels-photo-4004376.jpeg%3Fauto%3Dcompress%26cs%3Dtinysrgb%26w%3D1260%26h%3D750%26dpr%3D2)
