Google’s Titans Give AI Human-Like Memory

TOPSHOT - Alphabet Inc. and Google CEO Sundar Pichai ](Photo by ALAIN JOCARD / AFP) (Photo by ALAIN ... [+] JOCARD/AFP via Getty Images)

Seven years and seven months ago, Google changed the world with the Transformer architecture, which lies at the heart of generative AI applications like OpenAI’s ChatGPT.

Now Google has unveiled a new architecture called Titans, a direct evolution of the Transformer that takes us a step closer to AI that can think like humans.

The Transformer architecture has no long-term memory, limiting its ability to retain and use information over extended periods - an essential part of human thought. Titans introduces a neural long-term memory, along with short-term memory and a surprise-based learning system—tools our own minds use to remember unexpected or pivotal events.

In simple terms, Transformers have a sort of “spotlight” (called the attention mechanism) that looks at only the most relevant words or data points in a sentence or dataset at any given moment. Titans still uses that spotlight but adds a huge “library” (the long-term memory module) that stores important historical information.

Leave a Comment