Tools for the era of experience - Nikhil R
I recent read this beautiful chapter from the unpublished book “Designing an Intelligence” by David Silver and Rich Sutton . It’s titled “Welcome to the Era of Experience” and you can read it here. The basic gist is this:
AI agents will learn by doing - continuously interacting with environments and learning from the results, (similar to how AlphaProof generated millions of mathematical proofs to exceed human performance). The postulate that this new era of training AI agents will be based on the following four characteristics:
Streams: Unlike today’s AI that operates in short, disconnected sessions, experiential agents will maintain ongoing streams of experience spanning years. A health agent connected to wearables could track sleep, activity, and diet over months, building personalized recommendations from long-term patterns rather than single interactions(even if the short term recommendation doesn’t work immediately).
Actions and Observations: These agents will move beyond text responses to act autonomously in physical and digital environments. They will use both human-friendly interfaces(Browseruse, CUA are steps in this direction) and machine-friendly APIs to explore, adapt, and discover new strategies. A scientific agent could control laboratory equipment, operate telescopes, and conduct experiments independently.
