We stand on the threshold of a new era in artificial intelligence that promises to achieve an unprecedented level of ability. A new generation of agents will acquire superhuman capabilities by learning predominantly from experience. This note explores the key characteristics that will define this upcoming era.

Original Paper: https://storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf

The Era of Human Data

• While imitating humans is enough to reproduce many human capabilities to a competent level, this approach in isolation has not and likely cannot achieve superhuman intelligence across many important topics and tasks.

The Era of Experience

• To progress significantly further, a new source of data is required.
• Agents will inhabit streams of experience, rather than short snippets of interaction.
• Their actions and observations will be richly grounded in the environment, rather than interacting via human dialogue alone.
• Their rewards will be grounded in their experience of the environment, rather than coming from human prejudgement.
• They will plan and/or reason about experience, rather than reasoning solely in human terms.

Streams

• An experiential agent can continue to learn throughout a lifetime (in the era of human data, little or no information carries over from one episode to the next).
• Human behaviour adapts from past experiences to self-correct and improve.
• An individual step may not provide immediate benefit, or may even be detrimental in the short term, but may nevertheless contribute in aggregate to longer term success.

Actions and Observations

• Agents in the era of experience will act autonomously in the real world.
• Human-privileged communication.

Rewards

• Human prejudgement → Grounded reward (arise from the environment itself).