Abstract
Humans are remarkable in their ability to quickly learn to perform complex tasks. Reinforcement learning (RL) has long been proposed as a model of human learning, and while leading machine RL models have surpassed human expertise at many classic board games and video games, they require vast experience to learn successfully—none of today's algorithms accounts for humans' ability to learn so many different tasks so quickly. We study human learning on 90 simple yet challenging video games, showing how people learn most games within a few minutes. To explain this behaviour, we propose a strong form of model-based RL, which we call theory-based RL, because it uses cognitively grounded intuitive theories—rich, abstract, causal representations of objects, agents and their interactions—to explore and model an environment and plan effectively to achieve goals. We instantiate the approach in an agent called EMPA (the exploring, modelling and planning agent). EMPA matches human learning efficiency, generalizing robustly to new game situations and levels, as humans do, and exhibiting similar exploration and learning dynamics. Our work points the way for future efforts to build more detailed behavioural models as well as more human-like learning of complex tasks in artificial intelligence systems.