Reinforcement learning and counterfactual reasoning explain adaptive behavior in a changing environment

Yunfeng ZhangUniversity of Oregon, Eugene, Oregon, United States
Jaehyon PaikPalo Alto Research Center (PARC)
Peter PirolliPalo Alto Research Center (PARC)

Abstract

Animals routinely adapt to changes in the environment in order to survive. Though reinforcement learning may play a role in such adaption, it is not clear that it is the only mechanism involved, as it is not well suited to producing rapid, relatively immediate changes in strategy in response to environmental changes. We explored the possible adaptive mechanisms underlying in a cognitive model of human behavior in a change detection experiment. Besides reinforcement learning, the model incorporates counterfactual reasoning to help learn the utility of different task strategies under different environmental conditions. The results show that the model can accurately explain human data and that counterfactual reasoning is key to reproducing the various effects observed in this change detection paradigm.

Files

Reinforcement learning and counterfactual reasoning explain adaptive behavior in a changing environment (0.9 MB)



Back to Table of Contents