There is growing evidence that self-directed learning--in which a person chooses what information to experience--is more successful than passive learning across a range of tasks. One explanation for this advantage is that self-directed learners focus on information that is useful for reducing their uncertainty. However, the process by which uncertainty leads to sampling decisions is unclear. While previous work has focused on supporting a single normative model of sampling, we present evidence that such decisions reflect evolving demands over the course of learning. As participants learned to differentiate perceptual categories we measured their uncertainty in how to classify items they sampled, and find distinct shifts in the type of information they choose to collect as learning progresses. Our results highlight the importance of evaluating models of rational information acquisition (and claims of confirmation bias) in the context of sequentially extended, self-directed learning tasks.