Children are drivers of their own discovery. To develop a complete characterization of the factors that drive exploration in early childhood, we must first understand how competing factors influence children's decision making. We investigated preschool-aged children's decision-making on explore-exploit tasks where the available information about the distribution of rewards was controlled. When probability information is unknown, children preferred to exploit known rewards over exploring unknown ones. However, performance in Experiment 2 shows that children can use probabilistic information to form accurate expectations about possible outcomes to effectively choose between exploiting and exploring. The degree to which individual children are ``exploratory'' is also shown to be consistent over weeks, suggesting that individual children have ``trait-like'' exploratory drives. On aggregate, children incorporate these individual tendencies towards exploration or exploitation with probability information; thus children readily form estimations of expected reward and use this information to guide efficient exploratory behavior.