Balancing exploration and exploitation is difficult, and across a wide variety of situations under-exploration of uncertain alternatives appears prevalent. We propose that one possible cause of under-exploration is present bias, whereby immediate rewards (like those gained from exploitation) loom larger than future rewards (like those gained from exploration). This possible cause of under-exploration is not addressed by past studies, in which choices generally yield token rewards that are converted to money at the end of the experiment, removing the inter-temporal aspect of the decision-making process. To address this issue, we developed an exploratory choice task with immediately-consumed rewards. We then tested whether whether imposing a temporal delay before the consumption of rewards increased exploration by decreasing present bias, and report on our results.