The effect of language on perception of the visual world is an ongoing debate among researchers. According to one account, labels facilitate categorization by highlighting commonalities among labeled objects. Yet according to another account, early in development labels attenuate categorization by interfering with visual processing. In the current study, 4-year-old children were trained on two contrasting categories that were either labeled or presented in silence. Children were trained to discriminate the categories by associating them with a target object (Experiment 1) or with a target label (Experiment 2). Results demonstrated that children were more likely to learn the visual categories when images were presented in silence than when labeled. Furthermore, there was no evidence that children in the label condition reliably learned the categories, casting doubt on the idea that labels facilitate category learning in children.