Language has been shown to influence the ability to form categories. Here we investigate whether linguistic labels are privileged compared to other types of cues (e.g., numbers or symbols), and whether labels exert their effects regardless of whether they are introduced intentionally. In a categorization task, we found that adults were more likely to use labels to determine category boundaries compared to numbers or symbols, and that these effects persisted in all intentionality manipulations. These findings suggest that labels have a powerful effect on categorization compared to other cues; most strikingly, labels (but not other cues) are used during categorization even when people are specifically asked to ignore them. These results provide novel support for the position that labels indicate category membership.