A child's first word is an important step towards language. Aggregated across children, the distribution of these first productive uses of language can act as a window into early cognitive and linguistic development. We investigate both the variability and predictability in children's first words across four new datasets. We find, first, that children's first words tend to emerge earlier than previously estimated: more than 75 percent of children produce their first word before their first birthday. Second, we find a high degree of consistency in the types of things children name in their first words, independent of the age at which they are produced. Finally, we show that the particular words that children produce first are predictable from two linguistic factors: input frequency and phonological complexity. Together, our results suggest a degree of independence between early conceptual and linguistic development.