The ‘sense of agency’—the feeling of doing—is a phenomenological experience that cannot be taken for granted in infants. Researching the development of this phenomenon hinges on its conceptualization. Although it may seem natural to ask “When do infants have sense of agency?”, this binary view on the presence (or absence) of a sense of agency seems conceptually problematic. Cognitive phenomenological research reveals that sense of agency is a complex set of agentive experiences with different contents (e.g. ‘the experience of acting’, ‘the experience of volition’) that are often conflated. Given that different contents presuppose different representational capacities, developmental psychology may better target the question “What senses of agency can infants have at different developmental stages?” We show how this re-conceptualization impacts the interpretation of existing empirical findings in infant research, and how it can improve our understanding of the developmental trajectory of infants’ experiences of agency.