Much problem-solving research has investigated if and why `two heads are better than one', but typically posits that any process gain is due to exposure to ideas provided by another person. This fails to acknowledge or investigate what the interaction itself contributes. Using an online version of the Alternative Uses Task, we compare situations in which people are passively exposed to what is said in a dialogue, or actively engaged in the dialogue, thus varying interactivity independently of informational content. Interacting participants produce more turns overall, but do not come up with more ideas. They also build on each other's ideas more and produce more complex ideas when they do so; following leads to elaboration -- but only with genuine interactivity. These results indicate that conversational mechanisms promote the exploration of a problem space and that merely counting the number of ideas would miss the importance of the interaction itself.