Evidence for overt visual attention to hand gestures as a function of redundancy and speech disfluency

Abstract

We investigated the effect of gesture redundancy and speech disfluency on listeners’ fixations to gestures. Participants watched a speaker producing a redundant or non-redundant gesture, while producing fluent or disfluent speech. Eye movements were recorded. Participants spent little time on a speaker’s gestures regardless of condition. Gesture redundancy and speech disfluency did not affect listeners’ percentage dwell time to a speaker’s gestures. However, listeners were more likely to fixate to a speaker’s gestures when they expected the gesture to be non-redundant. Listeners were also more likely to fixate to a speaker’s gestures when the speaker was disfluent. Thus, listeners allocate overt visual attention based on the expected usefulness of a speaker’s gestures, although evidence does not suggest that they spend more time fixating on these gestures. Furthermore, listeners are sensitive to disfluency in a speaker’s utterance and change how they attend to gestures based on qualities of the speech.


Back to Table of Contents