Cross-Modal Grounding of Meaning

Kawai ChuiNational Chengchi University, Taipei City, Wenshan, Taiwan


The study investigates how the speaker grounds meaning for a referent by gestural repetition along with speech in conversation. Each stretch of talk encompasses the whole joint action during which similar gestures to depict the same referents are produced across turns. A particular cross-modal grounding strategy was found: The previous speaker’s gesture is replicated to form a semantic foundation shared by the participants; upon such common ground in gesture, the speaker readily conveys new meaning with a new lexical expression. The use of the cross-modal grounding strategy facilitates the expression of shared knowledge in gesture and new meaning in speech within a clause. It also bears out the bilateral process of speaking: During the construction of utterance, the speaker grounds new meaning by considering the addressee’s current state of knowledge as revealed by gestural repetition. In turn, the addressee informs the speaker about his/her acknowledgement of the newly established meaning.


Cross-Modal Grounding of Meaning (326 KB)

Back to Table of Contents