Special Coverage

Beat Gestures Can Make You Hear Different Words (32)

Hans Rutger Bosker (Max Planck Institute for Psycholinguistics) and David Peeters (Max Planck Institute for Psycholinguistics & Tilburg University)

Summary by Brett Myers, Digital Content Associate Editor

This recap is part of a special series of session summaries from the Psychonomic Society's 61st Annual Meeting. To read the rest of the series, click here.

Your Gesturing Can Impact My Perception

Hans Rutger Bosker describes his work with David Peeters on beat gestures in communication.

He describes face-to-face communication as being filled with auditory and visual cues. Beat gestures are the visual cues that we make with body movements to accent prominent syllables in our speech. These gestures are tied very closely to speech prosody. He uses an example from a Boris Johnson speech to demonstrate this strong alignment.

Beat gestures have been shown to enhance the meta-linguistic processing of sentences, where gestures help listeners understand the meaning of an utterance. However, there’s been no evidence that gestures can affect the acoustic level of speech perception. Bosker introduces the question: can beat gestures influence the speech sounds that we hear?

To do this, they designed a “manual McGurk effect” task. In a traditional McGurk effect, the visual cue of lips can shape the sounds that we hear. With the manual version, Bosker asks if hand gestures can have a similar effect.

In one experiment, the researchers focused on Dutch pseudo-words, in which they could control the acoustic stress placement. They took an acoustically ambiguous version of the pseudo-words, and they manipulated the alignment of the beat gesture with either the first or second syllable. Participants heard stress that was associated with the gesture placement, even though the acoustics were identical across gestures.

In another experiment, they tested if gestures could influence the perception of vowel length. They used pseudo-words that were ambiguous for having short or long vowels in the first syllable. Again, they placed beat gestures on either the first or second syllable. They found that listeners perceived the vowel as shorter when the gesture was on the first syllable and longer when it was on the second syllable.

Therefore, they showed that beat gestures influence the perception of acoustic stress, as well as the auditory perception of vowels. So, remember … sanitize your hands, and be careful how you speak with them.

via GIPHY