Linking Language Perception and Production: Latest Insights and Future Directions
30 August 2022, 9 - 10:40 AM Central European Time

Held in conjunction with the
22nd European Society for Cognitive Psychology Conference (ESCoP)
29 August - 1 September 2022 | Lille, France

Organizers:

Efthymia (Effie) Kapnoula	Arthur Samuel
Basque Center on Cognition, Brain and Language, Spain   e.kapnoula@bcbl.edu	Basque Center on Cognition, Brain and Language, Spain  a.samuel@bcbl.edu

Description 
This symposium will bring together leading experts on different aspects of the relationship between language production and perception. Clara Martin will present compelling evidence that prediction during comprehension relies on the production system, while Victor Ferreira will further discuss how accurate and inaccurate predictions affect language processing in the context of a unifying perception/production framework. Martin Pickering will present data on joint language production, addressing the question of whether representations are shared between production and comprehension. Melissa Baese-Berk will also talk about the representations involved in production and perception, but in the context of perceptual learning of novel speech sounds, showing evidence for a disruptive role of production. Finally, Sophie Scott will delve into the neural systems that underlie speech perception and production, clarifying and constraining their relationship. This list of talks covers a broad range of topics, while maintaining high overlap, thus assuring an engaging and productive discussion.

Prediction Is Production: The Missing Link between Language Production and Comprehension 
Clara Martin 
Basque Center on Cognition, Brain and Language (BCBL) & Ikerbasque

Readers predict upcoming words during sentence processing, when the sentence context is highly constrained. Several frameworks propose that the production system is implicitly used for prediction during comprehension. We tested this argument by comparing two groups of readers. The critical group read constrained sentences while pronouncing the syllable /ta/ on each word, aimed to block the production system. The control group read the same sentences, while performing tongue-tapping. This 2 group was also performing double-tasking, but without taxing the production system. Half of the sentences contained the most expected target word, half of them an unexpected one. Electroencephalography was recorded, and Event-Related Potentials were measured, time-locked on the article preceding the target noun. The control group showed significant N400 effects on the preceding article, but not the critical group. Those results suggest that blocking the production system hinders prediction during sentence comprehension, which supports the claim that prediction is production. 

      Co-author(s):
       Francesca Branzi, University of Liverpool & Moshe Bar, Bar-Ilan University 

To Predict or Not To Predict, That Is the...Sandwich: Effects of Close Probability and Semantic Similarity on Failed Predictions  
Victor Ferreira
University of California, San Diego

Recently, much work has investigated prediction during language processing, in part because successful prediction is beneficial. Less work has investigated when predictions go wrong. Predicting wrongly can be costly, but the cost may depend on how wrong the prediction was. Here, we ask whether predicting almost-correctly is better than predicting completely incorrectly, and if so, if predicting almost correctly is better than not predicting at all. Results showed that when a predicted ending is replaced with a related picture, it is named faster than when it is replaced with an unrelated picture, but that related picture is not named more quickly than when it appears after a low-constraint sentence. Thus, predicting almost correctly is better than predicting completely incorrectly, but it's not better than not predicting at all. This carries implications for current accounts that unify production and comprehension via prediction. 

      Co-author(s):
       Julie Bannon, McMaster University & Tamar Gollan, University of California, San Diego 

Shared Representations Across Language Production and Comprehension: Evidence from Joint Language Production
Martin J. Pickering
University of Edinburgh

We introduce joint language production as the study of the mechanisms involved in producing language jointly with another real or assumed speaker, and use it to study whether representations are shared between production and comprehension. We discuss two series of experiments in which participants take longer to produce words (Gambi et al., 2015) and sentences (Gambi et al., 2021) if they believe they have a partner who is also producing words or sentences than if they do not. However, it does not seem to matter whether they believe their partner is producing the same word/syntactic structure as them or 3 not. We propose that people use the same format to represent their intention to speak and their partner’s intention to speak, and interpret this claim in terms of an account in which comprehenders predict speakers’ intentions with the mechanisms that they use to produce language (Pickering & Gambi, 2018).

      Co-author(s):
      Chiara Gambi, Cardiff University

The Disruption of Perceptual Learning After Production of Novel Speech Sounds 
Melissa Baese-Berk
University of Oregon

When naïve learners produce novel speech sounds, their perceptual learning of these sounds is disrupted, compared to learners who only hear, but do not produce these sounds (Baese-Berk, 2019; Baese-Berk & Samuel, 2016). Some of this work suggests that a combination of linguistic and broader cognitive factors impact the presence and size of this disruption; however, the specific underlying causes of this disruption remain unclear. In this talk, I will begin by establishing that a disruption to perceptual learning occurs for a wide variety of novel speech sounds, including stop consonants and lexical tone. Next, I will describe ongoing work which isolates specific lexical and cognitive factors that may contribute to this disruption. Finally, I will discuss next directions for this work, focusing specifically on the evolution over time of the perception-production relationship during speech sound learning.

What Are the Neural Links Between Speech Perception and Speech Production, and What Are Their Functions?  
Sophie Scott
University College London (UCL) - Institute of Cognitive Neuroscience

In this talk I will map out the basic neural systems that underpin speech perception and production networks. I will delineate speech perception networks in the dorsolateral temporal lobes, and examine how these link into production networks via caudal auditory pathways. I will demonstrate how these 4 relate functionally to aspects of speech production, for example in the detection of, and compensation for, distortions in the perceptual consequences of speech production (e.g. delayed auditory feedback). I will delineate speech production networks, with an emphasis on cortical fields involved in the controlled production of speech. I will show how primary motor cortex, premotor cortex and supplementary motor cortex are also recruited during perceptual tasks, and explore what these non-auditory fields may be contributing to speech processing. I will argue that we need a nuanced model of what these different links between speech perception and production networks contribute to spoken language processing.