Balancing Prediction and Sensory Input in Speech Comprehension: The Spatiotemporal Dynamics of Word Recognition in Context

Spoken word recognition in context is remarkably fast and accurate, with recognition times of around 200ms, typically well before the end of the word. The neurocomputational mechanisms underlying these contextual effects are still poorly understood. This study combines source-localised electro- and magnetoencephalographic (EMEG) measures of real-time brain activity with multivariate Representational Similarity Analysis (RSA) to determine directly the timing and computational content of the processes evoked as spoken words are heard in context, and to evaluate the respective roles of bottom-up and predictive processing mechanisms in the integration of sensory and contextual constraints. Male and female human participants heard simple (modifier-noun) English phrases that varied in the degree of semantic constraint that the modifier (W1) exerted on the noun (W2), as in pairs like yellow banana We used gating tasks to generate estimates of the probabilistic predictions generated by these constraints as well as measures of their interaction with the bottom-up perceptual input for W2. RSA models of these measures were tested against EMEG brain data across a bilateral fronto-temporo-parietal language network. Consistent with probabilistic predictive processing accounts we found early activation of semantic constraints in frontal cortex (LBA45) as W1 was heard. The effects of these constraints (at 100ms post W2 onset in L middle temporal gyrus and at 140ms in L Heschl's gyrus) were only detectable, however, after the initial phonemes of W2 had been heard. Within an overall predictive processing framework, bottom-up sensory inputs are still required to achieve early and robust spoken word recognition in context.

SIGNIFICANCE STATEMENT

Human listeners recognise spoken words in natural speech contexts with remarkable speed and accuracy, often identifying a word well before all of it has been heard. In this study we investigate the brain systems that support this important capacity, using neuroimaging techniques that can track real-time brain activity during speech comprehension. This makes it possible to locate the brain areas that generate predictions about upcoming words and to show how these expectations are integrated with the evidence provided by the speech being heard. We use the timing and localisation of these effects to provide the most specific account to date of how the brain achieves an optimal balance between prediction and sensory input in the interpretation of spoken language.