Decoding the neural representation of story meanings across languages

Hum Brain Mapp. 2017 Dec;38(12):6096-6106. doi: 10.1002/hbm.23814. Epub 2017 Sep 20.

Abstract

Drawing from a common lexicon of semantic units, humans fashion narratives whose meaning transcends that of their individual utterances. However, while brain regions that represent lower-level semantic units, such as words and sentences, have been identified, questions remain about the neural representation of narrative comprehension, which involves inferring cumulative meaning. To address these questions, we exposed English, Mandarin, and Farsi native speakers to native language translations of the same stories during fMRI scanning. Using a new technique in natural language processing, we calculated the distributed representations of these stories (capturing the meaning of the stories in high-dimensional semantic space), and demonstrate that using these representations we can identify the specific story a participant was reading from the neural data. Notably, this was possible even when the distributed representations were calculated using stories in a different language than the participant was reading. Our results reveal that identification relied on a collection of brain regions most prominently located in the default mode network. These results demonstrate that neuro-semantic encoding of narratives happens at levels higher than individual semantic units and that this encoding is systematic across both individuals and languages. Hum Brain Mapp 38:6096-6106, 2017. © 2017 Wiley Periodicals, Inc.

Keywords: knowledge representation; language neuroscience; machine learning; natural language processing; semantics.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • Brain / diagnostic imaging
  • Brain / physiology*
  • Brain Mapping
  • Comprehension / physiology*
  • Culture
  • Female
  • Humans
  • Magnetic Resonance Imaging
  • Male
  • Multilingualism*
  • Narration*
  • Neuropsychological Tests
  • Pattern Recognition, Visual / physiology
  • Psycholinguistics
  • Reading*
  • Semantics*
  • Translating
  • Young Adult