Narratives: fMRI data for evaluating models of naturalistic language comprehension

How does the human brain construct narratives from a sequence of spoken words? Here we present a benchmark fMRI dataset for evaluating neural models for naturalistic speech perception and narrative comprehension. The ‘Narratives’ collection comprises over 500 scanning runs across more than 300 subjects. The story stimuli comprise over 20 spoken narratives ranging from 3 minutes to ~1 hour in duration, for ~4.5 hours of unique stimuli. Overall, this yields over 300,000 TRs across all subjects and stories, or ~5 days of fMRI data. All data and materials have been standardized and staged for public release. The stories span a variety of media, including commercially-produced radio and internet broadcasts, authors reading written works, professional storytellers performing in front of live audiences, and experimental subjects verbally recalling previous events. Alongside the stimuli, we provide time-stamped word-level transcripts created using a semi-supervised forced-alignment algorithm. MRI data have been organized according to the machine-readable Brain Imaging Data Structure (BIDS) with exhaustive metadata. Anonymized subject labels are linked across stories and include demographic and behavioral variables including age, gender, group or condition, and comprehension score (where available). MRI data are provided with various levels of preprocessing, including volumetric and surface-based spatial normalization, spatial smoothing, and temporal filtering with confound regressors. To more effectively aggregate data across subjects and stories, we developed a variant of hyperalignment that capitalizes on intersubject functional correlation (ISFC) analysis to derive a single connectivity-based shared response space across disjoint datasets. To validate the quality of the data, we applied intersubject correlation (ISC) analysis to each dataset. First, we measured ISCs in early auditory cortex to evaluate the temporal alignment of responses and identify outliers. Next, we performed whole-brain ISC analysis, revealing consistent responses throughout a network of cortical areas supporting language and event representation. Well-curated, naturalistic data have tremendous potential for re-use, and we hope the community will benefit from these data in disentangling the neural mechanisms of language comprehension.