BrainMagick

This is the code for the Decoding perceived speech from non-invasive brain recordings
paper, published at Nature Machine Intelligence in 2023. In this work, we present a novel method to perform high performance decoding of perceived speech from non invasive recordings. Inspired by CLIP, we use a contrastive loss between a learnt representation of the brain signals (EEG or MEG) and a representation Wav2vec 2.0 of candidate audio segments. We achieve up to a top-1 accuracy of 44% on the Gwilliams dataset.

For an up to date version, checkout our repository facebookresearch/brainmagick.