A zero-shot learning approach to the development of brain-computer interfaces for image retrieval

Ben McCartney*, Jesus Martinez-Del-Rincon, Barry Devereux, Brian Murphy

*Corresponding author for this work

Research output: Contribution to journalArticle

106 Downloads (Pure)

Abstract

Brain decoding—the process of inferring a person’s momentary cognitive state from their brain activity—has enormous potential in the field of human-computer interaction. In this study we propose a zero-shot EEG-to-image brain decoding approach which makes use of state-of-the-art EEG preprocessing and feature selection methods, and which maps EEG activity to biologically inspired computer vision and linguistic models. We apply this approach to solve the problem of identifying viewed images from recorded brain activity in a reliable and scalable way. We demonstrate competitive decoding accuracies across two EEG datasets, using a zero-shot learning framework more applicable to real-world image retrieval than traditional classification techniques.

Original languageEnglish
Article numbere0214342
JournalPLoS ONE
Volume14
Issue number9
Early online date16 Sep 2019
DOIs
Publication statusPublished - 2019

Keywords

  • brain-computer interfaces
  • EEG
  • deep learning

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)

Fingerprint Dive into the research topics of 'A zero-shot learning approach to the development of brain-computer interfaces for image retrieval'. Together they form a unique fingerprint.

  • Student Theses

    Optimising Image Feature Zero-Shot-Learning with EEG

    Author: McCartney, B., Jul 2020

    Supervisor: Martinez del Rincon, J. (Supervisor) & Devereux, B. (Supervisor)

    Student thesis: Doctoral ThesisDoctor of Philosophy

    Cite this