Optimising image feature zero-shot-learning with EEG

  • Ben McCartney

Student thesis: Doctoral ThesisDoctor of Philosophy


In this thesis, brain-computer interfaces (BCIs) are proposed as the future of our communication with machines. Once our thoughts can be directly understood and actioned, huge gains in terms of productivity and usability from consumer to professional applications can be envisioned. However, in its current state, BCI systems are limited in use to research and medical applications (in assistive technologies and in diagnoses).

This project presents a BCI system for image retrieval where the system is able to identify the specific image that the user is looking at. In contrast to the majority of BCI research, we contribute work under several real world restrictions. Namely, our processing must be fully automated, use a recording technology available to consumers, and should be able to search for images not present during training, i.e. we apply zero-shot learning.

To achieve this goal, we first investigate a discriminative feature extraction process for EEG brain data and establish a baseline evaluation of exemplar and category-level decoding. Once competitive decoding rates are achieved, we extend this framework to address our zero-shot restriction by making use of computer vision, linguistic, and traditional machine learning techniques. Finally, we demonstrate improvement over these results by designing a set of metric-learning-based deep neural network architectures, representing a more data-driven and state-of-the-art approach. This results in the first time a multi-modal network based on metric learning has been applied to BCI image retrieval. Our network resulted in a peak decoding rate of 5.85\% which we demonstrate is well situated against existing studies in order to give context in a field where appropriate result comparison is difficult.
Date of AwardJul 2020
Original languageEnglish
Awarding Institution
  • Queen's University Belfast
SupervisorJesus Martinez-del-Rincon (Supervisor) & Barry Devereux (Supervisor)


  • EEG
  • Machine Learning
  • BCI
  • Zero-shot Learning

Cite this