Visual and Semantic Feature Spaces for Zero-shot Image Decoding

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Downloads (Pure)

Abstract

Large image galleries are notoriously difficult to categorise and navigate without a great deal of manual organisation. We present a new methodology that could enable a Brain-Computer Interface (BCI) system for image retrieval, as an alternative to the automated tagging approaches recently emerging. We propose a regression framework that allows us to effectively map EEG signals into a visuo-semantic feature space, where the stimulus image can be matched. We use a set of intermediate features ranging from low-level edge detection, to the semantic properties of an image’s depicted content, reflecting theoretical and empirical knowledge of how the brain processes visual input and extracts meaning from it. A real-world system would need to generalise to images not seen during training, so we use zero-shot learning to decode image features from brain activity. Using our approach under this challenging setting, we were able to decode left-out individual stimuli at a rate significantly above chance.
Original languageEnglish
Title of host publicationIrish Machine Vision and Image Processing Conference Proceedings 2018: Proceedings
PublisherIrish Pattern Recognition & Classification Society
Pages25-32
ISBN (Print)978-0-9934207-3-3
Publication statusPublished - 2018
EventIrish Machine Vision and Image Processing Conference - Ulster University, Belfast, United Kingdom
Duration: 29 Aug 201831 Aug 2018
Conference number: 2018

Conference

ConferenceIrish Machine Vision and Image Processing Conference
Abbreviated titleIMVIP
CountryUnited Kingdom
CityBelfast
Period29/08/201831/08/2018

Fingerprint Dive into the research topics of 'Visual and Semantic Feature Spaces for Zero-shot Image Decoding'. Together they form a unique fingerprint.

  • Student Theses

    Optimising Image Feature Zero-Shot-Learning with EEG

    Author: McCartney, B., Jul 2020

    Supervisor: Martinez del Rincon, J. (Supervisor) & Devereux, B. (Supervisor)

    Student thesis: Doctoral ThesisDoctor of Philosophy

    Cite this

    McCartney, B., Martinez del Rincon, J., Devereux, B., & Murphy, B. (2018). Visual and Semantic Feature Spaces for Zero-shot Image Decoding. In Irish Machine Vision and Image Processing Conference Proceedings 2018: Proceedings (pp. 25-32). Irish Pattern Recognition & Classification Society. https://iprcs.scss.tcd.ie/pdf/IMVIP2018Book.pdf#page=35