Visual and Semantic Feature Spaces for Zero-shot Image Decoding

Benjamin McCartney, Jesus Martinez del Rincon, Barry Devereux, Brian Murphy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

63 Downloads (Pure)

Abstract

Large image galleries are notoriously difficult to categorise and navigate without a great deal of manual organisation. We present a new methodology that could enable a Brain-Computer Interface (BCI) system for image retrieval, as an alternative to the automated tagging approaches recently emerging. We propose a regression framework that allows us to effectively map EEG signals into a visuo-semantic feature space, where the stimulus image can be matched. We use a set of intermediate features ranging from low-level edge detection, to the semantic properties of an image’s depicted content, reflecting theoretical and empirical knowledge of how the brain processes visual input and extracts meaning from it. A real-world system would need to generalise to images not seen during training, so we use zero-shot learning to decode image features from brain activity. Using our approach under this challenging setting, we were able to decode left-out individual stimuli at a rate significantly above chance.
Original languageEnglish
Title of host publicationIrish Machine Vision and Image Processing Conference Proceedings 2018: Proceedings
PublisherIrish Pattern Recognition & Classification Society
Pages25-32
ISBN (Print)978-0-9934207-3-3
Publication statusPublished - 2018
Event20th Irish Machine Vision and Image Processing Conference 2018 - Ulster University, Belfast, United Kingdom
Duration: 29 Aug 201831 Aug 2018

Conference

Conference20th Irish Machine Vision and Image Processing Conference 2018
Abbreviated titleIMVIP 2018
Country/TerritoryUnited Kingdom
CityBelfast
Period29/08/201831/08/2018

Fingerprint

Dive into the research topics of 'Visual and Semantic Feature Spaces for Zero-shot Image Decoding'. Together they form a unique fingerprint.

Cite this