Machine learning and electroencephalography for enhanced learning in human-computer interaction

Abstract

Human-computer interaction has become fundamental to modern society. Improvements in this field have extensive potential across a number of important application domains, such as the capacity to augment human efficiency in the industry sector, evolve technology consumption in recreational settings, and transform learning environments in education. A successful human-computer interface requires a complex synthesis of multiple sophisticated technologies. Despite a rich and diverse corpus of literature that investigates various components and approaches, there are many elements under exploited. In particular, electroencephalography and machine learning are powerful tools that can assist computer interpretation of human behaviour and amplify communication protocols more generally. This thesis looks at different techniques for leveraging electroencephalography and machine learning in human-computer interaction to improve pedagogical technology, utilising driving as a training scenario. The work investigates the relationships between neurological features, learning, memory, and task performance; making effective predictions of vehicle trajectory based on participant behaviour data; and explaining neural networks to elevate transparency of computer reasoning. Multiple contributions are made in these research areas. Specifically, the P300 is demonstrated as a marker of working memory in virtual training environments, extending understanding of this neural response. Then, delta and theta band activity are demonstrated as modulating with different aspects of driving performance, allowing improvements in behavioural analysis. Furthermore, integrating neurological data into time series recurrent neural networks is verified as a valid technique for improving vehicle trajectory predictions. Finally, a contribution is made to the AI explanation literature, whereby the genetic algorithm is implemented to explain video classifiers.

Date of Award	Jul 2023
Original language	English
Awarding Institution	Queen's University Belfast
Supervisor	Karen Rafferty (Supervisor), Youcheng Sun (Supervisor) & Seán McLoone (Supervisor)