Mapping design of movement sonification to enhance motor skill acquisition

Abstract

Learning new skilled actions can be conceptualised as tuning into the control information crucial to coordinating the successful completion of the to-be-learned movements. According to ecological psychology, this control information exists in the environment and learners gradually tune their movements and perception to pick up the relevant information variables. Therefore, it may be possible to provide the learner with additional information beyond what they would perceive when attempting the task by itself – augmented feedback; in the hope that this will guide their attention towards the control variables of the skill. The focus of this thesis is a form of such augmented feedback – sonification. The method digitally transforms movement information into sounds and has been shown to improve the learning of numerous movement-based skills. Since sonification is supposed to guide learners to the information crucial to the successful completion of the task then it naturally follows that the design of sonification will determine its effectivity. Yet very few studies in sonification research tested these essential aspects of sonification design – what variables to sonify and what sounds to use in sonification.

In a series of 4 motor learning studies, we attempted to address these design questions. The first study directly compared two types of sounds to try to start answering the question of what sounds to use in sonification. Two subsequent studies attempted to develop a framework for identifying candidate movement variables as input for sonification. Our last study utilised an already developed model-based approach and its use in other research to identify the crucial information and tested sonification of this information.

Despite no significant benefits of sonification as augmented feedback in these studies, we discuss the indications that sonification and sounds that are used might require a particular type of information to be effective – timing-related information that the auditory system is particularly suited to perceive. Moreover, it might be possible that sonification encourages exploration of the tasks and might lead to more robust improvements in performance when the feedback is withdrawn or when transferring the skill to other tasks. Overall, our results highlight the importance of careful consideration for both the type of information that is sonified and the type of sounds that deliver the information. Finally, we noted that we observed large differences between individuals and the need to take these differences into account through an advanced methodology and possibly sonification design. Taken together, an improved design of sonification could lead to even further and more effective application of sonification in sports or rehabilitation.

Date of Award	Dec 2023
Original language	English
Awarding Institution	Queen's University Belfast
Sponsors	Northern Ireland Department for the Economy
Supervisor	Matthew Rodger (Supervisor) & Joost C. Dessing (Supervisor)