Evidence for communicative compensation in debt advice with reduced multimodality

Nicole Andelic, Aidan Feeney, Gary McKeown

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Downloads (Pure)

Abstract

Research has found that professional advice with empathy displays and signs of listening lead to more successful outcomes. These skills are typically displayed through visual nonverbal signals, whereas reduced multimodal contexts have to use other strategies to compensate for the lack of visual nonverbal information. Debt advice is often a highly emotional scenario but to date there has been no research comparing fully multimodal with reduced multimodal debt advice. The aim of the current study was to compare explicit emotional content (as expressed verbally) and implicit emotional content (as expressed through paralinguistic cues) in face to face (FTF) and telephone debt advice recordings. Twenty-two debt advice recordings were coded as emotional or functional and processed through emotion recognition software. The analysis found that FTF recordings included more explicit emotion than telephone recordings did. However, linear mixed effects modelling found substantially higher levels of arousal and slightly lower levels of valence in telephone advice. Interaction analyses found that emotional speech in FTF advice was characterised by lower levels of arousal than during functional speech, whereas emotional speech in telephone advice had higher levels of arousal than in functional speech. We can conclude that there are differences in emotional content when comparing full and reduced multimodal debt advice. Furthermore, as telephone advice cannot avail of visual nonverbal signals, it seems to compensate by using nonverbal cues present in the voice.
Original languageEnglish
Title of host publicationICMI '19 2019 International Conference on Multimodal Interaction
PublisherAssociation for Computing Machinery (ACM)
Pages210-219
ISBN (Electronic)978-1-4503-6860-5
DOIs
Publication statusPublished - 14 Oct 2019

Fingerprint

Telephone
Arousal
Cues
Emotions
Research
Software

Cite this

Andelic, N., Feeney, A., & McKeown, G. (2019). Evidence for communicative compensation in debt advice with reduced multimodality. In ICMI '19 2019 International Conference on Multimodal Interaction (pp. 210-219). Association for Computing Machinery (ACM). https://doi.org/10.1145/3340555.3353757
Andelic, Nicole ; Feeney, Aidan ; McKeown, Gary. / Evidence for communicative compensation in debt advice with reduced multimodality. ICMI '19 2019 International Conference on Multimodal Interaction. Association for Computing Machinery (ACM), 2019. pp. 210-219
@inproceedings{ce43f06195ca4854ac3e212d43a84260,
title = "Evidence for communicative compensation in debt advice with reduced multimodality",
abstract = "Research has found that professional advice with empathy displays and signs of listening lead to more successful outcomes. These skills are typically displayed through visual nonverbal signals, whereas reduced multimodal contexts have to use other strategies to compensate for the lack of visual nonverbal information. Debt advice is often a highly emotional scenario but to date there has been no research comparing fully multimodal with reduced multimodal debt advice. The aim of the current study was to compare explicit emotional content (as expressed verbally) and implicit emotional content (as expressed through paralinguistic cues) in face to face (FTF) and telephone debt advice recordings. Twenty-two debt advice recordings were coded as emotional or functional and processed through emotion recognition software. The analysis found that FTF recordings included more explicit emotion than telephone recordings did. However, linear mixed effects modelling found substantially higher levels of arousal and slightly lower levels of valence in telephone advice. Interaction analyses found that emotional speech in FTF advice was characterised by lower levels of arousal than during functional speech, whereas emotional speech in telephone advice had higher levels of arousal than in functional speech. We can conclude that there are differences in emotional content when comparing full and reduced multimodal debt advice. Furthermore, as telephone advice cannot avail of visual nonverbal signals, it seems to compensate by using nonverbal cues present in the voice.",
author = "Nicole Andelic and Aidan Feeney and Gary McKeown",
year = "2019",
month = "10",
day = "14",
doi = "10.1145/3340555.3353757",
language = "English",
pages = "210--219",
booktitle = "ICMI '19 2019 International Conference on Multimodal Interaction",
publisher = "Association for Computing Machinery (ACM)",
address = "United States",

}

Andelic, N, Feeney, A & McKeown, G 2019, Evidence for communicative compensation in debt advice with reduced multimodality. in ICMI '19 2019 International Conference on Multimodal Interaction. Association for Computing Machinery (ACM), pp. 210-219. https://doi.org/10.1145/3340555.3353757

Evidence for communicative compensation in debt advice with reduced multimodality. / Andelic, Nicole; Feeney, Aidan; McKeown, Gary.

ICMI '19 2019 International Conference on Multimodal Interaction. Association for Computing Machinery (ACM), 2019. p. 210-219.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Evidence for communicative compensation in debt advice with reduced multimodality

AU - Andelic, Nicole

AU - Feeney, Aidan

AU - McKeown, Gary

PY - 2019/10/14

Y1 - 2019/10/14

N2 - Research has found that professional advice with empathy displays and signs of listening lead to more successful outcomes. These skills are typically displayed through visual nonverbal signals, whereas reduced multimodal contexts have to use other strategies to compensate for the lack of visual nonverbal information. Debt advice is often a highly emotional scenario but to date there has been no research comparing fully multimodal with reduced multimodal debt advice. The aim of the current study was to compare explicit emotional content (as expressed verbally) and implicit emotional content (as expressed through paralinguistic cues) in face to face (FTF) and telephone debt advice recordings. Twenty-two debt advice recordings were coded as emotional or functional and processed through emotion recognition software. The analysis found that FTF recordings included more explicit emotion than telephone recordings did. However, linear mixed effects modelling found substantially higher levels of arousal and slightly lower levels of valence in telephone advice. Interaction analyses found that emotional speech in FTF advice was characterised by lower levels of arousal than during functional speech, whereas emotional speech in telephone advice had higher levels of arousal than in functional speech. We can conclude that there are differences in emotional content when comparing full and reduced multimodal debt advice. Furthermore, as telephone advice cannot avail of visual nonverbal signals, it seems to compensate by using nonverbal cues present in the voice.

AB - Research has found that professional advice with empathy displays and signs of listening lead to more successful outcomes. These skills are typically displayed through visual nonverbal signals, whereas reduced multimodal contexts have to use other strategies to compensate for the lack of visual nonverbal information. Debt advice is often a highly emotional scenario but to date there has been no research comparing fully multimodal with reduced multimodal debt advice. The aim of the current study was to compare explicit emotional content (as expressed verbally) and implicit emotional content (as expressed through paralinguistic cues) in face to face (FTF) and telephone debt advice recordings. Twenty-two debt advice recordings were coded as emotional or functional and processed through emotion recognition software. The analysis found that FTF recordings included more explicit emotion than telephone recordings did. However, linear mixed effects modelling found substantially higher levels of arousal and slightly lower levels of valence in telephone advice. Interaction analyses found that emotional speech in FTF advice was characterised by lower levels of arousal than during functional speech, whereas emotional speech in telephone advice had higher levels of arousal than in functional speech. We can conclude that there are differences in emotional content when comparing full and reduced multimodal debt advice. Furthermore, as telephone advice cannot avail of visual nonverbal signals, it seems to compensate by using nonverbal cues present in the voice.

U2 - 10.1145/3340555.3353757

DO - 10.1145/3340555.3353757

M3 - Conference contribution

SP - 210

EP - 219

BT - ICMI '19 2019 International Conference on Multimodal Interaction

PB - Association for Computing Machinery (ACM)

ER -

Andelic N, Feeney A, McKeown G. Evidence for communicative compensation in debt advice with reduced multimodality. In ICMI '19 2019 International Conference on Multimodal Interaction. Association for Computing Machinery (ACM). 2019. p. 210-219 https://doi.org/10.1145/3340555.3353757