With the ubiquitous network connectivity and the proliferation of mobile devices, people are increasingly consuming digital contents from social media driven music sharing platforms (e.g., YouTube, Soundcloud). In this paper, we study a novel problem of connotation-Aware music retrieval that focuses on the connotation which expresses the implicit feeling or emotion beyond the explicit content in artworks. Our goal is to automatically retrieve relevant music on social media based on the connotation of visual inputs (e.g., images, photos) provided by the users. The problem is challenging as it requires the accurate identification of the implicit connotation from both images and music pieces, and the precise matching of the identified connotation across different data modalities. We develop a connotation-Aware music retrieval (CaMR) framework to address the above challenges. Evaluation results from a real-world social media dataset demonstrate that the CaMR framework can retrieve music that is highly relevant to the connotation of the input image.
|Title of host publication||Proceedings of the 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2020)|
|Editors||Martin Atzmuller, Michele Coscia, Rokia Missaoui|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||5|
|Publication status||Published - 24 Mar 2021|
|Event||12th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2020 - Virtual, Online, Netherlands|
Duration: 07 Dec 2020 → 10 Dec 2020
|Name||Proceedings of the 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2020|
|Conference||12th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2020|
|Period||07/12/2020 → 10/12/2020|
Bibliographical noteFunding Information:
This research is supported in part by the National Science Foundation under Grant No. CNS-1845639, CNS-1831669, Army Research Office under Grant W911NF-17-1-0409. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.
© 2020 IEEE.
Copyright 2021 Elsevier B.V., All rights reserved.
ASJC Scopus subject areas
- Artificial Intelligence
- Computer Networks and Communications
- Information Systems and Management
- Social Psychology