CaMR: Towards Connotation-Aware Music Retrieval on Social Media with Visual Inputs

Lanyu Shang, Zhang Daniel Yue, Khan Siamul Karim, Jialie Shen, Dong Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With the ubiquitous network connectivity and the proliferation of mobile devices, people are increasingly consuming digital contents from social media driven music sharing platforms (e.g., YouTube, Soundcloud). In this paper, we study a novel problem of connotation-Aware music retrieval that focuses on the connotation which expresses the implicit feeling or emotion beyond the explicit content in artworks. Our goal is to automatically retrieve relevant music on social media based on the connotation of visual inputs (e.g., images, photos) provided by the users. The problem is challenging as it requires the accurate identification of the implicit connotation from both images and music pieces, and the precise matching of the identified connotation across different data modalities. We develop a connotation-Aware music retrieval (CaMR) framework to address the above challenges. Evaluation results from a real-world social media dataset demonstrate that the CaMR framework can retrieve music that is highly relevant to the connotation of the input image.

Original languageEnglish
Title of host publicationProceedings of the 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2020)
EditorsMartin Atzmuller, Michele Coscia, Rokia Missaoui
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages425-429
Number of pages5
ISBN (Electronic)9781728110561
DOIs
Publication statusPublished - 24 Mar 2021
Event12th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2020 - Virtual, Online, Netherlands
Duration: 07 Dec 202010 Dec 2020

Publication series

NameProceedings of the 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2020
PublisherIEEE
ISSN (Print)2473-9928
ISSN (Electronic)2473-991X

Conference

Conference12th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2020
CountryNetherlands
CityVirtual, Online
Period07/12/202010/12/2020

Bibliographical note

Funding Information:
This research is supported in part by the National Science Foundation under Grant No. CNS-1845639, CNS-1831669, Army Research Office under Grant W911NF-17-1-0409. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.

Publisher Copyright:
© 2020 IEEE.

Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Information Systems and Management
  • Social Psychology
  • Communication

Fingerprint

Dive into the research topics of 'CaMR: Towards Connotation-Aware Music Retrieval on Social Media with Visual Inputs'. Together they form a unique fingerprint.

Cite this