A Perception-Inspired Deep Learning Framework for Predicting Perceptual Texture Similarity

Ying Gao, Yanhai Gan, Lin Qi, Huiyu Zhou, Xinghui Dong, Junyu Dong*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Similarity learning plays a fundamental role in the fields of multimedia retrieval and pattern recognition. Prediction of perceptual similarity is a challenging task as in most cases we lack human labeled ground-truth data and robust models to mimic human visual perception. Although in the literature, some studies have been dedicated to similarity learning, they mainly focus on the evaluation of whether or not two images are similar, rather than prediction of perceptual similarity which is consistent with human perception. Inspired by the human visual perception mechanism, we here propose a novel framework in order to predict perceptual similarity between two texture images. Our proposed framework is built on the top of Convolutional Neural Networks (CNNs). The proposed framework considers both powerful features and perceptual characteristics of contours extracted from the images. The similarity value is computed by aggregating resemblances between the corresponding convolutional layer activations of the two texture maps. Experimental results show that the predicted similarity values are consistent with the human-perceived similarity data.

Original languageEnglish
Pages (from-to)3714-3726
Number of pages13
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number10
Publication statusPublished - 30 Sep 2020
Externally publishedYes

Bibliographical note

Funding Information:
Manuscript received June 12, 2019; revised September 10, 2019; accepted September 25, 2019. Date of publication September 30, 2019; date of current version October 2, 2020. The work of J. Dong was supported in part by the National Key Research and development Program of China under Grant 2018AAA0100602, and in part by the National Natural Science Foundation of China (NSFC) under Grant 41576011, Grant U1706218, and Grant 41927805. The work of H. Zhou was supported in part by the U.K. Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/N011074/1, in part by the Royal Society-Newton Advanced Fellowship under Grant NA160342, and in part by the European Unions Horizon 2020 Research and Innovation Program through the Marie-Sklodowska-Curie under Grant 720325. The work of L. Qi was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 61501417. This article was recommended by Associate Editor H. Lu. (Corresponding author: Junyu Dong.) Y. Gao, Y. Gan, L. Qi, and J. Dong are with the Department of Information Science and Technology, Ocean University of China, Qingdao 266100, China (e-mail: dongjunyu@ouc.edu.cn).

Publisher Copyright:
© 1991-2012 IEEE.

Copyright 2020 Elsevier B.V., All rights reserved.


  • convolutional neural networks
  • perceptual similarity
  • Similarity learning
  • texture similarity

ASJC Scopus subject areas

  • Media Technology
  • Electrical and Electronic Engineering


Dive into the research topics of 'A Perception-Inspired Deep Learning Framework for Predicting Perceptual Texture Similarity'. Together they form a unique fingerprint.

Cite this