Texture Classification Using Pair-Wise Difference Pooling-Based Bilinear Convolutional Neural Networks

Xinghui Dong*, Huiyu Zhou, Junyu Dong

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


Texture is normally represented by aggregating local features based on the assumption of spatial homogeneity. Effective texture features are always the research focus even though both hand-crafted and deep learning approaches have been extensively investigated. Motivated by the success of Bilinear Convolutional Neural Networks (BCNNs) in fine-grained image recognition, we propose to incorporate the BCNN with the Pair-wise Difference Pooling (i.e. BCNN-PDP) for texture classification. The BCNN-PDP is built on top of a set of feature maps extracted at a convolutional layer of the pre-trained CNN. Compared with the outer product used by the original BCNN feature set, the pair-wise difference not only captures the pair-wise relationship between two sets of features but also encodes the difference between each pair of features. Considering the importance of the gradient data to the representation of image structures, we further generalise the BCNN-PDP feature set to two sets of feature maps computed from the original image and its gradient magnitude map respectively, i.e. the Fused BCNN-PDP (F-BCNN-PDP) feature set. In addition, the BCNN-PDP can be applied to two different CNNs and is referred to as the Asymmetric BCNN-PDP (A-BCNN-PDP). The three PDP-based BCNN feature sets can also be extracted at multiple scales. Since the dimensionality of the BCNN feature vectors is very high, we propose a new yet simple Block-wise PCA (BPCA) method in order to derive more compact feature vectors. The proposed methods are tested on seven different datasets along with 21 baseline feature sets. The results show that the proposed feature sets are superior, or at least comparable, to their counterparts across different datasets.

Original languageEnglish
Article number9181467
Pages (from-to)8776-8790
Number of pages15
JournalIEEE Transactions on Image Processing
Publication statusPublished - 31 Aug 2020

Bibliographical note

Funding Information:
Manuscript received July 8, 2019; revised May 3, 2020; accepted August 19, 2020. Date of publication August 31, 2020; date of current version September 8, 2020. The work of Xinghui Dong was supported by the Young Taishan Scholars Program. The work of Huiyu Zhou was supported in part by U.K. the Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/N011074/1, in part by the Royal Society-Newton Advanced Fellowship under Grant NA160342, and in part by the European Union’s Horizon 2020 Research and Innovation Program through the Marie-Sklodowska-Curie Grant 720325. The work of Junyu Dong was supported in part by the National Key Research and Development Program of China under Grant 2018AAA0100602 and in part by the National Natural Science Foundation of China under Grant U1706218. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Xiaoming Liu. (Corresponding authors: Xinghui Dong; Junyu Dong.) Xinghui Dong and Junyu Dong are with the Department of Computer Science, Ocean University of China, Qingdao 266100, China (e-mail: dongxinghui@gmail.com; dongjunyu@ouc.edu.cn).

Publisher Copyright:
© 1992-2012 IEEE.

Copyright 2020 Elsevier B.V., All rights reserved.


  • BCNNs
  • CNNs
  • texture
  • Texture classification
  • texture features

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Texture Classification Using Pair-Wise Difference Pooling-Based Bilinear Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this