Grayscale-Thermal Tracking via Inverse Sparse Representation-Based Collaborative Encoding

Bin Kang, Dong Liang*, Wan Ding, Huiyu Zhou, Wei Ping Zhu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


Grayscale-thermal tracking has attracted a great deal of attention due to its capability of fusing two different yet complementary target observations. Existing methods often consider extracting the discriminative target information and exploring the target correlation among different images as two separate issues, ignoring their interdependence. This may cause tracking drifts in challenging video pairs. This paper presents a collaborative encoding model called joint correlation and discriminant analysis based inver-sparse representation (JCDA-InvSR) to jointly encode the target candidates in the grayscale and thermal video sequences. In particular, we develop a multi-objective programming to integrate the feature selection and the multi-view correlation analysis into a unified optimization problem in JCDA-InvSR, which can simultaneously highlight the special characters of the grayscale and thermal targets through alternately optimizing two aspects: the target discrimination within a given image and the target correlation across different images. For robust grayscale-thermal tracking, we also incorporate the prior knowledge of target candidate codes into the SVM based target classifier to overcome the overfitting caused by limited training labels. Extensive experiments on GTOT and RGBT234 datasets illustrate the promising performance of our tracking framework.

Original languageEnglish
Pages (from-to)3401-3415
Number of pages15
JournalIEEE Transactions on Image Processing
Publication statusPublished - 24 Dec 2020
Externally publishedYes

Bibliographical note

Funding Information:
Manuscript received May 15, 2019; revised October 5, 2019; accepted December 4, 2019. Date of publication December 24, 2019; date of current version January 30, 2020. This work was supported in part by the National Key R&D Program of China under Grant 2017YFB0802300, in part by the National Natural Science Foundation of China (NSFC) under Grant 61801242, Grant 61571240, Grant 61601223, Grant 61871235, Grant 61876093, and Grant 61802206, the National Science Foundation (NSF) of Jiangsu Province under Grant BK20170915, Grant BK20181393, and Grant BK20161072, and in part by the Priority Academic Program Development of Jiangsu Higher Education Institutions. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Mireille Boutin. (Corresponding author: Dong Liang.) B. Kang is with the Department of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210023, China.

Publisher Copyright:
© 1992-2012 IEEE.

Copyright 2020 Elsevier B.V., All rights reserved.


  • discriminant analysis
  • feature selection
  • Grayscale-thermal tracking
  • inverse sparse representation

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Grayscale-Thermal Tracking via Inverse Sparse Representation-Based Collaborative Encoding'. Together they form a unique fingerprint.

Cite this