Object-Adaptive LSTM Network for Visual Tracking

Yihan Du, Yan Yan, Si Chen, Yang Hua, Hanzi Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)
839 Downloads (Pure)


Convolutional Neural Networks (CNNs) have shown outstanding performance in visual object tracking. However, most of classification-based tracking methods using CNNs are timeconsuming due to expensive computation of complex online finetuning and massive feature extractions. Besides, these methods suffer from the problem of over-fitting since the training and testing stages of CNN models are based on the videos from the same domain. Recently, matching-based tracking methods (such as Siamese networks) have shown remarkable speed superiority, while they cannot well address target appearance variations and complex scenes for inherent lack of online adaptability and background information. In this paper, we propose a novel object-adaptive LSTM network, which can effectively exploit sequence dependencies and dynamically adapt to the temporal object variations via constructing an intrinsic model for object appearance and motion. In addition, we develop an efficient strategy for proposal selection, where the densely sampled proposals are firstly pre-evaluated using the fast matching-based method and then the well-selected high-quality proposals are fed to the sequence-specific learning LSTM network. This strategy enables our method to adaptively track an arbitrary object and operate faster than conventional CNN-based classification tracking methods. To the best of our knowledge, this is the
first work to apply an LSTM network for classification in visual object tracking. Experimental results on OTB and TC-128 benchmarks show that the proposed method achieves state-ofthe-art performance, which exhibits great potentials of recurrent structures for visual object tracking.
Original languageEnglish
Title of host publicationInternational Conference on Pattern Recognition: Proceedings
ISBN (Electronic)978-1-5386-3788-3
Publication statusPublished - 29 Nov 2018

Publication series

NameInternational Conference on Pattern Recognition (ICPR): Proceedings
ISSN (Print)1051-4651


Dive into the research topics of 'Object-Adaptive LSTM Network for Visual Tracking'. Together they form a unique fingerprint.

Cite this