Self-supervised multi-object tracking with cycle-consistency

Yuanhang Yin, Yang Hua, Tao Song, Ruhui Ma*, Haibing Guan

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Multi-object tracking is a challenging video task that requires both locating the objects in the frames and associating the objects among the frames, which usually utilizes the tracking-by-detection paradigm. Supervised multi-object tracking methods have made stunning progress recently, however, the expensive annotation costs for bounding boxes and track ID labels limit the robustness and generalization ability of these models. In this paper, we learn a novel multi-object tracker using only unlabeled videos by designing a self-supervisory learning signal for an association model. Specifically, inspired by the cycle-consistency used in video correspondence learning, we propose to track the objects forwards and backwards, i.e., each detection in the first frame is supposed to be matched with itself after the forward-backward tracking. We utilize this cycle-consistency as the self-supervisory learning signal for our proposed multi-object tracker. Experiments conducted on the MOT17 dataset show that our model is effective in extracting discriminative association features, and our tracker achieves competitive performance compared to other trackers using the same pre-generated detections, including UNS20 [1], Tracktor++ [2], FAMNet [8], and CenterTrack [31].

Original languageEnglish
Title of host publicationMultiMedia Modeling - 29th International Conference, MMM 2023, Proceedings
EditorsDuc-Tien Dang-Nguyen, Cathal Gurrin, Alan F. Smeaton, Martha Larson, Stevan Rudinac, Minh-Son Dao, Christoph Trattner, Phoebe Chen
PublisherSpringer Science and Business Media Deutschland GmbH
Pages483-495
Number of pages13
Volume13834
ISBN (Electronic)9783031278181
ISBN (Print)9783031278174
DOIs
Publication statusPublished - 31 Mar 2023
Event29th International Conference on MultiMedia Modeling, MMM 2023 - Bergen, Norway
Duration: 09 Jan 202312 Jan 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13834 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference29th International Conference on MultiMedia Modeling, MMM 2023
Country/TerritoryNorway
CityBergen
Period09/01/202312/01/2023

Keywords

  • Cycle-consistency
  • Multi-object Tracking
  • Self-supervised learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Self-supervised multi-object tracking with cycle-consistency'. Together they form a unique fingerprint.

Cite this