Latent label mining for group activity recognition in basketball videos

Lifang Wu, Zeyu Li, Ye Xiang*, Meng Jian, Jialie Shen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
101 Downloads (Pure)


Motion information has been widely exploited for group activity recognition in sports video. However, in order to model and extract the various motion information between the adjacent frames, existing algorithms only use the coarse video-level labels as supervision cues. This may lead to the ambiguity of extracted features and the omission of changing rules of motion patterns that are also important sports video recognition. In this paper, a latent label mining strategy for group activity recognition in basketball videos is proposed. The authors' novel strategy allows them to obtain the latent labels set for marking different frames in an unsupervised way, and build the frame-level and video-level representations with two separate levels of supervision signal. Firstly, the latent labels of motion patterns are digged using the unsupervised hierarchical clustering technique. The generated latent labels are then taken as the frame-level supervision signal to train a deep CNN for the frame-level features extraction. Lastly, the frame-level features are fed into an LSTM network to build the spatio-temporal representation for group activity recognition. Experimental results on the public NCAA dataset demonstrate that the proposed algorithm achieves state-of-the-art performance.

Original languageEnglish
JournalIET Image Processing
Early online date23 Jul 2021
Publication statusEarly online date - 23 Jul 2021

Bibliographical note

Funding Information:
This work was supported in part by the National Natural Science Foundation of China (61976010, 61802011, 61702022), Beijing Municipal Education Committee Science Foundation (KM201910005024), and Beijing University of Technology Ri Xin Cultivation Project.

Publisher Copyright:
© 2021 The Authors. IET Image Processing published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology

Copyright 2021 Elsevier B.V., All rights reserved.



ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering


Dive into the research topics of 'Latent label mining for group activity recognition in basketball videos'. Together they form a unique fingerprint.

Cite this