A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video

Meng Ding; Ning Li; Ziang Song; Ruixing Zhang; Xiaxia Zhang; Huiyu Zhou

doi:10.1109/ICECE51594.2020.9353008

A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video

Meng Ding, Ning Li^*, Ziang Song, Ruixing Zhang, Xiaxia Zhang, Huiyu Zhou

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In recent year, due to motility and wide coverage, unmanned aerial vehicle (UAV) has been widely applied in surveillance system. Human action recognition in UAV video is essential for surveillance video understanding. However, existing action recognition methods suffer from heavy computing, which makes it hard to deploy in real applications. In this paper, a lightweight action recognition method for UAV video(LARMUV) is proposed. This method is based on TSN and adopt Mo-bileNetV3 as backbone, which greatly reduces amount of computing and parameters. Self-attention mechanism is adopted to capture temporal structure among different frames. For loss function, Focal Loss is used to putting more focus on hard, misclassified examples. Last but not least, knowledge distillation is employed to enhance the performance of our model, which transfer knowledge from a larger teacher model to student model. Experimental results on HMDB51, UCF101 and UAV dataset show that our method can achieve competitive performance compared to baseline methods while run in real-time mode.

Original language	English
Title of host publication	2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	181-185
Number of pages	5
ISBN (Electronic)	9781728192581
DOIs	https://doi.org/10.1109/ICECE51594.2020.9353008
Publication status	Published - 17 Feb 2021
Externally published	Yes
Event	3rd IEEE International Conference on Electronics and Communication Engineering, ICECE 2020 - Virtual, Xi'an, China Duration: 14 Dec 2020 → 16 Dec 2020

Publication series

Name	2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020

Conference

Conference	3rd IEEE International Conference on Electronics and Communication Engineering, ICECE 2020
Country	China
City	Virtual, Xi'an
Period	14/12/2020 → 16/12/2020

Bibliographical note

Funding Information:
This work received support from Science and Technology on Electro-optic Control Laboratory and Aviation Science Foundation Project (ASFC-20175152036) and Key Project on Artificial intelligence(1004-56XZA19008). The authors are also grateful for the support of their colleagues at the Key Laboratory of Radar Imaging and Microwave Photonics, Ministry of Education.

Publisher Copyright:
© 2020 IEEE.

Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.

Keywords

action recognition
MobileNetV3
self-attention
UAV

ASJC Scopus subject areas

Computer Networks and Communications
Hardware and Architecture
Electrical and Electronic Engineering

Access to Document

10.1109/ICECE51594.2020.9353008Licence: Unspecified

Fingerprint Dive into the research topics of 'A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video'. Together they form a unique fingerprint.

Cite this

Ding, M., Li, N., Song, Z., Zhang, R., Zhang, X., & Zhou, H. (2021). A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video. In 2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020 (pp. 181-185). [9353008] (2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICECE51594.2020.9353008

Ding, Meng ; Li, Ning ; Song, Ziang ; Zhang, Ruixing ; Zhang, Xiaxia ; Zhou, Huiyu. / A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video. 2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020. Institute of Electrical and Electronics Engineers Inc., 2021. pp. 181-185 (2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020).

@inproceedings{612d022d2d87424f8b998e7fa6fd382a,

title = "A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video",

abstract = "In recent year, due to motility and wide coverage, unmanned aerial vehicle (UAV) has been widely applied in surveillance system. Human action recognition in UAV video is essential for surveillance video understanding. However, existing action recognition methods suffer from heavy computing, which makes it hard to deploy in real applications. In this paper, a lightweight action recognition method for UAV video(LARMUV) is proposed. This method is based on TSN and adopt Mo-bileNetV3 as backbone, which greatly reduces amount of computing and parameters. Self-attention mechanism is adopted to capture temporal structure among different frames. For loss function, Focal Loss is used to putting more focus on hard, misclassified examples. Last but not least, knowledge distillation is employed to enhance the performance of our model, which transfer knowledge from a larger teacher model to student model. Experimental results on HMDB51, UCF101 and UAV dataset show that our method can achieve competitive performance compared to baseline methods while run in real-time mode. ",

keywords = "action recognition, MobileNetV3, self-attention, UAV",

author = "Meng Ding and Ning Li and Ziang Song and Ruixing Zhang and Xiaxia Zhang and Huiyu Zhou",

note = "Funding Information: This work received support from Science and Technology on Electro-optic Control Laboratory and Aviation Science Foundation Project (ASFC-20175152036) and Key Project on Artificial intelligence(1004-56XZA19008). The authors are also grateful for the support of their colleagues at the Key Laboratory of Radar Imaging and Microwave Photonics, Ministry of Education. Publisher Copyright: {\textcopyright} 2020 IEEE. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.; 3rd IEEE International Conference on Electronics and Communication Engineering, ICECE 2020 ; Conference date: 14-12-2020 Through 16-12-2020",

year = "2021",

month = feb,

day = "17",

doi = "10.1109/ICECE51594.2020.9353008",

language = "English",

series = "2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "181--185",

booktitle = "2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020",

}

Ding, M, Li, N, Song, Z, Zhang, R, Zhang, X & Zhou, H 2021, A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video. in 2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020., 9353008, 2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020, Institute of Electrical and Electronics Engineers Inc., pp. 181-185, 3rd IEEE International Conference on Electronics and Communication Engineering, ICECE 2020, Virtual, Xi'an, China, 14/12/2020. https://doi.org/10.1109/ICECE51594.2020.9353008

A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video. / Ding, Meng; Li, Ning; Song, Ziang; Zhang, Ruixing; Zhang, Xiaxia; Zhou, Huiyu.

2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020. Institute of Electrical and Electronics Engineers Inc., 2021. p. 181-185 9353008 (2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video

AU - Ding, Meng

AU - Li, Ning

AU - Song, Ziang

AU - Zhang, Ruixing

AU - Zhang, Xiaxia

AU - Zhou, Huiyu

N1 - Funding Information: This work received support from Science and Technology on Electro-optic Control Laboratory and Aviation Science Foundation Project (ASFC-20175152036) and Key Project on Artificial intelligence(1004-56XZA19008). The authors are also grateful for the support of their colleagues at the Key Laboratory of Radar Imaging and Microwave Photonics, Ministry of Education. Publisher Copyright: © 2020 IEEE. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.

PY - 2021/2/17

Y1 - 2021/2/17

N2 - In recent year, due to motility and wide coverage, unmanned aerial vehicle (UAV) has been widely applied in surveillance system. Human action recognition in UAV video is essential for surveillance video understanding. However, existing action recognition methods suffer from heavy computing, which makes it hard to deploy in real applications. In this paper, a lightweight action recognition method for UAV video(LARMUV) is proposed. This method is based on TSN and adopt Mo-bileNetV3 as backbone, which greatly reduces amount of computing and parameters. Self-attention mechanism is adopted to capture temporal structure among different frames. For loss function, Focal Loss is used to putting more focus on hard, misclassified examples. Last but not least, knowledge distillation is employed to enhance the performance of our model, which transfer knowledge from a larger teacher model to student model. Experimental results on HMDB51, UCF101 and UAV dataset show that our method can achieve competitive performance compared to baseline methods while run in real-time mode.

AB - In recent year, due to motility and wide coverage, unmanned aerial vehicle (UAV) has been widely applied in surveillance system. Human action recognition in UAV video is essential for surveillance video understanding. However, existing action recognition methods suffer from heavy computing, which makes it hard to deploy in real applications. In this paper, a lightweight action recognition method for UAV video(LARMUV) is proposed. This method is based on TSN and adopt Mo-bileNetV3 as backbone, which greatly reduces amount of computing and parameters. Self-attention mechanism is adopted to capture temporal structure among different frames. For loss function, Focal Loss is used to putting more focus on hard, misclassified examples. Last but not least, knowledge distillation is employed to enhance the performance of our model, which transfer knowledge from a larger teacher model to student model. Experimental results on HMDB51, UCF101 and UAV dataset show that our method can achieve competitive performance compared to baseline methods while run in real-time mode.

KW - action recognition

KW - MobileNetV3

KW - self-attention

KW - UAV

U2 - 10.1109/ICECE51594.2020.9353008

DO - 10.1109/ICECE51594.2020.9353008

M3 - Conference contribution

AN - SCOPUS:85102141849

T3 - 2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020

SP - 181

EP - 185

BT - 2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 3rd IEEE International Conference on Electronics and Communication Engineering, ICECE 2020

Y2 - 14 December 2020 through 16 December 2020

ER -

Ding M, Li N, Song Z, Zhang R, Zhang X, Zhou H. A Lightweight Action Recognition Method for Unmanned-Aerial-Vehicle Video. In 2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020. Institute of Electrical and Electronics Engineers Inc. 2021. p. 181-185. 9353008. (2020 IEEE 3rd International Conference on Electronics and Communication Engineering, ICECE 2020). https://doi.org/10.1109/ICECE51594.2020.9353008