Multimodal gait recognition for neurodegenerative diseases

Aite Zhao, Jianbo Li, Junyu Dong, Lin Qi, Qianni Zhang, Ning Li, Xin Wang, Huiyu Zhou

Research output: Contribution to journalArticlepeer-review

40 Citations (Scopus)


In recent years, single modality-based gait recognition has been extensively explored in the analysis of medical images or other sensory data, and it is recognized that each of the established approaches has different strengths and weaknesses. As an important motor symptom, gait disturbance is usually used for diagnosis and evaluation of diseases; moreover, the use of multimodality analysis of the patient’s walking pattern compensates for the one-sidedness of single modality gait recognition methods that only learn gait changes in a single measurement dimension. The fusion of multiple measurement resources has demonstrated promising performance in the identification of gait patterns associated with individual diseases. In this article, as a useful tool, we propose a novel hybrid model to learn the gait differences between three neurodegenerative diseases, between patients with different severity levels of Parkinson’s disease, and between healthy individuals and patients, by fusing and aggregating data from multiple sensors. A spatial feature extractor (SFE) is applied to generating representative features of images or signals. In order to capture temporal information from the two modality data, a new correlative memory neural network (CorrMNN) architecture is designed for extracting temporal features. Afterward, we embed a multiswitch discriminator to associate the observations with individual state estimations. Compared with several state-of-the-art techniques, our proposed framework shows more accurate classification results.

Original languageEnglish
Pages (from-to)9439-9453
Number of pages15
JournalIEEE Transactions on Cybernetics
Issue number9
Early online date11 Mar 2021
Publication statusPublished - Sept 2022
Externally publishedYes


  • Correlation
  • Correlative memory neural network (CorrMNN)
  • Diseases
  • Feature extraction
  • gait recognition
  • Gait recognition
  • Hidden Markov models
  • multiswitch discriminator
  • Neural networks
  • neurodegenerative diseases (NDDs)
  • Parkinson's disease (PD)
  • Sensors

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Multimodal gait recognition for neurodegenerative diseases'. Together they form a unique fingerprint.

Cite this