A novel vSLAM framework with unsupervised semantic segmentation based on adversarial transfer learning

Sheng Jin, Liang Chen*, Rongchuan Sun, Seán McLoone

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

200 Downloads (Pure)


Significant progress has been made in the field of visual Simultaneous Localization and Mapping (vSLAM) systems. However, the localization accuracy of vSLAM can be significantly reduced in dynamic applications with mobile robots or passengers. In this paper, a novel semantic SLAM framework in dynamic environments is proposed to improve the localization accuracy. We incorporate a semantic segmentation model into the Oriented FAST and Rotated BRIEF-SLAM2 (ORB-SLAM2) system to filter out dynamic feature points, but we encounter one main challenge, i.e. the performance of a segmentation network well-trained with labeled datasets may decrease seriously in a real application without any labeled data due to the inconsistency between the source domain and the target domain. Therefore, we proposed an unsupervised semantic segmentation model with a Residual Neural Network (ResNet) structure, which is trained by the adversarial transfer learning method in the multi-level feature spaces.
This work may be the first to perform multi-level feature space adversarial transfer learning for the semantic SLAM task in dynamic environments. In order to evaluate our method, images of indoor scenes from three datasets are used as the source domain, and the dynamic sequences of the TUM dataset are used as the target domain. The extensive experimental results show favorable performance against the state-of-the-art methods in terms of the absolute trajectory accuracy and image semantic segmentation quality.
Original languageEnglish
Article number106153
Number of pages17
JournalApplied Soft Computing
Early online date03 Feb 2020
Publication statusPublished - 01 May 2020


Dive into the research topics of 'A novel vSLAM framework with unsupervised semantic segmentation based on adversarial transfer learning'. Together they form a unique fingerprint.

Cite this