A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier

Sunanda Das, Md. Samir Imtiaz, Nieb Hasan Neom, Nazmul Siddique, Hui Wang

Research output: Contribution to journalArticlepeer-review

36 Citations (Scopus)
87 Downloads (Pure)

Abstract

Sign language is the comprehensive medium of mass communication for hearing and speaking impaired individuals. As they can’t speak or hear, they aren’t able to use sound or vocal signals as an information medium for their communication. Rather, they are bound to exchange visual signals to express their feeling in their day-to-day life. For this, they use various body language mainly hand gestures as sign language. Sign language fundamentals can be largely divided into two parts namely digits (numerals) and characters (alphabetical). In this paper, we proposed a hybrid model consisting of a deep transfer learning-based convolutional neural network with a random forest classifier for the automatic recognition of Bangla Sign Language (numerals and alphabets). The overall performance of the presented system is verified on ‘Ishara-Bochon’ and ‘Ishara-Lipi’ datasets. ‘Ishara-Bochon’ and ‘Ishara-Lipi’ are datasets of isolated numerals and alphabets respectively which are the first complete multipurpose open-access dataset for Bangla Sign Language (BSL). Besides, we also proposed a background elimination algorithm that removes unnecessary features from the sign images. Along with the proposed background elimination technique, the system is able to achieve accuracy, precision, recall, f1-score values of 91.67%, 93.64%, 91.67%, 91.47% for character recognition and 97.33%, 97.89%, 97.33%, 97.37% for digit recognition respectively. The detailed experimental analysis assures the feasibility and effectiveness of the proposed system for BSL recognition.

Original languageEnglish
Article number118914
Number of pages14
JournalExpert Systems with Applications
Volume213
Issue numberPart B
Early online date17 Oct 2022
DOIs
Publication statusPublished - 01 Mar 2023

Fingerprint

Dive into the research topics of 'A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier'. Together they form a unique fingerprint.

Cite this