BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task

Pritam Deka; Nayan Jyoti Kalita; Shikhar Kumar Sarma

BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task

Pritam Deka
, Nayan Jyoti Kalita
, Shikhar Kumar Sarma

School of Electronics, Electrical Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Downloads (Pure)

Abstract

Language identification has recently gained research interest in code-mixed languages due to the extensive use of social media among people. People who speak multiple languages tend to use code-mixed languages when communicating with each other. It has become necessary to identify the languages in such code-mixed environment to detect hate speeches, fake news, misinformation or disinformation and for tasks such as sentiment analysis. In this work, we have proposed a BERT-based approach for language identification in the CoLI-Kanglish shared task at ICON 2022. Our approach achieved 86% weighted average F-1 score and a macro average F-1 score of 57% in the test set.

Original language	English
Title of host publication	Proceedings of the 19th International Conference on Natural Language Processing (ICON): Shared Task on Word Level Language Identification in Code-mixed Kannada-English Texts
Publisher	Association for Computational Linguistics
Pages	12-17
Number of pages	6
ISBN (Electronic)	9781959429388
Publication status	Published - Dec 2022
Event	19th International Conference on Natural Language Processing (ICON 2022) - Delhi, India Duration: 15 Dec 2022 → 18 Dec 2022

Conference

Conference	19th International Conference on Natural Language Processing (ICON 2022)
Country/Territory	India
City	Delhi
Period	15/12/2022 → 18/12/2022

Access to Document

BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task
Copyright 2022 Association for Computational Linguistics. This is an open access article published under a Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium, provided the author and source are cited.
Final published version, 167 KBLicence: CC BY

https://aclanthology.org/2022.icon-wlli.3/Licence: CC BY

Cite this

Deka, P., Kalita, N. J., & Sarma, S. K. (2022). BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task. In Proceedings of the 19th International Conference on Natural Language Processing (ICON): Shared Task on Word Level Language Identification in Code-mixed Kannada-English Texts (pp. 12-17). Association for Computational Linguistics. https://aclanthology.org/2022.icon-wlli.3/

Deka, Pritam ; Kalita, Nayan Jyoti ; Sarma, Shikhar Kumar. / BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task. Proceedings of the 19th International Conference on Natural Language Processing (ICON): Shared Task on Word Level Language Identification in Code-mixed Kannada-English Texts. Association for Computational Linguistics, 2022. pp. 12-17

@inproceedings{d5375b5058834bda9ca834a871c801ee,

title = "BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task",

abstract = "Language identification has recently gained research interest in code-mixed languages due to the extensive use of social media among people. People who speak multiple languages tend to use code-mixed languages when communicating with each other. It has become necessary to identify the languages in such code-mixed environment to detect hate speeches, fake news, misinformation or disinformation and for tasks such as sentiment analysis. In this work, we have proposed a BERT-based approach for language identification in the CoLI-Kanglish shared task at ICON 2022. Our approach achieved 86\% weighted average F-1 score and a macro average F-1 score of 57\% in the test set. ",

author = "Pritam Deka and Kalita, \{Nayan Jyoti\} and Sarma, \{Shikhar Kumar\}",

year = "2022",

month = dec,

language = "English",

pages = "12--17",

booktitle = "Proceedings of the 19th International Conference on Natural Language Processing (ICON): Shared Task on Word Level Language Identification in Code-mixed Kannada-English Texts",

publisher = "Association for Computational Linguistics",

note = "19th International Conference on Natural Language Processing (ICON 2022) ; Conference date: 15-12-2022 Through 18-12-2022",

}

Deka, P, Kalita, NJ & Sarma, SK 2022, BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task. in Proceedings of the 19th International Conference on Natural Language Processing (ICON): Shared Task on Word Level Language Identification in Code-mixed Kannada-English Texts. Association for Computational Linguistics, pp. 12-17, 19th International Conference on Natural Language Processing (ICON 2022), Delhi, India, 15/12/2022. <https://aclanthology.org/2022.icon-wlli.3/>

BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task. / Deka, Pritam; Kalita, Nayan Jyoti; Sarma, Shikhar Kumar.
Proceedings of the 19th International Conference on Natural Language Processing (ICON): Shared Task on Word Level Language Identification in Code-mixed Kannada-English Texts. Association for Computational Linguistics, 2022. p. 12-17.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task

AU - Deka, Pritam

AU - Kalita, Nayan Jyoti

AU - Sarma, Shikhar Kumar

PY - 2022/12

Y1 - 2022/12

N2 - Language identification has recently gained research interest in code-mixed languages due to the extensive use of social media among people. People who speak multiple languages tend to use code-mixed languages when communicating with each other. It has become necessary to identify the languages in such code-mixed environment to detect hate speeches, fake news, misinformation or disinformation and for tasks such as sentiment analysis. In this work, we have proposed a BERT-based approach for language identification in the CoLI-Kanglish shared task at ICON 2022. Our approach achieved 86% weighted average F-1 score and a macro average F-1 score of 57% in the test set.

AB - Language identification has recently gained research interest in code-mixed languages due to the extensive use of social media among people. People who speak multiple languages tend to use code-mixed languages when communicating with each other. It has become necessary to identify the languages in such code-mixed environment to detect hate speeches, fake news, misinformation or disinformation and for tasks such as sentiment analysis. In this work, we have proposed a BERT-based approach for language identification in the CoLI-Kanglish shared task at ICON 2022. Our approach achieved 86% weighted average F-1 score and a macro average F-1 score of 57% in the test set.

M3 - Conference contribution

SP - 12

EP - 17

BT - Proceedings of the 19th International Conference on Natural Language Processing (ICON): Shared Task on Word Level Language Identification in Code-mixed Kannada-English Texts

PB - Association for Computational Linguistics

T2 - 19th International Conference on Natural Language Processing (ICON 2022)

Y2 - 15 December 2022 through 18 December 2022

ER -

BERT-based language identification in code-mix Kannada-English text at the CoLI-Kanglish shared task

Abstract

Conference

Access to Document

Fingerprint

Cite this