Multimodal neural machine translation for low-resource language pairs using synthetic data

Koel Dutta Chowdhury; Mohammed Hasanuzzaman; Qun Liu

doi:10.18653/v1/W18-3405

Multimodal neural machine translation for low-resource language pairs using synthetic data

Koel Dutta Chowdhury, Mohammed Hasanuzzaman, Qun Liu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

26 Citations (Scopus)

10 Downloads (Pure)

Abstract

In this paper, we investigate the effectiveness of training a multimodal neural machine translation (MNMT) system with image features for a low-resource language pair, Hindi and English, using synthetic data. A three-way parallel corpus which contains bilingual texts and corresponding images is required to train a MNMT system with image features. However, such a corpus is not available for low resource language pairs. To address this, we developed both a synthetic training dataset and a manually curated development/test dataset for Hindi based on an existing English-image parallel corpus. We used these datasets to build our image description translation system by adopting state-of-the-art MNMT models. Our results show that it is possible to train a MNMT system for low-resource language pairs through the use of synthetic data and that such a system can benefit from image features.

Original language	English
Title of host publication	Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP
Publisher	Association for Computational Linguistics
Pages	33-42
Number of pages	10
ISBN (Electronic)	9781948087476
DOIs	https://doi.org/10.18653/v1/W18-3405
Publication status	Published - 19 Jul 2018
Externally published	Yes
Event	ACL Workshop on Deep Learning Approaches for Low-Resource NLP 2018 - Melbourne, Australia Duration: 19 Jul 2018 → 19 Jul 2018

Publication series

Name	ACL Proceedings
ISSN (Print)	0736-587X

Conference

Conference	ACL Workshop on Deep Learning Approaches for Low-Resource NLP 2018
Abbreviated title	ACL DeepLo Workshop 2018
Country/Territory	Australia
City	Melbourne
Period	19/07/2018 → 19/07/2018

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Access to Document

10.18653/v1/W18-3405Licence: CC BY

Multimodal neural machine translation for low-resource language pairs using synthetic data
Copyright 2018 The Authors. This is an open access article published under a Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium, provided the author and source are cited.
Final published version, 313 KBLicence: CC BY

Cite this

@inproceedings{4024f40b52ae438b8cf7322f3a6fa664,

title = "Multimodal neural machine translation for low-resource language pairs using synthetic data",

abstract = "In this paper, we investigate the effectiveness of training a multimodal neural machine translation (MNMT) system with image features for a low-resource language pair, Hindi and English, using synthetic data. A three-way parallel corpus which contains bilingual texts and corresponding images is required to train a MNMT system with image features. However, such a corpus is not available for low resource language pairs. To address this, we developed both a synthetic training dataset and a manually curated development/test dataset for Hindi based on an existing English-image parallel corpus. We used these datasets to build our image description translation system by adopting state-of-the-art MNMT models. Our results show that it is possible to train a MNMT system for low-resource language pairs through the use of synthetic data and that such a system can benefit from image features.",

author = "Chowdhury, {Koel Dutta} and Mohammed Hasanuzzaman and Qun Liu",

year = "2018",

month = jul,

day = "19",

doi = "10.18653/v1/W18-3405",

language = "English",

series = "ACL Proceedings",

publisher = "Association for Computational Linguistics",

pages = "33--42",

booktitle = "Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP",

note = "ACL Workshop on Deep Learning Approaches for Low-Resource NLP 2018, ACL DeepLo Workshop 2018 ; Conference date: 19-07-2018 Through 19-07-2018",

}

Chowdhury, KD, Hasanuzzaman, M & Liu, Q 2018, Multimodal neural machine translation for low-resource language pairs using synthetic data. in Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP. ACL Proceedings, Association for Computational Linguistics, pp. 33-42, ACL Workshop on Deep Learning Approaches for Low-Resource NLP 2018, Melbourne, Victoria, Australia, 19/07/2018. https://doi.org/10.18653/v1/W18-3405

Multimodal neural machine translation for low-resource language pairs using synthetic data. / Chowdhury, Koel Dutta; Hasanuzzaman, Mohammed; Liu, Qun.
Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP. Association for Computational Linguistics, 2018. p. 33-42 (ACL Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Multimodal neural machine translation for low-resource language pairs using synthetic data

AU - Chowdhury, Koel Dutta

AU - Hasanuzzaman, Mohammed

AU - Liu, Qun

PY - 2018/7/19

Y1 - 2018/7/19

N2 - In this paper, we investigate the effectiveness of training a multimodal neural machine translation (MNMT) system with image features for a low-resource language pair, Hindi and English, using synthetic data. A three-way parallel corpus which contains bilingual texts and corresponding images is required to train a MNMT system with image features. However, such a corpus is not available for low resource language pairs. To address this, we developed both a synthetic training dataset and a manually curated development/test dataset for Hindi based on an existing English-image parallel corpus. We used these datasets to build our image description translation system by adopting state-of-the-art MNMT models. Our results show that it is possible to train a MNMT system for low-resource language pairs through the use of synthetic data and that such a system can benefit from image features.

AB - In this paper, we investigate the effectiveness of training a multimodal neural machine translation (MNMT) system with image features for a low-resource language pair, Hindi and English, using synthetic data. A three-way parallel corpus which contains bilingual texts and corresponding images is required to train a MNMT system with image features. However, such a corpus is not available for low resource language pairs. To address this, we developed both a synthetic training dataset and a manually curated development/test dataset for Hindi based on an existing English-image parallel corpus. We used these datasets to build our image description translation system by adopting state-of-the-art MNMT models. Our results show that it is possible to train a MNMT system for low-resource language pairs through the use of synthetic data and that such a system can benefit from image features.

U2 - 10.18653/v1/W18-3405

DO - 10.18653/v1/W18-3405

M3 - Conference contribution

AN - SCOPUS:85077627118

T3 - ACL Proceedings

SP - 33

EP - 42

BT - Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP

PB - Association for Computational Linguistics

T2 - ACL Workshop on Deep Learning Approaches for Low-Resource NLP 2018

Y2 - 19 July 2018 through 19 July 2018

ER -

Multimodal neural machine translation for low-resource language pairs using synthetic data

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this