Text-image-video summary generation using joint integer linear programming

Anubhav Jangra; Adam Jatowt; Mohammad Hasanuzzaman; Sriparna Saha

doi:10.1007/978-3-030-45442-5_24

Text-image-video summary generation using joint integer linear programming

Anubhav Jangra, Adam Jatowt^*, Mohammad Hasanuzzaman, Sriparna Saha

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

20 Citations (Scopus)

Abstract

Automatically generating a summary for asynchronous data can help users to keep up with the rapid growth of multi-modal information on the Internet. However, the current multi-modal systems usually generate summaries composed of text and images. In this paper, we propose a novel research problem of text-image-video summary generation (TIVS). We first develop a multi-modal dataset containing text documents, images and videos. We then propose a novel joint integer linear programming multi-modal summarization (JILP-MMS) framework. We report the performance of our model on the developed dataset.

Original language	English
Title of host publication	Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings
Editors	Joemon M. Jose, Emine Yilmaz, João Magalhães, Pablo Castells, Nicola Ferro, Mário J. Silva, Flávio Martins
Publisher	Springer Singapore
Pages	190-198
Number of pages	9
ISBN (Electronic)	9783030454425
ISBN (Print)	9783030454418
DOIs	https://doi.org/10.1007/978-3-030-45442-5_24
Publication status	Published - 08 Apr 2020
Externally published	Yes
Event	42nd European Conference on IR Research, ECIR 2020 - Lisbon, Portugal Duration: 14 Apr 2020 → 17 Apr 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12036 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	42nd European Conference on IR Research, ECIR 2020
Country/Territory	Portugal
City	Lisbon
Period	14/04/2020 → 17/04/2020

Bibliographical note

Funding Information:
Acknowledgement. Dr. Sriparna Saha would like to acknowledge the support of Early Career Research Award of Science and Engineering Research Board (SERB) of Department of Science and Technology, India to carry out this research. Mohammed Hasanuzzaman would like to acknowledge ADAPT Centre for Digital Content Technology which is funded under the SFI Research Centres Programme (Grant 13/RC/2106).

Publisher Copyright:
© Springer Nature Switzerland AG 2020.

Keywords

Integer Linear Programming
Multi-modal summarization

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-45442-5_24Licence: Unspecified

Cite this

Jangra, A., Jatowt, A., Hasanuzzaman, M., & Saha, S. (2020). Text-image-video summary generation using joint integer linear programming. In J. M. Jose, E. Yilmaz, J. Magalhães, P. Castells, N. Ferro, M. J. Silva, & F. Martins (Eds.), Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings (pp. 190-198). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12036 LNCS). Springer Singapore. https://doi.org/10.1007/978-3-030-45442-5_24

Jangra, Anubhav ; Jatowt, Adam ; Hasanuzzaman, Mohammad et al. / Text-image-video summary generation using joint integer linear programming. Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings. editor / Joemon M. Jose ; Emine Yilmaz ; João Magalhães ; Pablo Castells ; Nicola Ferro ; Mário J. Silva ; Flávio Martins. Springer Singapore, 2020. pp. 190-198 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a18e0043512c494ead0dd058b316fe61,

title = "Text-image-video summary generation using joint integer linear programming",

abstract = "Automatically generating a summary for asynchronous data can help users to keep up with the rapid growth of multi-modal information on the Internet. However, the current multi-modal systems usually generate summaries composed of text and images. In this paper, we propose a novel research problem of text-image-video summary generation (TIVS). We first develop a multi-modal dataset containing text documents, images and videos. We then propose a novel joint integer linear programming multi-modal summarization (JILP-MMS) framework. We report the performance of our model on the developed dataset.",

keywords = "Integer Linear Programming, Multi-modal summarization",

author = "Anubhav Jangra and Adam Jatowt and Mohammad Hasanuzzaman and Sriparna Saha",

note = "Funding Information: Acknowledgement. Dr. Sriparna Saha would like to acknowledge the support of Early Career Research Award of Science and Engineering Research Board (SERB) of Department of Science and Technology, India to carry out this research. Mohammed Hasanuzzaman would like to acknowledge ADAPT Centre for Digital Content Technology which is funded under the SFI Research Centres Programme (Grant 13/RC/2106). Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2020.; 42nd European Conference on IR Research, ECIR 2020 ; Conference date: 14-04-2020 Through 17-04-2020",

year = "2020",

month = apr,

day = "8",

doi = "10.1007/978-3-030-45442-5_24",

language = "English",

isbn = "9783030454418",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Singapore",

pages = "190--198",

editor = "Jose, {Joemon M.} and Emine Yilmaz and Jo{\~a}o Magalh{\~a}es and Pablo Castells and Nicola Ferro and Silva, {M{\'a}rio J.} and Fl{\'a}vio Martins",

booktitle = "Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings",

address = "Singapore",

}

Jangra, A, Jatowt, A, Hasanuzzaman, M & Saha, S 2020, Text-image-video summary generation using joint integer linear programming. in JM Jose, E Yilmaz, J Magalhães, P Castells, N Ferro, MJ Silva & F Martins (eds), Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12036 LNCS, Springer Singapore, pp. 190-198, 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, 14/04/2020. https://doi.org/10.1007/978-3-030-45442-5_24

Text-image-video summary generation using joint integer linear programming. / Jangra, Anubhav; Jatowt, Adam; Hasanuzzaman, Mohammad et al.
Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings. ed. / Joemon M. Jose; Emine Yilmaz; João Magalhães; Pablo Castells; Nicola Ferro; Mário J. Silva; Flávio Martins. Springer Singapore, 2020. p. 190-198 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12036 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Text-image-video summary generation using joint integer linear programming

AU - Jangra, Anubhav

AU - Jatowt, Adam

AU - Hasanuzzaman, Mohammad

AU - Saha, Sriparna

N1 - Funding Information: Acknowledgement. Dr. Sriparna Saha would like to acknowledge the support of Early Career Research Award of Science and Engineering Research Board (SERB) of Department of Science and Technology, India to carry out this research. Mohammed Hasanuzzaman would like to acknowledge ADAPT Centre for Digital Content Technology which is funded under the SFI Research Centres Programme (Grant 13/RC/2106). Publisher Copyright: © Springer Nature Switzerland AG 2020.

PY - 2020/4/8

Y1 - 2020/4/8

N2 - Automatically generating a summary for asynchronous data can help users to keep up with the rapid growth of multi-modal information on the Internet. However, the current multi-modal systems usually generate summaries composed of text and images. In this paper, we propose a novel research problem of text-image-video summary generation (TIVS). We first develop a multi-modal dataset containing text documents, images and videos. We then propose a novel joint integer linear programming multi-modal summarization (JILP-MMS) framework. We report the performance of our model on the developed dataset.

AB - Automatically generating a summary for asynchronous data can help users to keep up with the rapid growth of multi-modal information on the Internet. However, the current multi-modal systems usually generate summaries composed of text and images. In this paper, we propose a novel research problem of text-image-video summary generation (TIVS). We first develop a multi-modal dataset containing text documents, images and videos. We then propose a novel joint integer linear programming multi-modal summarization (JILP-MMS) framework. We report the performance of our model on the developed dataset.

KW - Integer Linear Programming

KW - Multi-modal summarization

U2 - 10.1007/978-3-030-45442-5_24

DO - 10.1007/978-3-030-45442-5_24

M3 - Conference contribution

AN - SCOPUS:85084176012

SN - 9783030454418

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 190

EP - 198

BT - Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings

A2 - Jose, Joemon M.

A2 - Yilmaz, Emine

A2 - Magalhães, João

A2 - Castells, Pablo

A2 - Ferro, Nicola

A2 - Silva, Mário J.

A2 - Martins, Flávio

PB - Springer Singapore

T2 - 42nd European Conference on IR Research, ECIR 2020

Y2 - 14 April 2020 through 17 April 2020

ER -

Jangra A, Jatowt A, Hasanuzzaman M, Saha S. Text-image-video summary generation using joint integer linear programming. In Jose JM, Yilmaz E, Magalhães J, Castells P, Ferro N, Silva MJ, Martins F, editors, Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings. Springer Singapore. 2020. p. 190-198. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-45442-5_24

Text-image-video summary generation using joint integer linear programming

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this