Topic modeling for unsupervised concept extraction and document ranking

V. S. Anoop; S. Asharaf; P. Deepak

doi:10.1007/978-3-319-68385-0_11

Topic modeling for unsupervised concept extraction and document ranking

V. S. Anoop^*, S. Asharaf, P. Deepak

^*Corresponding author for this work

School of Electronics, Electrical Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

This paper proposes a framework which induces semantically rich concepts from probabilistically generated topics by a topic modeling algorithm. In this method an off-the-shelf tool has been used to extract noun-phrases as word bi-grams and tri-grams from the static document corpus and then models the topics using Latent Dirichlet Allocation algorithm. Additionally, we show that a small extension to our proposed framework can better rank documents in a large collection, which is a well studied area in information retrieval. Experiments conducted on three real world datasets show that this proposed framework outperforms state-of-the-art methods used for extracting concepts and ranking documents. When compared with the baselines chosen, our proposed concept extraction method showed an increased f-measure in the range of 16.65% to 22.04% and the proposed topic modeling guided document retrieval method showed 7.6%–16.61% increase in f-measure.

Original language	English
Title of host publication	Proceedings of the Third International Symposium on Intelligent Systems Technologies and Applications (ISTA’17)
Editors	Sabu M. Thampi, Alex Pappachen James, Stefano Berretti, Jayanta Mukhopadhyay, Sushmita Mitra, Kuan-Ching Li
Publisher	Springer Verlag
Pages	123-135
Number of pages	13
ISBN (Electronic)	9783319683850
ISBN (Print)	9783319683843
DOIs	https://doi.org/10.1007/978-3-319-68385-0_11
Publication status	Published - 20 Oct 2017
Event	3rd International Symposium on Intelligent Systems Technologies and Applications, ISTA’17 - Udupi, India Duration: 13 Sept 2017 → 16 Sept 2017

Publication series

Name	Advances in Intelligent Systems and Computing
Volume	683
ISSN (Print)	2194-5357
ISSN (Electronic)	2194-5365

Conference

Conference	3rd International Symposium on Intelligent Systems Technologies and Applications, ISTA’17
Country/Territory	India
City	Udupi
Period	13/09/2017 → 16/09/2017

Bibliographical note

Publisher Copyright:
© Springer International Publishing AG 2018.

Keywords

Concept extraction
Document ranking
Latent dirichlet allocation
Topic modeling

ASJC Scopus subject areas

Control and Systems Engineering
General Computer Science

Access to Document

10.1007/978-3-319-68385-0_11Licence: Unspecified

Cite this

Anoop, V. S., Asharaf, S., & Deepak, P. (2017). Topic modeling for unsupervised concept extraction and document ranking. In S. M. Thampi, A. P. James, S. Berretti, J. Mukhopadhyay, S. Mitra, & K.-C. Li (Eds.), Proceedings of the Third International Symposium on Intelligent Systems Technologies and Applications (ISTA’17) (pp. 123-135). (Advances in Intelligent Systems and Computing; Vol. 683). Springer Verlag. https://doi.org/10.1007/978-3-319-68385-0_11

Anoop, V. S. ; Asharaf, S. ; Deepak, P. / Topic modeling for unsupervised concept extraction and document ranking. Proceedings of the Third International Symposium on Intelligent Systems Technologies and Applications (ISTA’17). editor / Sabu M. Thampi ; Alex Pappachen James ; Stefano Berretti ; Jayanta Mukhopadhyay ; Sushmita Mitra ; Kuan-Ching Li. Springer Verlag, 2017. pp. 123-135 (Advances in Intelligent Systems and Computing).

@inproceedings{49c19932957f45b68bd4fe8305b859ef,

title = "Topic modeling for unsupervised concept extraction and document ranking",

abstract = "This paper proposes a framework which induces semantically rich concepts from probabilistically generated topics by a topic modeling algorithm. In this method an off-the-shelf tool has been used to extract noun-phrases as word bi-grams and tri-grams from the static document corpus and then models the topics using Latent Dirichlet Allocation algorithm. Additionally, we show that a small extension to our proposed framework can better rank documents in a large collection, which is a well studied area in information retrieval. Experiments conducted on three real world datasets show that this proposed framework outperforms state-of-the-art methods used for extracting concepts and ranking documents. When compared with the baselines chosen, our proposed concept extraction method showed an increased f-measure in the range of 16.65% to 22.04% and the proposed topic modeling guided document retrieval method showed 7.6%–16.61% increase in f-measure.",

keywords = "Concept extraction, Document ranking, Latent dirichlet allocation, Topic modeling",

author = "Anoop, {V. S.} and S. Asharaf and P. Deepak",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2018.; 3rd International Symposium on Intelligent Systems Technologies and Applications, ISTA{\textquoteright}17 ; Conference date: 13-09-2017 Through 16-09-2017",

year = "2017",

month = oct,

day = "20",

doi = "10.1007/978-3-319-68385-0_11",

language = "English",

isbn = "9783319683843",

series = "Advances in Intelligent Systems and Computing",

publisher = "Springer Verlag",

pages = "123--135",

editor = "Thampi, {Sabu M.} and James, {Alex Pappachen} and Stefano Berretti and Jayanta Mukhopadhyay and Sushmita Mitra and Kuan-Ching Li",

booktitle = "Proceedings of the Third International Symposium on Intelligent Systems Technologies and Applications (ISTA{\textquoteright}17)",

address = "Germany",

}

Anoop, VS, Asharaf, S & Deepak, P 2017, Topic modeling for unsupervised concept extraction and document ranking. in SM Thampi, AP James, S Berretti, J Mukhopadhyay, S Mitra & K-C Li (eds), Proceedings of the Third International Symposium on Intelligent Systems Technologies and Applications (ISTA’17). Advances in Intelligent Systems and Computing, vol. 683, Springer Verlag, pp. 123-135, 3rd International Symposium on Intelligent Systems Technologies and Applications, ISTA’17, Udupi, India, 13/09/2017. https://doi.org/10.1007/978-3-319-68385-0_11

Topic modeling for unsupervised concept extraction and document ranking. / Anoop, V. S.; Asharaf, S.; Deepak, P.
Proceedings of the Third International Symposium on Intelligent Systems Technologies and Applications (ISTA’17). ed. / Sabu M. Thampi; Alex Pappachen James; Stefano Berretti; Jayanta Mukhopadhyay; Sushmita Mitra; Kuan-Ching Li. Springer Verlag, 2017. p. 123-135 (Advances in Intelligent Systems and Computing; Vol. 683).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Topic modeling for unsupervised concept extraction and document ranking

AU - Anoop, V. S.

AU - Asharaf, S.

AU - Deepak, P.

N1 - Publisher Copyright: © Springer International Publishing AG 2018.

PY - 2017/10/20

Y1 - 2017/10/20

N2 - This paper proposes a framework which induces semantically rich concepts from probabilistically generated topics by a topic modeling algorithm. In this method an off-the-shelf tool has been used to extract noun-phrases as word bi-grams and tri-grams from the static document corpus and then models the topics using Latent Dirichlet Allocation algorithm. Additionally, we show that a small extension to our proposed framework can better rank documents in a large collection, which is a well studied area in information retrieval. Experiments conducted on three real world datasets show that this proposed framework outperforms state-of-the-art methods used for extracting concepts and ranking documents. When compared with the baselines chosen, our proposed concept extraction method showed an increased f-measure in the range of 16.65% to 22.04% and the proposed topic modeling guided document retrieval method showed 7.6%–16.61% increase in f-measure.

AB - This paper proposes a framework which induces semantically rich concepts from probabilistically generated topics by a topic modeling algorithm. In this method an off-the-shelf tool has been used to extract noun-phrases as word bi-grams and tri-grams from the static document corpus and then models the topics using Latent Dirichlet Allocation algorithm. Additionally, we show that a small extension to our proposed framework can better rank documents in a large collection, which is a well studied area in information retrieval. Experiments conducted on three real world datasets show that this proposed framework outperforms state-of-the-art methods used for extracting concepts and ranking documents. When compared with the baselines chosen, our proposed concept extraction method showed an increased f-measure in the range of 16.65% to 22.04% and the proposed topic modeling guided document retrieval method showed 7.6%–16.61% increase in f-measure.

KW - Concept extraction

KW - Document ranking

KW - Latent dirichlet allocation

KW - Topic modeling

U2 - 10.1007/978-3-319-68385-0_11

DO - 10.1007/978-3-319-68385-0_11

M3 - Conference contribution

AN - SCOPUS:85032702838

SN - 9783319683843

T3 - Advances in Intelligent Systems and Computing

SP - 123

EP - 135

BT - Proceedings of the Third International Symposium on Intelligent Systems Technologies and Applications (ISTA’17)

A2 - Thampi, Sabu M.

A2 - James, Alex Pappachen

A2 - Berretti, Stefano

A2 - Mukhopadhyay, Jayanta

A2 - Mitra, Sushmita

A2 - Li, Kuan-Ching

PB - Springer Verlag

T2 - 3rd International Symposium on Intelligent Systems Technologies and Applications, ISTA’17

Y2 - 13 September 2017 through 16 September 2017

ER -

Anoop VS, Asharaf S, Deepak P. Topic modeling for unsupervised concept extraction and document ranking. In Thampi SM, James AP, Berretti S, Mukhopadhyay J, Mitra S, Li KC, editors, Proceedings of the Third International Symposium on Intelligent Systems Technologies and Applications (ISTA’17). Springer Verlag. 2017. p. 123-135. (Advances in Intelligent Systems and Computing). doi: 10.1007/978-3-319-68385-0_11

Topic modeling for unsupervised concept extraction and document ranking

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this