Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding

Brian Murphy; Partha Pratim Talukdar; Tom Mitchell

Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding

Brian Murphy
, Partha Pratim Talukdar
, Tom Mitchell

School of Electronics, Electrical Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

128 Citations (Scopus)

563 Downloads (Pure)

Abstract

In this paper, we introduce an application of matrix factorization to produce corpus-derived, distributional
models of semantics that demonstrate cognitive plausibility. We find that word representations
learned by Non-Negative Sparse Embedding (NNSE), a variant of matrix factorization, are sparse,
effective, and highly interpretable. To the best of our knowledge, this is the first approach which
yields semantic representation of words satisfying these three desirable properties. Though extensive
experimental evaluations on multiple real-world tasks and datasets, we demonstrate the superiority
of semantic models learned by NNSE over other state-of-the-art baselines.

Original language	English
Title of host publication	International Conference on Computational Linguistics (COLING 2012), Mumbai, India
Publisher	Association for Computational Linguistics
Pages	1933-1949
Number of pages	17
Publication status	Published - Dec 2012

Keywords

distributional semantics,inter-,neuro-semantics,pretability,sparse coding,vector-space models,word embeddings

Access to Document

Learning Effective and Interpretable Semantic Models using Non-Negative Sparse EmbeddingFinal published version, 396 KB

http://aclweb.org/anthology/C/C12/C12-1118.pdf

Cite this

@inproceedings{774536f9073142ea9c80c3ee6dcccad9,

title = "Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding",

abstract = "In this paper, we introduce an application of matrix factorization to produce corpus-derived, distributional models of semantics that demonstrate cognitive plausibility. We find that word representations learned by Non-Negative Sparse Embedding (NNSE), a variant of matrix factorization, are sparse, effective, and highly interpretable. To the best of our knowledge, this is the first approach which yields semantic representation of words satisfying these three desirable properties. Though extensive experimental evaluations on multiple real-world tasks and datasets, we demonstrate the superiority of semantic models learned by NNSE over other state-of-the-art baselines.",

keywords = "distributional semantics,inter-,neuro-semantics,pretability,sparse coding,vector-space models,word embeddings",

author = "Brian Murphy and Talukdar, \{Partha Pratim\} and Tom Mitchell",

year = "2012",

month = dec,

language = "English",

pages = "1933--1949",

booktitle = "International Conference on Computational Linguistics (COLING 2012), Mumbai, India",

publisher = "Association for Computational Linguistics",

}

Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding. / Murphy, Brian; Talukdar, Partha Pratim; Mitchell, Tom.
International Conference on Computational Linguistics (COLING 2012), Mumbai, India. Association for Computational Linguistics, 2012. p. 1933-1949.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding

AU - Murphy, Brian

AU - Talukdar, Partha Pratim

AU - Mitchell, Tom

PY - 2012/12

Y1 - 2012/12

N2 - In this paper, we introduce an application of matrix factorization to produce corpus-derived, distributional models of semantics that demonstrate cognitive plausibility. We find that word representations learned by Non-Negative Sparse Embedding (NNSE), a variant of matrix factorization, are sparse, effective, and highly interpretable. To the best of our knowledge, this is the first approach which yields semantic representation of words satisfying these three desirable properties. Though extensive experimental evaluations on multiple real-world tasks and datasets, we demonstrate the superiority of semantic models learned by NNSE over other state-of-the-art baselines.

AB - In this paper, we introduce an application of matrix factorization to produce corpus-derived, distributional models of semantics that demonstrate cognitive plausibility. We find that word representations learned by Non-Negative Sparse Embedding (NNSE), a variant of matrix factorization, are sparse, effective, and highly interpretable. To the best of our knowledge, this is the first approach which yields semantic representation of words satisfying these three desirable properties. Though extensive experimental evaluations on multiple real-world tasks and datasets, we demonstrate the superiority of semantic models learned by NNSE over other state-of-the-art baselines.

KW - distributional semantics,inter-,neuro-semantics,pretability,sparse coding,vector-space models,word embeddings

M3 - Conference contribution

SP - 1933

EP - 1949

BT - International Conference on Computational Linguistics (COLING 2012), Mumbai, India

PB - Association for Computational Linguistics

ER -

Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding

Abstract

Keywords

Access to Document

Fingerprint

Cite this