Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model

Steven Derby, Paul Miller, Brian Murphy, Barry Devereux

Research output: Contribution to conference › Paper

Abstract

In this work, we evaluate latent semantic knowledge present in the LSTM activation patterns produced before and after the word of interest. We evaluate whether these activations predict human similarity ratings, human-derived property knowledge, and brain imaging data. In this way, we test the model{'}s ability to encode important semantic information relevant to word prediction, and it{'}s relationship with human cognitive semantic representations.
Original languageEnglish
Number of pages3
DOIs
Publication statusPublished - 29 Jun 2019

Fingerprint

Semantics
Chemical activation
Brain
Imaging techniques

Cite this

Derby, Steven ; Miller, Paul ; Murphy, Brian ; Devereux, Barry. / Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model. 3 p.
@conference{0b98eb5275a040d19e0726cd94626f3e,
title = "Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model",
abstract = "In this work, we evaluate latent semantic knowledge present in the LSTM activation patterns produced before and after the word of interest. We evaluate whether these activations predict human similarity ratings, human-derived property knowledge, and brain imaging data. In this way, we test the model{'}s ability to encode important semantic information relevant to word prediction, and it{'}s relationship with human cognitive semantic representations.",
author = "Steven Derby and Paul Miller and Brian Murphy and Barry Devereux",
year = "2019",
month = "6",
day = "29",
doi = "10.18653/v1/w18-5449",
language = "English",

}

Derby, S, Miller, P, Murphy, B & Devereux, B 2019, 'Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model'. https://doi.org/10.18653/v1/w18-5449

Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model. / Derby, Steven; Miller, Paul; Murphy, Brian; Devereux, Barry.

2019.

Research output: Contribution to conference › Paper

TY - CONF

T1 - Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model

AU - Derby, Steven

AU - Miller, Paul

AU - Murphy, Brian

AU - Devereux, Barry

PY - 2019/6/29

Y1 - 2019/6/29

N2 - In this work, we evaluate latent semantic knowledge present in the LSTM activation patterns produced before and after the word of interest. We evaluate whether these activations predict human similarity ratings, human-derived property knowledge, and brain imaging data. In this way, we test the model{'}s ability to encode important semantic information relevant to word prediction, and it{'}s relationship with human cognitive semantic representations.

AB - In this work, we evaluate latent semantic knowledge present in the LSTM activation patterns produced before and after the word of interest. We evaluate whether these activations predict human similarity ratings, human-derived property knowledge, and brain imaging data. In this way, we test the model{'}s ability to encode important semantic information relevant to word prediction, and it{'}s relationship with human cognitive semantic representations.

UR - http://www.mendeley.com/research/representation-word-meaning-intermediate-projection-layer-neural-language-model

UR - http://www.mendeley.com/research/representation-word-meaning-intermediate-projection-layer-neural-language-model

U2 - 10.18653/v1/w18-5449

DO - 10.18653/v1/w18-5449

M3 - Paper

ER -

Derby S, Miller P, Murphy B, Devereux B. Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model. 2019. https://doi.org/10.18653/v1/w18-5449

Access to Document