Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model

In this work, we evaluate latent semantic knowledge present in the LSTM activation patterns produced before and after the word of interest. We evaluate whether these activations predict human similarity ratings, human-derived property knowledge, and brain imaging data. In this way, we test the model{'}s ability to encode important semantic information relevant to word prediction, and it{'}s relationship with human cognitive semantic representations.