Feature2Vec: Distributional Semantic Modelling of Human Property Knowledge

Research output: Contribution to conferenceAbstract

Abstract

In computational linguistics, the distributional hypothesis of word meaning has allowed us to construct distributional semantic models by mining large corpora of text to extract co-occurrence statistics of words. Neural approaches such as word2vec learn two sets of matrix representations from mined word-context counts, by using dot product between target and context vector with a sigmoid function to measure the probability of positive association. By using negative sampling techniques and gradient descent optimization, we can learn an approximation of word meaning. Our goal is to construct vector representations for a set of human-derived properties by using a neural topology, similar to the skip-gram word2vec, which uses the target word to predict the surrounding windowed context from mined statistics. Surveying a number of concepts for human interpretable features is costly and time-consuming, but unsupervised learning of vector space models from text data is cheap and accessible. We learn feature meaning by sampling a tiny subset of a pretrained set of word embeddings for which we know the properties. Negative sampling along with gradient descent applied only to the matrix of representations allows us to learn feature meaning in relation to the pretrained word vectors. In this case, a word and a feature have meaningful association if their vectors are close together, which we can measure using cosine similarity. Ranking these features for a given concept, we can extract salient features for the word. Furthermore, since these come from a wider vector space model, we can sample unseen words for features. The process allows us to extract possible feature of words, which could make further surveying the concepts for properties much faster. Active learning would then allow us to repeat this process with a larger lexicon which could be then surveyed again, this time with a higher probability of correctly sampling features.
Original languageEnglish
Number of pages1
Publication statusPublished - 02 Sep 2019
EventInternational Conference of the Royal Statistical Society (RSS 2019) - Belfast, United Kingdom
Duration: 02 Sep 201905 Sep 2019
https://events.rss.org.uk/rss/frontend/reg/thome.csp?pageID=83705&ef_sel_menu=1647&eventID=270

Conference

ConferenceInternational Conference of the Royal Statistical Society (RSS 2019)
Abbreviated titleRSS 2019
CountryUnited Kingdom
CityBelfast
Period02/09/201905/09/2019
Internet address

Fingerprint Dive into the research topics of 'Feature2Vec: Distributional Semantic Modelling of Human Property Knowledge'. Together they form a unique fingerprint.

Cite this