TreeLSTM with Tag-Aware Hypernetwork for Sentence Representation

Chunlin Xu, Hui Wang, Shengli Wu, Zhiwei Lin

Research output: Contribution to journalArticlepeer-review

5 Citations (Scopus)
114 Downloads (Pure)


Tree-structured neural networks, such as TreeLSTM and its variants, have proven effective for learning semantic representations of sentences, which are useful for a variety of tasks in natural language processing such as text categorisation, text semantic matching and machine translation. These neural network models take as inputs parse trees of sentences, which are generated by a language parser. However, most existing tree-structured neural network models lack the ability of distinguishing different syntactic compositions, thus the expressive power of these models is limited. Moreover, the syntactic knowledge provided by Part-of-Speech tags in a parse tree has not been fully utilised in existing tree-structured neural network models. It is expected that such syntactic knowledge should help distinguish syntactic compositions, so should result in better semantic representation.

This paper proposes a novel neural network model, TagHyperTreeLSTM, which contains two components, a tag-aware hypernetwork and a sentence encoder. The tag-aware hypernetwork, which accepts tags as inputs, generates the parameters of the sentence encoder dynamically in order to distinguish different syntactic compositions. The sentence encoder, which accepts words as inputs, generates the final sentence representation.
Original languageEnglish
Pages (from-to)11-20
Early online date28 Dec 2020
Publication statusPublished - 28 Apr 2021


  • Sentence representation, Text classification, Text semantic matching, Dynamic composition, TreeLSTM


Dive into the research topics of 'TreeLSTM with Tag-Aware Hypernetwork for Sentence Representation'. Together they form a unique fingerprint.

Cite this