Interpreting Clinical Narrative Diagnosis Models with Sentence Importance

Research output: Contribution to conferenceAbstractpeer-review


Despite advances in the application of deep neural networks to various kinds of medical data, extracting information from unstructured textual sources remains a challenging task. Using a dataset of de-identified clinical letters gathered at a memory clinic, we evaluate recurrent neural networks (RNNs) on their ability to predict patients’ diagnoses of ‘Dementia’, ‘Mild Cognitive Impairment’ or ‘Non-impaired’. This classification framework can also have applications in the automatic identification of patients as candidates for clinical trials.

After showing that models trained on state-of-the-art sentence-level embeddings outperform both word-level models and a recent benchmark model that fine-tunes a pre-trained general-domain language model, we probe sentence embedding models in order to reveal interpretable insights into the types of sentence-level representations the RNNs build. Specifically, we take a measure of sentence importance with respect to a given class and identify clusters of sentences in the embedding space that correlate strongly with importance scores for each class. Extracting the most frequent phrases within each group of sentence representations shows that the model is sensitive to sentences that cluster around semantic concepts such as a patient’s level of geriatric depression and how independent the patient is in their daily activities.

In addition to showing which sentences in a document are most informative about the patient’s condition, our method can identify the types of sentences that can lead the model to make incorrect diagnoses.
Original languageEnglish
Number of pages1
Publication statusPublished - 02 Sep 2019
EventInternational Conference of the Royal Statistical Society (RSS 2019) - Belfast, United Kingdom
Duration: 02 Sep 201905 Sep 2019


ConferenceInternational Conference of the Royal Statistical Society (RSS 2019)
Abbreviated titleRSS 2019
Country/TerritoryUnited Kingdom
Internet address


Dive into the research topics of 'Interpreting Clinical Narrative Diagnosis Models with Sentence Importance'. Together they form a unique fingerprint.

Cite this