Comparison of deep learning models for natural language processing-based classification of non-English head CT reports

Yiftach Barash, Gennadiy Guralnik, Noam Tau, Shelly Soffer, Tal Levy, Orit Shimon, Eyal Zimlichman, Eli Konen, Eyal Klang

Research output: Contribution to journalArticlepeer-review

17 Scopus citations

Abstract

Purpose: Natural language processing (NLP) can be used for automatic flagging of radiology reports. We assessed deep learning models for classifying non-English head CT reports. Methods: We retrospectively collected head CT reports (2011–2018). Reports were signed in Hebrew. Emergency department (ED) reports of adult patients from January to February for each year (2013–2018) were manually labeled. All other reports were used to pre-train an embedding layer. We explored two use cases: (1) general labeling use case, in which reports were labeled as normal vs. pathological; (2) specific labeling use case, in which reports were labeled as with and without intra-cranial hemorrhage. We tested long short-term memory (LSTM) and LSTM-attention (LSTM-ATN) networks for classifying reports. We also evaluated the improvement of adding Word2Vec word embedding. Deep learning models were compared with a bag-of-words (BOW) model. Results: We retrieved 176,988 head CT reports for pre-training. We manually labeled 7784 reports as normal (46.3%) or pathological (53.7%), and 7.1% with intra-cranial hemorrhage. For the general labeling, LSTM-ATN-Word2Vec showed the best results (AUC = 0.967 ± 0.006, accuracy 90.8% ± 0.01). For the specific labeling, all methods showed similar accuracies between 95.0 and 95.9%. Both LSTM-ATN-Word2Vec and BOW had the highest AUC (0.970). Conclusion: For a general use case, word embedding using a large cohort of non-English head CT reports and ATN improves NLP performance. For a more specific task, BOW and deep learning showed similar results. Models should be explored and tailored to the NLP task.

Original languageEnglish
Pages (from-to)1247-1256
Number of pages10
JournalNeuroradiology
Volume62
Issue number10
DOIs
StatePublished - 1 Oct 2020
Externally publishedYes

Keywords

  • Attention
  • Deep learning
  • Emergency service, hospital
  • Natural language processing
  • Tomography, X-ray computed

Fingerprint

Dive into the research topics of 'Comparison of deep learning models for natural language processing-based classification of non-English head CT reports'. Together they form a unique fingerprint.

Cite this