Optimizing Embedding Space with Sub-categorical Supervised Pre-training: A Theoretical Approach Towards Improving Sepsis Prediction

Tingyi Wanyan, Mingquan Lin, Ying Ding, Benjamin Glicksberg, Fei Wang, Yifan Peng

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Supervised contrastive learning provides superior performance over self-supervised learning by considering label information in classification tasks. However, this process suffers from collapsing embedding space since the positive samples are randomly selected from the labeled group and are pulled together. In this work, we theoretically guarantee that any pre-training methods that maintain a mixture of sub-class distribution could consistently outperform supervised contrastive pre-training. Furthermore, based on our theoretical analysis, we propose a new pre-training method by adopting an efficient Expectation Maximization learning strategy. Finally, we empirically evaluated our proposed method of sepsis prediction from the PhysioNet/Computing in Cardiology Challenge dataset and showed its superior performance to the state-of-the-art from various perspectives.

Original languageEnglish
Title of host publicationProceedings - 2023 IEEE 11th International Conference on Healthcare Informatics, ICHI 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages101-110
Number of pages10
ISBN (Electronic)9798350302639
DOIs
StatePublished - 2023
Event11th IEEE International Conference on Healthcare Informatics, ICHI 2023 - Houston, United States
Duration: 26 Jun 202329 Jun 2023

Publication series

NameProceedings - 2023 IEEE 11th International Conference on Healthcare Informatics, ICHI 2023

Conference

Conference11th IEEE International Conference on Healthcare Informatics, ICHI 2023
Country/TerritoryUnited States
CityHouston
Period26/06/2329/06/23

Keywords

  • Pre-training
  • Self-supervised pre-training
  • Supervised contrastive learning

Fingerprint

Dive into the research topics of 'Optimizing Embedding Space with Sub-categorical Supervised Pre-training: A Theoretical Approach Towards Improving Sepsis Prediction'. Together they form a unique fingerprint.

Cite this