Assessing Endoscopic Response in Locally Advanced Rectal Cancer Treated with Total Neoadjuvant Therapy: Development and Validation of a Highly Accurate Convolutional Neural Network

Hannah Williams, Hannah M. Thompson, Christina Lee, Aneesh Rangnekar, Jorge T. Gomez, Maria Widmar, Iris H. Wei, Emmanouil P. Pappou, Garrett M. Nash, Martin R. Weiser, Philip B. Paty, J. Joshua Smith, Harini Veeraraghavan, Julio Garcia-Aguilar

Research output: Contribution to journalArticlepeer-review

Abstract

Background: Rectal tumors display varying degrees of response to total neoadjuvant therapy (TNT). We evaluated the performance of a convolutional neural network (CNN) in interpreting endoscopic images of either a non-complete response to TNT or local regrowth during watch-and-wait surveillance. Methods: Endoscopic images from stage II/III rectal cancers treated with TNT from 2012 to 2020 at a single institution were retrospectively reviewed. Images were labelled as Tumor or No Tumor based on endoscopy timing (before, during, or after treatment) and the tumor’s endoluminal response. A CNN was trained using ResNet-50 architecture. The area under the curve (AUC) was analyzed during training and for two test sets. The main test set included images of tumors treated with TNT. The other contained images of local regrowth. The model’s performance was compared to sixteen surgeons and surgical trainees who evaluated 119 images for evidence of tumor. Fleiss’ kappa was calculated by respondent experience level. Results: A total of 2717 images from 288 patients were included; 1407 (51.8%) contained tumor. The AUC was 0.99, 0.98, and 0.92 for training, main test, and local regrowth test sets. The model performed on par with surgeons of all experience levels for the main test set. Interobserver agreement was good (k = 0.71–0.81). All groups outperformed the model in identifying tumor from images of local regrowth. Interobserver agreement was fair to moderate (k= 0.24–0.52). Conclusions: A highly accurate CNN matched the performance of colorectal surgeons in identifying a noncomplete response to TNT. However, the model demonstrated suboptimal accuracy when analyzing images of local regrowth.

Original languageEnglish
JournalAnnals of Surgical Oncology
DOIs
StateAccepted/In press - 2024
Externally publishedYes

Keywords

  • Artificial intelligence
  • Endoscopy
  • Locally advanced rectal cancer
  • Nonoperative management
  • Total neoadjuvant therapy

Fingerprint

Dive into the research topics of 'Assessing Endoscopic Response in Locally Advanced Rectal Cancer Treated with Total Neoadjuvant Therapy: Development and Validation of a Highly Accurate Convolutional Neural Network'. Together they form a unique fingerprint.

Cite this