Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation

Objective Nasal endoscopy is integral to the evaluation of sinonasal disorders. However, prior studies have shown significant variability in the interrater agreement of nasal endoscopy interpretation among practicing rhinologists. The objective of the current study is to evaluate the interrater agreement of nasal endoscopy among otolaryngology residents from a single training program at baseline and following an educational intervention. Methods Eleven otolaryngology residents completed nasal endoscopy grading forms for eight digitally recorded nasal endoscopic examinations. An instructional lecture reviewing nasal endoscopy interpretation was subsequently provided. The residents then completed grading forms for eight different nasal endoscopic examinations. Interrater agreement among residents for the pre- and postlecture videos was calculated using the unweighted Fleiss' kappa (Kf) statistic and intraclass correlation agreement (ICC). Results Interrater agreement improved from a baseline level of fair (Kf range 0.268-0.383) to a posteducational level of moderate (Kf range 0.401-0.547) for nasal endoscopy findings of middle meatus mucosa, middle turbinate mucosa, middle meatus discharge, sphenoethmoid recess mucosa, sphenoethmoid recess discharge, and atypical lesions (ICC, P < 0.001). The baseline level of agreement for evaluation of nasal septum deviation was poor/fair and did not improve following educational intervention. Conclusions This study demonstrates a limited baseline level of interrater agreement of nasal endoscopy interpretation among otolaryngology residents. The interrater agreement for the majority of the characteristics that were evaluated improved after educational intervention. Further study is needed to improve nasal endoscopy interpretation.