Audience response system facilitates prediction of scores on in-training examination

Kaushal H. Shah, Jaime Jordan, Katherine Jahnes, David P. Lisbon, Lucienne Lutfy-Clayton, Grant Wei, Gary Winkel, Sally A. Santen

Research output: Contribution to journalArticlepeer-review


Introduction: Audience response systems (ARS) are increasingly popular; however, their contribution to education is not completely clear. Our study found that scores from review quizzes delivered by an ARS correlate with in-training exam (ITE) scores and are viewed positively by residents. This information may be useful in identifying poor performers early so that targeted educational interventions can be made. The objective was to determine if scores on review quizzes delivered by an ARS correlate with ITE scores and to obtain participant feedback on use of the ARS for ITE preparation. Methods: This was a prospective observational study of emergency medicine (EM) residents at six accredited EM residency programs. Subjects included residents who had taken previous ITEs. Subjects participated in bimonthly review sessions using an ARS. Twelve review quizzes were administered, each consisting of 10 multiple-choice questions. After the ITE, subjects completed an attitudinal survey consisting of six Likert-scale items and one yes/no item. We used a mixed linear model to analyze the data, accounting for prior 2012 ITE scores and nesting due to institution. Results: Among 192 participants, 135 (70.3%) completed the ITE in both 2012 and 2013; we analyzed their data for the first objective. Results from the mixed linear model indicate that the total mean score on the review quizzes was a significant [t(127) = 6.68; p < 0.001] predictor of the 2013 ITE after controlling for the 2012 ITE score. One hundred forty-six (76.0%) participants completed the attitudinal survey; 96% of respondents stated that they would like ARS to be used more often in resident education. Respondents felt the sessions aided in learning (mean 7.7/10), assisted in preparation for the ITE (mean 6.7/10), and helped identify content areas of weakness (mean 7.6/10). Conclusion: Our results suggest that scores from review quizzes delivered by an audience response system correlate with in-training exam scores and is viewed positively by residents.

Original languageEnglish
Pages (from-to)525-530
Number of pages6
JournalWestern Journal of Emergency Medicine
Issue number3
StatePublished - Apr 2017


  • Audience response system
  • Graduate medical education
  • Medical education
  • Test preparation


Dive into the research topics of 'Audience response system facilitates prediction of scores on in-training examination'. Together they form a unique fingerprint.

Cite this