Development and Validation of an Assessment Tool for Competency in Critical Care Ultrasound

Paru Patrawalla, Lewis Ari Eisen, Ariel Shiloh, Brijen J. Shah, Oleksandr Savenkov, Wendy Wise, Laura Evans, Paul Mayo, Demian Szyld

Research output: Contribution to journalArticlepeer-review

21 Scopus citations


BACKGROUND: Point-of-care ultrasound is an emerging technology in critical care medicine. Despite requirements for critical care medicine fellowship programs to demonstrate knowledge and competency in point-of-care ultrasound, tools to guide competency-based training are lacking.

OBJECTIVE: We describe the development and validity arguments of a competency assessment tool for critical care ultrasound.

METHODS: A modified Delphi method was used to develop behaviorally anchored checklists for 2 ultrasound applications: "Perform deep venous thrombosis study (DVT)" and "Qualify left ventricular function using parasternal long axis and parasternal short axis views (Echo)." One live rater and 1 video rater evaluated performance of 28 fellows. A second video rater evaluated a subset of 10 fellows. Validity evidence for content, response process, and internal consistency was assessed.

RESULTS: An expert panel finalized checklists after 2 rounds of a modified Delphi method. The DVT checklist consisted of 13 items, including 1.00 global rating step (GRS). The Echo checklist consisted of 14 items, and included 1.00 GRS for each of 2 views. Interrater reliability evaluated with a Cohen kappa between the live and video rater was 1.00 for the DVT GRS, 0.44 for the PSLA GRS, and 0.58 for the PSSA GRS. Cronbach α was 0.85 for DVT and 0.92 for Echo.

CONCLUSIONS: The findings offer preliminary evidence for the validity of competency assessment tools for 2 applications of critical care ultrasound and data on live versus video raters.

Original languageEnglish
Pages (from-to)567-573
Number of pages7
JournalJournal of graduate medical education
Issue number4
StatePublished - 1 Dec 2015


Dive into the research topics of 'Development and Validation of an Assessment Tool for Competency in Critical Care Ultrasound'. Together they form a unique fingerprint.

Cite this