Development of webcam-collected and artificial-intelligence-derived social and cognitive performance measures for neurodevelopmental genetic syndromes

Thomas W. Frazier, Robyn M. Busch, Patricia Klaas, Katherine Lachlan, Shafali Jeste, Alexander Kolevzon, Eva Loth, Jacqueline Harris, Leslie Speer, Tom Pepper, Kristin Anthony, J. Michael Graglia, Christal G. Delagrammatikas, Sandra Bedrosian-Sermone, Constance Smith-Hicks, Katie Huba, Robert Longyear, Lee Anne Green-Snyder, Frederick Shic, Mustafa SahinCharis Eng, Antonio Y. Hardan, Mirko Uljarević

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


This study focused on the development and initial psychometric evaluation of a set of online, webcam-collected, and artificial intelligence-derived patient performance measures for neurodevelopmental genetic syndromes (NDGS). Initial testing and qualitative input was used to develop four stimulus paradigms capturing social and cognitive processes, including social attention, receptive vocabulary, processing speed, and single-word reading. The paradigms were administered to a sample of 375 participants, including 163 with NDGS, 56 with idiopathic neurodevelopmental disability (NDD), and 156 neurotypical controls. Twelve measures were created from the four stimulus paradigms. Valid completion rates varied from 87 to 100% across measures, with lower but adequate completion rates in participants with intellectual disability. Adequate to excellent internal consistency reliability (α = 0.67 to 0.95) was observed across measures. Test–retest reproducibility at 1-month follow-up and stability at 4-month follow-up was fair to good (r = 0.40–0.73) for 8 of the 12 measures. All gaze-based measures showed evidence of convergent and discriminant validity with parent-report measures of other cognitive and behavioral constructs. Comparisons across NDGS groups revealed distinct patterns of social and cognitive functioning, including people with PTEN mutations showing a less impaired overall pattern and people with SYNGAP1 mutations showing more attentional, processing speed, and social processing difficulties relative to people with NFIX mutations. Webcam-collected performance measures appear to be a reliable and potentially useful method for objective characterization and monitoring of social and cognitive processes in NDGS and idiopathic NDD. Additional validation work, including more detailed convergent and discriminant validity analyses and examination of sensitivity to change, is needed to replicate and extend these observations.

Original languageEnglish
Article numbere32058
JournalAmerican Journal of Medical Genetics, Part C: Seminars in Medical Genetics
Issue number3
StatePublished - Sep 2023


  • eye tracking
  • facial expressions
  • genetic syndromes
  • neurodevelopment
  • webcam


Dive into the research topics of 'Development of webcam-collected and artificial-intelligence-derived social and cognitive performance measures for neurodevelopmental genetic syndromes'. Together they form a unique fingerprint.

Cite this