Rater agreement on the ISCSCI motor and sensory scores obtained before and after formal training in testing technique

Background/Objective: The purpose of this study is to report the results of rater agreement for the International Standards for Neurological Classification of Spinal Cord Injury (ISCSCI) motor and sensory scores before and after training in the testing technique. Methods: Six raters performed sequential motor and sensory examinations on 5 adolescents with SCI according to the ISCSCI manual. After completion of the first examinations, all raters were provided with a half-day formal training session on testing techniques, after which the raters repeated the examinations, Intraclass correlation coefficients (ICCs) and 95% confidence intervals (CIs) were calculated to provide parameters for ICC interpretation: >0.90 = high agreement; 0.75 to 0.90 = moderate agreement; <0.75 = poor agreement. Results: After training, there was improvement in rater agreement of summed motor scores (MS) from ICC = 0.809 to 0.862 and discrimination scores from ICC = 0.786 to 0.892. There was moderate rater agreement for light touch scores (LTS) before and after training. After training, there was improvement in 95% CIs except for ICCs for LTS, but for all ICCs, the lower 95% Cl value remained less than 0.75. Conclusions: Training improved rater agreement on MS and discrimination, but 95% CIs remained unacceptably wide. The positive effect of training in motor and sensory testing techniques is supported by the study data. Unlike previous studies that have suggested the ISCSCI has acceptable reliability for clinical trials, the results of this study do not fully support the use of the ISCSCI for clinical trials without better standardization to establish a lower 95% CI value of at least 0.75.