Abstract
Demonstrated are several methods to develop and to evaluate classification based on symptom lists. Methods are derived from simple counts, logistic regression analysis, Quality Receiver Operating Characteristic (QROC) analyses or a combination. Nine approaches were compared: Best Single Item, Count of Entire Symptom list, Count of Refined Symptom list, Logistic Regression All In Score, Logistic Regression Backward Score, Logistic Regression Backward Count, Logistic Regression Forward Score, Logistic Regression Forward Count and QROC Decision Tree. The validation criterion and symptom lists were based on a computer diagnosis of Attention Deficit Hyperactivity Disorder (ADHD), and the measure of agreement between classification and criterion was the weighted kappa. We present the argument that sensitivity and specificity are uncalibrated measures of test performance. Valuable indices of test quality resulted from calibrating sensitivity and specificity by the proportion of test positives. When plotted, the new indices create a QROC plane which facilitates symptom evaluation and combination. Of all the scoring approaches, the Logistic Regression All In was optimally efficient. For ease of use and practicality, a simple count of predictive items may be the optimal choice. Implications for the evaluation of DSM-III-R symptom lists are discussed. We recommend that, instead of reporting scores at a set cut-off point, the full range of scores should be illustrated in a QROC plane. This permits others to evaluate and select cut-off points that are optimal for the research or clinical situation.
Original language | English |
---|---|
Pages (from-to) | 223-236 |
Number of pages | 14 |
Journal | International Journal of Methods in Psychiatric Research |
Volume | 5 |
Issue number | 4 |
State | Published - 1996 |
Externally published | Yes |
Keywords
- Classification
- Diagnostic efficiency
- Logistic regression analysis
- Receiver operating characteristic analysis
- Screening