Independent validation test of the vote-counting strategy used to rank biomarkers from published studies

Aim: Vote counting is frequently used in meta-analyses to rank biomarker candidates, but to our knowledge, there have been no independent assessments of its validity. Here, we used predictions from a recent meta-analysis to determine how well number of supporting studies, combined sample size and mean fold change performed as vote-counting strategy criteria. Materials & methods: Fifty miRNAs previously ranked for their ability to distinguish lung cancer tissue from normal were assayed by RT-qPCR using 45 paired tumor-normal samples. Results: Number of supporting studies predicted biomarker performance (p = 0.0006; r = 0.44), but sample size and fold change did not (p > 0.2). Conclusion: Despite limitations, counting the number supporting studies appears to be an effective criterion for ranking biomarkers. Predictions based on sample size and fold change provided little added value. External validation studies should be conducted to establish the performance characteristics of strategies used to rank biomarkers.