Performance adjusted risks: A method to improve the quality of algorithm performance while allowing all to play

Objective: Differences in quality among ultrasound nuchal translucency providers or laboratories can profoundly affect Down syndrome screening results. A new method, performance adjusted risks (PAR), is developed to allow for such differences and improve performance. Methods: Individual provider and laboratory marker distribution parameters are compared with national expectations. The maximum absolute deviation over the operating range is used to derive a handicap and weighting factors. Down syndrome risks obtained from commercial software for individual women can be corrected using the weights. Those with the biggest handicap will have the greatest correction. Five theoretical examples are used to illustrate handicap calculation and prospective combined test results on 32 Down syndrome and 7205 unaffected pregnancies are used to indicate the influence of weighting on performance. Results: In the theoretical examples, a 10% systematic change (inaccuracy) in one or more markers or a similar change in the standard deviation (imprecision) yielded a handicap ranging from 4 to 11. Over the operating range, the individual risk reduced 40% or increased 250%. When the prospective combined test results were artificially adjusted to create 10% inaccurracy in all three markers, the detection rate was only 59% but after PAR weighting this increased to 75%. Conclusions: PAR recognizes that not all providers are equal, and perfection is unrealistic. Using this approach all 'can play' while patients are protected from poor performance.