The Net Reclassification Index (NRI): A Misleading Measure of Prediction Improvement Even with Independent Test Data Sets

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningfagfællebedømt

Margaret S. Pepe, Jing Fan, Ziding Feng, Thomas Gerds, Jorgen Hilden

The Net Reclassification Index (NRI) is a very popular measure for evaluating the improvement in prediction performance gained by adding a marker to a set of baseline predictors. However, the statistical properties of this novel measure have not been explored in depth. We demonstrate the alarming result that the NRI statistic calculated on a large test dataset using risk models derived from a training set is likely to be positive even when the new marker has no predictive information. A related theoretical example is provided in which an incorrect risk function that includes an uninformative marker is proven to erroneously yield a positive NRI. Some insight into this phenomenon is provided. Since large values for the NRI statistic may simply be due to use of poorly fitting risk models, we suggest caution in using the NRI as the basis for marker evaluation. Other measures of prediction performance improvement, such as measures derived from the receiver operating characteristic curve, the net benefit function, and the Brier score, cannot be large due to poorly fitting risk functions.
OriginalsprogEngelsk
TidsskriftStatistics in BioSciences
Vol/bind7
Udgave nummer2
Sider (fra-til)282-295
Antal sider14
ISSN1867-1764
DOI
StatusUdgivet - okt. 2015

ID: 140154597