Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers

By Michael J. Pencina, Ewout W. Steyerberg, Ralph B. D’Agostino. Statistics in Medicine, 2010

2,465 citations recorded by the source corpus.

Appropriate quantification of added usefulness offered by new markers included in risk prediction algorithms is a problem of active research and debate. Standard methods, including statistical significance and c statistic are useful but not sufficient. Net reclassification improvement (NRI) offers a simple intuitive way of quantifying improvement offered by new markers and has been gaining popularity among researchers. However, several aspects of the NRI have not been studied in sufficient detail.

Subfields: Economics and Econometrics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/cb8531867f895658de6ef62fa29ff989c391bff8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Statistics and Probability

Citations in the corpus

Towards better clinical prediction models: seven steps for development and an ABCD for validation. (2014, neutral): “predictions to be meaningful (45) (46).”
Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker (2012, methodology): “To circumvent this problem, a version of the NRI that does not require stratification of the population into risk group may be used.(87) Alternatively, the integrated discrimination improvement (IDI) may be useful.”
A unified inference procedure for a class of measures to assess improvement in risk prediction systems with survival data (2012, methodology): “Z0I t0/ > 0 j T 0 > t0 (2) proposed by Pencina et al. [17] as an extension of NRI discussed in [13].”
Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist (2014, background): “Reclassification measures, such as net reclassification improvement or index (NRI), evaluate whether a single biomarker has any incremental value to a prediction model [124,126].”
Dynamic data during hypotensive episode improves mortality predictions among patients with sepsis and hypotension (2013, methodology): “Therefore, we calculated the Net Reclassification Improvement (NRI) (37) (38), which measures the ability of a new model to reclassify a high risk individual as higher risk, and a low risk individual as lower risk, for our model as compared to APACHE-IV.”
Galectin-3, a Marker of Cardiac Fibrosis, Predicts Incident Heart Failure in the Community (2012, methodology): “We estimated the integrated discrimiation improvement (IDI) and the category-free net reclasification improvement (NRI) metric for the addition of al-3 in fully adjusted models (25,26).”
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research (2013, background): “These different statistics are mathematically related, however [51,52].”
Further insight into the incremental value of new markers: the interpretation of performance measures and the importance of clinical context. (2012, background): “They pay particular attention to the change in the area under the receiver operating characteristic curve (ΔAUC), integrated discrimination improvement (IDI) (2), and continuous net reclassification improvement (NRI(>0)) (3).”
A predictive model for progression of chronic kidney disease to kidney failure. (2011, methodology): “To evaluate the effect of definition of risk categories on reclassification, we calculated NRI for alternative risk thresholds in CKD stages 3 and 4 and using an alternative method that does not require categories.(25) To evaluate the effect of the competing risk of mortality before kidney failure o”
Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer. (2016, methodology): “Therefore, the incremental value of histologic grade as an additional candidate predictor was evaluated, with C-index and calibration curve derived and the net reclassification improvement (NRI) calculated.(27,28)”