External validation of a Cox prognostic model: principles and methods

By Patrick Royston, Douglas G. Altman. BMC Medical Research Methodology, 2013

868 citations recorded by the source corpus.

BackgroundA prognostic model should not enter clinical practice unless it has been demonstrated that it performs a useful role. External validation denotes evaluation of model performance in a sample independent of that used to develop the model. Unlike for logistic regression models, external validation of Cox models is sparsely treated in the literature. Successful validation of a model means achieving satisfactory discrimination and calibration (prediction accuracy) in the validation sample. Validating Cox models is not straightforward because event probabilities are estimated relative to an unspecified baseline function.MethodsWe describe statistical approaches to external validation of a published Cox model according to the level of published information, specifically (1) the prognostic index only, (2) the prognostic index together with Kaplan-Meier curves for risk groups, and (3) the first two plus the baseline survival curve (the estimated survival function at the mean prognostic index across the sample). The most challenging task, requiring level 3 information, is assessing calibration, for which we suggest a method of approximating the baseline survival function.ResultsWe apply the methods to two comparable datasets in primary breast cancer, treating one as derivation and the other as validation sample. Results are presented for discrimination and calibration. We demonstrate plots of survival probabilities that can assist model evaluation.ConclusionsOur validation methods are applicable to a wide range of prognostic studies and provide researchers with a toolkit for external validation of a published Cox model.

Subfields: Cancer Research, Oncology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/63cfc1b471288f1110c9ae0f5de164f56acc888e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cancer Research

In Oncology

In Statistics and Probability

Citations in the corpus

Sample size considerations for the external validation of a multivariable prognostic model: a resampling study (2015, methodology): “To externally validate a model is to evaluate its predictive performance (calibration and discrimination) using a separate data set from that used to develop the model [9].”
External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges (2016, neutral): “38”
DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network (2016, background): “Medical researchers use survival models to evaluate the signiﬁcance of prognostic variables in outcomes such as death or cancer recurrence and subsequently inform patients of their treatment options [1, 2, 3, 4].”
External validation of multivariable prediction models: a systematic review of methodological conduct and reporting (2014, background): “The two key components characterising the performance of a prediction model are calibration and discrimination [14,15,34].”
Deep Survival : The Deep Cox Proportional Hazards Network (2016, background): “Medical researchers rely on survival models to evaluate the significance of prognostic variables in outcomes such as death or cancer recurrence and to inform treatment options [1, 2, 3, 4].”
Towards better clinical prediction models: seven steps for development and an ABCD for validation. (2014, methodology): “We propose four key measures in the assessment of the validation of prediction models, related to calibration, discrimination and clinical usefulness (32) (33) (34) (35).”
External validation of prognostic models: what, why, how, when and where? (2020, methodology): “Calibration determines whether the absolute predicted risks are similar to the observed risks [21].”