Notes on the C-statistic as a measure of predictive performance

The area under the receiver operating characteristic (ROC) curve or \(C\)-statistic can be defined as the probability of correctly classifying a case-control pair. If the predictor is computed from many independent variables, the following results hold (McKeigue, Stat Methods Med Res 2017, in press):-

• The \(C\)-statistic is a function of the expected log-likelihood ratio \(\Lambda\) favouring case status over control status in a case, or control over case status in a control. When natural logarithms are used, this function is \[\begin{equation} C = 1 - \Phi \left(-\sqrt{\Lambda} \right) \end{equation}\] where \(\Phi\) is the standard gaussian cumulative distribution function. This can be inverted to express \(\Lambda\) as a function of \(C\):- \[\begin{equation} \Lambda = \left[\Phi^{-1} \left( C \right) \right]^2 \end{equation}\]
• The sampling distribution of the log-likelihood ratio is gaussian with variance \(2 \Lambda\).

\(\Lambda\) can be interpreted as the average information about case-control status that is conveyed by the predictor. Mathematically it is a Kullback-Leibler divergence, which can take any non-negative value. The \(C\) statistic can be interpreted as a mapping of \(\Lambda\) to the interval from 0.5 to 1.

These results have practical implications:-

1. The frequent complaint that the \(C\)-statistic does not tell us how well the predictor will perform at low or high levels of risk is not valid for a predictor based on many independent risk factors. From the \(C\)-statistic, we can calculate any quantile of the likelihood ratio, and we can multiply this likelihood ratio by the prior odds of disease to obtain the posterior odds.

2. If the \(C\)-statistic is transformed to express predictive performance on the the scale of expected log-likelihood ratio, the contributions of independent predictors can be added.

3. If interval estimates (confidence intervals or credible intervals) are required, it is easier to calculate them on the scale of \(\Lambda\), then transform to the scale of \(C\).

In defining the \(C\)-statistic as the probability of correctly classifying a case-control pair, we would usually expect these case-control pairs to be matched for age and sex. If risk varies markedly with age and sex, and the predictive performance of a model that includes age and sex is calculated as the area under the ROC curve, the \(C\)-statistic may be misleadingly high. Working on the scale of expected log-likelihood ratio overcomes this problem, because the contributions of independent predictors are additive on this scale.