| Title: | Heller Test of Equal AUCs for a New Marker in Nested Models |
|---|---|
| Description: | Tests whether new markers improve the area under the receiver operating characteristic curve beyond established markers in nested models, using the maximum rank correlation estimator and the weighted chi-square null distribution of Heller, Seshan, Moskowitz and Gonen (2017) <doi:10.1093/biostatistics/kxw045>. The weights are estimated from the bootstrap covariance of the rank correlation estimate (or from a sandwich formula) and the curvature of the kernel smoothed area. The null hypothesis is that the best linear ordering of subjects does not use the new markers, and the test keeps its asymptotic level under any strictly increasing link. Clustered observations are handled by a cluster bootstrap. The likelihood ratio test, Wald tests with a robust covariance and the DeLong test are computed alongside for comparison. |
| Authors: | Satoshi Nakamura [aut, cre] |
| Maintainer: | Satoshi Nakamura <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.2.2 |
| Built: | 2026-11-18 00:10:52 UTC |
| Source: | https://github.com/cran/aucnull |
Tests whether adding the new markers Z to the established markers
X improves the area under the ROC curve of the best linear predictor.
The statistic is 2 n (A_full - A_reduced) with both areas obtained
by maximum rank correlation on a kernel smoothed objective. Under the null
hypothesis that the best linear ordering does not use Z, the
statistic converges to a weighted sum of independent chi-square variables
with one degree of freedom (Heller et al. 2017, Theorem 1); the weights are
the eigenvalues of -V_gamma %*% solve(D^{gamma gamma}), where
V_gamma is the asymptotic covariance of the rank correlation estimate
of the coefficients of Z and D is the Hessian of the smoothed
area. Because the weights do not involve the link, the test keeps its asymptotic level for
any strictly increasing link, which is not true of the likelihood ratio test
when the logistic link is misspecified.
aucnull_test( y, X, Z, method = c("boot", "plugin"), B = 200, expo = 1/3, n_sim = 1e+05, seed = NULL, compare = TRUE, cluster = NULL, hessian_threshold = -0.01 )aucnull_test( y, X, Z, method = c("boot", "plugin"), B = 200, expo = 1/3, n_sim = 1e+05, seed = NULL, compare = TRUE, cluster = NULL, hessian_threshold = -0.01 )
y |
binary outcome (0/1 or logical). |
X |
established markers: a numeric vector or matrix (n by p). The first column is the anchor whose coefficient is fixed at one; it must be continuous. Its orientation is chosen by fitting the reduced model with the anchor and with its negative and keeping the orientation with the larger smoothed area (for p = 1 this amounts to an empirical AUC of at least 0.5). Factors must be converted to numeric columns beforehand. |
Z |
new markers: a numeric vector or matrix (n by q). |
method |
|
B |
number of bootstrap replicates for |
expo |
bandwidth exponent for the statistic and the curvature; the
bandwidth is |
n_sim |
number of Monte Carlo draws used to evaluate the weighted
chi-square distribution when |
seed |
optional seed for the bootstrap; the random number generator state is restored on exit. |
compare |
logical; if |
cluster |
optional vector of cluster identifiers of length n. |
hessian_threshold |
identifiability threshold on the standardized scale:
if the smallest eigenvalue of the curvature of the smoothed area in the
coefficients of |
Two estimators of V_gamma are offered. "boot" (recommended)
uses n times the bootstrap covariance of the rank correlation
estimate. "plugin" uses the sandwich form solve(D) Sigma_g solve(D)
with Sigma_g estimated from the per observation gradients of the
Hajek projection of the smoothed area, computed at the bandwidth
sd(X[,1]) n^(-1/3) regardless of expo.
Clustered data (for example several lesions per patient) are handled through
cluster: the bootstrap then resamples clusters, the plugin covariance
sums the projections within clusters, and the comparison tests use a cluster
robust covariance. The limiting distribution is unchanged; the asymptotics are
in the number of clusters, so the weights are poorly estimated when there are
few clusters. Rows with missing values in y, X, Z or
cluster are dropped with a message.
an object of class "aucnull_test": a list with the statistic,
the weights lambda, the p value, the rank correlation estimates,
the two smoothed areas, the number of clusters, and (if compare = TRUE)
the comparison tests.
Heller G, Seshan VE, Moskowitz CS, Gonen M (2017). Inference for the difference in the area under the ROC curve derived from nested binary regression models. Biostatistics 18, 260-274.
Han AK (1987). Non-parametric analysis of a generalized regression model. Journal of Econometrics 35, 303-316.
Sherman RP (1993). The limiting distribution of the maximum rank correlation estimator. Econometrica 61, 123-137.
Demler OV, Pencina MJ, D'Agostino RB (2012). Misuse of DeLong test to compare AUCs for nested models. Statistics in Medicine 31, 2577-2587.
set.seed(1) n <- 200 x <- rnorm(n); z <- rnorm(n) y <- rbinom(n, 1, plogis(1.2 * x)) # z carries no information aucnull_test(y, x, z, B = 20) # use the default B = 200 in practice # clustered data: two observations per cluster id <- rep(1:100, each = 2) aucnull_test(y, x, z, B = 20, cluster = id)set.seed(1) n <- 200 x <- rnorm(n); z <- rnorm(n) y <- rbinom(n, 1, plogis(1.2 * x)) # z carries no information aucnull_test(y, x, z, B = 20) # use the default B = 200 in practice # clustered data: two observations per cluster id <- rep(1:100, each = 2) aucnull_test(y, x, z, B = 20, cluster = id)
Provided for comparison only. It is valid for two fixed scores on the same subjects and is not valid for the linear predictors of nested models fitted on the same data (Demler et al. 2012).
delong_test(s1, s2, y)delong_test(s1, s2, y)
s1, s2
|
scores. |
y |
binary outcome. |
list with the two empirical areas, their difference, the standard error, the z statistic and the two sided p value.
Maximizes the smoothed area smooth_auc of the linear predictor
X[,1] + X[,-1] %*% eta + Z %*% gamma over (eta, gamma).
The coefficient of the first established marker is fixed at one for
identifiability (Han 1987; Sherman 1993). Pass Z = NULL for the
reduced model.
mrc_fit(y, X, Z = NULL, h, start = NULL, reltol = 1e-10)mrc_fit(y, X, Z = NULL, h, start = NULL, reltol = 1e-10)
y |
binary outcome. |
X |
numeric matrix (n by p) of established markers, p >= 1. The first column should be oriented so that larger values indicate a case. |
Z |
numeric matrix (n by q) of new markers, or |
h |
bandwidth. |
start |
optional starting values of length p - 1 + q; by default the
logistic estimates scaled to the normalization are supplied by
|
reltol |
convergence tolerance passed to |
a list with theta (named coefficient vector, excluding the
fixed first coefficient), auc (maximized smoothed area), the
objective function f (returns the smoothed area at a parameter
vector) and convergence.
aucnull_test objectsPrint method for aucnull_test objects
## S3 method for class 'aucnull_test' print(x, digits = 4, ...)## S3 method for class 'aucnull_test' print(x, digits = 4, ...)
x |
an object returned by |
digits |
number of significant digits (currently unused; four are shown). |
... |
further arguments, ignored. |
x, invisibly.
Returns P(sum lambda_j chi2_1j > x). Exact for a single weight;
Monte Carlo otherwise.
pweightedchisq(x, lambda, n_sim = 1e+05)pweightedchisq(x, lambda, n_sim = 1e+05)
x |
value of the statistic. |
lambda |
vector of weights. |
n_sim |
number of Monte Carlo draws when |
upper tail probability.
Computes the kernel smoothed Mann and Whitney statistic
,
as used by Heller et al. (2017) to obtain a differentiable objective.
smooth_auc(score, y, h)smooth_auc(score, y, h)
score |
numeric vector of scores (higher means more likely to be a case). |
y |
binary outcome (0/1, or logical). |
h |
bandwidth. With |
a single number in (0, 1).