Introduction to ssp.quantreg: Subsampling for Quantile Regression

library(subsampling)

This vignette introduces the usage of ssp.quantreg. The statistical theory and algorithms behind this implementation can be found in the relevant reference papers.

Quantile regression aims to estimate conditional quantiles by minimizing the following loss function:

$$ \min_{\beta} L(\beta) = \frac{1}{N} \sum_{i=1}^{N} \rho_\tau \left( y_i - \beta^\top x_i \right) = \frac{1}{N} \sum_{i=1}^{N} \left( y_i - \beta^\top x_i \right) \left\{ \tau - I \left( y_i < \beta^\top x_i \right) \right\}, $$ where τ is the quantile of interest, y is the response variable, x is covariates vector and N is the number of observations in full dataset.

The idea of subsampling methods is as follows: instead of fitting the model on the size N full dataset, a subsampling probability is assigned to each observation and a smaller, informative subsample is drawn. The model is then fitted on the subsample to obtain an estimator with reduced computational cost.

Terminology

  • Full dataset: The whole dataset used as input.

  • Full data estimator: The estimator obtained by fitting the model on the full dataset.

  • Subsample: A subset of observations drawn from the full dataset.

  • Subsample estimator: The estimator obtained by fitting the model on the subsample.

  • Subsampling probability (π): The probability assigned to each observation for inclusion in the subsample.

Example

We introduce ssp.quantreg with simulated data. X contains d = 6 covariates drawn from multinormal distribution and Y is the response variable. The full data size is N = 1 × 104. The interested quantile τ = 0.75.

set.seed(1)
N <- 1e4
tau <- 0.75
beta.true <- rep(1, 7)
d <- length(beta.true) - 1
corr  <- 0.5
sigmax  <- matrix(0, d, d)
for (i in 1:d) for (j in 1:d) sigmax[i, j] <- corr^(abs(i-j))
X <- MASS::mvrnorm(N, rep(0, d), sigmax)
err <- rnorm(N, 0, 1) - qnorm(tau)
Y <- beta.true[1] + X %*% beta.true[-1] + err * rowMeans(abs(X))
data <- as.data.frame(cbind(Y, X))
colnames(data) <- c("Y", paste("V", 1:ncol(X), sep=""))
formula <- Y ~ .
head(data)
#>            Y          V1          V2         V3         V4          V5
#> 1  3.2538275  0.04903563  0.03653273 -0.2117768  0.8739927  0.98805763
#> 2  0.3007946 -0.14207288 -1.15194889 -0.5345835  0.1808184  0.71585268
#> 3  3.4726715 -0.77028541  0.21386411  0.4376943  1.3864691  0.61171516
#> 4 -8.3576405 -2.67054518 -1.50329615 -0.9850075  0.1603506 -0.91122179
#> 5 -0.4599822  0.06629031 -0.58973412 -0.3738513 -0.7433070 -0.07916941
#> 6  2.2812710  0.58562537 -0.38872983  0.4672438  0.8128496  1.75186481
#>           V6
#> 1  0.6753897
#> 2  0.3671586
#> 3  1.0002106
#> 4 -0.7965586
#> 5  0.9402104
#> 6 -0.3151219

Key Arguments

The function usage is

ssp.quantreg(
  formula,
  data,
  subset = NULL,
  tau = 0.5,
  n.plt,
  n.ssp,
  B = 5,
  boot = TRUE,
  criterion = "optL",
  sampling.method = "withReplacement",
  likelihood = c("weighted"),
  control = list(...),
  contrasts = NULL,
  ...
)

The core functionality of ssp.quantreg revolves around three key questions:

  • How are subsampling probabilities computed? (Controlled by the criterion argument)

  • How is the subsample drawn? (Controlled by the sampling.method argument)

  • How is the likelihood adjusted to correct for bias? (Controlled by the likelihood argument)

criterion

criterion stands for the criterion we choose to compute the sampling probability for each observation. The choices of criterion include optL(default) and uniform. In optL, the optimal subsampling probability is by minimizing a transformation of the asymptotic variance of subsample estimator. uniform is a baseline method.

sampling.method

The options for the sampling.method argument include withReplacement (default) and poisson. withReplacement stands for drawing n.ssp subsamples from full dataset with replacement, using the specified subsampling probabilities. poisson stands for drawing subsamples one by one by comparing the subsampling probability with a realization of uniform random variable U(0, 1). The expected number of drawn samples are n.ssp.

likelihood

The available choice for likelihood in ssp.quantreg is weighted. It takes the inverse of sampling probabblity as the weights in likelihood function to correct the bias introduced by unequal subsampling probabilities.

boot and B

An option for drawing B subsamples (each with expected size n.ssp) and deriving subsample estimator and asymptotic covariance matrix based on them. After getting β̂b on the b-th subsample, b = 1, …B, it calculates

$$ \hat{\beta}_I = \frac{1}{B} \sum_{b=1}^{B} \hat{\beta}_{b} $$ as the final subsample estimator and $$ \hat{V}(\hat{\beta}_I) = \frac{1}{r_{ef} B (B - 1)} \sum_{b=1}^{B} \left( \hat{\beta}_{b} - \hat{\beta}_I \right)^{\otimes 2}, $$ where ref is a correction term for effective subsample size since the observations in each subsample can be replicated. For more details, see Wang and Ma (2021).

Outputs

After drawing subsample(s), ssp.quantreg utilizes quantreg::rq to fit the model on the subsample(s). Arguments accepted by quantreg::rq can be passed through ... in ssp.quantreg.

Below are two examples demonstrating the use of ssp.quantreg with different configurations.

B <- 5
n.plt <- 200
n.ssp <- 200
ssp.results1 <- ssp.quantreg(formula, 
                             data, 
                             tau = tau, 
                             n.plt = n.plt,
                             n.ssp = n.ssp,
                             B = B, 
                             boot = TRUE, 
                             criterion = 'optL',
                             sampling.method = 'withReplacement', 
                             likelihood = 'weighted'
                             )

ssp.results2 <- ssp.quantreg(formula, 
                             data, 
                             tau = tau, 
                             n.plt = n.plt,
                             n.ssp = n.ssp,
                             B = B, 
                             boot = FALSE, 
                             criterion = 'optL',
                             sampling.method = 'withReplacement', 
                             likelihood = 'weighted'
                             )

Returned object

The returned object contains estimation results and index of drawn subsample in the full dataset.

names(ssp.results1)
#> [1] "model.call"            "coef.plt"              "coef"                 
#> [4] "cov"                   "index.plt"             "index.ssp"            
#> [7] "N"                     "subsample.size.expect" "terms"
summary(ssp.results1)
#> Model Summary
#> 
#> 
#> Call:
#> 
#> ssp.quantreg(formula = formula, data = data, tau = tau, n.plt = n.plt, 
#>     n.ssp = n.ssp, B = B, boot = TRUE, criterion = "optL", sampling.method = "withReplacement", 
#>     likelihood = "weighted")
#> 
#> Subsample Size:
#> [1] 1000
#> 
#> Coefficients:
#> 
#>           Estimate Std. Error z value Pr(>|z|)
#> Intercept   0.9801     0.0398 24.6242  <0.0001
#> V1          1.0197     0.0348 29.3105  <0.0001
#> V2          1.0092     0.0304 33.1692  <0.0001
#> V3          0.9572     0.0284 33.6474  <0.0001
#> V4          0.9597     0.0422 22.7536  <0.0001
#> V5          1.0507     0.0289 36.3173  <0.0001
#> V6          0.9735     0.0304 31.9773  <0.0001
summary(ssp.results2)
#> Model Summary
#> 
#> 
#> Call:
#> 
#> ssp.quantreg(formula = formula, data = data, tau = tau, n.plt = n.plt, 
#>     n.ssp = n.ssp, B = B, boot = FALSE, criterion = "optL", sampling.method = "withReplacement", 
#>     likelihood = "weighted")
#> 
#> Subsample Size:
#> [1] 1000
#> 
#> Coefficients:
#> 
#>           Estimate
#> Intercept   1.0012
#> V1          1.0077
#> V2          0.9831
#> V3          1.0389
#> V4          1.0425
#> V5          0.9442
#> V6          0.9526

Some key returned variables:

  • index.plt and index are the row indices of drawn pilot subsamples and optimal subsamples in the full data.

  • coef.ssp is the subsample estimator for β and coef is the linear combination of coef.plt (pilot estimator) and coef.ssp.

  • cov.ssp and cov are estimated covariance matrices of coef.ssp and coef. If boot=FALSE, covariance matrix would not be estimated and a size n.ssp * B subsample would be drawn.

See the help documentation of ssp.quantreg for details.

References

Wang, Haiying, and Yanyuan Ma. 2021. “Optimal Subsampling for Quantile Regression in Big Data.” Biometrika 108 (1): 99–112.