Title: | Inference on Prototypes from Clusters of Features |
---|---|
Description: | Procedures for testing for group-wide signal in clusters of variables. Tests can be performed for single groups in isolation (univariate) or multiple groups together (multivariate). Specific tests include the exact and approximate (un)selective likelihood ratio tests described in Reid et al (2015), the selective F test and marginal screening prototype test of Reid and Tibshirani (2015). User may pre-specify columns to be included in prototype formation, or allow the function to select them itself. A mixture of these two is also possible. Any variable selection is accounted for using the selective inference framework. Options for non-sampling and hit-and-run null reference distributions. |
Authors: | Stephen Reid |
Maintainer: | Stephen Reid <[email protected]> |
License: | GPL (>= 2) |
Version: | 1.2 |
Built: | 2024-12-22 06:43:25 UTC |
Source: | CRAN |
Procedures for testing for group-wide signal in clusters of variables. Tests can be perfromed for single groups in isolation (univariate) or multiple groups together (multivariate). Specific tests include the exact and approximate (un)selective likelihood ratio (ELR, ALR) tests described in Reid et al (2015), the selective F test and marginal screening prototype test of Reid and Tibshirani (2015). User may prespecify columns to be included in prototype formation, or allow the function to select them itself. A mixture of these two is also possible. Any variable selection is accounted for using the selective inference framework introduced in Lee et al (2013) and further developed in Lee and Taylor (2014). Options for non-sampling and hit-and-run null reference distrbutions. Tests are examples of selected model tests, a notion introduced in Fithian et al (2015).
Package: | prototest |
Type: | Package |
Version: | 1.0 |
Date: | 2015-11-12 |
License: | GPL (>= 2) |
Only two functions provided: prototest.univariate
(for tests with a single group in isolation) and prototest.multivariate
(for tests with multiple groups simultaneously). Each function provides options to perform one of the ELR, ALR, F or marginal screening prototype tests. User may specify which columns are to be used in prototype construction, or leave it for the function to select. Valid tests are performed in the event of variable selection. User has option to use non-sampling null reference distributions (where available) or hit-and-run references.
Stephen Reid
Maintainer: Stephen Reid <[email protected]>
Reid, S. and Tibshirani, R. (2015) Sparse regression and marginal testing using cluster prototypes. http://arxiv.org/pdf/1503.00334v2.pdf. Biostatistics doi:10.1093/biostatistics/kxv049
Reid, S., Taylor, J. and Tibshirani, R. (2015) A general framework for estimation and inference from clusters of features. Available online: http://arxiv.org/abs/1511.07839
Lee, J.D., Sun, D.L., Sun, Y. and Taylor, J.E. (2013) Exact post-selection inference, with application to the lasso. http://arxiv.org/pdf/1311.6238v6.pdf. Annals of Statistics (to appear)
Lee, J.D. and Taylor, J.E. (2014) Exact Post Model Selection Inference for Marginal Screening. http://arxiv.org/pdf/1402.5596v2.pdf
Fithian, W., Sun, D.L. and Taylor, J.E. (2015) Optimal Inference After Model Selection. http://arxiv.org/pdf/1410.2597v2.pdf
require (prototest) ### generate data set.seed (12345) n = 100 p = 80 X = matrix (rnorm(n*p, 0, 1), ncol=p) beta = rep(0, p) beta[1:3] = 0.1 # three signal variables: number 1, 2, 3 signal = apply(X, 1, function(col){sum(beta*col)}) intercept = 3 y = intercept + signal + rnorm (n, 0, 1) ### treat all columns as if in same group and test for signal # non-selective ELR test with nuisance intercept elr = prototest.univariate (X, y, "ELR", selected.col=1:5) # selective F test with nuisance intercept; non-sampling f.test = prototest.univariate (X, y, "F", lambda=0.01, hr.iter=0) print (elr) print (f.test) ### assume variables occur in 4 equally sized groups num.groups = 4 groups = rep (1:num.groups, each=p/num.groups) # selective ALR test -- select columns 21-25 in 2nd group; test for signal in 1st; hit-and-run alr = prototest.multivariate(X, y, groups, 1, "ALR", 21:25, lambda=0.005, hr.iter=20000) # non-selective MS test -- specify first column in each group; test for signal in 1st ms = prototest.multivariate(X, y, groups, 1, "MS", c(1,21,41,61)) print (alr) print (ms)
require (prototest) ### generate data set.seed (12345) n = 100 p = 80 X = matrix (rnorm(n*p, 0, 1), ncol=p) beta = rep(0, p) beta[1:3] = 0.1 # three signal variables: number 1, 2, 3 signal = apply(X, 1, function(col){sum(beta*col)}) intercept = 3 y = intercept + signal + rnorm (n, 0, 1) ### treat all columns as if in same group and test for signal # non-selective ELR test with nuisance intercept elr = prototest.univariate (X, y, "ELR", selected.col=1:5) # selective F test with nuisance intercept; non-sampling f.test = prototest.univariate (X, y, "F", lambda=0.01, hr.iter=0) print (elr) print (f.test) ### assume variables occur in 4 equally sized groups num.groups = 4 groups = rep (1:num.groups, each=p/num.groups) # selective ALR test -- select columns 21-25 in 2nd group; test for signal in 1st; hit-and-run alr = prototest.multivariate(X, y, groups, 1, "ALR", 21:25, lambda=0.005, hr.iter=20000) # non-selective MS test -- specify first column in each group; test for signal in 1st ms = prototest.multivariate(X, y, groups, 1, "MS", c(1,21,41,61)) print (alr) print (ms)
prototest
object
Generic print
method for prototest
objects
## S3 method for class 'prototest' print(x, ...)
## S3 method for class 'prototest' print(x, ...)
x |
object of type |
... |
other parameters passed to |
Prints the test statistic and p-value associated with the prototest
object x
.
Stephen Reid
prototest.univariate
, prototest.multivariate
require (prototest) ### generate data set.seed (12345) n = 100 p = 80 X = matrix (rnorm(n*p, 0, 1), ncol=p) beta = rep(0, p) beta[1:3] = 2 # three signal variables: number 1, 2, 3 signal = apply(X, 1, function(col){sum(beta*col)}) intercept = 3 y = intercept + signal + rnorm (n, 0, 1) ### treat all columns as if in same group and test for signal # non-selective ELR test with nuisance intercept elr = prototest.univariate (X, y, "ELR", selected.col=1:5) print (elr)
require (prototest) ### generate data set.seed (12345) n = 100 p = 80 X = matrix (rnorm(n*p, 0, 1), ncol=p) beta = rep(0, p) beta[1:3] = 2 # three signal variables: number 1, 2, 3 signal = apply(X, 1, function(col){sum(beta*col)}) intercept = 3 y = intercept + signal + rnorm (n, 0, 1) ### treat all columns as if in same group and test for signal # non-selective ELR test with nuisance intercept elr = prototest.univariate (X, y, "ELR", selected.col=1:5) print (elr)
Perform prototype or F tests for significance of groups of predictors in the multivariate model. Choose either exact or approximate likelihood ratio prototype tests (ELR) or (ALR) or F test or marginal screening prototype test. Options for selective or non-selective tests. Further options for non-sampling or hit-and-run reference distributions for selective tests.
prototest.multivariate(x, y, groups, test.group, type = c("ELR", "ALR", "F", "MS"), selected.col = NULL, lambda, mu = NULL, sigma = 1, hr.iter = 50000, hr.burn.in = 5000, verbose = FALSE, tol = 10^-8)
prototest.multivariate(x, y, groups, test.group, type = c("ELR", "ALR", "F", "MS"), selected.col = NULL, lambda, mu = NULL, sigma = 1, hr.iter = 50000, hr.burn.in = 5000, verbose = FALSE, tol = 10^-8)
x |
input matrix of dimension n-by-p, where p is the number of predictors over all predictor groups of interest. Will be mean centered and standardised before tests are performed. |
y |
response variable. Vector of length n, assumed to be quantitative. |
groups |
group membership of the columns of |
test.group |
group label for which we test nullity. Should be one of the values seen in |
type |
type of test to be performed. Can select one at a time. Options include the exact and approximate likelihood ratio prototype tests of Reid et al (2015) (ELR, ALR), the F test and the marginal screening prototype test of Reid and Tibshirani (2015) (MS). Default is ELR. |
selected.col |
preselected columns selected by the user. Vector of indices in the set {1, 2, ... p}. Used in conjunction with |
lambda |
regularisation parameter for the lasso fit. Same for each group. Must be supplied when at least one group has unspecified columns in |
mu |
mean parameter for the response. See Details below. If supplied, it is first subtracted from the response to yield a zero-mean (at the population level) vector for which we proceed with testing. If |
sigma |
error standard deviation for the response. See Details below. Must be supplied. If not, it is assumed to be 1. Required for computation of some of the test statistics. |
hr.iter |
number of hit-and-run samples required in the reference distribution of the a selective test. Applies only if |
hr.burn.in |
number of burn-in hit-and-run samples. These are generated first so as to make subsequent hit-and-run realisations less dependent on the observed response. Samples are then discarded and do not inform the null reference distribution. |
verbose |
should progress be printed? |
tol |
convergence threshold for iterative optimisation procedures. |
The model underpinning each of the tests is
where and K is the number of predictor groups.
depends on the particular test considered.
In particular, for the ELR, ALR and F tests, we have , where
.
is the input matrix reduced to the columns with indices in the set M.
is the set of indices selected from considering group k of predictors in isolation. This set is either provided by the user (via
selected.col
) or is selected automatically (if selected.col
is NULL
). If the former, a non-selective test is performed; if the latter, a selective test is performed, with the restrictions , as set out in Lee et al (2015) and stacked as in Reid and Tibshirani (2015).
For the marginal screening prototype (MS) test, where
is the
column of
x
and , where
is the set of indices in the overall predictor set corresponding to predictors in the
group.
All tests test the null hypothesis , where
is supplied by the user via
test.group
. Details of each are described in Reid et al (2015).
A list with the following four components:
ts |
The value of the test statistic on the observed data. |
p.val |
Valid p-value of the test. |
selected.col |
Vector with columns selected for prototype formation in the test. If initially |
y.hr |
Matrix with hit-and-run replications of the response. If sampled selective test was not performed, this will be |
Stephen Reid
Reid, S. and Tibshirani, R. (2015) Sparse regression and marginal testing using cluster prototypes. http://arxiv.org/pdf/1503.00334v2.pdf. Biostatistics doi:10.1093/biostatistics/kxv049
Reid, S., Taylor, J. and Tibshirani, R. (2015) A general framework for estimation and inference from clusters of features. Available online: http://arxiv.org/abs/1511.07839.
require (prototest) ### generate data set.seed (12345) n = 100 p = 80 X = matrix (rnorm(n*p, 0, 1), ncol=p) beta = rep(0, p) beta[1:3] = 0.1 # three signal variables: number 1, 2, 3 signal = apply(X, 1, function(col){sum(beta*col)}) intercept = 3 y = intercept + signal + rnorm (n, 0, 1) ### treat all columns as if in same group and test for signal # non-selective ELR test with nuisance intercept elr = prototest.univariate (X, y, "ELR", selected.col=1:5) # selective F test with nuisance intercept; non-sampling f.test = prototest.univariate (X, y, "F", lambda=0.01, hr.iter=0) print (elr) print (f.test) ### assume variables occur in 4 equally sized groups num.groups = 4 groups = rep (1:num.groups, each=p/num.groups) # selective ALR test -- select columns 21-25 in 2nd group; test for signal in 1st; hit-and-run alr = prototest.multivariate(X, y, groups, 1, "ALR", 21:25, lambda=0.005, hr.iter=20000) # non-selective MS test -- specify first column in each group; test for signal in 1st ms = prototest.multivariate(X, y, groups, 1, "MS", c(1,21,41,61)) print (alr) print (ms)
require (prototest) ### generate data set.seed (12345) n = 100 p = 80 X = matrix (rnorm(n*p, 0, 1), ncol=p) beta = rep(0, p) beta[1:3] = 0.1 # three signal variables: number 1, 2, 3 signal = apply(X, 1, function(col){sum(beta*col)}) intercept = 3 y = intercept + signal + rnorm (n, 0, 1) ### treat all columns as if in same group and test for signal # non-selective ELR test with nuisance intercept elr = prototest.univariate (X, y, "ELR", selected.col=1:5) # selective F test with nuisance intercept; non-sampling f.test = prototest.univariate (X, y, "F", lambda=0.01, hr.iter=0) print (elr) print (f.test) ### assume variables occur in 4 equally sized groups num.groups = 4 groups = rep (1:num.groups, each=p/num.groups) # selective ALR test -- select columns 21-25 in 2nd group; test for signal in 1st; hit-and-run alr = prototest.multivariate(X, y, groups, 1, "ALR", 21:25, lambda=0.005, hr.iter=20000) # non-selective MS test -- specify first column in each group; test for signal in 1st ms = prototest.multivariate(X, y, groups, 1, "MS", c(1,21,41,61)) print (alr) print (ms)
Perform prototype or F tests for significance of groups of predictors in the univariate model. Choose either exact or approximate likelihood ratio prototype tests (ELR) or (ALR) or F test or marginal screening prototype test. Options for selective or non-selective tests. Further options for non-sampling or hit-and-run null reference distributions for selective tests.
prototest.univariate(x, y, type = c("ALR", "ELR", "MS", "F"), selected.col = NULL, lambda, mu = NULL, sigma = 1, hr.iter = 50000, hr.burn.in = 5000, verbose = FALSE, tol = 10^-8)
prototest.univariate(x, y, type = c("ALR", "ELR", "MS", "F"), selected.col = NULL, lambda, mu = NULL, sigma = 1, hr.iter = 50000, hr.burn.in = 5000, verbose = FALSE, tol = 10^-8)
x |
input matrix of dimension n-by-p, where p is the number of predictors in a single predetermined group of predictors. Will be mean centered and standardised before tests are performed. |
y |
response variable. Vector of length emphn, assumed to be quantitative. |
type |
type of test to be performed. Can only select one at a time. Options include the exact and approximate likelihood ratio prototype tests of Reid et al (2015) (ELR, ALR), the F test and the marginal screening prototype test of Reid and Tibshirani (2015) (MS). Default is ELR. |
selected.col |
preselected columns specified by user. Vector of indices in the set {1, 2, ..., p}. If specified, a non-selective (classical) version of the chosen test it performed. In particular, this means the classicial |
lambda |
regularisation parameter for the lasso fit. Must be supplied when |
mu |
mean parameter for the response. See Details below. If supplied, it is first subtracted from the response to yield a mean-zero (at the population level) vector for which we proceed with testing. If |
sigma |
error standard deviation for the response. See Details below. Must be supplied. If not, it is assumed to be 1. Required for the computation of some of the test statistics. |
hr.iter |
number of hit-and-run samples required in the reference distrbution of a selective test. Applies only if |
hr.burn.in |
number of burn-in hit-and-run samples. These are generated first so as to make subsequent hit-and-run realisations less dependent on the observed response. Samples are then discarded and do not inform the null reference distribution. |
verbose |
should progress be printed? |
tol |
convergence threshold for iterative optimisation procedures. |
The model underpinning each of the tests is
where and
depends on the particular test considered.
In particular, for the ELR, ALR and F tests, we have , where
.
is the input matrix reduced to the columns in the set M, which, in turn, is either provided by the user (via
selected.col
) or selected by the lasso (if selected.col
is NULL
). If the former, a non-selective test is performed; if the latter, a selective test is performed, with the restrictions , as set out in Lee et al (2015).
For the marginal screening prototype (MS) test, where
is the
column of
x
and .
All tests test the null hypothesis . Details of each are described in Reid et al (2015).
A list with the following four components:
ts |
The value of the test statistic on the observed data. |
p.val |
Valid p-value of the test. |
selected.col |
Vector with columns selected. If initially |
y.hr |
Matrix with hit-and-run replications of the response. If sampled selective test was not performed, this will be |
Stephen Reid
Reid, S. and Tibshirani, R. (2015) Sparse regression and marginal testing using cluster prototypes. http://arxiv.org/pdf/1503.00334v2.pdf. Biostatistics doi:10.1093/biostatistics/kxv049
Reid, S., Taylor, J. and Tibshirani, R. (2015) A general framework for estimation and inference from clusters of features. Available online: http://arxiv.org/abs/1511.07839.
require (prototest) ### generate data set.seed (12345) n = 100 p = 80 X = matrix (rnorm(n*p, 0, 1), ncol=p) beta = rep(0, p) beta[1:3] = 0.1 # three signal variables: number 1, 2, 3 signal = apply(X, 1, function(col){sum(beta*col)}) intercept = 3 y = intercept + signal + rnorm (n, 0, 1) ### treat all columns as if in same group and test for signal # non-selective ELR test with nuisance intercept elr = prototest.univariate (X, y, "ELR", selected.col=1:5) # selective F test with nuisance intercept; non-sampling f.test = prototest.univariate (X, y, "F", lambda=0.01, hr.iter=0) print (elr) print (f.test) ### assume variables occur in 4 equally sized groups num.groups = 4 groups = rep (1:num.groups, each=p/num.groups) # selective ALR test -- select columns 21-25 in 2nd group; test for signal in 1st; hit-and-run alr = prototest.multivariate(X, y, groups, 1, "ALR", 21:25, lambda=0.005, hr.iter=20000) # non-selective MS test -- specify first column in each group; test for signal in 1st ms = prototest.multivariate(X, y, groups, 1, "MS", c(1,21,41,61)) print (alr) print (ms)
require (prototest) ### generate data set.seed (12345) n = 100 p = 80 X = matrix (rnorm(n*p, 0, 1), ncol=p) beta = rep(0, p) beta[1:3] = 0.1 # three signal variables: number 1, 2, 3 signal = apply(X, 1, function(col){sum(beta*col)}) intercept = 3 y = intercept + signal + rnorm (n, 0, 1) ### treat all columns as if in same group and test for signal # non-selective ELR test with nuisance intercept elr = prototest.univariate (X, y, "ELR", selected.col=1:5) # selective F test with nuisance intercept; non-sampling f.test = prototest.univariate (X, y, "F", lambda=0.01, hr.iter=0) print (elr) print (f.test) ### assume variables occur in 4 equally sized groups num.groups = 4 groups = rep (1:num.groups, each=p/num.groups) # selective ALR test -- select columns 21-25 in 2nd group; test for signal in 1st; hit-and-run alr = prototest.multivariate(X, y, groups, 1, "ALR", 21:25, lambda=0.005, hr.iter=20000) # non-selective MS test -- specify first column in each group; test for signal in 1st ms = prototest.multivariate(X, y, groups, 1, "MS", c(1,21,41,61)) print (alr) print (ms)