| Title: | Methods for Closed Testing with Simes Inequality, in Particular Hommel's Method |
|---|---|
| Description: | Provides methods for closed testing using Simes local tests. In particular, calculates adjusted p-values for Hommel's multiple testing method, and provides lower confidence bounds for true discovery proportions. A robust but more conservative variant of the closed testing procedure that does not require the assumption of Simes inequality is also implemented. The methods have been described in detail in Goeman et al (2016) <arXiv:1611.06739v2>. |
| Authors: | Jelle Goeman, Rosa Meijer, Thijmen Krebs |
| Maintainer: | Jelle Goeman <[email protected]> |
| License: | GPL (>= 2) |
| Version: | 1.6 |
| Built: | 2024-10-31 06:46:58 UTC |
| Source: | CRAN |
Provides methods for closed testing using Simes local tests. In particular, calculates adjusted p-values for Hommel's multiple testing method, and provides lower confidence bounds for true discovery proportions. A robust but more conservative variant of the closed testing procedure that does not require the assumption of Simes inequality is also implemented.
The starting point of the package is the hommel function. See examples there.
Jelle Goeman, Rosa Meijer, Thijmen Krebs.
Maintainer: Jelle Goeman <[email protected]>
Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika, 75(2), 383-386.
Meijer, R., Krebs, T., Solari, A., & Goeman, J. (2016). Simultaneous Control of All False Discovery Proportions by an Extension of Hommel's Method. arXiv preprint arXiv:1611.06739.
Calculates the concentration set threshold. The concentration set is the subset of the p-values that contains all discoveries at confidence level 1-alpha.
concentration (hommel, alpha=0.05)concentration (hommel, alpha=0.05)
hommel |
The hommel object for the complete multiple testing problem. |
alpha |
The type I error allowed. |
Returns a p-value. P-values larger than that value contain no discoveries in this data set at this level of alpha and may be disregarded.
Jelle Goeman.
Goeman, J., Meijer, R., Krebs, T., & Solari, A. (2016). Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing. arXiv preprint arXiv:1611.06739.
#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. # Use Simes'test (simes = TRUE) or Hommel's robust test (simes = FALSE) hom <- hommel(pvalues, simes = TRUE) # Find the concentration set bound threshold <- concentration(hom) # Find the concentration set itself set <- pvalues <= threshold sum(set)#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. # Use Simes'test (simes = TRUE) or Hommel's robust test (simes = FALSE) hom <- hommel(pvalues, simes = TRUE) # Find the concentration set bound threshold <- concentration(hom) # Find the concentration set itself set <- pvalues <= threshold sum(set)
Calculates adjusted p-values of Hommel's method efficiently.
hommel (p, simes = TRUE)hommel (p, simes = TRUE)
p |
A vector of p-values. |
simes |
If |
Returns a hommel-class object.
Jelle Goeman, Rosa Meijer, Thijmen Krebs.
Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika, 75(2), 383-386.
Meijer, R. J., Krebs, T. J., & Goeman, J. J. (2019). Hommel's procedure in linear time. Biometrical Journal, 61(1), 73-82.
Goeman, J., Meijer, R., Krebs, T., & Solari, A. (2016). Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing. arXiv preprint arXiv:1611.06739.
#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. # Use Simes'test (simes = TRUE) or Hommel's robust test (simes = FALSE) hom <- hommel(pvalues, simes = TRUE) hom summary(hom) # retrieve familywise error adjusted p-values p.adjust(hom) # Find lower confidence bounds # for the number of false null hypotheses (discoveries) # In any subset of interest discoveries(hom, 1:2)#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. # Use Simes'test (simes = TRUE) or Hommel's robust test (simes = FALSE) hom <- hommel(pvalues, simes = TRUE) hom summary(hom) # retrieve familywise error adjusted p-values p.adjust(hom) # Find lower confidence bounds # for the number of false null hypotheses (discoveries) # In any subset of interest discoveries(hom, 1:2)
The class hommel is the output of a call to hommel. It stores the adjusted p-values of all elementary hypotheses, as well as information needed to quickly calculate other results about the underlying closed testing procedure.
These slots are not meant to be directly accessed by the user.
p:Object of class "numeric". Stores original (unadjusted) p-values for all hypotheses.
sorter:Object of class "integer". Stores the permutation to sort p.
jumpalpha:Object of class "numeric". Stores the alpha-values where the function h jumps.
adjusted:Object of class "numeric". Stores adjusted p-values for all elementary hypotheses.
simesfactor:Object of class "numeric". The denominator of the local test that is used.
simes:Object of class "logical". Is TRUE if the adjusted p-values are calculated based on Simes' test that assumes the Simes inequality. If FALSE if they are based on Hommel's robust test.
(hommel): Retrieves adjusted pvalues for all elementary hypotheses.
(hommel, alpha=0.05): Summarizes the results of hommel's method for the specified level of alpha.
Jelle Goeman, Rosa Meijer, Thijmen Krebs.
Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika, 75(2), 383-386.
Meijer, R. J., Krebs, T. J., & Goeman, J. J. (2019). Hommel's procedure in linear time. Biometrical Journal, 61(1), 73-82.
Goeman, J., Meijer, R., Krebs, T., & Solari, A. (2016). Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing. arXiv preprint arXiv:1611.06739.
#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. # Use Simes'test (simes = TRUE) or Hommel's robust test (simes = FALSE) hom <- hommel(pvalues, simes = TRUE) hom summary(hom) # retrieve familywise error adjusted p-values p.adjust(hom) # Find lower confidence bounds # for the number of false null hypotheses (discoveries) # In any subset of interest discoveries(hom, 1:2)#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. # Use Simes'test (simes = TRUE) or Hommel's robust test (simes = FALSE) hom <- hommel(pvalues, simes = TRUE) hom summary(hom) # retrieve familywise error adjusted p-values p.adjust(hom) # Find lower confidence bounds # for the number of false null hypotheses (discoveries) # In any subset of interest discoveries(hom, 1:2)
An intersection hypothesis is defined for any collection of null hypotheses as the hypothesis that is true if and only if the proportion of null hypotheses are less than the selected threshold. A test for an intersection hypothesis is called a local test. This function calculates familywise error adjusted p-values for local tests. Familywise error control is over all possible local tests simultaneously.
localtest (hommel, ix, tdp)localtest (hommel, ix, tdp)
hommel |
The hommel object for the complete multiple testing problem. |
ix |
The selection of hypotheses defining the intersection hypothesis. Any selector that works on the original vector of p-values (index, negative index, logical, names) is allowed. If ix is missing, the global null hypothesis is used. |
tdp |
The threshold value for the intersection hypothesis which should be a proportion. |
Returns the familywise error adjusted p-value.
Jelle Goeman.
Meijer, R. J., Krebs, T. J., & Goeman, J. J. (2019). Hommel's procedure in linear time. Biometrical Journal, 61(1), 73-82.
Goeman, J., Meijer, R., Krebs, T., & Solari, A. (2016). Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing. arXiv preprint arXiv:1611.06739.
#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. hom <- hommel(pvalues, simes = TRUE) # Test any intersection hypothesis of interest against any chosen threshold localtest(hom, tdp=0.2) localtest(hom, 1:3, 0.8)#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. hom <- hommel(pvalues, simes = TRUE) # Test any intersection hypothesis of interest against any chosen threshold localtest(hom, tdp=0.2) localtest(hom, 1:3, 0.8)
Calculates (1-alpha) lower confidence bounds for the number of false null hypotheses (discoveries) or for the true discovery proportion, and (1-alpha) upper confidence bounds for the false discovery proportions. These bounds are for arbitrary subsets, and are simultaneous over all subsets. They are valid even if the subset is chosen after seeing the data.
discoveries (hommel, ix, incremental=FALSE, alpha=0.05) fdp (hommel, ix, incremental=FALSE, alpha=0.05) tdp (hommel, ix, incremental=FALSE, alpha=0.05)discoveries (hommel, ix, incremental=FALSE, alpha=0.05) fdp (hommel, ix, incremental=FALSE, alpha=0.05) tdp (hommel, ix, incremental=FALSE, alpha=0.05)
hommel |
The hommel object for the complete multiple testing problem. |
ix |
The selection of hypotheses considered. Any selector that works on the original vector of p-values (index, negative index, logical, names) is allowed. If missing, all hypotheses are selected. |
incremental |
If TRUE, the number of discoveries (of fdp or tdp) is reported for incremental sets formed by adding the indices from ix in order. If FALSE, reports the numbr of discoveries in the full set only. |
alpha |
The type I error allowed. |
Returns a single integer (discoveries) or proportion (fdp, tdp), or a vector if incremental = TRUE. These represent the lower confidence bound (discoveries, tdp) or upper confidence bound (fdp). The confidence intervals are one-sided: the upper confidence bound for discoveries is the size of the set, and for tdp 1. The lower bound for fdp is 0.
Jelle Goeman, Rosa Meijer, Thijmen Krebs, Mitra Ebrahimpoor, Xu Chen.
Goeman, J., Meijer, R., Krebs, T., & Solari, A. (2016). Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing. arXiv preprint arXiv:1611.06739.
#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. # Use Simes'test (simes = TRUE) or Hommel's robust test (simes = FALSE) hom <- hommel(pvalues, simes = TRUE) hom summary(hom) # Find lower confidence bounds # for the number of false null hypotheses (discoveries) # In any subset of interest discoveries(hom, 1:2) # a bound for the false discovery proportion: set <- 1:5 fdp(hom, set) # or incremental for sets ix = (1), (1,2), (1,2,3), (1,2,3,4) and (1,2,3,4,5) fdp(hom, set, incremental = TRUE) # use alpha=0.5 to obtain a conservative point estimate tdp(hom, set, alpha=0.5)#Generate a vector of pvalues set.seed(102) m <- 10 pvalues <- c(runif(0.5*m,0,0.02), runif(0.5*m,0,1)) # First step: create a hommel object. # Use Simes'test (simes = TRUE) or Hommel's robust test (simes = FALSE) hom <- hommel(pvalues, simes = TRUE) hom summary(hom) # Find lower confidence bounds # for the number of false null hypotheses (discoveries) # In any subset of interest discoveries(hom, 1:2) # a bound for the false discovery proportion: set <- 1:5 fdp(hom, set) # or incremental for sets ix = (1), (1,2), (1,2,3), (1,2,3,4) and (1,2,3,4,5) fdp(hom, set, incremental = TRUE) # use alpha=0.5 to obtain a conservative point estimate tdp(hom, set, alpha=0.5)