Package 'TestCor' reference manual

Title:	FWER and FDR Controlling Procedures for Multiple Correlation Tests
Description:	Different multiple testing procedures for correlation tests are implemented. These procedures were shown to theoretically control asymptotically the Family Wise Error Rate (Roux (2018) <https://tel.archives-ouvertes.fr/tel-01971574v1>) or the False Discovery Rate (Cai & Liu (2016) <doi:10.1080/01621459.2014.999157>). The package gather four test statistics used in correlation testing, four FWER procedures with either single step or stepdown versions, and four FDR procedures.
Authors:	Gannaz Irene [aut, cre]
Maintainer:	Gannaz Irene <[email protected]>
License:	GPL (>= 2)
Version:	0.0.2.2
Built:	2024-12-11 07:01:23 UTC
Source:	CRAN

FWER and FDR controlling procedures for multiple correlation tests

Description

The package compiles some multiple testing procedures which theoretically control asymptotically the FWER in the framework of correlation testing. Four tests statistics can be considered: the empirical correlation, the Student statistics, the Fisher's z-transform and the usual Gaussian statistics considering random variables $(X_i-mean(X_i))(X_j-mean(X_j))$ . Four methods are implemented: Bonferroni (1935)'s, Šidák (1967)'s, Romano & Wolf (2005)'s bootstrap and (Drton & Perlman (2007)'s procedure based on the asymptotic distributions of the test statistics, called MaxTinfty. The package also includes some multiple testing procedures which are related to the control of the FDR : Cai & Liu (2016)'s procedures called LCT-N and LCT-B -which have been proven to control the FDR for correlation tests- and Benjamini & Hochberg (1995)'s -which has no theoretical results in correlation testing.

Details

Consider $\lbrace \bold{X}_\ell = (X_{1\ell},\dots X_{p\ell}),\; \ell=1,...,n\rbrace$ a set of $n$ independent and identically distributed $R^p$ -valued random variables. Denote data the array containing $\lbrace\mathbf{X_\ell},\; \ell=1,\dots,n\rbrace$ , with observation indexes $l$ in row. The aim is to test simultaneously

$(H_{0ij})~ Cor(X_i,X_j)=0 {~~against~~} (H_{1ij})~ Cor(X_i,X_j)\neq 0,~~ i,j=1,..., p,~ i<j.$

Four tests statistics are implemented: the empirical correlation, the Student statistics, the Fisher's z-transform and the usual test statistics on expectancy considering the product of random variables. They are available in function eval_stat. Next, two main types of procedures are available:

Asymptotically FWER controlling procedures:

Bonferroni (1935)'s method, Šidák (1967)'s procedure, Romano & Wolf (2005)'s bootstrap procedure and Drton & Perlman (2007)'s procedure. A description of these methods can be found in Chapter 5 of Roux (2018). To apply these procedures, function ApplyFwerCor can be used as follows:

ApplyFwerCor(data,alpha,stat_test,method), with alpha the desired level of control for FDR and stat_test, method respectively the kind of test statistic and the FDR controlling method. The function returns the list of indexes $\lbrace (i,j), i < j \rbrace$ for which null hypothesis $(H_{0ij})$ is rejected.

Asymptotically FDR controlling procedures:

Cai & Liu(2016)'s two procedures and Benjamini & Hochberg (1995)'s procedure (with no theoretical proof for the latest). To apply these procedures, use function ApplyFdrCor as follows: ApplyFdrCor(data,alpha,stat_test,method) with alpha the desired level of control for FWER and stat_test, method respectively the kind of test statistic and the FDR controlling method. The function returns the list of indexes $\lbrace (i,j), i < j \rbrace$ for which null hypothesis $(H_{0ij})$ is rejected.

Functions SimuFwer and SimuFdr provide simulations of Gaussian random variables for a given correlation matrix and return estimated FWER, FDR, Power and true discovery rate obtained applying one of the procedure above. Some example of results obtained can be found in Chapter 6 of Roux (2018).

Author(s)

Irene Gannaz

Maintainer: Irene Gannaz <[email protected]>

References

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological), 289-300, https://doi.org/10.1111/j.2517-6161.1995.tb02031.x.

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240, https://doi.org/10.1080/01621459.2014.999157.

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449, https://doi.org/10.1214/088342307000000113.

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108, https://doi.org/10.1198/016214504000000539.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Šidák, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626-633.

Examples


# Parameters for simulations
Nsimu  <- 100                # number of Monte-Carlo simulations
seqn   <- seq(100,400,100)   # sample sizes
p      <- 10                 # number of random variables considered
rho    <- 0.3                # value of non-zero correlations
seed   <- 156724
 
corr_theo <- diag(1,p)       # the correlation matrix
corr_theo[1,2:p] <- rho
corr_theo[2:p,1] <- rho               

# Parameters for multiple testing procedure
stat_test <- 'empirical'     # test statistics for correlation tests
method <- 'BootRW'           # FWER controlling procedure
SD <- FALSE                  # logical determining if stepdown is applied
alpha  <- 0.05               # FWER threshold 
Nboot  <- 100                # number of bootstrap or simulated samples

# Simulations and application of the chosen procedure
res <- matrix(0,nrow=length(seqn),ncol=5)
for(i in 1:length(seqn)){
    temp <- SimuFwer(corr_theo,n=seqn[i],Nsimu=Nsimu,alpha=alpha,stat_test=stat_test,
           method='BootRW',Nboot=Nboot,stepdown=SD,seed=seed)
    res[i,] <- temp
}
rownames(res) <- seqn
colnames(res) <- names(temp)

# Display results
par(mfrow=c(1,2))
plot(seqn,res[,'fwer'],type='b',ylim=c(0,max(alpha*1.1,max(res[,'fwer']))),
    main='FWER',ylab='fwer',xlab='number of observations')
plot(seqn,res[,'sensitivity'],type='b',ylim=c(0,1.1),
    main='Power',ylab='sensitivity',xlab='number of observations')

# Parameters for simulations
Nsimu  <- 100                # number of Monte-Carlo simulations
seqn   <- seq(100,400,100)   # sample sizes
p      <- 10                 # number of random variables considered
rho    <- 0.3                # value of non-zero correlations
seed   <- 156724
 
corr_theo <- diag(1,p)       # the correlation matrix
corr_theo[1,2:p] <- rho
corr_theo[2:p,1] <- rho               

# Parameters for multiple testing procedure
stat_test <- 'empirical'     # test statistics for correlation tests
method <- 'BootRW'           # FWER controlling procedure
SD <- FALSE                  # logical determining if stepdown is applied
alpha  <- 0.05               # FWER threshold 
Nboot  <- 100                # number of bootstrap or simulated samples

# Simulations and application of the chosen procedure
res <- matrix(0,nrow=length(seqn),ncol=5)
for(i in 1:length(seqn)){
    temp <- SimuFwer(corr_theo,n=seqn[i],Nsimu=Nsimu,alpha=alpha,stat_test=stat_test,
           method='BootRW',Nboot=Nboot,stepdown=SD,seed=seed)
    res[i,] <- temp
}
rownames(res) <- seqn
colnames(res) <- names(temp)

# Display results
par(mfrow=c(1,2))
plot(seqn,res[,'fwer'],type='b',ylim=c(0,max(alpha*1.1,max(res[,'fwer']))),
    main='FWER',ylab='fwer',xlab='number of observations')
plot(seqn,res[,'sensitivity'],type='b',ylim=c(0,1.1),
    main='Power',ylab='sensitivity',xlab='number of observations')

Applies multiple testing procedures built to control (asymptotically) the FDR for correlation testing.

Description

Applies multiple testing procedures built to control (asymptotically) the FDR for correlation testing. Some have no theoretical proofs for tests on a correlation matrix.

Usage

ApplyFdrCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  method = "LCTnorm",
  Nboot = 1000,
  vect = FALSE,
  arr.ind = FALSE
)
ApplyFdrCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  method = "LCTnorm",
  Nboot = 1000,
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`method`	choice between 'LCTnorm' and 'LCTboot' developped by Cai & Liu (2016), 'BH', traditional Benjamini-Hochberg's procedure Benjamini & Hochberg (1995)'s and 'BHboot', Benjamini-Hochberg (1995)'s procedure with bootstrap evaluation of p-values
`Nboot`	number of iterations for bootstrap p-values evaluation
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing rows and columns of significant correlations
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with repspect to level alpha

Value

Returns either

logicals indicating if the corresponding correlation is significant, as a vector or a matrix depending on vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
res <- ApplyFdrCor(data,stat_test='empirical',method='LCTnorm')
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
res <- ApplyFdrCor(data,stat_test='empirical',method='LCTnorm')
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)

Applies multiple testing procedures controlling (asymptotically) the FWER for tests on a correlation matrix.

Description

Applies multiple testing procedures controlling (asymptotically) the FWER for tests on a correlation matrix. Methods are described in Chapter 5 of Roux (2018).

Usage

ApplyFwerCor(
  data,
  alpha = NULL,
  stat_test = "empirical",
  method = "Sidak",
  Nboot = 1000,
  stepdown = TRUE,
  vect = FALSE,
  logical = stepdown,
  arr.ind = FALSE
)
ApplyFwerCor(
  data,
  alpha = NULL,
  stat_test = "empirical",
  method = "Sidak",
  Nboot = 1000,
  stepdown = TRUE,
  vect = FALSE,
  logical = stepdown,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing (used if logical=TRUE)
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`method`	choice between 'Bonferroni', 'Sidak', 'BootRW', 'MaxTinfty'
`Nboot`	number of iterations for Monte-Carlo of bootstrap quantile evaluation
`stepdown`	logical, if TRUE a stepdown procedure is applied
`vect`	if TRUE returns a vector of adjusted p-values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing the adjusted p-values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected if `stepdown=TRUE` and `logical=FALSE`, returns a list of successive p-values.
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with repspect to level alpha

Value

Returns either

the adjusted p-values, as a vector or a matrix, depending on vect
logicals indicating if the corresponding correlation is significant if logical=TRUE, as a vector or a matrix depending on vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Šidák, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626-633.

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
(res <- ApplyFwerCor(data,stat_test='empirical',method='Bonferroni',stepdown=FALSE))
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
(res <- ApplyFwerCor(data,stat_test='empirical',method='Bonferroni',stepdown=FALSE))
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)

Applies an oracle version of MaxTinfty procedure described in Drton & Perlman (2007) for correlation testing.

Description

Applies oracle MaxTinfty procedure described in Drton & Perlman (2007) which controls asymptotically the FWER for tests on a correlation matrix. It needs the true correlation matrix.

Usage

ApplyFwerCor_oracle(
  data,
  corr_theo,
  alpha = c(),
  stat_test = "empirical",
  method = "MaxTinfty",
  Nboot = 1000,
  stepdown = TRUE,
  vect = FALSE,
  logical = stepdown,
  arr.ind = FALSE
)
ApplyFwerCor_oracle(
  data,
  corr_theo,
  alpha = c(),
  stat_test = "empirical",
  method = "MaxTinfty",
  Nboot = 1000,
  stepdown = TRUE,
  vect = FALSE,
  logical = stepdown,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`corr_theo`	true matrix of correlations
`alpha`	level of multiple testing (used if logical=TRUE)
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`method`	only 'MaxTinfty' implemented
`Nboot`	number of iterations for Monte-Carlo of bootstrap quantile evaluation
`stepdown`	logical, if TRUE a stepdown procedure is applied
`vect`	if TRUE returns a vector of adjusted p-values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing the adjusted p-values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected if `stepdown=TRUE` and `logical=FALSE`, returns a list of successive p-values.
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with repspect to level alpha

Value

Returns either

the adjusted p-values, as a vector or a matrix, depending on vect (unavailable with stepdown)
logicals indicating if the corresponding correlation is significant if logical=TRUE, as a vector or a matrix depending on vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

Oracle estimation of the quantile is used, based on the true correlation matrix

References

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values:
(res <- ApplyFwerCor_oracle(data,corr_theo,stat_test='empirical',Nboot=1000,stepdown=FALSE))
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values:
(res <- ApplyFwerCor_oracle(data,corr_theo,stat_test='empirical',Nboot=1000,stepdown=FALSE))
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)

Benjamini & Hochberg (1995)'s procedure for correlation testing with bootstrap evaluation of p-values.

Description

Benjamini & Hochberg (1995)'s procedure on the correlation matrix entries with bootstrap evaluation of p-values (no theoretical proof of control).

Usage

BHBootCor(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  Nboot = 100,
  vect = FALSE,
  arr.ind = FALSE
)
BHBootCor(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  Nboot = 100,
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`Nboot`	number of iterations for bootstrap quantile evaluation
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing TRUE/FALSE values for each entry of the correlation matrix
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

a vector or a matrix of logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, if arr.ind=FALSE,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BHBootCor(data,alpha,stat_test='empirical',arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BHBootCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Benjamini & Hochberg (1995)'s procedure for correlation testing.

Description

Benjamini & Hochberg (1995)'s procedure on the correlation matrix entries (no theoretical proof of control).

Usage

BHCor(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  vect = FALSE,
  arr.ind = FALSE
)
BHCor(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to vectorize(cor(data)) if FALSE, returns an array containing TRUE/FALSE values for each entry of the correlation matrix
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BHCor(data,alpha,stat_test='empirical',arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BHCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Bonferroni multiple testing procedure for correlations.

Description

Bonferroni multiple testing procedure for correlations.

Usage

BonferroniCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)
BonferroniCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing (used if logical=TRUE)
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`vect`	if TRUE returns a vector of adjusted p-values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing the adjusted p-values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

the adjusted p-values, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
corr_theo <- diag(1,p)
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- BonferroniCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
BonferroniCor(data,alpha,stat_test='empirical',arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
corr_theo <- diag(1,p)
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- BonferroniCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
BonferroniCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Bonferroni multiple testing method for correlations with stepdown procedure.

Description

Bonferroni multiple testing method for correlations with stepdown procedure.

Usage

BonferroniCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)
BonferroniCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	4 test statistics are available: 'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing TRUE/FALSE values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected if FALSE, returns a list of successive p-values : element [[i+1]] of the list giving the p-values evaluated on the non-rejected hypothesis at step [[i]]; p-values are either as a vector or a list depending on `vect`
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BonferroniCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- BonferroniCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})  
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BonferroniCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- BonferroniCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})

Bootstrap multiple testing method of Romano & Wolf (2005) for correlations.

Description

Multiple testing method based on the evaluation of quantile by bootstrap in the initial dataset (Romano & Wolf (2005)).

Usage

BootRWCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)
BootRWCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing (used if logical=TRUE)
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`Nboot`	number of iterations for Monte-Carlo quantile evaluation
`vect`	if TRUE returns a vector of adjusted p-values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing the adjusted p-values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

the adjusted p-values, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- BootRWCor(data,stat_test='empirical',Nboot=1000)
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
BootRWCor(data,alpha,stat_test='empirical',Nboot=1000,arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- BootRWCor(data,stat_test='empirical',Nboot=1000)
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
BootRWCor(data,alpha,stat_test='empirical',Nboot=1000,arr.ind=TRUE)

Boootstrap multiple testing method of Romano & Wolf (2005) for correlations, with stepdown procedure.

Description

Multiple testing method based on the evaluation of quantile by bootstrap in the initial dataset (Romano & Wolf (2005)), with stepdown procedure.

Usage

BootRWCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)
BootRWCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	4 test statistics are available: 'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`Nboot`	number of iterations for Bootstrap quantile evaluation
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing TRUE/FALSE values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected if FALSE, returns a list of successive p-values : element [[i+1]] of the list giving the p-values evaluated on the non-rejected hypothesis at step [[i]]; p-values are either as a vector or a list depending on `vect`
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BootRWCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- BootRWCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})  
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BootRWCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- BootRWCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})

Returns the theoretical covariance of empirical correlations.

Description

Returns the theoretical covariance of empirical correlations.

Usage

covD2nd(r)
covD2nd(r)

Arguments

`r`	a correlation matrix

Value

Returns the theoretical covariance of 2nd order statistics, $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$ .

Examples

p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covD2nd(corr_theo)
p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covD2nd(corr_theo)

Returns the theoretical covariance of empirical correlations.

Description

Returns the theoretical covariance of empirical correlations.

Usage

covDcor(r)
covDcor(r)

Arguments

`r`	a correlation matrix

Value

Returns the theoretical covariance of empirical correlations.

References

Aitkin, M. A. (1969). Some tests for correlation matrices. Biometrika, 443-446.

Examples

p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covDcor(corr_theo)
p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covDcor(corr_theo)

Returns the theoretical covariance of test statistics for correlation testing.

Description

Returns the theoretical covariance of test statistics for correlation testing.

Usage

covDcorNorm(cor_mat, stat_test = "empirical")
covDcorNorm(cor_mat, stat_test = "empirical")

Arguments

`cor_mat`	A correlation matrix
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$

Value

Returns the theoretical covariance of the test statistics.

Examples

p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covDcorNorm(corr_theo,stat_test='student')
p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covDcorNorm(corr_theo,stat_test='student')

Evaluates the test statistics for tests on correlation matrix entries.

Description

Evaluates the test statistics for tests on correlation matrix entries.

Usage

eval_stat(data, type = "empirical")
eval_stat(data, type = "empirical")

Arguments

`data`	matrix of observations
`type`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$

Value

Returns the test statistics for correlation testing.

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
stat <- eval_stat(data,'fisher')
n <- 100
p <- 10
corr_theo <- diag(1,p)
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
stat <- eval_stat(data,'fisher')

Bootstrap procedure LCT-B proposed by Cai & Liu (2016) for correlation testing.

Description

Bootstrap procedure LCT-B proposed by Cai & Liu (2016) for correlation testing.

Usage

LCTboot(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  Nboot = 100,
  vect = FALSE,
  arr.ind = FALSE
)
LCTboot(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  Nboot = 100,
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`Nboot`	number of iterations for bootstrap quantile evaluation
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing TRUE/FALSE values for each entry of the correlation matrix
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240.

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
LCTboot(data,alpha,stat_test='empirical',Nboot=100,arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
LCTboot(data,alpha,stat_test='empirical',Nboot=100,arr.ind=TRUE)

Procedure LCT-N proposed by Cai & Liu (2016) for correlation testing.

Description

Procedure LCT-N proposed by Cai & Liu (2016) for correlation testing.

Usage

LCTnorm(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  vect = FALSE,
  arr.ind = FALSE
)
LCTnorm(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing TRUE/FALSE values for each entry of the correlation matrix
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240.

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
LCTnorm(data,alpha,stat_test='empirical',arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
LCTnorm(data,alpha,stat_test='empirical',arr.ind=TRUE)

Multiple testing method of Drton & Perlman (2007) for correlations.

Description

Multiple testing method based on the evaluation of quantile by simulation of observations from the asymptotic distribution (Drton & Perlman (2007)).

Usage

maxTinftyCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  OmegaChap = covDcorNorm(cor(data), stat_test),
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)
maxTinftyCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  OmegaChap = covDcorNorm(cor(data), stat_test),
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing (used if logical=TRUE)
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`Nboot`	number of iterations for Monte-Carlo quantile evaluation
`OmegaChap`	matrix of covariance of empirical correlations used for quantile evaluation; optional, useful for oracle estimation and step-down
`vect`	if TRUE returns a vector of adjusted p-values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing the adjusted p-values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

the adjusted p-values, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- maxTinftyCor(data,stat_test='empirical',Nboot=1000)
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
res <- maxTinftyCor(data,alpha,stat_test='empirical',Nboot=1000,arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- maxTinftyCor(data,stat_test='empirical',Nboot=1000)
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
res <- maxTinftyCor(data,alpha,stat_test='empirical',Nboot=1000,arr.ind=TRUE)

Multiple testing method of Drton & Perlman (2007) for correlations, with stepdown procedure.

Description

Multiple testing method based on the evaluation of quantile by simulation of observations from the asymptotic distribution (Drton & Perlman (2007)), with stepdown procedure.

Usage

maxTinftyCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  OmegaChap = covDcorNorm(cor(data), stat_test),
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)
maxTinftyCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  OmegaChap = covDcorNorm(cor(data), stat_test),
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	4 test statistics are available: 'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`Nboot`	number of iterations for Monte-Carlo quantile evaluation
`OmegaChap`	matrix of covariance of test statistics; optional, useful for oracle estimation and step-down
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing TRUE/FALSE values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected if FALSE, returns a list of successive p-values : element [[i+1]] of the list giving the p-values evaluated on the non-rejected hypothesis at step [[i]]; p-values are either as a vector or a list depending on `vect`
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
maxTinftyCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- maxTinftyCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})  
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
maxTinftyCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- maxTinftyCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})

Sidak multiple testing procedure for correlations.

Description

Sidak multiple testing procedure for correlations.

Usage

SidakCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)
SidakCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing (used if logical=TRUE)
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`vect`	if TRUE returns a vector of adjusted p-values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing the adjusted p-values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

the adjusted p-values, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Šidák, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626-633.

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- SidakCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
SidakCor(data,alpha,stat_test='empirical',arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- SidakCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
SidakCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Sidak multiple testing method for correlations with stepdown procedure.

Description

Sidak multiple testing method for correlations with stepdown procedure.

Usage

SidakCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)
SidakCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing
`stat_test`	4 test statistics are available: 'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`vect`	if TRUE returns a vector of TRUE/FALSE values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing TRUE/FALSE values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected if FALSE, returns a list of successive p-values : element [[i+1]] of the list giving the p-values evaluated on the non-rejected hypothesis at step [[i]]; p-values are either as a vector or a list depending on `vect`
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

References

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Šidák, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626-633.

Examples

  
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
SidakCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- SidakCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})  
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
SidakCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- SidakCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})

Simulates Gaussian data with a given correlation matrix and applies a FDR controlling procedure on the correlations.

Description

Simulates Gaussian data with a given correlation matrix and applies a FDR controlling procedure on the correlations.

Usage

SimuFdr(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "LCTnorm",
  Nboot = 1000,
  seed = NULL
)
SimuFdr(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "LCTnorm",
  Nboot = 1000,
  seed = NULL
)

Arguments

`corr_theo`	the correlation matrix of Gaussien data simulated
`n`	sample size
`Nsimu`	number of simulations
`alpha`	level of multiple testing
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ 'gaussian' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`method`	choice between 'LCTnorm' and 'LCTboot', developped by Cai & Liu (2016), 'BH', traditional Benjamini-Hochberg (1995)'s procedure, and 'BHboot', Benjamini-Hochberg (1995)'s procedure with bootstrap evaluation of pvalues
`Nboot`	number of iterations for Monte-Carlo of bootstrap quantile evaluation
`seed`	seed for the Gaussian simulations

Value

Returns a line vector containing estimated values for fwer, fdr, sensitivity, specificity and accuracy.

References

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

Nsimu <- 1000
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFdr(corr_theo,n,Nsimu,alpha,stat_test='empirical',method='LCTnorm')
Nsimu <- 1000
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFdr(corr_theo,n,Nsimu,alpha,stat_test='empirical',method='LCTnorm')

Simulates Gaussian data with a given correlation matrix and applies a FWER controlling procedure on the correlations.

Description

Simulates Gaussian data with a given correlation matrix and applies a FWER controlling procedure on the correlations.

Usage

SimuFwer(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "Sidak",
  Nboot = 1000,
  stepdown = TRUE,
  seed = NULL
)
SimuFwer(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "Sidak",
  Nboot = 1000,
  stepdown = TRUE,
  seed = NULL
)

Arguments

`corr_theo`	the correlation matrix of Gaussien data simulated
`n`	sample size
`Nsimu`	number of simulations
`alpha`	level of multiple testing
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ 'gaussian' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`method`	choice between 'Bonferroni', 'Sidak', 'BootRW', 'MaxTinfty'
`Nboot`	number of iterations for Monte-Carlo of bootstrap quantile evaluation
`stepdown`	logical, if TRUE a stepdown procedure is applied
`seed`	seed for the Gaussian simulations

Value

Returns a line vector containing estimated values for fwer, fdr, sensitivity, specificity and accuracy.

References

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Westfall, P.H. & Young, S. (1993) Resampling-based multiple testing: Examples and methods for p-value adjustment, John Wiley & Sons, vol. 279.

Examples

Nsimu <- 1000 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFwer(corr_theo,n,Nsimu,alpha,stat_test='empirical',method='Bonferroni',stepdown=FALSE)
Nsimu <- 1000 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFwer(corr_theo,n,Nsimu,alpha,stat_test='empirical',method='Bonferroni',stepdown=FALSE)

Simulates Gaussian data with a given correlation matrix and applies oracle MaxTinfty on the correlations.

Description

Simulates Gaussian data with a given correlation matrix and applies oracle MaxTinfty (i.e. Drton & Perlman (2007)'s procedure with the true correlation matrix) on the correlations.

Usage

SimuFwer_oracle(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "MaxTinfty",
  Nboot = 1000,
  stepdown = TRUE,
  seed = NULL
)
SimuFwer_oracle(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "MaxTinfty",
  Nboot = 1000,
  stepdown = TRUE,
  seed = NULL
)

Arguments

`corr_theo`	the correlation matrix of Gaussien data simulated
`n`	sample size
`Nsimu`	number of simulations
`alpha`	level of multiple testing
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ 'gaussian' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`method`	only 'MaxTinfty' available
`Nboot`	number of iterations for Monte-Carlo of bootstrap quantile evaluation
`stepdown`	logical, if TRUE a stepdown procedure is applied
`seed`	seed for the Gaussian simulations

Value

Returns a line vector containing estimated values for fwer, fdr, sensitivity, specificity and accuracy.

References

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Examples

Nsimu <- 1000
n <- 50
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFwer_oracle(corr_theo,n,Nsimu,alpha,stat_test='empirical',stepdown=FALSE,Nboot=100)
Nsimu <- 1000
n <- 50
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFwer_oracle(corr_theo,n,Nsimu,alpha,stat_test='empirical',stepdown=FALSE,Nboot=100)

Uncorrected testing procedure for correlations.

Description

Uncorrected testing procedure for correlations.

Usage

UncorrectedCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)
UncorrectedCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

`data`	matrix of observations
`alpha`	level of multiple testing (used if logical=TRUE)
`stat_test`	'empirical' $\sqrt{n}abs(corr)$ 'fisher' $\sqrt{n-3}1/2\log( (1+corr)/(1-corr) )$ 'student' $\sqrt{n-2}abs(corr)/\sqrt(1-corr^2)$ '2nd.order' $\sqrt{n}*mean(Y)/sd(Y)$ with $Y=(X_i-mean(X_i))(X_j-mean(X_j))$
`vect`	if TRUE returns a vector of adjusted p-values, corresponding to `vectorize(cor(data))`; if FALSE, returns an array containing the adjusted p-values for each entry of the correlation matrix
`logical`	if TRUE, returns either a vector or a matrix where each element is equal to TRUE if the corresponding null hypothesis is rejected, and to FALSE if it is not rejected
`arr.ind`	if TRUE, returns the indexes of the significant correlations, with respect to level alpha

Value

Returns

the non-adjusted p-values, as a vector or a matrix depending of the value of vect,
an array containing indexes $\lbrace(i,j),\,i<j\rbrace$ for which correlation between variables $i$ and $j$ is significant, if arr.ind=TRUE.

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# p-values
res <- UncorrectedCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
UncorrectedCor(data,alpha,stat_test='empirical',arr.ind=TRUE)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# p-values
res <- UncorrectedCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
UncorrectedCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Returns an upper-triangle matrix, without the diagonal, containing the elements of a given vector.

Description

Returns an upper-triangle matrix, without the diagonal, containing the elements of a given vector.

Usage

unvectorize(vect)
unvectorize(vect)

Arguments

vect

A vector containing the upper triangle of a matrix, without the diagonal

Value

Returns an upper-triangle matrix where each entry is given by the vector containing the upper triangle of a matrix, without the diagonal.

Examples

unvectorize(1:10)
unvectorize(1:10)

Returns a vector containing the upper triangle of a matrix, without the diagonal.

Description

Returns a vector containing the upper triangle of a matrix, without the diagonal.

Usage

vectorize(mat)
vectorize(mat)

Arguments

mat

a square matrix

Value

Returns a vector containing the upper triangle of a matrix, without the diagonal.

Examples

vectorize(matrix(1:9,3,3))
vectorize(matrix(1:9,3,3))

Returns the indexes of an upper triangular matrix with logical entries.

Description

Returns the indexes of an upper triangular matrix with logical entries.

Usage

whichCor(mat)
whichCor(mat)

Arguments

mat

A matrix with logical entries in the upper triangular part

Value

Returns the indexes of the upper triangular part where the entries are TRUE

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
res <- ApplyFwerCor(data,stat_test='empirical',method='Bonferroni',stepdown=FALSE)
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
res <- ApplyFwerCor(data,stat_test='empirical',method='Bonferroni',stepdown=FALSE)
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)

Package 'TestCor'

Help Index

FWER and FDR controlling procedures for multiple correlation tests

Description

Details

Author(s)

References

Examples

Applies multiple testing procedures built to control (asymptotically) the FDR for correlation testing.

Description

Usage

Arguments

Value

References

See Also

Examples

Applies multiple testing procedures controlling (asymptotically) the FWER for tests on a correlation matrix.

Description

Usage

Arguments

Value

References

See Also

Examples

Applies an oracle version of MaxTinfty procedure described in Drton & Perlman (2007) for correlation testing.

Description

Usage

Arguments

Value

References

See Also

Examples

Benjamini & Hochberg (1995)'s procedure for correlation testing with bootstrap evaluation of p-values.

Description

Usage

Arguments

Value

References

See Also

Examples

Benjamini & Hochberg (1995)'s procedure for correlation testing.

Description

Usage

Arguments

Value

References

See Also

Examples

Bonferroni multiple testing procedure for correlations.

Description

Usage

Arguments

Value

References

See Also

Examples

Bonferroni multiple testing method for correlations with stepdown procedure.

Description

Usage

Arguments

Value

References

See Also

Examples

Bootstrap multiple testing method of Romano & Wolf (2005) for correlations.

Description

Usage

Arguments

Value

References

See Also

Examples

Boootstrap multiple testing method of Romano & Wolf (2005) for correlations, with stepdown procedure.

Description

Usage

Arguments

Value

References

See Also

Examples