| Title: | Flexible Randomization-Based Inference |
|---|---|
| Description: | Allows the user to conduct randomization-based inference for a wide variety of experimental scenarios. The package leverages a potential outcomes framework to output randomization-based p-values and null intervals for test statistics geared toward any estimands of interest, according to the specified null and alternative hypotheses. Users can define custom randomization schemes so that the randomization distributions are accurate for their experimental settings. The package also creates visualizations of randomization distributions and can test multiple test statistics simultaneously. |
| Authors: | Joseph J. Lee and Tirthankar Dasgupta |
| Maintainer: | Joseph J. Lee <[email protected]> |
| License: | GPL-2 |
| Version: | 1.0.4 |
| Built: | 2024-12-25 06:30:26 UTC |
| Source: | CRAN |
Calculates the analysis of variance F statistic.
anovaF(y, w, calcOptions = NULL)anovaF(y, w, calcOptions = NULL)
y |
a vector or matrix of outcomes. |
w |
a vector or matrix of assignments. |
calcOptions |
a list of options for calculating the analysis of variance (ANOVA) F statistic (if necessary).
|
Returns the F statistic calculated in an analysis of variance of a linear model with no interaction terms.
An analysis of variance F statistic.
Joseph J. Lee and Tirthankar Dasgupta
# 1 treatment factor with 3 levels # Assignments and outcomes w <- c(1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3) y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12, 9, 9) anovaF(y, w) # 1 treatment factor with 3 levels, with block # Assignments, blocks, and outcomes w <- c(1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3) x <- c(1, 1, 2, 2, 1, 1, 2, 2, 1, 1, 2, 2) y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12, 9, 9) anovaF(y, w, calcOptions = list(block = x))# 1 treatment factor with 3 levels # Assignments and outcomes w <- c(1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3) y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12, 9, 9) anovaF(y, w) # 1 treatment factor with 3 levels, with block # Assignments, blocks, and outcomes w <- c(1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3) x <- c(1, 1, 2, 2, 1, 1, 2, 2, 1, 1, 2, 2) y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12, 9, 9) anovaF(y, w, calcOptions = list(block = x))
Randomly draws a specified number of assignment vectors or matrices according to a randomized block design.
blockRand(w, nrand, block)blockRand(w, nrand, block)
w |
a vector or matrix of assignments. |
nrand |
a number specifying the desired number of random assignments. |
block |
a vector of block designations. |
Assignments are randomly permuted within each block.
If w is a matrix, the permutations occur by row.
A list of random assignment vectors or matrices.
Joseph J. Lee and Tirthankar Dasgupta
w1 <- c(0, 1, 0, 1, 0, 1, 0, 1) block <- c(0, 0, 0, 0, 1, 1, 1, 1) blockRand(w1, nrand = 5, block) w2 <- c(0, 0, 1, 1, 0, 0, 1, 1) blockRand(w = cbind(w1, w2), nrand = 5, block)w1 <- c(0, 1, 0, 1, 0, 1, 0, 1) block <- c(0, 0, 0, 0, 1, 1, 1, 1) blockRand(w1, nrand = 5, block) w2 <- c(0, 0, 1, 1, 0, 0, 1, 1) blockRand(w = cbind(w1, w2), nrand = 5, block)
Randomly draws a specified number of assignment vectors or matrices according to a completely randomized design.
completeRand(w, nrand)completeRand(w, nrand)
w |
a vector or matrix of assignments. |
nrand |
a number specifying the desired number of random assignments. |
If w is a matrix, the permutations occur by row.
A list of random assignment vectors or matrices.
Joseph J. Lee and Tirthankar Dasgupta
w1 <- c(0, 0, 0, 0, 1, 1, 1, 1) completeRand(w1, nrand = 5) w2 <- c(0, 1, 0, 1, 0, 1, 0, 1) completeRand(w = cbind(w1, w2), nrand = 5)w1 <- c(0, 0, 0, 0, 1, 1, 1, 1) completeRand(w1, nrand = 5) w2 <- c(0, 1, 0, 1, 0, 1, 0, 1) completeRand(w = cbind(w1, w2), nrand = 5)
Calculates potential outcomes under modified assignments, according to the specified constant treatment effect(s).
constEffect(y, w, w_new, poOptions)constEffect(y, w, w_new, poOptions)
y |
a vector or matrix of outcomes. |
w |
a vector or matrix of assignments. |
w_new |
a vector or matrix of modified assignments. |
poOptions |
a list of options for calculating potential outcomes.
|
A vector of potential outcomes under the modified assignments.
Joseph J. Lee and Tirthankar Dasgupta
# 1 treatment factor with 2 levels # Assignments w <- c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1) # Modified Assignments w_new <- c(1, 1, 1, 1, 1, 0, 0, 0, 0, 0) # Outcomes y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12) constEffect(y, w, w_new, poOptions = list(tau = 2)) # 2 treatment factors, each with 2 levels # Assignments w1 <- c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1) w2 <- c(0, 1, 0, 1, 0, 1, 0, 1, 0, 1) w <- cbind(w1, w2) # Modified assignments w1_new <- c(1, 1, 1, 1, 1, 0, 0, 0, 0, 0) w2_new <- c(1, 0, 1, 0, 1, 0, 1, 0, 1, 0) w_new <- cbind(w1_new, w2_new) # Outcomes y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12) constEffect(y, w, w_new, poOptions = list(tau = c(2, -1)))# 1 treatment factor with 2 levels # Assignments w <- c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1) # Modified Assignments w_new <- c(1, 1, 1, 1, 1, 0, 0, 0, 0, 0) # Outcomes y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12) constEffect(y, w, w_new, poOptions = list(tau = 2)) # 2 treatment factors, each with 2 levels # Assignments w1 <- c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1) w2 <- c(0, 1, 0, 1, 0, 1, 0, 1, 0, 1) w <- cbind(w1, w2) # Modified assignments w1_new <- c(1, 1, 1, 1, 1, 0, 0, 0, 0, 0) w2_new <- c(1, 0, 1, 0, 1, 0, 1, 0, 1, 0) w_new <- cbind(w1_new, w2_new) # Outcomes y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12) constEffect(y, w, w_new, poOptions = list(tau = c(2, -1)))
Calculates the difference of mean observed outcomes for a specified treatment factor and specified pair of comparison levels.
diffMeans(y, w, calcOptions = NULL)diffMeans(y, w, calcOptions = NULL)
y |
a vector or matrix of outcomes. |
w |
a vector or matrix of assignments. |
calcOptions |
a list of options for calculating the difference of mean outcomes (if necessary).
|
The difference of mean observed outcomes.
Joseph J. Lee and Tirthankar Dasgupta
# 1 treatment factor with 2 levels # Assignments and outcomes w <- c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1) y <- c(4, 6, 5, 6, 4, 11, 11, 9, 10, 9) diffMeans(y, w) # Equals 5 # 2 treatment factors, each with 3 levels # Assignments and outcomes w1 <- c(1, 2, 3, 1, 2, 3, 1, 2, 3) w2 <- c(1, 2, 3, 2, 3, 1, 3, 1, 2) w <- cbind(w1, w2) y <- c(4, 6, 5, 7, 4, 7, 11, 9, 10) diffMeans( y, w, calcOptions = list(factor = 2, pair = c(1, 3)) ) # Equals 0# 1 treatment factor with 2 levels # Assignments and outcomes w <- c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1) y <- c(4, 6, 5, 6, 4, 11, 11, 9, 10, 9) diffMeans(y, w) # Equals 5 # 2 treatment factors, each with 3 levels # Assignments and outcomes w1 <- c(1, 2, 3, 1, 2, 3, 1, 2, 3) w2 <- c(1, 2, 3, 2, 3, 1, 3, 1, 2) w <- cbind(w1, w2) y <- c(4, 6, 5, 7, 4, 7, 11, 9, 10) diffMeans( y, w, calcOptions = list(factor = 2, pair = c(1, 3)) ) # Equals 0
Calculates the differences of mean observed outcomes for multiple specified treatment factors and specified pairs of comparison levels.
diffMeansVector(y, w, calcOptions)diffMeansVector(y, w, calcOptions)
y |
a vector or matrix of outcomes. |
w |
a vector or matrix of assignments. |
calcOptions |
a list of options for calculating the vector of differences of mean outcomes.
|
If unspecified, calcOptions$pairs defaults to c(0,1).
If calcOptions$factors is specified, its length must equal the number of rows specified in calcOptions$pairs.
If unspecified, calcOptions$factors defaults to an appropriately-sized vector of 1's.
A vector of differences of mean observed outcomes.
Joseph J. Lee and Tirthankar Dasgupta
# 2 treatment factors, each with 3 levels # Assignments, outcomes, and desired pairs w1 <- c(1, 2, 3, 1, 2, 3, 1, 2, 3) w2 <- c(1, 2, 3, 2, 3, 1, 3, 1, 2) w <- cbind(w1, w2) y <- c(4, 6, 5, 7, 4, 7, 11, 9, 10) diffMeansVector( y, w, calcOptions = list( factors = c(1, 1, 2), pairs = rbind(c(1, 2), c(2, 3), c(1, 3)) ) ) # Equals c(-1, 1, 0)# 2 treatment factors, each with 3 levels # Assignments, outcomes, and desired pairs w1 <- c(1, 2, 3, 1, 2, 3, 1, 2, 3) w2 <- c(1, 2, 3, 2, 3, 1, 3, 1, 2) w <- cbind(w1, w2) y <- c(4, 6, 5, 7, 4, 7, 11, 9, 10) diffMeansVector( y, w, calcOptions = list( factors = c(1, 1, 2), pairs = rbind(c(1, 2), c(2, 3), c(1, 3)) ) ) # Equals c(-1, 1, 0)
Randomly draws an assignment vector or matrix according to the isomorphic Latin square design, for a specified number of permutations.
latinRand(w, nrand, row, col)latinRand(w, nrand, row, col)
w |
a vector or matrix of assignments. |
nrand |
a number specifying the desired number of random assignments. |
row |
a vector of row designations. |
col |
a vector of column designations. |
Assignments are randomly permuted along rows and columns such that the Latin square design is preserved.
If w is a matrix, the permutations occur by row.
A list of random isomorphic assignment vectors or matrices.
Joseph J. Lee and Tirthankar Dasgupta
w <- c( "C", "D", "B", "A", "A", "B", "D", "C", "D", "C", "A", "B", "B", "A", "C", "D" ) row <- rep(1:4, 4) col <- c(rep(1, 4), rep(2, 4), rep(3, 4), rep(4, 4)) latinRand(w, nrand = 5, row, col)w <- c( "C", "D", "B", "A", "A", "B", "D", "C", "D", "C", "A", "B", "B", "A", "C", "D" ) row <- rep(1:4, 4) col <- c(rep(1, 4), rep(2, 4), rep(3, 4), rep(4, 4)) latinRand(w, nrand = 5, row, col)
Calculates randomization-based intervals under the null hypothesis for the specified test statistics and coverage levels.
randInterval(results, coverage = 0.95)randInterval(results, coverage = 0.95)
results |
a resultant object of the |
coverage |
a number specifying the desired coverage level (defaults to |
If multiple tests are conducted simultaneously, users should be wary of multiple comparisons and make adjustments accordingly (e.g., Bonferroni corrections).
A randomization-based interval (or multiple intervals) for the test statistic(s) under the null hypothesis with the specified coverage level.
Joseph J. Lee and Tirthankar Dasgupta
# Completely randomized design example # with one treatment factor at two levels w <- c(rep(0, 5), rep(1, 5)) y <- rnorm(10, mean = 0, sd = 1) # Two-sided test twoSidedTest <- randTest(y, w, nrand = 50, calcTestStat = diffMeans) randInterval(twoSidedTest) # Reading comprehension pre- and post-test example data(reading) # Testing within-block pairwise effects readingTest <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = withinBlockEffects, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ), randOptions = list(type = "block", block = reading$Block) ) randInterval(readingTest)# Completely randomized design example # with one treatment factor at two levels w <- c(rep(0, 5), rep(1, 5)) y <- rnorm(10, mean = 0, sd = 1) # Two-sided test twoSidedTest <- randTest(y, w, nrand = 50, calcTestStat = diffMeans) randInterval(twoSidedTest) # Reading comprehension pre- and post-test example data(reading) # Testing within-block pairwise effects readingTest <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = withinBlockEffects, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ), randOptions = list(type = "block", block = reading$Block) ) randInterval(readingTest)
Randomization-based p-values and null intervals for a wide variety of experimental scenarios, with corresponding visualizations.
| Package: | randomizationInference |
| Type: | Package |
| Version: | 1.0.4 |
| Date: | 2022-05-17 |
| License: | GPL-2 |
| Main functions: | randTest, randPlot, randInterval
|
The randomizationInference package conducts randomization-based inference for a wide variety
of experimental scenarios. The package leverages a potential outcomes framework to output
randomization-based p-values and null intervals for test statistics geared toward any estimands of
interest, according to the specified null and alternative hypotheses. Users can define custom
randomization schemes so that the randomization distributions are accurate for their experimental
settings. The package also creates visualizations of randomization distributions and can test
multiple test statistics simultaneously.
Joseph J. Lee and Tirthankar Dasgupta
Maintainer: Joseph J. Lee <[email protected]>
Wu, C. F. J. and Hamada, M. (2009) Experiments, Planning, Analysis and Optimization (2nd ed), Wiley.
Moore, David S., and George P. McCabe (1989). Introduction to the Practice of Statistics.
# Completely randomized design example # with one treatment factor at two levels w <- c(rep(0, 5), rep(1, 5)) y <- rnorm(10, mean = 0, sd = 1) # Two-sided test twoSidedTest <- randTest(y, w, nrand = 50, calcTestStat = diffMeans) randInterval(twoSidedTest) randPlot(twoSidedTest) # One-sided test oneSidedTest <- randTest( y, w, nrand = 50, calcTestStat = diffMeans, alternative = "greater" ) # Two=sided test with non-zero null hypothesis nonZeroTest <- randTest( y, w, nrand = 50, calcTestStat = diffMeans, calcPO = constEffect, poOptions = list(tau = 2), null = 2 ) # Randomized block design example # with one treatment factor at three levels x <- rep(1:3, 4) w_block <- rep(1:4, 3) y_block <- rnorm(12, mean = x, sd = 1) blockTest <- randTest( y_block, w_block, nrand = 50, calcTestStat = anovaF, calcOptions = list(block = x), randOptions = list(type = "block", block = x) ) randInterval(blockTest) randPlot(blockTest) # 4x4 Latin square example (from the Wu/Hamada reference) row <- rep(1:4, 4) col <- c(rep(1, 4), rep(2, 4), rep(3, 4), rep(4, 4)) w_latin <- c( "C", "D", "B", "A", "A", "B", "D", "C", "D", "C", "A", "B", "B", "A", "C", "D" ) y_latin <- c( 235, 236, 218, 268, 251, 241, 227, 229, 234, 273, 274, 226, 195, 270, 230, 225 ) latinTest <- randTest( y_latin, w_latin, nrand = 50, calcTestStat = anovaF, calcOptions = list(row = row, col = col), randOptions = list(type = "Latin", row = row, col = col) ) randInterval(latinTest) randPlot(latinTest) # User-defined randomization example # Partial randomization: first four assignments are fixed # Due to physical limitations # User-defined randomization function # Input: number of random assignments, function options # Output: list of random assignments myRand <- function(nrand, userOptions = NULL){ w_fixed = c(0, 0, 1, 1) lapply(1:nrand, function(i) c(w_fixed, sample(rep(0:1, 5)))) } w_user <- c(c(0, 0, 1, 1), c(0, 1, 1, 0, 0, 0, 1, 1, 0, 1)) # observed assignment y_user <- rnorm(14, mean = 0, sd = 1) userTest <- randTest( y_user, w_user, nrand = 50, calcTestStat = diffMeans, randOptions = list(type = "user.defined"), userRand = myRand ) randInterval(userTest) randPlot(userTest) # 2^3 factorial design example # three treatment factors (OT, CP, and ST) at two levels each OT <- c(-1, -1, -1, -1, 1, 1, 1, 1) CP <- c(-1, -1, 1, 1, -1, -1, 1, 1) ST <- rep(c(-1, 1), 4) w_fac <- cbind(OT, CP, ST) y_fac <- c(67, 79, 61, 75, 59, 90, 52, 87) # Testing the main effect of factor "OT" facTest1 <- randTest( y_fac, w_fac, nrand = 50, calcTestStat = diffMeans, calcOptions = list(factor = 1, pair = c(-1, 1)) ) # Testing all three main effects simultaneously facTest2 <- randTest( y_fac, w_fac, nrand = 50, calcTestStat = diffMeansVector, calcOptions = list( factors = 1:3, pairs = matrix(rep(c(-1, 1), 3), ncol = 2, byrow = TRUE) ) ) # Testing all contrasts simultaneously w_facNew <- cbind(OT, CP, ST, OT*CP, OT * ST, CP * ST, OT * CP * ST) facTest3 <- randTest( y_fac, w_facNew, nrand = 50, calcTestStat = diffMeansVector, calcOptions = list( factors = 1:7, pairs = matrix(rep(c(-1, 1), 7), ncol = 2, byrow = TRUE) ) ) randInterval(facTest3) randPlot(facTest3, plotDim = c(2, 4)) # Reading comprehension pre- and post-test example data(reading) # Ignoring blocks readingTest1 <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = anovaF ) # Testing within-block pairwise effects readingTest2 <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = withinBlockEffects, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ), randOptions = list(type = "block", block = reading$Block) ) randInterval(readingTest2) randPlot(readingTest2, plotDim = c(2, 3))# Completely randomized design example # with one treatment factor at two levels w <- c(rep(0, 5), rep(1, 5)) y <- rnorm(10, mean = 0, sd = 1) # Two-sided test twoSidedTest <- randTest(y, w, nrand = 50, calcTestStat = diffMeans) randInterval(twoSidedTest) randPlot(twoSidedTest) # One-sided test oneSidedTest <- randTest( y, w, nrand = 50, calcTestStat = diffMeans, alternative = "greater" ) # Two=sided test with non-zero null hypothesis nonZeroTest <- randTest( y, w, nrand = 50, calcTestStat = diffMeans, calcPO = constEffect, poOptions = list(tau = 2), null = 2 ) # Randomized block design example # with one treatment factor at three levels x <- rep(1:3, 4) w_block <- rep(1:4, 3) y_block <- rnorm(12, mean = x, sd = 1) blockTest <- randTest( y_block, w_block, nrand = 50, calcTestStat = anovaF, calcOptions = list(block = x), randOptions = list(type = "block", block = x) ) randInterval(blockTest) randPlot(blockTest) # 4x4 Latin square example (from the Wu/Hamada reference) row <- rep(1:4, 4) col <- c(rep(1, 4), rep(2, 4), rep(3, 4), rep(4, 4)) w_latin <- c( "C", "D", "B", "A", "A", "B", "D", "C", "D", "C", "A", "B", "B", "A", "C", "D" ) y_latin <- c( 235, 236, 218, 268, 251, 241, 227, 229, 234, 273, 274, 226, 195, 270, 230, 225 ) latinTest <- randTest( y_latin, w_latin, nrand = 50, calcTestStat = anovaF, calcOptions = list(row = row, col = col), randOptions = list(type = "Latin", row = row, col = col) ) randInterval(latinTest) randPlot(latinTest) # User-defined randomization example # Partial randomization: first four assignments are fixed # Due to physical limitations # User-defined randomization function # Input: number of random assignments, function options # Output: list of random assignments myRand <- function(nrand, userOptions = NULL){ w_fixed = c(0, 0, 1, 1) lapply(1:nrand, function(i) c(w_fixed, sample(rep(0:1, 5)))) } w_user <- c(c(0, 0, 1, 1), c(0, 1, 1, 0, 0, 0, 1, 1, 0, 1)) # observed assignment y_user <- rnorm(14, mean = 0, sd = 1) userTest <- randTest( y_user, w_user, nrand = 50, calcTestStat = diffMeans, randOptions = list(type = "user.defined"), userRand = myRand ) randInterval(userTest) randPlot(userTest) # 2^3 factorial design example # three treatment factors (OT, CP, and ST) at two levels each OT <- c(-1, -1, -1, -1, 1, 1, 1, 1) CP <- c(-1, -1, 1, 1, -1, -1, 1, 1) ST <- rep(c(-1, 1), 4) w_fac <- cbind(OT, CP, ST) y_fac <- c(67, 79, 61, 75, 59, 90, 52, 87) # Testing the main effect of factor "OT" facTest1 <- randTest( y_fac, w_fac, nrand = 50, calcTestStat = diffMeans, calcOptions = list(factor = 1, pair = c(-1, 1)) ) # Testing all three main effects simultaneously facTest2 <- randTest( y_fac, w_fac, nrand = 50, calcTestStat = diffMeansVector, calcOptions = list( factors = 1:3, pairs = matrix(rep(c(-1, 1), 3), ncol = 2, byrow = TRUE) ) ) # Testing all contrasts simultaneously w_facNew <- cbind(OT, CP, ST, OT*CP, OT * ST, CP * ST, OT * CP * ST) facTest3 <- randTest( y_fac, w_facNew, nrand = 50, calcTestStat = diffMeansVector, calcOptions = list( factors = 1:7, pairs = matrix(rep(c(-1, 1), 7), ncol = 2, byrow = TRUE) ) ) randInterval(facTest3) randPlot(facTest3, plotDim = c(2, 4)) # Reading comprehension pre- and post-test example data(reading) # Ignoring blocks readingTest1 <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = anovaF ) # Testing within-block pairwise effects readingTest2 <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = withinBlockEffects, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ), randOptions = list(type = "block", block = reading$Block) ) randInterval(readingTest2) randPlot(readingTest2, plotDim = c(2, 3))
Plots observed test statistics against their randomization distributions and against the randomization-based intervals for their corresponding estimands.
randPlot( results, coverage = 0.95, breaks = 10, plotDim = c(length(results$obs_stat), 1) )randPlot( results, coverage = 0.95, breaks = 10, plotDim = c(length(results$obs_stat), 1) )
results |
a resultant object of the |
coverage |
a number specifying the desired coverage level (defaults to |
breaks |
a value specifying the histogram break pattern (defaults to 10). |
plotDim |
a numeric vector (of length 2) specifying the desired plot dimensions (if necessary, defaults to |
One plot will be displayed for each test statistic specified by results.
Observed test statistic values are demarcated by solid red lines. Comparison values under the null hypothesis are demarcated by solid black lines. Randomization-based interval bounds for the specified coverage level are demarcated by dotted blue lines.
Setting plotDim = c(a, b) is equivalent to specifying par(mfrow=c(a, b)). Plot dimensions are automatically reset to c(1, 1) afterward.
If multiple test statistics are tested simultaneously, plotDim may need to be specified suitably for the plots to be displayed.
If multiple tests are conducted simultaneously, users should be wary of multiple comparisons and make adjustments accordingly (e.g., Bonferroni corrections).
The plot(s) described in details will be displayed.
Joseph J. Lee and Tirthankar Dasgupta
# Completely randomized design example # with one treatment factor at two levels w <- c(rep(0, 5), rep(1, 5)) y <- rnorm(10, mean = 0, sd = 1) # Two-sided test twoSidedTest <- randTest(y, w, nrand = 50, calcTestStat = diffMeans) randPlot(twoSidedTest) # Reading comprehension pre- and post-test example data(reading) # Testing within-block pairwise effects readingTest <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = withinBlockEffects, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ), randOptions = list(type = "block", block = reading$Block) ) randPlot(readingTest, breaks = 20, plotDim = c(2, 3))# Completely randomized design example # with one treatment factor at two levels w <- c(rep(0, 5), rep(1, 5)) y <- rnorm(10, mean = 0, sd = 1) # Two-sided test twoSidedTest <- randTest(y, w, nrand = 50, calcTestStat = diffMeans) randPlot(twoSidedTest) # Reading comprehension pre- and post-test example data(reading) # Testing within-block pairwise effects readingTest <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = withinBlockEffects, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ), randOptions = list(type = "block", block = reading$Block) ) randPlot(readingTest, breaks = 20, plotDim = c(2, 3))
Conducts randomization-based hypothesis tests according to the specified test statistic(s), assignment mechanism, and null and alternative hypotheses.
randTest( y, w, nrand, calcTestStat, calcOptions = NULL, calcPO = zeroEffect, poOptions = NULL, randOptions = list( type = c("complete", "block", "Latin", "user.defined"), block = NULL, row = NULL, col = NULL ), userRand = NULL, userOptions = NULL, null = NULL, alternative = c("two.sided", "greater", "less") )randTest( y, w, nrand, calcTestStat, calcOptions = NULL, calcPO = zeroEffect, poOptions = NULL, randOptions = list( type = c("complete", "block", "Latin", "user.defined"), block = NULL, row = NULL, col = NULL ), userRand = NULL, userOptions = NULL, null = NULL, alternative = c("two.sided", "greater", "less") )
y |
a vector or matrix of outcomes. |
w |
a vector (or matrix, when multiple treatment factors are present) of assignments. |
nrand |
a number specifying the desired number of random hypothetical assignments. |
calcTestStat |
a function to calculate the specified test statistic(s). |
calcOptions |
a list of options for calculating the test statistic(s) (if necessary). |
calcPO |
a function to calculate potential outcomes (defaults to |
poOptions |
a list of options for calculating potential outcomes (if necessary). |
randOptions |
a list of options governing the random assignment mechanism. |
userRand |
a function to generate random assignments (if necessary). |
userOptions |
a list of options governing the user-defined random assignment mechanism (if necessary). |
null |
a number or numeric vector specifying the comparison value(s) under the null hypothesis. |
alternative |
a character string specifying the alternative hypothesis, must be one of the following: |
The inputs to calcTestStat are y, w, and calcOptions (if necessary).
The output of calcTestStat is a number or numeric vector representing the test statistic value(s).
Several common test statistics (diffMeans, anovaF, diffMeansVector) are built-in for convenience.
calcTestStat can also be a custom function, as long as the inputs and output are as described above.
If multiple test statistics are tested simultaneously, users should be wary of multiple comparisons and make appropriate adjustments (e.g., Bonferroni corrections).
The inputs to calcPO are y, w, w_new (a vector or matrix of modified assignments), and poOptions (if necessary).
The output of calcPO is a vector of potential outcomes under the modified assignments.
Two common potential outcome calculations (zeroEffect (default) and constEffect) are built-in for convenience.
calcPO can also be a custom function, as long as the inputs and output are as described above.
If unspecified, randOptions$type defaults to "complete".
If randOptions$type equals "block", then randOptions$block must be specified.
If randOptions$type equals "Latin", then randOptions$row and randOptions$col must be specified.
If randOptions$type equals "user.defined", then userRand must be specified.
The inputs to userRand are nrand and userOptions (if necessary).
If null is specified, its length must equal the length of the output of calcTestStat.
If unspecified, null defaults to 0 or an appropriately sized vector of 0's.
alternative = "greater" is the alternative that the test statistic is greater than the comparison value.
A list containing the following elements:
perm_stats |
a vector or matrix of hypothetical test statistic values. |
obs_stat |
the observed test statistic value(s). |
null |
a number or numeric vector specifying the comparison value(s) under the null hypothesis. |
alternative |
a character string specifying the alternative hypothesis. |
pvalue |
the randomization-based p-value(s). |
Joseph J. Lee and Tirthankar Dasgupta
Wu, C. F. J. and Hamada, M. (2009) Experiments, Planning, Analysis and Optimization (2nd ed), Wiley.
Moore, David S., and George P. McCabe (1989). Introduction to the Practice of Statistics. Original source: study conducted by Jim Baumann and Leah Jones of the Purdue University Education Department.
diffMeans, anovaF, diffMeansVector,
zeroEffect, constEffect,
completeRand, blockRand, latinRand,
randInterval, randPlot
# Completely randomized design example # with one treatment factor at two levels w <- c(rep(0, 5), rep(1, 5)) y <- rnorm(10, mean = 0, sd = 1) # Two-sided test twoSidedTest <- randTest(y, w, nrand = 50, calcTestStat = diffMeans) # One-sided test oneSidedTest <- randTest( y, w, nrand = 50, calcTestStat = diffMeans, alternative = "greater" ) # Two=sided test with non-zero null hypothesis nonZeroTest <- randTest( y, w, nrand = 50, calcTestStat = diffMeans, calcPO = constEffect, poOptions = list(tau = 2), null = 2 ) # Randomized block design example # with one treatment factor at three levels x <- rep(1:3, 4) w_block <- rep(1:4, 3) y_block <- rnorm(12, mean = x, sd = 1) blockTest <- randTest( y_block, w_block, nrand = 50, calcTestStat = anovaF, calcOptions = list(block = x), randOptions = list(type = "block", block = x) ) # 4x4 Latin square example (from the Wu/Hamada reference) row <- rep(1:4, 4) col <- c(rep(1, 4), rep(2, 4), rep(3, 4), rep(4, 4)) w_latin <- c( "C", "D", "B", "A", "A", "B", "D", "C", "D", "C", "A", "B", "B", "A", "C", "D" ) y_latin <- c( 235, 236, 218, 268, 251, 241, 227, 229, 234, 273, 274, 226, 195, 270, 230, 225 ) latinTest <- randTest( y_latin, w_latin, nrand = 50, calcTestStat = anovaF, calcOptions = list(row = row, col = col), randOptions = list(type = "Latin", row = row, col = col) ) # 2^3 factorial design example # three treatment factors (OT, CP, and ST) at two levels each OT <- c(-1, -1, -1, -1, 1, 1, 1, 1) CP <- c(-1, -1, 1, 1, -1, -1, 1, 1) ST <- rep(c(-1, 1), 4) w_fac <- cbind(OT, CP, ST) y_fac <- c(67, 79, 61, 75, 59, 90, 52, 87) # Testing the main effect of factor "OT" facTest1 <- randTest( y_fac, w_fac, nrand = 50, calcTestStat = diffMeans, calcOptions = list(factor = 1, pair = c(-1, 1)) ) # Testing all three main effects simultaneously facTest2 <- randTest( y_fac, w_fac, nrand = 50, calcTestStat = diffMeansVector, calcOptions = list( factors = 1:3, pairs = matrix(rep(c(-1, 1), 3), ncol = 2, byrow = TRUE) ) ) # Testing all contrasts simultaneously w_facNew <- cbind(OT, CP, ST, OT * CP, OT * ST, CP * ST, OT * CP * ST) facTest3 <- randTest( y_fac, w_facNew, nrand = 50, calcTestStat = diffMeansVector, calcOptions = list( factors = 1:7, pairs = matrix(rep(c(-1, 1), 7), ncol = 2, byrow = TRUE) ) ) # Reading comprehension pre- and post-test example data(reading) # Ignoring blocks readingTest1 <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = anovaF ) # Testing within-block pairwise effects readingTest2 <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = withinBlockEffects, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ), randOptions = list(type = "block", block = reading$Block) )# Completely randomized design example # with one treatment factor at two levels w <- c(rep(0, 5), rep(1, 5)) y <- rnorm(10, mean = 0, sd = 1) # Two-sided test twoSidedTest <- randTest(y, w, nrand = 50, calcTestStat = diffMeans) # One-sided test oneSidedTest <- randTest( y, w, nrand = 50, calcTestStat = diffMeans, alternative = "greater" ) # Two=sided test with non-zero null hypothesis nonZeroTest <- randTest( y, w, nrand = 50, calcTestStat = diffMeans, calcPO = constEffect, poOptions = list(tau = 2), null = 2 ) # Randomized block design example # with one treatment factor at three levels x <- rep(1:3, 4) w_block <- rep(1:4, 3) y_block <- rnorm(12, mean = x, sd = 1) blockTest <- randTest( y_block, w_block, nrand = 50, calcTestStat = anovaF, calcOptions = list(block = x), randOptions = list(type = "block", block = x) ) # 4x4 Latin square example (from the Wu/Hamada reference) row <- rep(1:4, 4) col <- c(rep(1, 4), rep(2, 4), rep(3, 4), rep(4, 4)) w_latin <- c( "C", "D", "B", "A", "A", "B", "D", "C", "D", "C", "A", "B", "B", "A", "C", "D" ) y_latin <- c( 235, 236, 218, 268, 251, 241, 227, 229, 234, 273, 274, 226, 195, 270, 230, 225 ) latinTest <- randTest( y_latin, w_latin, nrand = 50, calcTestStat = anovaF, calcOptions = list(row = row, col = col), randOptions = list(type = "Latin", row = row, col = col) ) # 2^3 factorial design example # three treatment factors (OT, CP, and ST) at two levels each OT <- c(-1, -1, -1, -1, 1, 1, 1, 1) CP <- c(-1, -1, 1, 1, -1, -1, 1, 1) ST <- rep(c(-1, 1), 4) w_fac <- cbind(OT, CP, ST) y_fac <- c(67, 79, 61, 75, 59, 90, 52, 87) # Testing the main effect of factor "OT" facTest1 <- randTest( y_fac, w_fac, nrand = 50, calcTestStat = diffMeans, calcOptions = list(factor = 1, pair = c(-1, 1)) ) # Testing all three main effects simultaneously facTest2 <- randTest( y_fac, w_fac, nrand = 50, calcTestStat = diffMeansVector, calcOptions = list( factors = 1:3, pairs = matrix(rep(c(-1, 1), 3), ncol = 2, byrow = TRUE) ) ) # Testing all contrasts simultaneously w_facNew <- cbind(OT, CP, ST, OT * CP, OT * ST, CP * ST, OT * CP * ST) facTest3 <- randTest( y_fac, w_facNew, nrand = 50, calcTestStat = diffMeansVector, calcOptions = list( factors = 1:7, pairs = matrix(rep(c(-1, 1), 7), ncol = 2, byrow = TRUE) ) ) # Reading comprehension pre- and post-test example data(reading) # Ignoring blocks readingTest1 <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = anovaF ) # Testing within-block pairwise effects readingTest2 <- randTest( y = reading$Diff1, w = reading$Group, nrand = 50, calcTestStat = withinBlockEffects, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ), randOptions = list(type = "block", block = reading$Block) )
Pre- and post-treatment reading comprehension test scores for 66 students randomly assigned to one of three methods for teaching reading comprehension.
data(reading)data(reading)
A data set of 66 students with 6 variables:
Subjectstudent ID numbers
Blockblock designations
Grouptreatment group assignments
Pre1pre-treatment reading comprehension test scores
Post1post-treatment reading comprehension test scores
Diff1changes in test score (improvement if positive)
Moore, David S., and George P. McCabe (1989). Introduction to the Practice of Statistics. Original source: study conducted by Jim Baumann and Leah Jones of the Purdue University Education Department.
data(reading)data(reading)
Calculates the differences of mean outcomes for multiple specified treatment factors and specified pairs of comparison levels, within the specified blocks.
withinBlockEffects(y, w, calcOptions)withinBlockEffects(y, w, calcOptions)
y |
a vector or matrix of outcomes. |
w |
a vector or matrix of assignments. |
calcOptions |
a list of options for calculating the difference of mean outcomes within blocks.
|
calcOptions$block should have the same length as y and w.
If unspecified, calcOptions$pairs defaults to c(0,1).
If calcOptions$factors is specified, its length must equal the number of rows specified in calcOptions$pairs.
If unspecified, calcOptions$factors defaults to an appropriately-sized vector of 1's.
calcOptions$blockindex should have the same length as calcOptions$factors.
A vector of differences of mean outcomes within blocks.
Joseph J. Lee and Tirthankar Dasgupta
Moore, David S., and George P. McCabe (1989). Introduction to the Practice of Statistics. Original source: study conducted by Jim Baumann and Leah Jones of the Purdue University Education Department.
# Reading comprehension pre- and post-test example data(reading) withinBlockEffects( y = reading$Diff1, w = reading$Group, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ) )# Reading comprehension pre- and post-test example data(reading) withinBlockEffects( y = reading$Diff1, w = reading$Group, calcOptions = list( block = reading$Block, pairs = rbind( c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat"), c("Basal", "DRTA"), c("Basal", "Strat"), c("DRTA", "Strat") ), blockindex = c(rep(1, 3), rep(2, 3)) ) )
Calculates potential outcomes under modified assignments, assuming zero treatment effect(s).
zeroEffect(y, w, w_new)zeroEffect(y, w, w_new)
y |
a vector or matrix of outcomes. |
w |
a vector or matrix of assignments. |
w_new |
a vector or matrix of modified assignments. |
A vector of potential outcomes under the modified assignments.
Joseph J. Lee and Tirthankar Dasgupta
# Assignments w <- c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1) # Modified Assignments w_new <- c(1, 1, 1, 1, 1, 0, 0, 0, 0, 0) # Outcomes y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12) zeroEffect(y, w, w_new) # Returns y as is# Assignments w <- c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1) # Modified Assignments w_new <- c(1, 1, 1, 1, 1, 0, 0, 0, 0, 0) # Outcomes y <- c(4, 6, 5, 7, 4, 7, 11, 9, 8, 12) zeroEffect(y, w, w_new) # Returns y as is