Package 'puniform'

Data from a meta-analysis infants' habituation to a give stimulus and their later cognitive ability (IQ)

Description

Results from 12 studies on the association between infants' habituation to a give stimulus and their later cognitive ability (IQ).

Format

The data frame contains two columns: ri (raw correlation coefficients) and ni (sample sizes).

Author(s)

Robbie C.M. van Aert [email protected]

Source

McCall, R. B., & Carriger, M. S. (1993). A meta-analysis of infant habituation and recognition memory performance as predictors of later IQ. Child Development, 64, 57-79. http://dx.doi.org/10.2307/1131437

---

diffprior

Description

Function for computing posterior probabilities based on the Snapshot Bayesian Hybrid Meta-Analysis Method or Snapshot Bayesian Meta-Analysis Method (uncorrected) for another than a uniform prior.

Usage

diffprior(prior, res.snap)

Arguments

prior	A vector of length four containing the prior probabilities for no, small, medium, and large true effect size.
res.snap	A data frame with posterior probabilities obtained with the snapshot or uncor.snapshot function.

Details

The function computes posterior probabilities for four true effect sizes (no, small, medium, and large) based on the snapshot or uncor.snapshot function for another than a uniform prior. For more information see van Aert and van Assen (2016).

Value

The diffprior function returns a data frame with posterior probabilities for no (p.0), small (p.sm), medium (p.me), and large (p.la) true effect size.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & van Assen, M.A.L.M. (2017). Bayesian evaluation of effect size after replicating an original study. PLoS ONE, 12(4), e0175302. doi:10.1371/journal.pone.0175302

Examples

### Example as presented on page 491 in Maxwell, Lau, and Howard (2015)
res.snap <- snapshot(ri=c(0.243, 0.114), ni=c(80, 172), alpha=.05)

### Prior probabilities with probablity for no effect twice as large as for the other true effects
prior <- c(0.4, 0.2, 0.2, 0.2)

### Compute posterior probabilities based on new prior
diffprior(prior = prior, res.snap = res.snap)

---

escompute

Description

Function that computes Hedges' g and its sampling variance for an one-sample mean and a two-independent means, Fisher's r-to-z transformed correlation coefficient and its sampling variance for a raw correlation coefficient and computes a p-value as in the primary studies was done.

Usage

escompute(
  mi,
  ri,
  ni,
  sdi,
  m1i,
  m2i,
  n1i,
  n2i,
  sd1i,
  sd2i,
  tobs,
  yi,
  vi,
  alpha,
  side,
  measure
)

Arguments

mi	A vector of group means for one-sample mean
ri	A vector of raw correlation coefficients
ni	A vector of sample sizes for one-sample mean
sdi	A vector of standard deviations for one-sample mean
m1i	A vector of means in group 1 for two-independent means
m2i	A vector of means in group 2 for two-independent means
n1i	A vector of sample sizes in group 1 for two-independent means
n2i	A vector of sample sizes in group 2 for two-independent means
sd1i	A vector of standard deviations in group 1 for two-independent means
sd2i	A vector of standard deviations in group 2 for two-independent means
tobs	A vector of t-values
yi	A vector of standardized effect sizes
vi	A vector of sampling variances belonging to the standardized effect sizes (yi)
alpha	A numerical value specifying the alpha level as used in primary studies
side	A character indicating the direction of the tested hypothesis in the primary studies (either "right" or "left")
measure	A character indicating what kind of effect size should be computed (Hedges' g or Fisher's r-to-z transformed correlation coefficients) and which arguments are used as input ("M", "MT", "MD", "MDT", or "COR"). See Details below.

Details

The measure argument has to be used to specify the desired effect size and what input parameters are used. There are six options:

• "M" for one-sample mean with mi, ni, sdi, alpha, and side as input parameters

• "MT" for one-sample mean with tobs, ni, alpha, and side as input parameters

• "MD" for two-sample mean with m1i, m2i, n1i, n2i, sd1i, sd2i, alpha, and side as input parameters

• "MDT" for two-sample mean with tobs, n1i, n2i, alpha, and side as input parameters

• "COR" for raw correlation coefficients with ri, ni, alpha, and side as input parameters

• "SPE" for user-specified standardized effect sizes and sampling variances with yi, vi, alpha, and side as input parameters

Value

Function returns a data frame with standardized effect sizes (yi), variances of these standardized effect sizes (vi), z-values (zval), p-values as computed in primary studies (pval), and critical z-values (zcv).

Author(s)

Robbie C.M. van Aert [email protected]

---

fe_ma

Description

Function that conducts fixed-effect meta-analysis

Usage

fe_ma(yi, vi)

Arguments

yi	A vector of standardized effect sizes
vi	A vector of sampling variances belonging to the standardized effect sizes (yi)

Details

The fe.ma function can be used for conducting fixed-effect meta-analysis on a set of studies

Value

est.fe	effect size estimate of fixed-effect meta-analysis
se.fe	standard error of estimate of fixed-effect meta-analysis
ci.lb.fe	lower bound 95% confidence interval
ci.ub.fe	upper bound 95% confidence interval
zval.fe	z-value of test of no effect
pval.fe	two-tailed p-value of test of no effect
pval.fe.one	one-tailed p-value of test of no effect
Qstat	test statistic of the Q-test
Qpval	p-value of the Q-test

---

fis_trans

Description

Function for transforming raw correlation coefficients to Fisher-z transformed correlation coefficients and vice versa.

Usage

fis_trans(r, fis)

Arguments

r	A numerical value being a raw correlation coefficient
fis	A numerical value being a Fisher-z transformed correlation coefficient

Author(s)

Robbie C.M. van Aert [email protected]

---

hybrid

Description

Function to statistically combine conventional and preregistered/replications studies by means of the hybrid methods as described in van Aert and van Assen (2018) and van Aert (2023).

Usage

hybrid(
  m1i,
  m2i,
  mi,
  ri,
  sd1i,
  sd2i,
  sdi,
  n1i,
  n2i,
  ni,
  tobs,
  yi,
  vi,
  conventional,
  mods = NULL,
  alpha = 0.05,
  side,
  control
)

Arguments

m1i	A vector of means in group 1 for two-independent means
m2i	A vector of means in group 2 for two-independent means
mi	A vector of group means for one-sample means
ri	A vector of raw correlations
sd1i	A vector of standard deviations in group 1 for two-independent means
sd2i	A vector of standard deviations in group 2 for two-independent means
sdi	A vector of standard deviations for one-sample means
n1i	A vector of sample sizes in group 1 for two-independent means
n2i	A vector of sample sizes in group 2 for two-independent means
ni	A vector of sample sizes for one-sample means and correlations
tobs	A vector of t-values
yi	A vector of standardized effect sizes (see Details)
vi	A vector of sampling variances belonging to the standardized effect sizes (see Details)
conventional	A vector indicating whether a study is a conventional study (indicated with a 1) and therefore was susceptible to bias. Studies not susceptible to bias are indicated with a 0
mods	A one-sided formula to specify the moderators to include. For example x1 can be included as moderator by specifying mods = ~ x1
alpha	A numerical value specifying the alpha level as used in the conventional study (default is 0.05, see Details)
side	A character indicating whether the observed effect size of the conventional studies are in the right-tail of the distribution (i.e., positive) or in the left-tail of the distribution (i.e., negative) (either "right" or "left")
control	An optional list of elements that give the user more control over the estimation procedures (see Note)

Details

Three different effect sizes can be used as input for the hybrid function: one-sample means, two-independent means, and raw correlation coefficients. Analyzing one-sample means and two-independent means can be done by either providing the function group means (mi or m1i and m2i), standard deviations (sdi or sd1i and sd2i), and sample sizes (ni or n1i and n2i) or t-values (tobs) and sample sizes (ni or n1i and n2i). Pearson correlation coefficients can be analyzed by supplying ri and ni to the hybrid function. These correlation coefficients are internally transformed to Fisher's z correlations before analyzing the data. The results in the output are of the Fisher's z transformed correlations. It is also possible to specify the standardized effect sizes and its sampling variances directly via the yi and vi arguments.

Two other arguments that need to be specified are side and conventional. side indicates whether the effect size in the conventional study was expected to be in the right-tail (side = "right" or in the left-tail (side = "left") of the distribution. The argument conventional has to be used to indicate which studies are conventional studies and expected to be susceptible to bias. A 1 indicates that a study was susceptible to bias and a 0 indicates that a study was not susceptible to bias.

It is assumed that two-tailed hypothesis tests were conducted in the conventional studies. In case one-tailed hypothesis tests were conducted in the conventional studies, the alpha level has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be supplied to the hybrid function.

Previous version

The usage of a previous version of the hybrid function was more restricted. Users could only apply the method to a single conventional study and replication. Before the addition of the extra functionality to also analyze multiple conventional studies and replications, data of the conventional study and replication were specified in vectors containing two elements with the first element being the data of the conventional study and the second one the data of the replication. In order to maintain backwards compatibility, it is still possible to analyze data like this by using the arguments m1i, m2i, mi, ri, sd1i, sd2i, sdi, n1i, n2i, ni, tobs. However, using the hybrid function in this way is now deprecated.

Value

k	total number of effect sizes
k.conventional	number of effect sizes of conventional studies
est	parameter estimates of the fixed effects of the hybrid method
tau2	estimate of the between-study variance in true effect size of hybrid method
se	standard error of the fixed effects and of the estimated between-study variance
ci.lb	lower bound of hybrid method's confidence interval of the average effect size
ci.ub	upper bound of hybrid method's confidence interval of the average effect size
L.0	test statistic of hybrid method's test of null-hypothesis of no effect. This is either a z-value (zval in the output) or a chi-square value (LR in the output) in case of a likelihood-ratio test
pval.0	p-value of hybrid method's test of null-hypothesis of no effect
tau2	estimate of the between-study variance in true effect size of hybrid method
tau2.lb	lower bound of hybrid method's confidence interval of the between-study variance
tau2.ub	upper bound of hybrid method's confidence interval of the between-study variance
L.het	test statistic of hybrid method's test of null-hypothesis of no heterogeneity This is either a z-value (zval in the output) or a chi-square value (LR in the output) in case of a likelihood-ratio test
pval.het	p-value of hybrid method's test of null-hypothesis of no heterogeneity
optim.info	model fitting results if the implementation of van Aert (2023) is used

The elements below are only returned if the deprecated implementation of van Aert and van Assen (2018) is used:

est.hyr	effect size estimate of hybridR method
ci.lb.hyr	lower bound of hybridR method's confidence interval of the effect size
ci.ub.hyr	upper bound of hybridR method's confidence interval of the effect size
L.0.hyr	test statistic of hybridR method's test of null-hypothesis of no effect
pval.0.hyr	p-value of hybridR method's test of null-hypothesis of no effect
pval.o	two-tailed p-value of conventional study
est.hy0	effect size estimate of hybrid0 method
ci.lb.hy0	lower bound of hybrid0 method's confidence interval of the effect size
ci.ub.hy0	upper bound of hybrid0 method's confidence interval of the effect size
L.0.hy0	test statistic of hybrid0 method's test of null-hypothesis of no effect
pval.0.hy0	two-tailed p-value of hybrid0 method's test of null-hypothesis of no effect
est.fe	effect size estimate based on fixed-effect meta-analysis
se.fe	standard error of effect size estimate based on traditional fixed-effect meta-analysis
zval.fe	test statistic of the null-hypothesis of no effect based on fixed-effect meta-analysis
pval.fe	two-tailed p-value of the null-hypothesis of no effect based on fixed-effect meta-analysis
ci.lb.fe	lower bound of confidence interval based on traditional fixed-effect meta-analysis
ci.ub.fe	upper bound of confidence interval based on fixed-effect meta-analysis
est.repl	effect size estimate of replication
se.repl	standard error of replication's effect size estimate
ci.lb.repl	lower bound of replication's confidence interval
ci.ub.repl	upper bound of replication's confidence interval
stat.repl	test statistic of replication for testing null-hypothesis of no effect
pval.repl	two-tailed p-value of replication for testing null-hypothesis of no effect

Note

The control argument in the hybrid function is an optional argument that gives the user more control over the estimation procedures. This can be especially useful if estimation of the method does not converge and NAs are returned by the function. The control argument should be specified as a list containing one or more elements. For example, control = list(verbose = TRUE) Default values are used if an element is not specified. The following elements can be specified by the user:

• int: A vector of length two that indicates the lower and upper bound of the interval that is used for estimating the effect size. The effect size estimate should be included in this interval. Its default values are -10 for the first element and the maximum effect size of a study included in the analysis + 1 as the second element. This control argument is only applicable to the implementation of van Aert and van Assen (2018).

• est.ci: A vector of length two indicating the values that are subtracted from and added to the estimate of the effect size for computing the 95% confidence intervals. Its default values are (50, 1). To give an example, estimates for the lower and upper bound around the effect size estimate are searched on the interval (est-50, est) and (est, est+1), respectively.

• tau2.ci A vector of length two indicating the values that are subtracted from and added to the estimate of the between-study variance for computing the 95% confidence intervals. Its default values are (0.5, 0.5). To give an example, estimates for the lower and upper bound around the effect size estimate are searched on the interval (tau2-0.5, tau2) and (tau2, tau2+0.5), respectively.

• tol: A number indicating the desired accuracy of the estimates. Its default value is .Machine$double.eps^0.25. This control argument is only applicable to the implementation of van Aert and van Assen (2018).

• verbose: A logical indicating whether information should be printed about the estimation procedure. Its default value is FALSE.

• par: Starting values for the optimization procedure in case of the implementation of van Aert (2023). The default values are zeros.

• implementation: A character indicating whether the implementation of van Aert and van Assen (2018) based on a single conventional and single replication should be used (implementation = "two") or whether the implementation of van Aert (2023) should be used that allows for including more than two studies (implementation = "multiple").

• optimizer: A character indicating the optimizer that is used if the implementation of van Aert (2023) is used. The default value is "Nelder-Mead". The optim function is used for optimization, so the optimization methods implemented in the optim function can be used. See the documentation of the optim function for more information.

• type: A character vector indicating whether Wald-based hypothesis tests and confidence intervals are preferred (type = "Wald") or likelihood-ratio tests and profile likelihood confidence intervals (type = "profile"). The default is "Wald/profile" which implies that Wald tests and confidence intervals are computed for the fixed effects and likelihood-ratio tests and profile likelihood confidence intervals for the between-study variance.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R. C. M., & van Assen, M. A. L. M. (2018). Examining reproducibility in psychology: A hybrid method for statistically combining a biased original study and replication. Behavior Research Methods, 50(4): 1515-1539. doi:10.3758/s13428-017-0967-6

van Aert, R.C.M. (2023). Empowering meta-analysis by taking advantage of preregistered studies: The extended hybrid meta-analysis method. Manuscript in preparation.

Examples

### Application using the implementation of van Aert (2023) with more than two studies.
# Note that these data come from the "metadat" R package that needs to be loaded.
# The data come from the meta-analyses by Lehmann et al. (2018). See the help 
# file of dat.lehmann2018 for more information about this meta-analysis.
library(metadat)
dat <- dat.lehmann2018

### Create a new object to indicate which studies were conventional studies and 
# which ones were preregistered
dat$conventional <- ifelse(dat$Preregistered == "Not Pre-Registered", 1, 0)

### Lehmann et al. split the analyses for males and females. We only use the data
# of females for this example.
red_romance_femalep <- dat[dat$Gender == "Females", ]

### Prepare data for the analysis
yi <- red_romance_femalep$yi
vi <- red_romance_femalep$vi
conventional <- red_romance_femalep$conventional

### Apply the hybrid method with Wald-type hypothesis tests and confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right")

### Apply the hybrid method with likelihood ratio hypothesis tests and profile 
# likelihood confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", 
control = list(type = "profile"))

### Include Color_Match as moderator
Color_Match <- red_romance_femalep$Color_Match

### Apply the hybrid method with Wald-type hypothesis tests and confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", mods = ~ Color_Match)

### Apply the hybrid method with likelihood ratio hypothesis tests and profile 
# likelihood confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", mods = ~ Color_Match, 
control = list(type = "profile"))

### Application using the implementation of van Aert and van Assen (2018). 
# The hybrid method is applied to example on page 5 of van Aert and van Assen 
# (2018).

pval <- c(0.03, 0.3) # p-value conventional study and replication
n1i <- n2i <- c(40, 80) # Sample sizes per group 
tobs <- qt(pval/2, df = n1i+n2i-2, lower.tail = FALSE) # Observed t-values 

### Apply hybrid method using the implementation of van Aert and van Assen (2018)
hybrid(tobs = tobs, n1i = n1i, n2i = n2i, side = "right", conventional = c(1,0), 
control = list(implementation = "two"))

---

Meta-plot

Description

Function to create meta-plots for two-independent means, raw correlations, and odds ratios. See van Assen et al. (2023) for more information.

Usage

meta_plot(
  m1i,
  m2i,
  sd1i,
  sd2i,
  n1i,
  n2i,
  gi,
  vgi,
  ri,
  ni,
  ai,
  bi,
  ci,
  di,
  alpha = 0.05,
  method_tau2 = "PM",
  nr_lines = "all",
  pub_bias = TRUE,
  main = "",
  cex.pch = 1
)

Arguments

m1i	A vector of means in group 1 for two-independent means
m2i	A vector of means in group 2 for two-independent means
sd1i	A vector of standard deviations in group 1 for two-independent means
sd2i	A vector of standard deviations in group 2 for two-independent means
n1i	A vector of sample sizes in group 1 for two-independent means
n2i	A vector of sample sizes in group 2 for two-independent means
gi	A vector of Hedges' g values for two-independent means if group means and standard deviations are not available
vgi	A vector of Hedges' g sampling variances for two-independent means if group means and standard deviations are not available
ri	A vector of raw correlations
ni	A vector of sample sizes if raw correlations are the effect size measure
ai	A vector of frequencies in upper left cell of 2x2 frequency table (see Details)
bi	A vector of frequencies in upper right cell of 2x2 frequency table (see Details)
ci	A vector of frequencies in lower left cell of 2x2 frequency table (see Details)
di	A vector of frequencies in lower right cell of 2x2 frequency table (see Details)
alpha	A numerical value specifying the alpha level as used in primary studies (default is 0.05 but see Details)
method_tau2	A character indicating the estimation method for the between-study variance in true effect size in the meta-analysis (default is "PM", but see Details)
nr_lines	A character indicating whether all primary study's effect sizes ("all", default) or a selection of primary study's effect sizes ("summary") are plotted (see Details)
pub_bias	A logical indicating whether the expected results of the cumulative meta-analysis based on a zero true effect in combination with extreme publication bias should be plotted. The default value is NA implying that these results are only included if at least 80% of the primary studies is statistically significant. These results are always included if this argument is set to TRUE and never included if this argument is set to FALSE
main	A character indicating the title of the plot (default is no title)
cex.pch	A numerical value to control the size of the points in the plot

Details

The meta_plot function assumes that two-tailed hypothesis tests were conducted in the primary studies. In case one-tailed hypothesis tests were conducted in the primary studies, the submitted alpha argument to the meta_plot function has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be submitted to the meta_plot function.

Different estimators can be used for estimating the between-study variance in true effect size. The default estimator is the Paule-Mandel estimator (Paule & Mandel, 1982), because this estimator was recommended in Veroniki et al. (2016) and Langan, Higgins, and Simmonds (2016). However, all estimators that are included in the rma.uni function of the metafor package can be used, because this function is called in the meta_plot function.

When nr_lines = "summary" is specified, the estimates of meta-analyses based on primary studies with sufficient statistical power are displayed. Next to the estimate and 95% confidence interval of the meta-analysis including all studies (leftmost), it shows these results for studies with sufficient statistical power (80%) to detect a large true effect size (left vertical line), medium true effect size (middle), and small true effect size (right). Note that the summary meta-plot is just the meta-plot with many meta-analyses and confidence intervals left out, and keeping the leftmost meta-analysis and those just immediately to the right of the vertical lines.

The meta-plot can be created for standardized mean differences by providing the function with the sample means (m1i and m21), the sample sizes (n1i and n2i), and the standard deviations (sd1i and sd2i) or by specifying the standardized mean differences (i.e., Hedges' g; gi) together with the corresponding sampling variances (vgi) and the sample sizes (n1i and n2i). Hedges' g standardized mean differences and corresponding sampling variances are computed in the function if the sample means, sample sizes, and standard deviations are provided.

For creating a meta-plot based on odds ratios as effect size measure, the 2x2 frequency table should follow a specific format. The reason for this is that the probability for the outcome of interest in the control conditions has to be estimated. Hence, the 2x2 frequency table should look like this:

	Outcome 1	Outcome 2
Group 1	ai	bi
Group 2	ci	di

Value

An invisibly returned data frame consisting of the submitted data and

yi	Standardized effect sizes used in the analyses
vi	Sampling variances of the standardized effect sizes used in the analyses
est_cum	Estimates of the cumulative meta-analyses
lb_cum	Lower bounds of the 95% confidence intervals of the cumulative meta-analyses
ub_cum	Upper bounds of the 95% confidence intervals of the cumulative meta-analyses
pub_est	Estimates of cumulative meta-analyses based on Mill's ratios
info	Information of a primary study (only for two-independent means)
stand_info	Standardized information of a primary study (only for two-independent means)
preci	Precision of a primary study (only for odds ratios)

Author(s)

Robbie C.M. van Aert [email protected]

References

Langan, D., Higgins, J. P. T., & Simmonds, M. (2016). Comparative performance of heterogeneity variance estimators in meta-analysis: A review of simulation studies. Research Synthesis Methods, 8(2), 181-198. doi:10.1002/jrsm.1198

Paule, R. C., & Mandel, J. (1982). Consensus values and weighting factors. Journal of Research of the National Bureau of Standards, 87(5), 377-385.

van Assen, ..., & van Aert (2023). The meta-plot: A graphical tool for interpreting the results of a meta-analysis. Zeitschrift fur Psychologie, 231(1), 65-78. doi:10.1027/2151-2604/a000513

Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., . . . Salanti, G. (2016). Methods to estimate the between-study variance and its uncertainty in meta-analysis. Research Synthesis Methods, 7(1), 55-79. doi:10.1002/jrsm.1164

Examples

### Load data from meta-analysis by McCall and Carriger (1993)
data(data.mccall93)

### Create meta-plot
meta_plot(ri = data.mccall93$ri, ni = data.mccall93$ni)

### Create summary meta-plot
meta_plot(ri = data.mccall93$ri, ni = data.mccall93$ni, nr_lines = "summary")

---

p-uniform*

Description

Function to apply the p-uniform* method for one-sample mean, two-independent means, and one raw correlation coefficient as described in van Aert and van Assen (2023).

Usage

puni_star(
  mi,
  ri,
  ni,
  sdi,
  m1i,
  m2i,
  n1i,
  n2i,
  sd1i,
  sd2i,
  tobs,
  yi,
  vi,
  alpha = 0.05,
  side,
  method = "ML",
  boot = FALSE,
  control
)

Arguments

mi	A vector of group means for one-sample means
ri	A vector of raw correlations
ni	A vector of sample sizes for one-sample means and correlations
sdi	A vector of standard deviations for one-sample means
m1i	A vector of means in group 1 for two-independent means
m2i	A vector of means in group 2 for two-independent means
n1i	A vector of sample sizes in group 1 for two-independent means
n2i	A vector of sample sizes in group 2 for two-independent means
sd1i	A vector of standard deviations in group 1 for two-independent means
sd2i	A vector of standard deviations in group 2 for two-independent means
tobs	A vector of t-values
yi	A vector of standardized effect sizes (see Details)
vi	A vector of sampling variances belonging to the standardized effect sizes (see Details)
alpha	A numerical value specifying the alpha level as used in primary studies (default is 0.05 but see Details).
side	A character indicating whether the effect sizes in the primary studies are in the right-tail of the distribution (i.e., positive) or in the left-tail of the distribution (i.e., negative) (either "right" or "left")
method	A character indicating the method to be used "ML" (default), "P", or "LNP"
boot	A logical indicating whether the p-value of testing whether the between-study variance is zero for methods P and LNP should be obtained by means of a parametric bootstrap. The default value is FALSE.
control	An optional list of elements that give the user more control over the optimization and root-finding algorithms (see Note)

Details

Three different effect size measures can be used as input for the puni_star function: one-sample means, two-independent means, and raw correlation coefficients. Analyzing one-sample means and two-independent means can be done by either providing the function group means (mi or m1i and m2i), standard deviations (sdi or sd1i and sd2i), and sample sizes (ni or n1i and n2i) or t-values (tobs) and sample sizes (ni or n1i and n2i). Both options should be accompanied with input for the arguments side, method, and alpha. See the Example section for examples. Raw correlation coefficients can be analyzed by supplying the raw correlation coefficients ri and sample sizes and ni to the puni_star function next to input for the arguments side, method, and alpha. Note that the method internally transforms the raw correlation coefficients to Fisher's z correlation coefficients. The output of the function also shows the results for the Fisher's z correlation coefficient. Hence, the results need to be transformed to raw correlation coefficients if this is preferred by the user.

It is also possible to specify the standardized effect sizes and its sampling variances directly via the yi and vi arguments. However, extensive knowledge about computing standardized effect sizes and its sampling variances is required and specifying standardized effect sizes and sampling variances is not recommended to be used if the p-values in the primary studies are not computed with a z-test. In case the p-values in the primary studies were computed with, for instance, a t-test, the p-values of a z-test and t-test do not exactly coincide and studies may be incorrectly included as a statistically significant or nonsignificant effect size. Furthermore, critical values in the primary studies are not transformed to critical z-values if yi and vi are used as input. This yields less accurate results.

The puni_star function assumes that two-tailed hypothesis tests were conducted in the primary studies. In case one-tailed hypothesis tests were conducted in the primary studies, the submitted alpha argument to the puni_star function has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be submitted to the puni_star function.

Note that only one effect size measure can be specified at a time. A combination of effect size measures usually causes true heterogeneity among effect sizes and including different effect size measures is therefore not recommended.

Selecting a method

Three different methods are currently implemented in the puni_star function. The ML method refers to maximum likelihood estimation of the effect size and the between-study variance. Profile likelihood confidence intervals around the estimates are computed by means of inverting the likelihood-ratio test. Likelihood-ratio tests are used for testing the null hypotheses of no effect and no between-study variance. The ML method is the recommended method for applying p-uniform*.

The two other methods (P and LNP) are moment based estimators. The method P is based on the distribution of the sum of independent uniformly distributed random variables (Irwin-Hall distribution) and the LNP method refers to Fisher's method (1950, Chapter 4). For these methods, a p-value for testing the null hypothesis of no between-study variance can also be obtained by means of a parametric bootstrap. This is necessary since the data is otherwise first used for estimating the effect size in the procedure for testing the null hypothesis of no between-study variance and then also used for computing a p-value. The test of no effect is not available for the methods P and LNP and the publication bias test for these methods is not yet implemented.

Value

est	p-uniform*'s effect size estimate
ci.lb	lower bound of p-uniform*'s 95% confidence interval of the effect size
ci.ub	upper bound of p-uniform*'s 95% confidence interval of the effect size
L.0	test statistic of p-uniform*'s test of the null hypothesis of no effect
pval.0	one-tailed p-value of p-uniform*'s test of null hypothesis of no effect
tau2	p-uniform*'s estimate of the between-study variance
tau2.lb	lower bound of p-uniform*'s 95% confidence interval of the between-study variance
tau2.ub	upper bound of p-uniform*'s 95% confidence interval of the between-study variance
L.het	test statistic of p-uniform*'s test of the null hypothesis of no between-study variance
pval.het	one-tailed p-value of p-uniform*'s test of null hypothesis of no between-study variance
pval.boot	one-tailed p-value of p-uniform*'s test of null hypothesis of no between-study variance obtained with a parametric bootstrap
...	a number of additional elements

Note

The control argument in the puni_star function is an optional argument that gives the user more control over the optimization and root-finding algorithms. This can be especially useful if estimation of the method does not converge and NAs are returned by the function. The control argument should be specified as a list containing one or more elements. For example, control = list(verbose = TRUE) Default values are used if an element is not specified. The following elements can be specified by the user:

• proc.ml: A character indicating with optimization procedure should be used for the method ML. The initial implementation of p-uniform* iteratively optimized the profile log-likelihood functions. As of version 0.2.6 of this package, the default optimization routine estimates both parameters at the same time. The old optimization procedure can be used by specifying proc.ml = "prof".

• stval.d: An integer that is the starting value of the effect size for estimation. This starting value is used for the method ML and is only needed when both parameters are estimated at the same time. See argument proc.ml for more information. The default value is the mean of the effect sizes in the meta-analysis.

• stval.tau: An integer that is the starting value of tau for estimation. This starting value is used for the methods ML, P, and LNP. The default value is the standard deviation of the effect sizes in the meta-analysis.

• int: A vector of length two that indicates the lower and upper bound of the interval that is used for estimating the effect size. The effect size estimate should be included in this interval. This interval is used for the methods ML, P, and LNP and its default values are (-2, 2).

• bounds.int A vector of length two that is used for determining the bounds for estimating the effect size with P and LNP. The default values are a function of the yi. The lower bound is the minimum yi minus 1 and the upper bound is the maximum yi plus 1. The effect size has to be between the lower and upper bound.

• tau.int: A vector of length two that indicates the lower and upper bound of the interval that is used for estimating the between-study variance. The estimate of the between-study variance should be included in this interval. This interval is used for the methods ML, P, and LNP and its default values are (0, 2).

• est.ci: A vector of length two indicating the values that are subtracted from and added to the estimate of the effect size for computing the 95% confidence intervals. This vector is used for the methods ML, P, and LNP and its default values are (3, 3). To give an example, estimates for the lower and upper bound around the effect size estimate are searched on the interval (est-3, est) and (est, est+3), respectively.

• tau.ci: A vector of length two indicating the values that are added to the estimate of the between-study variance for computing the 95% confidence intervals. This vector is used for the methods ML, P, and LNP and its default values are (3, 1).

• tol: A number indicating the desired accuracy of the estimates. This number is used for the methods ML, P, and LNP and its default value is 0.001.

• maxit: An integer indicating the maximum number of iterations that is used for estimating the effect size and between-study variance. This number is used for the methods ML, P, and LNP and its default value is 300.

• fnscale: An integer that can be used for scaling the log-likelihood value in the optimization. fnscale is one of the control arguments of the optim() function that is internally used for optimization when the ML method is used and both parameters are estimated at the same time. See the argument proc.ml above #' and the documentation of the optim() function for more information.

• verbose: A logical indicating whether information should be printed about the algorithm for estimating the effect size and between-study variance. This logical is used for the methods ML, P, and LNP and its default value is FALSE.

• reps: An integer indicating the number of bootstrap replications for computing the bootstrapped p-value for the test of no between-study variance. This integer is used for the methods P and LNP and its default value is 1000.

Author(s)

Robbie C.M. van Aert [email protected]

References

Fisher, R.A. (1950). Statistical methods for research workers (11th ed.). London: Oliver & Boyd.

van Aert, R.C.M., & van Assen, M.A.L.M. (2023). Correcting for publication bias in a meta-analysis with the p-uniform* method. Manuscript submitted for publication. Preprint: https://osf.io/preprints/bitss/zqjr9/

Examples

### Generate data for one-sample mean with mu = 0.2 and tau^2 = 0.01
set.seed(123)
ni <- rep(50, 25)
sdi <- rep(1, 25)
ui <- rnorm(25, mean = 0.2, sd = 0.1)
mi <- rnorm(25, mean = ui, sd = sdi/sqrt(ni))
tobs <- mi/(sdi/sqrt(ni))

### Apply p-uniform* method using sample means
puni_star(mi = mi, ni = ni, sdi = sdi, alpha = 0.05, side = "right", method = "ML")

### Apply p-uniform* method using t-values
puni_star(tobs = tobs, ni = ni, alpha = 0.05, side = "right", method = "ML")

---

p-uniform

Description

Function to apply the p-uniform method for one-sample mean, two-independent means, and one raw correlation coefficient as described in van Assen, van Aert, and Wicherts (2015) and van Aert, Wicherts, and van Assen (2016).

Usage

puniform(
  mi,
  ri,
  ni,
  sdi,
  m1i,
  m2i,
  n1i,
  n2i,
  sd1i,
  sd2i,
  tobs,
  yi,
  vi,
  alpha = 0.05,
  side,
  method = "P",
  plot = FALSE
)

Arguments

mi	A vector of group means for one-sample means
ri	A vector of raw correlations
ni	A vector of sample sizes for one-sample means and correlations
sdi	A vector of standard deviations for one-sample means
m1i	A vector of means in group 1 for two-independent means
m2i	A vector of means in group 2 for two-independent means
n1i	A vector of sample sizes in group 1 for two-independent means
n2i	A vector of sample sizes in group 2 for two-independent means
sd1i	A vector of standard deviations in group 1 for two-independent means
sd2i	A vector of standard deviations in group 2 for two-independent means
tobs	A vector of t-values
yi	A vector of standardized effect sizes (see Details)
vi	A vector of sampling variances belonging to the standardized effect sizes (see Details)
alpha	A numerical value specifying the alpha level as used in primary studies (default is 0.05, see Details).
side	A character indicating whether the effect sizes in the primary studies are in the right-tail of the distribution (i.e., positive) or in the left-tail of the distribution (i.e., negative) (either "right" or "left")
method	A character indicating the method to be used "P" (default), "LNP", "LN1MINP", "KS", "AD", or "ML"
plot	A logical indicating whether a plot showing the relation between observed and expected p-values has to be rendered (default is TRUE)

Details

Three different effect size measures can be used as input for the puniform function: one-sample means, two-independent means, and raw correlation coefficients. Analyzing one-sample means and two-independent means can be done by either providing the function group means (mi or m1i and m2i), standard deviations (sdi or sd1i and sd2i), and sample sizes (ni or n1i and n2i) or t-values (tobs) and sample sizes (ni or n1i and n2i). Both options should be accompanied with input for the arguments side, method, and alpha. See the Example section for examples. Raw correlation coefficients can be analyzed by supplying the raw correlation coefficients ri and sample sizes and ni to the puni_star function next to input for the arguments side, method, and alpha. Note that the method internally transforms the raw correlation coefficients to Fisher's z correlation coefficients. The output of the function also shows the results for the Fisher's z correlation coefficient. Hence, the results need to be transformed to raw correlation coefficients if this is preferred by the user.

It is also possible to specify the standardized effect sizes and its sampling variances directly via the yi and vi arguments. However, extensive knowledge about computing standardized effect sizes and its sampling variances is required and specifying standardized effect sizes and sampling variances is not recommended to be used if the p-values in the primary studies are not computed with a z-test. In case the p-values in the primary studies were computed with, for instance, a t-test, the p-values of a z-test and t-test do not exactly coincide and studies may be incorrectly included in the analyses. Furthermore, critical values in the primary studies cannot be transformed to critical z-values if yi and vi are used as input. This yields less accurate results.

The puniform function assumes that two-tailed hypothesis tests were conducted in the primary studies. In case one-tailed hypothesis tests were conducted in the primary studies, the alpha level has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be submitted to p-uniform.

Note that only one effect size measure can be specified at a time. A combination of effect size measures usually causes true heterogeneity among effect sizes and including different effect size measures is therefore not recommended.

Six different estimators can be used when applying p-uniform. The P method is based on the distribution of the sum of independent uniformly distributed random variables (Irwin-Hall distribution) and is the recommended estimator (van Aert et al., 2016). The ML estimator refers to effect size estimation with maximum likelihood. Profile likelihood confidence intervals are computed, and likelihood ratio tests are used for the test of no effect and publication bias test if ML is used. The LNP estimator refers to Fisher's method (1950, Chapter 4) for combining p-values and the LN1MINP estimator first computes 1 - p-value in each study before applying Fisher's method on these transformed p-values (van Assen et al., 2015). KS and AD respectively use the Kolmogorov-Smirnov test (Massey, 1951) and the Anderson-Darling test (Anderson & Darling, 1954) for testing whether the (conditional) p-values follow a uniform distribution.

Value

est	p-uniform's effect size estimate
ci.lb	lower bound of p-uniform's confidence interval
ci.ub	upper bound of p-uniform's confidence interval
ksig	number of significant studies
L.0	test statistic of p-uniform's test of null-hypothesis of no effect (for method "P" a z-value)
pval.0	one-tailed p-value of p-uniform's test of null-hypothesis of no effect
L.pb	test statistic of p-uniform's publication bias test
pval.pb	one-tailed p-value of p-uniform's publication bias test
est.fe	effect size estimate based on traditional fixed-effect meta-analysis
se.fe	standard error of effect size estimate based on traditional fixed-effect meta-analysis
zval.fe	test statistic of the null-hypothesis of no effect based on traditional fixed-effect meta-analysis
pval.fe	one-tailed p-value of the null-hypothesis of no effect based on traditional fixed-effect meta-analysis
ci.lb.fe	lower bound of confidence interval based on traditional fixed-effect meta-analysis
ci.ub.fe	ci.ub.fe upper bound of confidence interval based on traditional fixed-effect meta-analysis
Qstat	test statistic of the Q-test for testing the null-hypothesis of homogeneity
Qpval	one-tailed p-value of the Q-test

Author(s)

Robbie C.M. van Aert [email protected]

References

Anderson, T. W., & Darling, D. A. (1954). A test of goodness of fit. Journal of the American Statistical Association, 49(268), 765-769.

Fisher, R. A. (1950). Statistical methods for research workers (11th ed.). London: Oliver & Boyd.

Massey, F. J. (1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Association, 46(253), 68-78.

Van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016). Conducting meta-analyses on p-values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science, 11(5), 713-729. doi:10.1177/1745691616650874

Van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015). Meta-analysis using effect size distributions of only statistically significant studies. Psychological Methods, 20(3), 293-309. doi: http://dx.doi.org/10.1037/met0000025

Examples

### Load data from meta-analysis by McCall and Carriger (1993)
data(data.mccall93)

### Apply p-uniform method
puniform(ri = data.mccall93$ri, ni = data.mccall93$ni, side = "right", 
method = "LNP", plot = TRUE)

### Generate example data for one-sample means design
set.seed(123)
ni <- 100
sdi <- 1
mi <- rnorm(8, mean = 0.2, sd = sdi/sqrt(ni))
tobs <- mi/(sdi/sqrt(ni))

### Apply p-uniform method based on sample means
puniform(mi = mi, ni = ni, sdi = sdi, side = "right", plot = FALSE)

### Apply p-uniform method based on t-values
puniform(ni = ni, tobs = tobs, side = "right", plot = FALSE)

---

req_ni_r

Description

Function for computing the required sample size for a replication based on the Snapshot Bayesian Hybrid Meta-Analysis Method for two-independent means and raw correlation coefficients.

Usage

req_ni_r(
  ri.o,
  ni.o,
  m1i.o,
  m2i.o,
  n1i.o,
  n2i.o,
  sd1i.o,
  sd2i.o,
  tobs.o,
  alpha,
  des.pprob,
  des.pow,
  lo = 4,
  hi = 1e+05
)

Arguments

ri.o	A numeric value containing the raw correlation coefficient of the original study
ni.o	An integer containing the sample size for the raw correlation coefficient
m1i.o	A numeric value containing the mean in group 1 of the original study for two-independent means
m2i.o	A numeric value containing the mean in group 2 of the original study for two-independent means
n1i.o	A numeric value containing the sample size in group 1 of the original study for two-independent means
n2i.o	A numeric value containing the sample size in group 2 of the original study for two-independent means
sd1i.o	A numeric value containing the standard deviation in group 1 of the original study for two-independent means
sd2i.o	A numeric value containing the standard deviation in group 2 of the original study for two-independent means
tobs.o	A numeric value containing the t-value of the original study
alpha	A numeric value specifying the alpha level as used in the original study
des.pprob	A numeric value specifying the posterior probability that an user desires to obtain for one of the four true effect sizes
des.pow	A numeric value specifying the probability of observing a posterior probability larger than des.pprob that an user desires to obtain for one of the four true effect sizes
lo	A numeric value specifying the lower bound of the search interval that is used for the optimization procedure (default is 4)
hi	A numeric value specifying the upper bound of the search interval that is used for the optimization procedure (default is 100,000)

Details

The function computes the required sample size for the replication based on the Snapshot Bayesian Hybrid Meta-Analysis Method for four true effect sizes (no, small, medium, and large). The required sample size is computed by optimizing $P(\pi_x \ge a)=b$ with $\pi_x$ being the posterior probability with x referring to no (0), small (S), medium (M), and large (L) true effect size and $a$ the desired posterior probability, and $b$ the desired probability of observing a posterior probability larger than $a$. The required sample size for the replication is computed with and without including information of the original study. Computing the required sample size with the Snapshot Bayesian Hybrid Meta-Analysis Method is akin to computing the required sample size with a power analysis in null hypothesis significance testing. For more information see van Aert and van Assen (2016).

The req.ni.r function assumes that a two-tailed hypothesis test was conducted in the original study. In case one-tailed hypothesis tests was conducted in the original study, the alpha level has to be multiplied by two. For example, if a one-tailed hypothesis test was conducted with an alpha level of .05, an alpha of 0.1 has to be submitted to req.ni.r.

Value

The req.ni.r function returns a 4x2 matrix with in the first column the required total sample size of the replication when information of the original study is taken into account and in the second column the required sample size if information of the original study is ignored.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & van Assen, M.A.L.M. (2016). Bayesian evaluation of effect size after replicating an original study. Manuscript submitted for publication.

Examples

### Example as presented on page 491 in Maxwell, Lau, and Howard (2015)
req_ni_r(ri.o = 0.243, ni.o = 80, alpha = .05, des.pprob = 0.75, des.pow = 0.8)

---

snapshot

Description

Function for applying Snapshot Bayesian Hybrid Meta-Analysis Method for two-independent means and raw correlation coefficients.

Usage

snapshot(ri, ni, m1i, m2i, n1i, n2i, sd1i, sd2i, tobs, alpha = 0.05)

Arguments

ri	A vector of length two containing the raw correlation coefficients of the original study and replication
ni	A vector of length two containing the sample size of the original study and replication for the raw correlation coefficient
m1i	A vector of length two containing the means in group 1 for the original study and replication for two-independent means
m2i	A vector of length two containing the means in group 2 for the original and replication for two-independent means
n1i	A vector of length two containing the sample sizes in group 1 for the original study and replication for two-independent means
n2i	A vector of length two containing the sample sizes in group 2 for the original study and replication for two-independent means
sd1i	A vector of length two containing the standard deviations in group 1 for the original study and replication for two-independent means
sd2i	A vector of length two containing the standard deviations in group 2 for the original study and replication for two-independent means
tobs	A vector of length two containing the t-values of the original study and replication
alpha	A numerical value specifying the alpha level as used in the original study (default is 0.05, see Details)

Details

The function computes posterior probabilities (assuming a uniform prior distribution) for four true effect sizes (no, small, medium, and large) based on an original study and replication while taking into account statistical significance in the original study. For more information see van Aert and van Assen (2016).

Two different effect size measures can be used as input for the snapshot function: two-independent means and raw correlation coefficients. Analyzing two-independent means can be done by either providing the function group means (m1i and m2i), standard deviations (sd1i and sd2i), and sample sizes (n1i and n2i) or t-values (tobs) and sample sizes (n1i and n2i). Both options should be accompanied with input for the argument alpha. See the Example section for an example. Raw correlation coefficients can be analyzed by supplying ri and ni to the snapshot function next to input for the argument alpha.

The snapshot function assumes that a two-tailed hypothesis test was conducted in the original study. In case a one-tailed hypothesis test was conducted in the original study, the alpha level has to be multiplied by two. For example, if a one-tailed hypothesis test was conducted with an alpha level of .05, an alpha of 0.1 has to be submitted to snapshot.

Value

The snapshot function returns a data frame with posterior probabilities for no (p.0), small (p.sm), medium (p.me), and large (p.la) true effect size.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & van Assen, M.A.L.M. (2017). Bayesian evaluation of effect size after replicating an original study. PLoS ONE, 12(4), e0175302. doi:10.1371/journal.pone.0175302

Examples

### Example as presented on page 491 in Maxwell, Lau, and Howard (2015)
snapshot(ri = c(0.243, 0.114), ni = c(80, 172))

---

snapshot_naive

Description

Function for applying Snapshot Bayesian Meta-Analysis Method (snapshot naive) for two-independent means and raw correlation coefficients.

Usage

snapshot_naive(ri, ni, m1i, m2i, n1i, n2i, sd1i, sd2i, tobs)

Arguments

ri	A vector of length two containing the raw correlation coefficients of the original study and replication
ni	A vector of length two containing the sample size of the original study and replication for the raw correlation coefficient
m1i	A vector of length two containing the means in group 1 for the original study and replication for two-independent means
m2i	A vector of length two containing the means in group 2 for the original and replication for two-independent means
n1i	A vector of length two containing the sample sizes in group 1 for the original study and replication for two-independent means
n2i	A vector of length two containing the sample sizes in group 2 for the original study and replication for two-independent means
sd1i	A vector of length two containing the standard deviations in group 1 for the original study and replication for two-independent means
sd2i	A vector of length two containing the standard deviations in group 2 for the original study and replication for two-independent means
tobs	A vector of length two containing the t-values of the original study and replication

Details

The function computes posterior probabilities (assuming a uniform prior distribution) for four true effect sizes (no, small, medium, and large) based on an original study and replication. For more information see van Aert and van Assen (2016).

Two different effect size measures can be used as input for the snapshot.naive function: two-independent means and raw correlation coefficients. Analyzing two-independent means can be done by either providing the function group means (m1i and m2i), standard deviations (sd1i and sd2i), and sample sizes (n1i and n2i) or t-values (tobs) and sample sizes (n1i and n2i).See the Example section for an example. Raw correlation coefficients can be analyzed by supplying ri and ni to the snapshot.naive.

Value

The snapshot.naive function returns a data frame with posterior probabilities for no (p.0), small (p.sm), medium (p.me), and large (p.la) true effect size.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & van Assen, M.A.L.M. (2017). Bayesian evaluation of effect size after replicating an original study. PLoS ONE, 12(4), e0175302. doi:10.1371/journal.pone.0175302

Examples

### Example as presented on page 491 in Maxwell, Lau, and Howard (2015)
snapshot_naive(ri = c(0.243, 0.114), ni = c(80, 172))

---

var_boot_fis

Description

Function for parametric bootstrapping procedure to estimate the variability in outcomes' effect size in case of Fisher-z transformed correlations as effect size measure.

Usage

var_boot_fis(ri, n, r, dv = 10, reps = 1000)

Arguments

ri	A vector with Pearson correlation coefficients in a primary study (see Details)
n	A numerical value specifying the total sample size of a primary study
r	A numerical value specifying the Pearson correlation coefficient between variables h and m (see Details)
dv	An integer specifying the total number of dependent measures (default is 10, see Details)
reps	An integer specifying the number of bootstrap replications (default is 1,000)

Details

In case of three variables (l, h, and m), overlapping Fisher-z transformed correlation coefficients can be computed between variables l and h and variables l and m. The function computes the variance of the two overlapping Fisher-z transformed correlation coefficients using a parametric bootstrap procedure. For more information see van Aert & Wicherts (2023).

The vector ri can contain a single Pearson correlation coefficient or multiple coefficients if information on more than one outcome is available. The integer dv is an optional argument to specify the expected number of outcomes used in a primary study. This argument can be any value between 2 and infinity. Larger values yield more accurate estimates of the variance but slow down the bootstrap procedure.

The variance that is computed with this function can be used to correct for outcome reporting bias by including the variance as a moderator in a (multivariate) meta-analysis. Please see van Aert & Wicherts (2023) for more information.

Value

The var_boot_fis function returns a numerical value that is the variance of multiple correlated Fisher-z transformed correlations.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & Wicherts, J.M. (2023). Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach. Behavior Research Methods.

Examples

### Compute variance for an artificial example
var_boot_fis(ri = 0, n = 100, r = 0.3)

---

var_boot_rmd

Description

Function for parametric bootstrapping procedure to estimate the variability in outcomes' effect size in case of raw mean difference as effect size measure.

Usage

var_boot_rmd(sd1i, sd2i, n1i, n2i, r, dv = 10, reps = 1000)

Arguments

sd1i	A vector of standard deviations of the outcomes in group 1 (see Details)
sd2i	A vector of standard deviations of the outcomes in group 2 (see Details)
n1i	An integer specifying the sample size of group 1
n2i	An integer specifying the sample size of group 2
r	A numerical value specifying the Pearson correlation coefficient between participants' scores on the different outcomes
dv	An integer specifying the total number of outcomes (default is 10, see Details)
reps	An integer specifying the number of bootstrap replications (default is 1,000)

Details

Multiple raw mean differences can be computed in case of two groups and multiple outcomes. The function estimates the variance of raw mean differences given a correlation among the outcomes using a parametric bootstrap procedure. For more information see van Aert & Wicherts (2023).

The vectors sd1i and sd2i can contain a single standard deviation or multiple standard deviations if information on more than one outcome is available. The integer dv is an optional argument to specify the expected number of outcomes used in a primary study. This argument can be any value between 2 and infinity. Larger values yield more accurate estimates of the variance but slow down the bootstrap procedure.

The variance that is estimated with this function can be used to correct for outcome reporting bias by including the variance as a moderator in a (multivariate) meta-analysis. Please see van Aert & Wicherts (2023) for more information.

Value

The var_boot_rmd function returns a numerical value that is an estimate of the variance of multiple correlated raw mean differences.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & Wicherts, J.M. (2023). Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach. Behavior Research Methods.

Examples

### Compute variance for an artificial example
var_boot_rmd(sd1i = c(0.8, 1.2), sd2i = c(0.85, 1.15), n1i = 100, n2i = 95, r = 0.3)

---

var_dif_fis

Description

Function for computing the variance of the difference between two overlapping Fisher-z transformed correlation coefficients.

Usage

var_dif_fis(n, r, rho)

Arguments

n	A numerical value specifying the total sample size of a primary study
r	A numerical value specifying the Pearson correlation coefficient between variables h and m (see Details)
rho	A numerical value specifying the Pearson correlation coefficient between variables l and h and variables h and m (see Details)

Details

In case of three variables (l, h, and m), overlapping Fisher-z transformed correlation coefficients can be computed between variables l and h and variables l and m. The function computes the variance of the difference between these two overlapping Fisher-z transformed correlations. For a derivation of this variance see van Aert & Wicherts (2023).

The variance that is computed with this function can be used to correct for outcome reporting bias by including the variance as a moderator in a (multivariate) meta-analysis. Please see van Aert & Wicherts (2023) for more information.

Value

The var_dif_fis function returns a numerical value that is the variance of the difference of two overlapping Fisher-z transformed correlations given n, r, and rho.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & Wicherts, J.M. (2023). Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach. Behavior Research Methods.

Examples

### Compute variance for an artificial example
var_dif_fis(n = 100, r = 0.3, rho = 0.5)

---

var_dif_rmd

Description

Function for computing the variance of the difference between two raw mean differences.

Usage

var_dif_rmd(sd1i, sd2i, n1i, n2i, r)

Arguments

sd1i	A vector of standard deviations of the outcomes in group 1 (see Details)
sd2i	A vector of standard deviations of the outcomes in group 2 (see Details)
n1i	An integer specifying the sample size of group 1
n2i	An integer specifying the sample size of group 2
r	A numerical value specifying the Pearson correlation coefficient between participants' scores on the different outcomes

Details

Multiple raw mean differences can be computed in case of two groups and multiple outcomes. The function computes the variance of the difference of two raw mean differences given a correlation between the outcomes. For a derivation of this variance see the supplemental materials of van Aert & Wicherts (2023).

The vectors sd1i and sd2i can contain a single standard deviation or multiple standard deviations if information on more than one outcome measure is available.

The variance that is computed with this function can be used to correct for outcome reporting bias by including the variance as a moderator in a (multivariate) meta-analysis. Please see van Aert & Wicherts (2023) for more information.

Value

The var_dif_rmd function returns a numerical value that is the variance of the difference of two raw mean differences given r.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & Wicherts, J.M. (2023). Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach. Behavior Research Methods.

Examples

### Compute variance for an artificial example
var_dif_rmd(sd1i = c(0.8, 1.2), sd2i = c(0.85, 1.15), n1i = 100, n2i = 95, r = 0.3)

---

var_pop

Description

Function for estimating the population variance of correlated outcomes' effect size within a primary study.

Usage

var_pop(v, r)

Arguments

v	A numerical value specifying the sampling variance of the effect size (see Details)
r	A numerical value specifying the Pearson correlation coefficient between the outcomes (see Details)

Details

This function estimates the population variance of the effect size of correlated outcomes within a study. That is, it estimates the population variance from a single draw of a multivariate normal distribution. The function assumes equal true effect size of all outcomes, equal sampling variances of the outcomes' effect size, and equal correlation (i.e., r) among the outcomes.

For a derivation of this estimator see van Aert & Wicherts (2023).

The variance that is computed with this function can be used to correct for outcome reporting bias by including the variance as a moderator in a (multivariate) meta-analysis. Please see van Aert & Wicherts (2023) for more information.

Value

The var_pop function returns a numerical value that is the estimate of the population variance of correlated outcomes' effect size given v and r.

Author(s)

Robbie C.M. van Aert [email protected]

References

van Aert, R.C.M. & Wicherts, J.M. (2023). Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach. Behavior Research Methods.

Examples

### Compute variance for an artificial example
var_pop(v = 0.1, r = 0.3)

---