Introduction

The two-sample t-test is the standard frequentist parametric procedure when the variate in each condition is continuous and when the data are normally distributed with the same population variance in each condition. It is not likely that the assumptions of the parametric analysis are strictly valid, so it is prudent to have some nonparametric procedures as alternatives. The median test and the Mann-Whitney U-test are two frequentist nonparametric procedures that are the conventional alternatives to the two-sample-t test.

The essential idea underlying the median test is to categorize all the scores so as to form a 2 × 2 table of frequencies. One classification (e.g., the rows) is based on the observation being above the combined median versus being at or below the median. The other classification (e.g. the columns) is based on the observation being from the experimental group (denoted as the E group) or being from the control group (denoted as the C group). In contrast to the median test, the Mann-Whitney test is based on U-statistics. The U_E statistic is the number of times an E-labelled score is larger than a C-labelled score, whereas the U_C statistic is the number of times the C variate is larger than the E variate. Because the U statistics are based on more fine-grained information than the median test, the median test is generally less powerful relative to the Mann-Whitney procedure. Yet, for precisely that reason, the median test has an appeal for some investigators: if a median test can detect differences between the two conditions, then it is convincing that those conditions are different.

In the DFBA package, the Bayesian counterpart to the median test is given by the dfba_median_test() function. This vignette on the dfba_median_test() contains discussion of the beta distribution, the binomial model, and Bayes factors; for more on those topics see the dfba_binomial() and dfba_beta_bayes_factor() vignettes.

Theoretical Framework for the Bayesian Median Test

Given n_E continuous scores from group E and n_C scores from group C, the display below helps to define the frequencies associated with the median test where the rows of the 2 × 2 table are organized in relation to the combined median.

$$ \begin{array}{c|cc|c} & E \hskip 1mm & C\hskip 1mm & \\ \hline above\,\,median & n_E^{above} & n_C^{above} & n_E^{above} + n_C^{above}\\ & & &\\ at\,\,or\,\,below\,\,median & n_E^{below} & n_C^{below} & n_E^{below} + n_C^{below}\\ \hline & & &\\ & n_E & n_C & n_E+n_C \end{array} $$

The frequentist median test examines, with a χ² statistic, the sharp null hypothesis that the proportion of responses above the combined median is the same for both the E and C groups (Siegel & Castellan, 1988). Chechile (2020) provided an alternative Bayesian analysis for this procedure. Rather than focusing the median test on an assessment of a point-null hypothesis, the Bayesian approach involves estimating a population parameter associated with an above-median response being from the E group. The frequencies in row 1 (above-median frequencies) are binomial frequencies in terms of the group origin (i.e., E versus C). From a Bayesian perspective, a population-level ϕ parameter can be defined for the proportion of E values that are above the combined sample median. Similarly, the frequencies for the scores at or below the combined sample median can also be examined; in that case, the corresponding population proportion in the E condition must be 1 − ϕ. Thus, it is sufficient only to examine the above-median frequencies to make an inference about the ϕ parameter. Thus the Bayesian form of the median test reduces to the binomial model, so the prior and posterior distributions for the population ϕ parameter belong to the beta family of distributions. The default prior for the dfba_median_test() function is the uniform distribution, i.e, a₀ = b₀ = 1. The posterior shape parameters for ϕ are a = a₀ + n_E^above and b = b₀ + n_C^above.¹

Because the number of scores in groups E and C might be very different, it is important to examine the ϕ parameter relative to an expected base-rate value. For example, suppose that there are n_E = 90 observed values from the E condition and n_C = 10 observed values for the C condition. Given that there are nine times more E scores than C scores in the sample, the assessment of the ϕ parameter should be to see if it is greater than .9. If there were no difference whatsoever between the E and C conditions in this hypothetical example, then about 90 percent or less of the above-median values would be expected from the E condition. If the posterior ϕ parameter were substantially greater than the group E sample base rate, then that would provide support for the hypothesis that group E has larger representation of the above-median responses in the population than does the C group.

Using the dfba_median_test() Function

There are four arguments for the dfba_median_test() function: E, C, a0, and b0. The E and C arguments are required vectors of continuous values for the data in the two independent conditions. The a0 and b0 arguments are the shape parameters for the prior beta distribution; each have default values of 1. Note: the default values a0 = 1 and b0 = 1 – resulting in a beta distribution with α = β = 1 – represent the uniform distribution.

group1 <- c(12.90, 10.84, 22.67, 10.64, 10.67, 10.79, 13.55, 10.95, 12.19, 12.76, 10.89, 11.02, 14.27, 13.98, 11.52, 13.49, 11.22, 15.07, 15.74, 19.00)

group2 <- c(4.63, 58.64, 5.07, 4.66, 4.13, 3.92, 3.39, 3.57, 3.56, 3.39)

dfba_median_test(E = group1, 
                 C = group2)
#> Descriptive Statistics 
#> ========================
#>   Observed frequencies: 
#>   E               C 
#>   20              10 
#>   Overall median: 
#>   10.985 
#>   Frequencies above the median: 
#>   E               C 
#>   14              1 
#>   Frequencies at or below the median: 
#>   E               C 
#>   6               9 
#> 
#> Bayesian Analyses
#> ========================
#>   Base rates for E and C responses:
#>   E               C
#>   0.666667        0.3333333 
#>   Analysis of above-median response rates for E and C:
#>   Posterior beta shape parameter for the phi parameter 
#>   a_post          b_post 
#>   15              2 
#>   Prior probability of exceeding base rate: 
#>   E               C 
#>   0.333333        0.6666667 
#>   Posterior probability of exceeding base rate: 
#>   E               C 
#>   0.986298        0.01370195 
#>   Bayes factor BF10 E > E_baserate: 
#>   143.9646 
#>   Bayes factor BF10 C > C_baserate 
#>   0.006946151

median_test_results<-dfba_median_test(E = group1, 
                                      C = group2)

dfba_binomial(median_test_results$nEabove, 
              median_test_results$nCabove)
#> Prior and Posterior Beta Shape Parameters: 
#> ========================
#>   Prior Beta Shape Parameters 
#>   a0              b0 
#>   1               1 
#>   Posterior Beta Shape Parameters:   
#>   a_post          b_post 
#>   15              2 
#> Estimates of the Binomial Population Rate Parameter 
#> ========================
#>   Posterior Mean 
#>   0.8823529 
#>   Posterior Median 
#>   0.897297 
#>   Posterior Mode 
#>   0.9333333 
#>   95% Equal-tail interval limits: 
#>   Lower Limit     Upper Limit 
#>   0.697679        0.9844864 
#>   95% Highest-density interval limits: 
#>   Lower Limit     Upper Limit 
#>   0.734172        0.9962847 
#> 
#> 

plot(dfba_binomial(median_test_results$nEabove, 
                   median_test_results$nCabove))

dfba_beta_bayes_factor(a_post = median_test_results$a_post, 
                       b_post = median_test_results$b_post, 
                       method = "point", 
                       H0 = 14/15)
#> Bayes Factor for Point Estimates 
#> ========================
#>   Point Null Hypothesis 
#>   0.9333333 
#>   Shape Parameters for Prior Beta Distribution 
#>   a0              b0 
#>   1               1 
#>   Shape Parameters for Posterior Beta Distribution 
#>   a_post          b_post 
#>   15              2 
#>   Prior Probability Density for Null Hypothesis 
#>   1 
#>   Posterior Probability Density for Null Hypothesis 
#>   6.090246 
#>   Bayes Factor Estimate for the Alternative over the Null Hypothesis 
#>   0.164197 
#>   Bayes Factor Estimate for the Null over the Alternative Hypothesis 
#>   6.090246

References

Chechile, R. A. (2020). Bayesian Statistics for Experimental Scientists: A General Introduction Using Distribution-Free Methods. Cambridge, MA: MIT Press.

Siegel, S., and Castellan N. J. (1988). Nonparametric Statistics for the Behavioral Sciences. New York: McGraw-Hill.