| Title: | Likelihood Based Inference |
|---|---|
| Description: | Maximum likelihood estimation and likelihood ratio test are essential for modern statistics. This package supports in calculating likelihood based inference. Reference: Pawitan Y. (2001, ISBN:0-19-850765-8). |
| Authors: | Kyun-Seop Bae [aut, cre, cph] |
| Maintainer: | Kyun-Seop Bae <[email protected]> |
| License: | GPL-3 |
| Version: | 0.2.1 |
| Built: | 2024-12-16 07:04:18 UTC |
| Source: | CRAN |
It conducts likelihood based inference.
Modern likelihood concept and maximum likelihood estimation are established by Fisher RA, while Likelihood Ratio Test (LRT) is established by Neyman J. Post-Fisher methods - generalized linear model, survival analysis, and mixed effects model - are all likelihood based. Inferences from the perspective of Fisherian and pure likelihoodist are suggested here.
Kyun-Seop Bae <[email protected]>
Wilks SS. The Large-sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses. Ann Math Stat. 1938;9(1):60-62.
Edwards AWF. Likelihood. 1972.
Fisher RA. Statistical Methods and Scientific Inference. 3e. 1973.
Bates DM, Watts DG. Nonlinear Regression Analysis and its Application. 1988.
Ruppert D, Cressie N, Carroll RJ. A Transformation/Weighting Model for Estimating Michaelis-Menten Parameters. Cornell University Technical Report 796. 1988.
Royall R. Statistical Evidence. 1997.
Pinheiro JC, Bates DM. Mixed Effects Models in S and S-PLUS. 2000.
Pawitan Y. In All Likelihood: Statistical Modelling and Inference Using Likelihood. 2001.
Lehmann EL. Fisher, Nayman, and the Creation of Classical Statistics. 2011.
Rohde CA. Introductory Statistical Inference with the Likelihood Function. 2014.
Held L, Bové DS. Likelihood and Bayesian Inference. 2020.
Likelihood based confidence interval of sd and variance assuming normal distribution. It usually shows narrower interval than convenrtional chi-square interval. This uses estimated likelihood, not profile likelihood.
LBCIvar(x, conf.level=0.95)LBCIvar(x, conf.level=0.95)
x |
a vector of observation |
conf.level |
confidence level |
It calculates (same height) likelihood based confidence interval of sd and variance assuming normal distribution in one group. The likelihood interval is asymmetric and there is no standard error in the output.
PE |
maximum likelihood estimate |
LL |
lower limit of likelihood interval |
UL |
upper limit of likelihood interval |
Kyun-Seop Bae [email protected]
LBCIvar(lh) (length(lh) - 1)*var(lh)/qchisq(c(0.975, 0.025), length(lh) - 1)LBCIvar(lh) (length(lh) - 1)*var(lh)/qchisq(c(0.975, 0.025), length(lh) - 1)
Likelihood interval of a proportion in one group
LIbin(y, n, k, conf.level=0.95, eps=1e-8)LIbin(y, n, k, conf.level=0.95, eps=1e-8)
y |
positive event count of a group |
n |
total count of a group |
k |
1/k likelihood interval will be calculated |
conf.level |
approximately corresponding confidence level. If k is specified, this is ignored. |
eps |
Values less than eps are considered as 0. |
It calculates likelihood interval of a proportion in one group. The likelihood interval is asymmetric and there is no standard error in the output. If you need percent scale, multiply the output by 100.
y |
positive (concerning) event count |
n |
total trial count |
PE |
maximum likelihood estimate on the proportion |
LL |
lower limit of likelihood interval |
UL |
upper limit of likelihood interval |
Kyun-Seop Bae [email protected]
Fisher RA. Statistical methods and scientific inference. 3e. 1973. pp68-76.
binom.test, prop.test
LIbin(3, 14, k=2) LIbin(3, 14, k=5) LIbin(3, 14, k=15) LIbin(3, 14) # binom.test(3, 14) # prop.test(3, 14)LIbin(3, 14, k=2) LIbin(3, 14, k=5) LIbin(3, 14, k=15) LIbin(3, 14) # binom.test(3, 14) # prop.test(3, 14)
Likelihood interval of mean and sd assuming normal distribution. This is estimated likelihood interval, not profile likelihood interval.
LInorm(x, k, conf.level=0.95, PLOT="", LOCATE=FALSE, Resol=201)LInorm(x, k, conf.level=0.95, PLOT="", LOCATE=FALSE, Resol=201)
x |
a vector of observation |
k |
1/k likelihood interval will be calculated |
conf.level |
approximately corresponding confidence level. If k is specified, this is ignored. |
PLOT |
"1d" for profile plot or "2d" for contour plot. |
LOCATE |
use locater. This works only with PLOT="2D" option. |
Resol |
resolution for plot. This works only with PLOT=TRUE option. |
It calculates likelihood interval of mean and sd assuming normal distribution in one group. There is no standard error in the output.
PE |
maximum likelihood estimate |
LL |
lower limit of likelihood interval |
UL |
upper limit of likelihood interval |
Kyun-Seop Bae [email protected]
x = c(-5.3, -4.5, -1.0, -0.7, 3.7, 3.9, 4.2, 5.5, 6.8, 7.4, 9.3) LInorm(x, k=1/0.15) # Pawitan Ex10-9 p289 LInorm(x) LInorm(x, PLOT="1d") LInorm(x, PLOT="2d", LOCATE=TRUE)x = c(-5.3, -4.5, -1.0, -0.7, 3.7, 3.9, 4.2, 5.5, 6.8, 7.4, 9.3) LInorm(x, k=1/0.15) # Pawitan Ex10-9 p289 LInorm(x) LInorm(x, PLOT="1d") LInorm(x, PLOT="2d", LOCATE=TRUE)
Likelihood interval of sd and variance assuming normal distribution. This is estimated likelihood interval, not profile likelihood interval.
LInormVar(x, k, conf.level=0.95)LInormVar(x, k, conf.level=0.95)
x |
a vector of observation |
k |
1/k likelihood interval will be calculated |
conf.level |
approximately corresponding confidence level. If k is specified, this is ignored. |
It calculates likelihood interval of sd and variance assuming normal distribution in one group. The likelihood interval is asymmetric and there is no standard error in the output.
PE |
maximum likelihood estimate |
LL |
lower limit of likelihood interval |
UL |
upper limit of likelihood interval |
Kyun-Seop Bae [email protected]
x = c(-5.3, -4.5, -1.0, -0.7, 3.7, 3.9, 4.2, 5.5, 6.8, 7.4, 9.3) LInormVar(x, k=1/0.15) # Pawitan Ex10-9 p289 LInormVar(x)x = c(-5.3, -4.5, -1.0, -0.7, 3.7, 3.9, 4.2, 5.5, 6.8, 7.4, 9.3) LInormVar(x, k=1/0.15) # Pawitan Ex10-9 p289 LInormVar(x)
Likelihood interval of lambda assuming Poisson distribution.
LIpois(x, k, n = 1, conf.level = 0.95, eps = 1e-8)LIpois(x, k, n = 1, conf.level = 0.95, eps = 1e-8)
x |
raw data vector or a mean value. If the length of x is 1, x is considered as a mean. |
k |
1/k likelihood interval will be calculated. |
n |
number of observations. If the length of x is 1, x is considered as the mean. |
conf.level |
approximately corresponding confidence level. If k is specified, this is ignored. |
eps |
estimated values less than this eps are considered as 0. |
It calculates likelihood interval of mean(lambda) assuming Poisson distribution. The likelihood interval is asymmetric and there is no standard error in the output.
PE |
maximum likelihood estimate on the lambda |
LL |
lower limit of likelihood interval |
UL |
upper limit of likelihood interval |
n |
number of observations |
k |
1/k likelihood interval provided |
logk |
log(k) of k value |
maxLL without factorial |
maximum log likelihood without factorial part |
Kyun-Seop Bae [email protected]
LIpois(4, k=15) # Fisher LIpois(4, k=exp(2)) # Edwards LIpois(4, k=1/0.15) # Pawitan LIpois(4, k=8) # Rhode LIpois(4, n=4) # Bae LIpois(4) # Bae # poisson.test(4) LIpois(4, k=32) # 0.7454614 11.7893612 LIpois(2.1, n=60) # 1.750222 2.493533LIpois(4, k=15) # Fisher LIpois(4, k=exp(2)) # Edwards LIpois(4, k=1/0.15) # Pawitan LIpois(4, k=8) # Rhode LIpois(4, n=4) # Bae LIpois(4) # Bae # poisson.test(4) LIpois(4, k=32) # 0.7454614 11.7893612 LIpois(2.1, n=60) # 1.750222 2.493533
Likelihood interval of sd and variance assuming normal distribution. This is estimated likelihood interval, not profile likelihood interval.
LIvar(s1, n1, k, conf.level=0.95)LIvar(s1, n1, k, conf.level=0.95)
s1 |
standard deviation of the sample |
n1 |
sample size |
k |
1/k likelihood interval will be calculated |
conf.level |
approximately corresponding confidence level. If k is specified, this is ignored. |
It calculates likelihood interval of sd and variance assuming normal distribution in one group. The likelihood interval is asymmetric and there is no standard error in the output.
PE |
maximum likelihood estimate on the population variance |
LL |
lower limit of likelihood interval |
UL |
upper limit of likelihood interval |
Kyun-Seop Bae [email protected]
x = c(-5.3, -4.5, -1.0, -0.7, 3.7, 3.9, 4.2, 5.5, 6.8, 7.4, 9.3) LInormVar(x) LIvar(sd(x), length(x))x = c(-5.3, -4.5, -1.0, -0.7, 3.7, 3.9, 4.2, 5.5, 6.8, 7.4, 9.3) LInormVar(x) LIvar(sd(x), length(x))
Likelihood interval of the ratio of two variances from two groups assuming normal distribution. Likelihood interval usually gives a narrower interval when the likelihood function is asymmetric.
LIvRatio(x, y, k, conf.level=0.95)LIvRatio(x, y, k, conf.level=0.95)
x |
observations from the first group, the test group, used for the numerator |
y |
observations from the second group, the control group, used for the denominator |
k |
1/k likelihood interval will be provided |
conf.level |
approximate confidence level |
It calculates likelihood interval of the ratio of two variances from two groups. Likelihood interval usually gives a narrower interval when the likelihood function is asymmetric.
PE |
maximum likelihood estimate on the ratio |
LL |
lower limit of likelihood interval |
UL |
upper limit of likelihood interval |
logk |
log(k) value used for LI |
maxLL |
maximum log likelihood |
conf.level |
approximate confidence level |
Kyun-Seop Bae [email protected]
LIvRatio(mtcars$drat, mtcars$wt) var.test(mtcars$drat, mtcars$wt) LIvRatio(mtcars$qsec, mtcars$wt) var.test(mtcars$qsec, mtcars$wt) LIvtest(sd(mtcars$qsec), nrow(mtcars), sd(mtcars$wt), nrow(mtcars))LIvRatio(mtcars$drat, mtcars$wt) var.test(mtcars$drat, mtcars$wt) LIvRatio(mtcars$qsec, mtcars$wt) var.test(mtcars$qsec, mtcars$wt) LIvtest(sd(mtcars$qsec), nrow(mtcars), sd(mtcars$wt), nrow(mtcars))
Likelihood interval of variance and sd assuming normal distribution. This is estimated likelihood interval, not profile likelihood interval.
LIvtest(s1, n1, s2, n2, k, conf.level=0.95)LIvtest(s1, n1, s2, n2, k, conf.level=0.95)
s1 |
sample standard deviation of the first group |
n1 |
sample size of the first group |
s2 |
sample standard deviation of the second group |
n2 |
sample size of the second group |
k |
1/k likelihood interval will be calculated |
conf.level |
approximate confidence level. If k is specified, this is ignored. |
It calculates likelihood interval of variance and sd using sufficient statistics. There is no standard error in the output.
PE |
maximum likelihood estimate on the ratio |
LL |
lower limit of likelihood interval |
UL |
upper limit of likelihood interval |
logk |
log(k) value used for LI |
maxLL |
maximum log likelihood |
conf.level |
approximate confidence level |
Kyun-Seop Bae [email protected]
LIvtest(10.5, 3529, 8.9, 5190) LIvtest(3, 10, 2, 10) LIvtest(3, 10, 2, 10, k=15)LIvtest(10.5, 3529, 8.9, 5190) LIvtest(3, 10, 2, 10) LIvtest(3, 10, 2, 10, k=15)
Likelihood ratio test with given fitting results, sample size, number of parameters, log-likelihoods, and alpha
LRT(n, pFull, pReduced, logLikFull, logLikReduced, alpha=0.05, Wilks=FALSE)LRT(n, pFull, pReduced, logLikFull, logLikReduced, alpha=0.05, Wilks=FALSE)
n |
number of observations |
pFull |
number of parameters of full model |
pReduced |
number of parameters of reduced model |
logLikFull |
log likelihood of full model |
logLikReduced |
log likelihood of reduced model |
alpha |
alpha value for type I error, significance level |
Wilks |
if TRUE, Wilks theorem (chi-square distribution) will be used, otherwise F distribution will be used. |
It performs likelihood ratio test with given fitting results. The default test is using F distribution. For small n (i.e. less than 100), you need to use F distribution. If the residuals are normally distributed, the delta -2 log likelihood (the difference between -2LL, the objective function value of each model) follows exactly an F-distribution, independent of sample size. When the distribution of the residuals is not normal (no matter what the distribution of the residuals is), it approaches a chi-square distribution as sample size increases (Wilks' theorem). The extreme distribution of the F-distribution (when the degrees of freedom in the denominator go to infinity) is chi-square distribution. The p-value from the F-distribution is slightly larger than the p-value from the chi-square distribution, meaning the F-distribution is more conservative. The difference decreases as sample size increases.
n |
number of observations |
paraFull |
number of parameters of full model |
paraReduced |
number of parameters of reduced model |
deltaPara |
difference of parameter counts |
cutoff |
cutoff, threshold, critical value of log-likelihood for the test |
deltaLogLik |
difference of log likelihood, if negative 0 is used. |
Chisq or Fval |
statistics according to the used distribution Chi-square of F |
pval |
p-value of null hypothesis. i.e. the reduced model is better. |
Verdict |
the model preferred. |
Kyun-Seop Bae [email protected]
Ruppert D, Cressie N, Carroll RJ. A Transformation/Weighting Model For Estimating Michaelis-Menten Parameters. School of Operations Research and Industrial Engineering, College of Engineering, Cornell University. Technical Report No. 796. May 1988.
Scheffé H. The Analysis of Variance. Wiley. 1959.
Wilks SS. The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses. Annals Math. Statist. 1938;9:60-62
LRT(20, 4, 2, -58.085, -60.087) LRT(20, 4, 2, -58.085, -60.087, Wilks=TRUE) LRT(20, 4, 2, -57.315, -66.159) LRT(20, 4, 2, -57.315, -66.159, Wilks=TRUE) r1 = lm(mpg ~ disp + drat + wt, mtcars) r2 = lm(mpg ~ disp + drat, mtcars) anova(r2, r1) LRT(nrow(mtcars), r1$rank, r2$rank, logLik(r1), logLik(r2))LRT(20, 4, 2, -58.085, -60.087) LRT(20, 4, 2, -58.085, -60.087, Wilks=TRUE) LRT(20, 4, 2, -57.315, -66.159) LRT(20, 4, 2, -57.315, -66.159, Wilks=TRUE) r1 = lm(mpg ~ disp + drat + wt, mtcars) r2 = lm(mpg ~ disp + drat, mtcars) anova(r2, r1) LRT(nrow(mtcars), r1$rank, r2$rank, logLik(r1), logLik(r2))
With a given vector, it performs likelihood ratio test which model - one or two group - is better.
OneTwo(x, alpha=0.05)OneTwo(x, alpha=0.05)
x |
a vector of numbers |
alpha |
alpha value for type I error, significance level |
It performs likelihood ratio test using both F distribution and Chi-square distribution (by Wilks' theorem).
Estimate |
n, Mean, SD for each group assumption and prior probability of each group in two group model |
Delta |
delta number of parameters and log-likelihoods |
Statistic |
Statistics from both the F distribution and Chi-square distribtuion. Cutoff is in terms of log-likelihood not the statistic. |
Kyun-Seop Bae [email protected]
OneTwo(c(7, 5, 17, 13, 16, 5, 7, 3, 8, 10, 8, 14, 14, 11, 14, 17, 2, 12, 15, 19)) OneTwo(c(5, 3, 0, 6, 5, 2, 6, 6, 4, 4, 15, 13, 18, 18, 19, 14, 19, 13, 19, 18))OneTwo(c(7, 5, 17, 13, 16, 5, 7, 3, 8, 10, 8, 14, 14, 11, 14, 17, 2, 12, 15, 19)) OneTwo(c(5, 3, 0, 6, 5, 2, 6, 6, 4, 4, 15, 13, 18, 18, 19, 14, 19, 13, 19, 18))
Odds ratio and its likelihood interval between two groups without stratification
ORLI(y1, n1, y2, n2, conf.level=0.95, k, eps=1e-8)ORLI(y1, n1, y2, n2, conf.level=0.95, k, eps=1e-8)
y1 |
positive event count of test (the first) group |
n1 |
total count of the test (the first) group. Maximum allowable value is 1e8. |
y2 |
positive event count of control (the second) group |
n2 |
total count of control (the second) group. Maximum allowable value is 1e8. |
conf.level |
approximate confidence level to calculate k when k is missing. |
k |
1/k likelihood interval will be provided |
eps |
absolute value less than eps is regarded as negligible |
It calculates risk (proportion) difference and its likelihood interval between the two groups. The likelihood interval is asymmetric, and there is no standard error in the output. This does not support stratification.
There is no standard error.
odd1 |
odd from the first group, y1/(n1 - y1) |
odd2 |
odd from the second group, y2/(n2 - y2) |
OR |
odds ratio, odd1/odd2 |
lower |
lower likelihood limit of OR |
upper |
upper likelihood limit of OR |
Kyun-Seop Bae [email protected]
ORLI(7, 10, 3, 10) ORLI(3, 10, 7, 10)ORLI(7, 10, 3, 10) ORLI(3, 10, 7, 10)
Risk difference and its likelihood interval between two groups without stratification
RDLI(y1, n1, y2, n2, conf.level=0.95, k, eps=1e-8)RDLI(y1, n1, y2, n2, conf.level=0.95, k, eps=1e-8)
y1 |
positive event count of test (the first) group |
n1 |
total count of the test (the first) group. Maximum allowable value is 1e8. |
y2 |
positive event count of control (the second) group |
n2 |
total count of control (the second) group. Maximum allowable value is 1e8. |
conf.level |
approximate confidence level to calculate k when k is missing. |
k |
1/k likelihood interval will be provided |
eps |
absolute value less than eps is regarded as negligible |
It calculates risk (proportion) difference and its likelihood interval between the two groups. The likelihood interval is asymmetric, and there is no standard error in the output. This does not support stratification.
There is no standard error.
p1 |
proportion from the first group, y1/n1 |
p2 |
proportion from the second group, y2/n2 |
RD |
risk difference, p1 - p2 |
lower |
lower likelihood limit of RD |
upper |
upper likelihood limit of RD |
Kyun-Seop Bae [email protected]
RDLI(7, 10, 3, 10) RDLI(3, 10, 7, 10)RDLI(7, 10, 3, 10) RDLI(3, 10, 7, 10)
Relative risk and its likelihood interval between two groups without stratification
RRLI(y1, n1, y2, n2, conf.level=0.95, k, eps=1e-8)RRLI(y1, n1, y2, n2, conf.level=0.95, k, eps=1e-8)
y1 |
positive event count of test (the first) group |
n1 |
total count of the test (the first) group. Maximum allowable value is 1e8. |
y2 |
positive event count of control (the second) group |
n2 |
total count of control (the second) group. Maximum allowable value is 1e8. |
conf.level |
approximate confidence level to calculate k when k is missing. |
k |
1/k likelihood interval will be provided |
eps |
absolute value less than eps is regarded as negligible |
It calculates relative risk and its likelihood interval between the two groups. The likelihood interval is asymmetric, and there is no standard error in the output. This does not support stratification.
There is no standard error.
p1 |
proportion from the first group, y1/n1 |
p2 |
proportion from the second group, y2/n2 |
RR |
relative risk, p1/p2 |
lower |
lower likelihood limit of RR |
upper |
upper likelihood limit of RR |
Kyun-Seop Bae [email protected]
RRLI(7, 10, 3, 10) RRLI(3, 10, 7, 10)RRLI(7, 10, 3, 10) RRLI(3, 10, 7, 10)