Title: | Runs Monte Carlo Markov Chain - With Either 'JAGS', 'nimble' or 'greta' - While Adjusting Burn-in and Thinning Parameters |
---|---|
Description: | The function runMCMC_btadjust() returns a mcmc.list object which is the output of a Markov Chain Monte Carlo obtained - from either 'JAGS', 'nimble' or 'greta' - after adjusting burn-in and thinning parameters to meet pre-specified criteria in terms of convergence & effective sample size. Used with 'nimble', runMCMC_btadjust() allows extra calculations (e.g. information criteria for model comparison and goodness-of-fit p-values for model diagnosis). |
Authors: | Frédéric Gosselin [cre, aut] (<https://orcid.org/0000-0003-3737-106X>, INRAE), Institut national de recherche pour l'agriculture, l'alimentation et l'environnement [cph] (INRAE) |
Maintainer: | Frédéric Gosselin <[email protected]> |
License: | CECILL-2.1 |
Version: | 1.1.2 |
Built: | 2024-11-27 06:53:29 UTC |
Source: | CRAN |
finds the couples of parameters of a MCMC.list object that have at least a minCorr level of (absolute) correlation
findMCMC_strong_corrs( mcmcList, corrMethod = "pearson", minCorr = 0.3, namesToRemove = NULL )
findMCMC_strong_corrs( mcmcList, corrMethod = "pearson", minCorr = 0.3, namesToRemove = NULL )
mcmcList |
R object of type mcmc.list that contains the MCMC output |
corrMethod |
character: designates the kind of correlation calculated among "pearson" (the default, for linear relationships), "spearman" (for monotone relationships) or "hoeffd" (for general associations - i.e. dependencies - between parameters) |
minCorr |
double, between 0 and 1: minimum level of (absolute) correlation to report. |
namesToRemove |
R object (can be a vector, matrix, array, list...) all components of which must be of character type: will remove parameters whose names partially match one of tghese components. |
In case corrMethod equals "hoeffd", the hoeffd function in the Hmisc package will be used. This can be very slow. Therefore a warning message is printed in this case.
## Not run: #generating data set.seed(1) y1000<-rnorm(n=1000,mean=600,sd=30) ModelData <-list(mass = y1000,nobs = length(y1000)) #writing the Jags code as a character chain in R modeltotransfer<-"model { # Priors population.mean ~ dunif(0,5000) population.sd ~ dunif(0,100) # Precision = 1/variance: Normal distribution parameterized by precision in Jags population.variance <- population.sd * population.sd precision <- 1 / population.variance # Likelihood for(i in 1:nobs){ mass[i] ~ dnorm(population.mean, precision) } }" #specifying the initial values ModelInits <- function() {list (population.mean = rnorm(1,600,90), population.sd = runif(1, 1, 30))} params <- c("population.mean", "population.sd", "population.variance") K<-3 set.seed(1) Inits<-lapply(1:K,function(x){ModelInits()}) # running runMCMC_btadjust with MCMC_language="Jags": set.seed(1) out.mcmc.Coda<-runMCMC_btadjust(MCMC_language="Jags", code=modeltotransfer, data=ModelData, Nchains=K, params=params, inits=Inits, niter.min=1000, niter.max=300000, nburnin.min=100, nburnin.max=200000, thin.min=1, thin.max=1000, neff.min=1000, conv.max=1.05, control=list(print.diagnostics=TRUE, neff.method="Coda")) findMCMC_strong_corrs(out.mcmc.Coda) ## End(Not run)
## Not run: #generating data set.seed(1) y1000<-rnorm(n=1000,mean=600,sd=30) ModelData <-list(mass = y1000,nobs = length(y1000)) #writing the Jags code as a character chain in R modeltotransfer<-"model { # Priors population.mean ~ dunif(0,5000) population.sd ~ dunif(0,100) # Precision = 1/variance: Normal distribution parameterized by precision in Jags population.variance <- population.sd * population.sd precision <- 1 / population.variance # Likelihood for(i in 1:nobs){ mass[i] ~ dnorm(population.mean, precision) } }" #specifying the initial values ModelInits <- function() {list (population.mean = rnorm(1,600,90), population.sd = runif(1, 1, 30))} params <- c("population.mean", "population.sd", "population.variance") K<-3 set.seed(1) Inits<-lapply(1:K,function(x){ModelInits()}) # running runMCMC_btadjust with MCMC_language="Jags": set.seed(1) out.mcmc.Coda<-runMCMC_btadjust(MCMC_language="Jags", code=modeltotransfer, data=ModelData, Nchains=K, params=params, inits=Inits, niter.min=1000, niter.max=300000, nburnin.min=100, nburnin.max=200000, thin.min=1, thin.max=1000, neff.min=1000, conv.max=1.05, control=list(print.diagnostics=TRUE, neff.method="Coda")) findMCMC_strong_corrs(out.mcmc.Coda) ## End(Not run)
returns a mcmc.list object which is the output of a Markov Chain Monte Carlo obtained after adjusting burn-in & thinning parameters to meet pre-specified criteria in terms of convergence & effective sample size - i.e. sample size adjusted for autocorrelation - of the MCMC output
runMCMC_btadjust( code = NULL, data = NULL, constants = NULL, model = NULL, MCMC_language = "Nimble", Nchains, inits = NULL, params = NULL, params.conv = NULL, params.save = NULL, niter.min = 100, niter.max = Inf, nburnin.min = 10, nburnin.max = Inf, thin.min = 1, thin.max = Inf, neff.min = NULL, neff.med = NULL, neff.mean = NULL, conv.max = NULL, conv.med = NULL, conv.mean = NULL, control = list(time.max = NULL, check.convergence = TRUE, check.convergence.firstrun = NULL, recheck.convergence = TRUE, convtype = NULL, convtype.Gelman = 2, convtype.Geweke = c(0.1, 0.5), convtype.alpha = 0.05, ip.nc = 0, neff.method = "Stan", Ncycles.target = 2, props.conv = c(0.25, 0.5, 0.75), min.Nvalues = NULL, min.thinmult = 1.1, force.niter.max = FALSE, force.time.max = FALSE, time.max.turns.off.niter.max = FALSE, safemultiplier.Nvals = 1.2, max.prop.decr.neff = 0.1, round.thinmult = TRUE, thinmult.in.resetMV.temporary = TRUE, check.thinmult = 2, decrease.thinmult.multiplier = 0.8, decrease.thinmult.threshold = 20, only.final.adapt.thin = FALSE, identifier.to.print = "", print.diagnostics = FALSE, conv.thorough.check = FALSE, print.thinmult = TRUE, innerprint = FALSE, seed = NULL, remove.fixedchains = TRUE, check.installation = TRUE, save.data = FALSE, conveff.final.allparams = TRUE), control.MCMC = list(confModel.expression.toadd = NULL, sampler = expression(hmc()), warmup = 1000, n.adapt = -1, RNG.names = c("base::Wichmann-Hill", "base::Marsaglia-Multicarry", "base::Super-Duper", "base::Mersenne-Twister"), n_cores = NULL, showCompilerOutput = FALSE, buildDerivs = FALSE, resetMV = FALSE, parallelize = FALSE, parallelizeInitExpr = expression(if (MCMC_language == "Nimble") { library("nimble") if (control.MCMC$APT) { library("nimbleAPT") } } else { NULL }), useConjugacy = FALSE, WAIC = FALSE, WAIC.Nsamples = 2000, WAIC.control = list(online = TRUE, dataGroups = NULL, marginalizeNodes = NULL, niterMarginal = 1000, convergenceSet = c(0.25, 0.5, 0.75), thin = TRUE, nburnin_extra = 0), APT = FALSE, APT.NTemps = 7, APT.initTemps = NULL, APT.tuneTemps = c(10, 0.7), APT.thinPrintTemps = expression(niter/5), includeAllStochNodes = FALSE, monitorAllStochNodes = FALSE, saveAllStochNodes = FALSE, includeParentNodes = FALSE, monitorParentNodes = FALSE, saveParentNodes = FALSE, extraCalculations = NULL) )
runMCMC_btadjust( code = NULL, data = NULL, constants = NULL, model = NULL, MCMC_language = "Nimble", Nchains, inits = NULL, params = NULL, params.conv = NULL, params.save = NULL, niter.min = 100, niter.max = Inf, nburnin.min = 10, nburnin.max = Inf, thin.min = 1, thin.max = Inf, neff.min = NULL, neff.med = NULL, neff.mean = NULL, conv.max = NULL, conv.med = NULL, conv.mean = NULL, control = list(time.max = NULL, check.convergence = TRUE, check.convergence.firstrun = NULL, recheck.convergence = TRUE, convtype = NULL, convtype.Gelman = 2, convtype.Geweke = c(0.1, 0.5), convtype.alpha = 0.05, ip.nc = 0, neff.method = "Stan", Ncycles.target = 2, props.conv = c(0.25, 0.5, 0.75), min.Nvalues = NULL, min.thinmult = 1.1, force.niter.max = FALSE, force.time.max = FALSE, time.max.turns.off.niter.max = FALSE, safemultiplier.Nvals = 1.2, max.prop.decr.neff = 0.1, round.thinmult = TRUE, thinmult.in.resetMV.temporary = TRUE, check.thinmult = 2, decrease.thinmult.multiplier = 0.8, decrease.thinmult.threshold = 20, only.final.adapt.thin = FALSE, identifier.to.print = "", print.diagnostics = FALSE, conv.thorough.check = FALSE, print.thinmult = TRUE, innerprint = FALSE, seed = NULL, remove.fixedchains = TRUE, check.installation = TRUE, save.data = FALSE, conveff.final.allparams = TRUE), control.MCMC = list(confModel.expression.toadd = NULL, sampler = expression(hmc()), warmup = 1000, n.adapt = -1, RNG.names = c("base::Wichmann-Hill", "base::Marsaglia-Multicarry", "base::Super-Duper", "base::Mersenne-Twister"), n_cores = NULL, showCompilerOutput = FALSE, buildDerivs = FALSE, resetMV = FALSE, parallelize = FALSE, parallelizeInitExpr = expression(if (MCMC_language == "Nimble") { library("nimble") if (control.MCMC$APT) { library("nimbleAPT") } } else { NULL }), useConjugacy = FALSE, WAIC = FALSE, WAIC.Nsamples = 2000, WAIC.control = list(online = TRUE, dataGroups = NULL, marginalizeNodes = NULL, niterMarginal = 1000, convergenceSet = c(0.25, 0.5, 0.75), thin = TRUE, nburnin_extra = 0), APT = FALSE, APT.NTemps = 7, APT.initTemps = NULL, APT.tuneTemps = c(10, 0.7), APT.thinPrintTemps = expression(niter/5), includeAllStochNodes = FALSE, monitorAllStochNodes = FALSE, saveAllStochNodes = FALSE, includeParentNodes = FALSE, monitorParentNodes = FALSE, saveParentNodes = FALSE, extraCalculations = NULL) )
code |
R object: code for the model that will be used to build the MCMC when |
data |
R list: a list that will contain the data when |
constants |
R list: a list that will contain the rest of the data (in addition to data) when |
model |
R object: should be the result of the |
MCMC_language |
character value: designates the |
Nchains |
integer value : the number of Markov chains to run in the MCMC. |
inits |
either NULL, a function or an R list, with |
params |
character vector: contains the names of the parameters to save at the end of the MCMC and to monitor for convergence and effective sample size;
inactive for convergence/effective sample size if |
params.conv |
character vector: contains the names of the parameters to monitor for convergence and effective sample size. |
params.save |
character vector: contains the names of the parameters to be saved at the end of the MCMC. |
niter.min |
integer value: the minimum number of iterations in each chain of the MCMC. |
niter.max |
integer value: the maximum number of iterations in each chain of the MCMC. Will stop the MCMC once the number of iterations will reach this limit. |
nburnin.min |
integer value: the minimum number of burn-in (=transitory) iterations in each chain of the MCMC. |
nburnin.max |
integer value: the maximum number of burn-in (=transitory) iterations in each chain of the MCMC. Will stay at this burn-in value once this limit is reached. |
thin.min |
integer value: the minimum value of the thin parameter of the MCMC. |
thin.max |
integer value: the maximum value of the thin parameter of the MCMC. Will stay at this thin value once this limit is reached. |
neff.min |
positive real number: minimum effective sample size - over parameters used to diagnose convergence & effective sample size- , as calculated with |
neff.med |
positive real number: median effective sample size - over parameters used to diagnose convergence & effective sample size- , as calculated with |
neff.mean |
positive real number: mean effective sample size - over parameters used to diagnose convergence & effective sample size-, as calculated with |
conv.max |
positive real number: maximum - over parameters used to diagnose convergence & effective sample size - convergence diagnostic, as calculated with |
conv.med |
positive real number: median - over parameters used to diagnose convergence & effective sample size - convergence diagnostic, as calculated with |
conv.mean |
positive real number: mean - over parameters used to diagnose convergence & effective sample size - convergence diagnostic, as calculated with |
control |
list of
|
control.MCMC |
list of MCMC control parameters: with the following components - that depend on
|
Recap:
If MCMC_language=="Nimble"
, the code, data and constants arguments should be specified according to the requirements of nimble
package.
If MCMC_language=="Jags"
, the code and data arguments need to be specified as required by rjags
package.
If MCMC_language=="Greta"
, the model argument must be specified and should be the result of the model
command in greta
package.
Details on check.convergence
:
If FALSE, no check of convergence at all, after nburnin.min
(& recheck.convergence
is put to FALSE & check.convergence.firstrun
is dominated by check.convergence
).
If TRUE, the convergence behavior is governed by check.convergence.firstrun
& recheck.convergence
.
Example for confModel.expression.toadd
component of control.MCMC
:confModel.expression.toadd<-expression({ConfModel[[i]]$removeSamplers(c("alpha","dzetad","beta","exper_bias[2]","exper_bias[3]","exper_precision[2]","exper_precision[3]"))
ConfModel[[i]]$addSampler(target = c("alpha","dzetad","beta"),type = "RW_block")
ConfModel[[i]]$addSampler(target = c("exper_bias[2]","exper_bias[3]"),type = "RW_block")
ConfModel[[i]]$addSampler(target = c("exper_precision[2]","exper_precision[3]"),type = "RW_block")
})
Remark for params, params.conv, params.save
:
in cases of parameters that are vectors, matrices... the params
vector can contain only the name of the vector or matrix... in which case all its components will be used. It can also contain the names of individual components.
a mcmc.list
object with attributes with the following components:
call.params
: a list containing most of the important arguments of the runMCMC_btadjust
call as well as either a summary of dimensions/lengths and mean of components of data
and constants
arguments or their entire values.
final.params
: a list with the parameters of the MCMC at the end of fitting:
converged
: logical: TRUE if model has converged when stopping the MCMC, FALSE otherwise
neffs.reached
: logical: TRUE if model has converged and reached the objectives in terms of effective sample size, FALSE otherwise
final.Nchains
: number of Markov chains finally retained - since some chains could be excluded if invariable.
removed.chains
: identity of the Markov chains finally removed - those that would have been invariable during the first cycle.
burnin
: number of iterations of the transient (burn-in) period
thin
: number of iterations used for thinning the final output
niter.tot
: total number of iterations (of each MCMC chain)
Nvalues
: number of saved values (over all the MCMC chains)
neff.min
: minimum number of effective values in the final MCMC (over params.conv)
neff.median
: median number of effective values (over params.conv)
WAIC
: results of the calculus of online WAIC (only if control.MCMC$WAIC and MCMC_language=="Nimble"
). One component per component of control.MCMC$control.WAIC. Each component has a WAIC component and then a WAICDetails component.
extraResults
: results of the implementation of control.MCMC$extraCalculations. Can be any kind of R object.
Temps
: results of the series of Temperatures met in APT (only if control.MCMC$APT and MCMC_language=="Nimble"
). One line for each end of $run (=MCMC run).
duration
: total duration (elapsed time) of the fit (in seconds)
duration.MCMC.preparation
: duration (elapsed time) of MCMC preparation (in seconds)
duration.MCMC.transient
: duration (elapsed time) of the MCMC transient (burn-in) phase (in seconds)
duration.MCMC.asymptotic
: duration (elapsed time) of the MCMC asymptotic phase (in seconds)
duration.MCMC.after
: duration (elapsed time) of the MCMC phase after the asymptotic phase of sampling (e.g. to calculate WAIC) (in seconds)
duration.btadjust
: duration (elapsed time) outside MCMC preparation & fitting (in seconds)
CPUduration
: total CPU duration (user+system, self+child if not NA - otherwise only self) of the fit (in seconds)
CPUduration.MCMC.preparation
: CPU duration (user+system, self+child if not NA - otherwise only self) of MCMC preparation (in seconds)
CPUduration.MCMC.transient
: CPU duration (user+system, self+child if not NA - otherwise only self) of the MCMC transient (burn-in) phase (in seconds)
CPUduration.MCMC.asymptotic
: CPU duration (user+system, self+child if not NA - otherwise only self) of the MCMC asymptotic phase (in seconds)
CPUduration.MCMC.after
: CPU duration (user+system, self+child if not NA - otherwise only self) of the MCMC phase after the asymptotic phase of sampling (e.g. to calculate WAIC) (in seconds)
CPUduration.btadjust
: CPU duration (user+system, self+child if not NA - otherwise only self) outside MCMC preparation & fitting (in seconds)
childCPUduration
: total child CPU duration (user+system) of the fit (in seconds)
childCPUduration.MCMC.preparation
: child CPU duration (user+system) of MCMC preparation (in seconds)
childCPUduration.MCMC.transient
: child CPU duration (user+system) of the MCMC transient (burn-in) phase (in seconds)
childCPUduration.MCMC.asymptotic
: child CPU duration (user+system) of the MCMC asymptotic phase (in seconds)
childCPUduration.MCMC.after
: child CPU duration (user+system) of the MCMC phase after the asymptotic phase of sampling (e.g. to calculate WAIC) (in seconds)
childCPUduration.btadjust
: child CPU duration (user+system) outside MCMC preparation & fitting (in seconds)
time
: time (from Sys.time) at the end of model fitting
final.diags
: a list with final diagnostics of the fit:
params
: parameters of the MCMC (burn-in, thin, niter...)
conv_synth
: synthetic output of convergence diagnostics
neff_synth
: synthetic output for calculations of effective sample sizes
conv
: raw convergence values for all the parameters being diagnosed if control$conveff.final.allparams is FALSE and all the parameters otherwise
neff
: raw effective sample size values for all the parameters being diagnosed if control$conveff.final.allparams is FALSE and all the parameters otherwise
sessionInfo
: a list containing the result of the call to sessionInfo() function at the end of runMCMC_btadjust function; contains info on the platform, versions of packages, R version...;
warnings
: a list of the warning messages issued during fitting; unsure it still works with this version
error
: a list with the error messages issued during fitting; unsure it still works with this version
# for examples with Nimble or Greta, see the Presentation Vignette. ## Not run: #generating data set.seed(1) y1000<-rnorm(n=1000,mean=600,sd=30) ModelData <-list(mass = y1000,nobs = length(y1000)) #writing the Jags code as a character chain in R modeltotransfer<-"model { # Priors population.mean ~ dunif(0,5000) population.sd ~ dunif(0,100) # Precision = 1/variance: Normal distribution parameterized by precision in Jags population.variance <- population.sd * population.sd precision <- 1 / population.variance # Likelihood for(i in 1:nobs){ mass[i] ~ dnorm(population.mean, precision) } }" #specifying the initial values ModelInits <- function() {list (population.mean = rnorm(1,600,90), population.sd = runif(1, 1, 30))} params <- c("population.mean", "population.sd", "population.variance") K<-3 set.seed(1) Inits<-lapply(1:K,function(x){ModelInits()}) # running runMCMC_btadjust with MCMC_language="Jags": set.seed(1) out.mcmc.Coda<-runMCMC_btadjust(MCMC_language="Jags", code=modeltotransfer, data=ModelData, Nchains=K, params=params, inits=Inits, niter.min=1000, niter.max=300000, nburnin.min=100, nburnin.max=200000, thin.min=1, thin.max=1000, neff.min=1000, conv.max=1.05, control=list(print.diagnostics=TRUE, neff.method="Coda")) summary(out.mcmc.Coda) ## End(Not run)
# for examples with Nimble or Greta, see the Presentation Vignette. ## Not run: #generating data set.seed(1) y1000<-rnorm(n=1000,mean=600,sd=30) ModelData <-list(mass = y1000,nobs = length(y1000)) #writing the Jags code as a character chain in R modeltotransfer<-"model { # Priors population.mean ~ dunif(0,5000) population.sd ~ dunif(0,100) # Precision = 1/variance: Normal distribution parameterized by precision in Jags population.variance <- population.sd * population.sd precision <- 1 / population.variance # Likelihood for(i in 1:nobs){ mass[i] ~ dnorm(population.mean, precision) } }" #specifying the initial values ModelInits <- function() {list (population.mean = rnorm(1,600,90), population.sd = runif(1, 1, 30))} params <- c("population.mean", "population.sd", "population.variance") K<-3 set.seed(1) Inits<-lapply(1:K,function(x){ModelInits()}) # running runMCMC_btadjust with MCMC_language="Jags": set.seed(1) out.mcmc.Coda<-runMCMC_btadjust(MCMC_language="Jags", code=modeltotransfer, data=ModelData, Nchains=K, params=params, inits=Inits, niter.min=1000, niter.max=300000, nburnin.min=100, nburnin.max=200000, thin.min=1, thin.max=1000, neff.min=1000, conv.max=1.05, control=list(print.diagnostics=TRUE, neff.method="Coda")) summary(out.mcmc.Coda) ## End(Not run)