| Title: | EM Algorithm for Lasso-Penalized Linear Mixed Models |
|---|---|
| Description: | Fits linear mixed models with a lasso penalty on the fixed-effects coefficients via an Expectation-Maximization (EM) algorithm. Random effects are treated as missing data at each E-step and the fixed-effects update is solved with a coordinate-descent lasso ('glmnet'), so the same penalized-likelihood machinery used for ordinary linear models extends naturally to clustered/longitudinal data. By default, EMLMLasso() fits the model over a data-driven grid of penalty values and selects the tuning parameter by Bayesian Information Criterion. This is a modification of the method of Oliveira, Schumacher, and Lachos (2026) <doi:10.1177/09622802251399913>, in which the tuning parameter is re-selected by cross-validation at every EM iteration. |
| Authors: | Fernanda Lang Schumacher [aut, cre], Daniela C.R. Oliveira [aut], Victor H. Lachos [aut], Marcos Santos de Oliveira [aut] |
| Maintainer: | Fernanda Lang Schumacher <[email protected]> |
| License: | GPL-3 |
| Version: | 0.1.0 |
| Built: | 2026-11-18 00:11:29 UTC |
| Source: | https://github.com/cran/EMLMLasso |
EMLMLasso fitExtract coefficients, fitted values or residuals from an EMLMLasso fit
## S3 method for class 'EMLMLasso' coef(object, refit = FALSE, ...) ## S3 method for class 'EMLMLasso' fitted(object, refit = FALSE, ...) ## S3 method for class 'EMLMLasso' residuals(object, refit = FALSE, ...)## S3 method for class 'EMLMLasso' coef(object, refit = FALSE, ...) ## S3 method for class 'EMLMLasso' fitted(object, refit = FALSE, ...) ## S3 method for class 'EMLMLasso' residuals(object, refit = FALSE, ...)
object |
an |
refit |
if |
... |
unused. |
For coef(), a named numeric vector of fixed-effect estimates,
one per column of the fixed-effects design matrix (including the
intercept when intercept = TRUE in EMLMLasso()); covariates shrunk
out by the lasso are exactly 0, or NA when refit = TRUE.
For fitted() and residuals(), a numeric vector of length equal to
the total number of observations, in the same order as the stored
response y (observations grouped by groupVar), with fitted values
combining the fixed effects (including implied_intercept when the data
were centred) and the predicted random effects.
Fits a linear mixed model with a lasso penalty on the fixed-effects
coefficients, using an Expectation-Maximization (EM) algorithm (at each
E-step, the random effects are treated as missing data and their
conditional means and covariances given the data are computed in closed
form; at each M-step,
the fixed effects are updated by a coordinate-descent lasso fit
(glmnet::glmnet()) on the random-effects-adjusted response, and the
variance components are updated in closed form). This is the package's main
function and the argument lambda decides whether it fits one model
or searches a grid and selects by BIC.
EMLMLasso( data, formFixed, groupVar, formRandom = ~1, lambda = NULL, intercept = FALSE, center = TRUE, n_lambdas = 30, lambda_min_ratio = 0.001, precision = 1e-04, MaxIter = 200, penalty_factor = NULL, standardize_x = TRUE, refit = FALSE, verbose = interactive() )EMLMLasso( data, formFixed, groupVar, formRandom = ~1, lambda = NULL, intercept = FALSE, center = TRUE, n_lambdas = 30, lambda_min_ratio = 0.001, precision = 1e-04, MaxIter = 200, penalty_factor = NULL, standardize_x = TRUE, refit = FALSE, verbose = interactive() )
data |
a data.frame containing all variables referenced by
|
formFixed |
two-sided formula for the fixed effects, e.g. |
groupVar |
character string naming the subject/cluster variable in |
formRandom |
one-sided formula for the random effects, e.g. |
lambda |
controls what gets fit:
|
intercept |
if |
center |
if |
n_lambdas |
number of |
lambda_min_ratio |
smallest lambda in the data-driven grid, as a
fraction of the largest (as in |
precision |
convergence tolerance on the Euclidean distance between
successive parameter vectors (beta, sigma2, D). Default |
MaxIter |
maximum number of EM iterations. Default |
penalty_factor |
optional vector of per-coefficient penalty
multipliers, in the same order as the columns of the fixed-effects
design matrix (see |
standardize_x |
if |
refit |
if |
verbose |
if |
Following Oliveira et al. (2026), the model is
, with
independent of , so the covariance for subject/cluster
is . lambda plays the
role of in the M-step of Algorithm 1 of Oliveira et al.
(2026) (their Equation 8) – the penalty passed directly to
glmnet::glmnet(). Following Equation 1 of the same paper, which
assumes a centred response and centred covariates, no intercept is fit by
default; instead, the response and covariates are centred before fitting
(see center).
This package implements the EMLMLasso algorithm of Oliveira, Schumacher
and Lachos (2026) with modifications. Most notably, the penalty
is selected by BIC over a grid of candidate values, with
held fixed within each EM fit, instead of being re-selected by
cross-validation at every EM iteration as in Algorithm 1 of the paper.
Convergence is also assessed on the change in parameter estimates rather
than on the relative change in the penalized log-likelihood. Results may
therefore differ from those reported in the paper.
Per-iteration cross-validation was dropped because it was slow (one
k-fold cross-validation per EM iteration) and, in simulations, produced a
lambda path that was not monotone in the number of active covariates.
See NEWS.md for further details.
An object of class "EMLMLasso", a list with elements:
named vector of estimated fixed-effects coefficients for the selected (or only) model.
the intercept implied by centring,
, or 0 if the data were not
centred (see center).
whether the response and covariates were centred.
estimated residual variance.
estimated random-effects covariance matrix.
matrix of estimated (empirical Bayes) random effects, one row per group.
the penalty value of the selected (or only) model.
the marginal (unpenalized) log-likelihood at each iteration, for the selected (or only) model.
information criteria for the selected (or only) model.
number of estimated parameters used in AIC and BIC: the
non-zero fixed effects, plus and the
unique elements of .
number of groups and total number of observations.
the penalty multipliers used.
number of EM iterations performed for the selected (or only) model.
whether the selected (or only) model's EM algorithm
reached precision within MaxIter iterations and its M-steps'
glmnet::glmnet() fits themselves converged (see
glmnet_converged). FALSE if MaxIter was reached first.
whether every M-step of the selected (or
only) model's glmnet::glmnet() calls converged.
the design matrices/response/group sizes used to fit the model.
the matched call and model specification.
TRUE if lambda triggered a grid search
(NULL or a vector), FALSE if a single lambda was fit directly.
the full grid of lambda values considered, or
NULL if search_performed is FALSE.
BIC and number of active (non-zero)
covariates at each grid value (NA at any lambda excluded because
its M-step's glmnet::glmnet() fit did not converge), or NULL if
search_performed is FALSE.
logical vector, one per grid value: whether
that lambda's fit converged and was used in the BIC comparison, or
NULL if search_performed is FALSE.
list of all fits considered, one per grid value
(including any excluded from BIC comparison, for inspection), or
NULL if search_performed is FALSE.
If refit = TRUE, this additionally carries refit_fit,
refit_coefficients, refit_formula and refit_singular for the
selected (or only) model – see the refit argument above.
Oliveira DCR, Schumacher FL, Lachos VH (2026). The use of the EM algorithm for regularization problems in high-dimensional linear mixed-effects models. Statistical Methods in Medical Research, 35(2), 240-254. doi:10.1177/09622802251399913
set.seed(1) p <- 5 dat <- simulate_lasso_lmm(n = 25, nj = 4, p = p, pstar = 2, seed = 1) fixedForm <- stats::as.formula(paste("y ~", paste0("x", seq_len(p), collapse = " + "))) # fit a single lambda directly fit <- EMLMLasso(dat, formFixed = fixedForm, groupVar = "id", formRandom = ~time, lambda = 0.1, MaxIter = 50, verbose = FALSE) fit coef(fit) # search a data-driven grid and select by BIC (the common case) search <- EMLMLasso(dat, formFixed = fixedForm, groupVar = "id", formRandom = ~time, n_lambdas = 8, MaxIter = 50, verbose = FALSE) search coef(search)set.seed(1) p <- 5 dat <- simulate_lasso_lmm(n = 25, nj = 4, p = p, pstar = 2, seed = 1) fixedForm <- stats::as.formula(paste("y ~", paste0("x", seq_len(p), collapse = " + "))) # fit a single lambda directly fit <- EMLMLasso(dat, formFixed = fixedForm, groupVar = "id", formRandom = ~time, lambda = 0.1, MaxIter = 50, verbose = FALSE) fit coef(fit) # search a data-driven grid and select by BIC (the common case) search <- EMLMLasso(dat, formFixed = fixedForm, groupVar = "id", formRandom = ~time, n_lambdas = 8, MaxIter = 50, verbose = FALSE) search coef(search)
Only applicable to an "EMLMLasso" object created with a BIC search
(lambda = NULL or a vector of candidate values, the default) – errors
informatively if x was fit at a single lambda instead, since there is
then no grid to plot.
## S3 method for class 'EMLMLasso' plot(x, ...)## S3 method for class 'EMLMLasso' plot(x, ...)
x |
an |
... |
further arguments passed to |
Invisibly returns x, the input "EMLMLasso" object, unchanged.
Called for its side effect of plotting BIC against the lambda grid, with
the selected lambda highlighted in red.
Gene expression and riboflavin production data on Bacillus subtilis, with a longitudinal (repeated-measures) structure recovered from the original microarray metadata, reproducing the real-data application in Oliveira, Schumacher and Lachos (2026), Section 5.
riboflavinV100riboflavinV100
A data frame with 71 rows (observations) and 103 columns.
strain (factor, 28 levels) is the B. subtilis strain/isolate a
given observation belongs to (the grouping/cluster variable), each
measured 2 to 4 times; time (numeric) is the time of measurement in
hours after induction (24, 30, 48 or 52); y (numeric) is
log-transformed riboflavin (vitamin B2) production rate, the response
variable. The remaining 100 columns are numeric log-expression levels,
one per gene (column names are the genes' probe identifiers, e.g.
YCIC_at) – the 100 genes of largest empirical variance across
these 71 observations (see Details).
The expression matrix and response are the standard riboflavin
dataset (Bühlmann, Kalisch and Meier, 2014), most commonly distributed
via the hdi package, which records 71 microarray samples and 4088
gene expression levels but exposes no grouping or time variable. The
original .CEL microarray file names (preserved as that object's row
names) encode the B. subtilis strain (e.g. "Fbat1070") and the
measurement time in hours (e.g. "PT24") each sample belongs to.
Parsing these names recovers exactly the longitudinal structure
described in Oliveira, Schumacher and Lachos (2026, Section 5): 28
strains, each measured 2-4 times, 71 observations in total.
Following prior work, genes are filtered to the 100 of largest
empirical variance across these 71 observations – the same
riboflavinV100 reduction used by the paper itself
(following Schelldorfer, Bühlmann and van de Geer, 2011), so that the
analysis setting matches the paper's (results differ from its Table 5
because of the change in selection; see EMLMLasso()) and
fitting is fast. See vignette("EMLMLasso-riboflavin") for a worked
example fitting EMLMLasso() to it.
Derived from the riboflavin dataset distributed with the hdi
package (https://github.com/cran/hdi), combined with strain/time
metadata parsed from that dataset's original microarray file names.
Oliveira DCR, Schumacher FL, Lachos VH (2026). "The use of the EM algorithm for regularization problems in high-dimensional linear mixed-effects models." Statistical Methods in Medical Research, 35(2), 240-254. doi:10.1177/09622802251399913
Bühlmann P, Kalisch M, Meier L (2014). "High-Dimensional Statistics with a View Toward Applications in Biology." Annual Review of Statistics and Its Application, 1, 255-278.
Schelldorfer J, Bühlmann P, van de Geer S (2011). "Estimation for
high-dimensional linear mixed-effects models using
-penalization." Scandinavian Journal of Statistics, 38(2),
197-214.
data(riboflavinV100) dim(riboflavinV100) table(table(riboflavinV100$strain)) # group sizes: 2 to 4 per strain # see vignette("EMLMLasso-riboflavin") for a full worked exampledata(riboflavinV100) dim(riboflavinV100) table(table(riboflavinV100$strain)) # group sizes: 2 to 4 per strain # see vignette("EMLMLasso-riboflavin") for a full worked example
Generates balanced longitudinal/clustered data from a linear mixed model
with p fixed-effect covariates (only the first pstar have a non-zero
coefficient) and a random intercept + random slope on time. This is the
data-generating mechanism used in the package examples, tests and
vignette (adapted from Scenario 1 of Oliveira et al. 2026, except that
covariates are drawn from N(0, 1) rather than N(6, 1) and then centred).
simulate_lasso_lmm( n = 50, nj = 5, p = 10, pstar = 4, beta_value = 1, sigmae = 1, D = NULL, seed = NULL )simulate_lasso_lmm( n = 50, nj = 5, p = 10, pstar = 4, beta_value = 1, sigmae = 1, D = NULL, seed = NULL )
n |
number of subjects/clusters. |
nj |
number of observations per subject, |
p |
total number of fixed-effect covariates. |
pstar |
number of covariates with a non-zero true coefficient
( |
beta_value |
the common value used for the non-zero true coefficients. |
sigmae |
true residual variance. |
D |
true 2x2 covariance matrix of the random intercept and random
slope on |
seed |
optional seed passed to |
A data.frame with columns id, time, y, and x1, ..., xp.
Oliveira DCR, Schumacher FL, Lachos VH (2026). The use of the EM algorithm for regularization problems in high-dimensional linear mixed-effects models. Statistical Methods in Medical Research, 35(2), 240-254. doi:10.1177/09622802251399913
dat <- simulate_lasso_lmm(n = 20, nj = 4, p = 6, pstar = 2, seed = 42) head(dat)dat <- simulate_lasso_lmm(n = 20, nj = 4, p = 6, pstar = 2, seed = 42) head(dat)