| Title: | Sparse Generalized Linear Model with L0 Approximation for Feature Selection |
|---|---|
| Description: | An efficient procedure for feature selection for generalized linear models with L0 penalty, including linear, logistic, Poisson, gamma, inverse Gaussian regression. Adaptive ridge algorithms are used to fit the models. |
| Authors: | Wenchuan Guo, Shujie Ma, Zhenqiu Liu |
| Maintainer: | Wenchuan Guo <[email protected]> |
| License: | GPL-2 |
| Version: | 0.1.6 |
| Built: | 2024-12-21 06:28:57 UTC |
| Source: | CRAN |
Print the coefficients from the model with the optimal lambda.
## S3 method for class 'cv.l0ara' coef(object, ...)## S3 method for class 'cv.l0ara' coef(object, ...)
object |
Fitted "cv.l0ara" object. |
... |
Not used argument. |
This function fit the model with the optimal lambda first and then print the coefficients. This function makes it easier to use the results to make a prediction or to see the fitted model.
The object returns the coefficients.
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
predict method and l0ara function.
Print the coefficients from the model.
## S3 method for class 'l0ara' coef(object, ...)## S3 method for class 'l0ara' coef(object, ...)
object |
Fitted "l0ara" object. |
... |
Not used argument. |
This function makes it easier to use the results to make a prediction or to see the fitted model.
The object returns the coefficients.
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
predict method and l0ara function.
Does k-fold cross-validation for l0ara, produces a plot, and returns the optimal lambda
cv.l0ara(x, y, family, lam, measure, nfolds, maxit, eps, seed)cv.l0ara(x, y, family, lam, measure, nfolds, maxit, eps, seed)
x |
Input matrix as in |
y |
Response variable as in |
family |
Response type as in |
lam |
A user supplied |
measure |
Loss function used for corss validation. |
nfolds |
Number of folds. Default value is 10. Smallest value is 3. |
maxit |
Maximum number of passes over the data for |
eps |
Convergence threshold. Default value is |
seed |
Seed of random number generator. |
This function calls l0ara nfolds times, each time leaving out 1/nfolds of the data. The cross-validation error is based on etiher mean square error (mse) or mean absolute error (mae).
An object with S3 class "cv.l0ara" containing:
cv.error |
The mean cross validated error for given lambda sequence |
cv.std |
The estimates of standard error of |
lam.min |
The lambda gives min cv.error |
lambda |
The lambda used |
measure |
Type of measure |
family |
Model used |
x |
Design matrix |
y |
Response variable |
name |
Full name of the measure |
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
l0ara, coef.cv.l0ara, plot.cv.l0ara methods.
#' # Linear regression # Generate design matrix and response variable n <- 100 p <- 40 x <- matrix(rnorm(n*p), n, p) beta <- c(1,0,2,3,rep(0,p-4)) noise <- rnorm(n) y <- x%*%beta+noise lam <- c(0.1, 0.3, 0.5) fit <- cv.l0ara(x, y, family="gaussian", lam, measure = "mse")#' # Linear regression # Generate design matrix and response variable n <- 100 p <- 40 x <- matrix(rnorm(n*p), n, p) beta <- c(1,0,2,3,rep(0,p-4)) noise <- rnorm(n) y <- x%*%beta+noise lam <- c(0.1, 0.3, 0.5) fit <- cv.l0ara(x, y, family="gaussian", lam, measure = "mse")
An adaptive ridge algorithm for feature selection with L0 penalty.
l0ara(x, y, family, lam, standardize, maxit, eps)l0ara(x, y, family, lam, standardize, maxit, eps)
x |
Input matrix, of dimension nobs x nvars; each row is an observation vector. |
y |
Response variable. Quantitative for |
family |
Response type(see above). |
lam |
A user supplied |
standardize |
Logical flag for data normalization. If |
maxit |
Maximum number of passes over the data for |
eps |
Convergence threshold. Default value is |
The sequence of models indexed by the parameter lambda is fit using adptive ridge algorithm. The objective function for generalized linear models (including family above) is defined to be
is the number of non-zero elements in . To select the "best" model with AIC or BIC criterion, let lambda to be 2 or log(n). This adaptive ridge algorithm is developed to approximate L0 penalized generalized linear models with sequential optimization and is efficient for high-dimensional data.
An object with S3 class "l0ara" containing:
beta |
A vector of coefficients |
df |
Number of nonzero coefficients |
iter |
Number of iterations |
lambda |
The lambda used |
x |
Design matrix |
y |
Response variable |
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
cv.l0ara, predict.l0ara, coef.l0ara, plot.l0ara methods.
# Linear regression # Generate design matrix and response variable n <- 100 p <- 40 x <- matrix(rnorm(n*p), n, p) beta <- c(1,0,2,3,rep(0,p-4)) noise <- rnorm(n) y <- x%*%beta+noise # fit sparse linear regression using BIC res.gaussian <- l0ara(x, y, family="gaussian", log(n)) # predict for new observations print(res.gaussian) predict(res.gaussian, newx=matrix(rnorm(3,p),3,p)) coef(res.gaussian) # Logistic regression # Generate design matrix and response variable n <- 100 p <- 40 x <- matrix(rnorm(n*p), n, p) beta <- c(1,0,2,3,rep(0,p-4)) prob <- exp(x%*%beta)/(1+exp(x%*%beta)) y <- rbinom(n, rep(1,n), prob) # fit sparse logistic regression res.logit <- l0ara(x, y, family="logit", 0.7) # predict for new observations print(res.logit) predict(res.logit, newx=matrix(rnorm(3,p),3,p)) coef(res.logit) # Poisson regression # Generate design matrix and response variable n <- 100 p <- 40 x <- matrix(rnorm(n*p), n, p) beta <- c(1,0,0.5,0.3,rep(0,p-4)) mu <- exp(x%*%beta) y <- rpois(n, mu) # fit sparse Poisson regression using AIC res.pois <- l0ara(x, y, family="poisson", 2) # predict for new observations print(res.pois) predict(res.pois, newx=matrix(rnorm(3,p),3,p)) coef(res.pois)# Linear regression # Generate design matrix and response variable n <- 100 p <- 40 x <- matrix(rnorm(n*p), n, p) beta <- c(1,0,2,3,rep(0,p-4)) noise <- rnorm(n) y <- x%*%beta+noise # fit sparse linear regression using BIC res.gaussian <- l0ara(x, y, family="gaussian", log(n)) # predict for new observations print(res.gaussian) predict(res.gaussian, newx=matrix(rnorm(3,p),3,p)) coef(res.gaussian) # Logistic regression # Generate design matrix and response variable n <- 100 p <- 40 x <- matrix(rnorm(n*p), n, p) beta <- c(1,0,2,3,rep(0,p-4)) prob <- exp(x%*%beta)/(1+exp(x%*%beta)) y <- rbinom(n, rep(1,n), prob) # fit sparse logistic regression res.logit <- l0ara(x, y, family="logit", 0.7) # predict for new observations print(res.logit) predict(res.logit, newx=matrix(rnorm(3,p),3,p)) coef(res.logit) # Poisson regression # Generate design matrix and response variable n <- 100 p <- 40 x <- matrix(rnorm(n*p), n, p) beta <- c(1,0,0.5,0.3,rep(0,p-4)) mu <- exp(x%*%beta) y <- rpois(n, mu) # fit sparse Poisson regression using AIC res.pois <- l0ara(x, y, family="poisson", 2) # predict for new observations print(res.pois) predict(res.pois, newx=matrix(rnorm(3,p),3,p)) coef(res.pois)
Produces curves from a fitted "cv.l0ara" object.
## S3 method for class 'cv.l0ara' plot(x, col = 3, ...)## S3 method for class 'cv.l0ara' plot(x, col = 3, ...)
x |
Fitted "cv.l0ara" object. |
col |
color of the dots. |
... |
Not used argument. |
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
predict, coef methods, cv.l0ara and l0ara function.
Two plots are availiable: a plot of fitted value against linear predictor; roc(auc) curve for family="logit".
## S3 method for class 'l0ara' plot(x, auc = FALSE, split = FALSE, col = 4, ...)## S3 method for class 'l0ara' plot(x, auc = FALSE, split = FALSE, col = 4, ...)
x |
Fitted "l0ara" object. |
auc |
logical; if |
split |
logical; if if |
col |
color of the dots. |
... |
Not used argument. |
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
predict, coef methods and l0ara function.
Make predictions from the model.
## S3 method for class 'l0ara' predict(object, newx, type = c("link", "response", "coefficients", "class"), ...)## S3 method for class 'l0ara' predict(object, newx, type = c("link", "response", "coefficients", "class"), ...)
object |
Fitted "l0ara" object. |
newx |
Matrix of new values for x at which predictions are to be made. Must be a matrix. |
type |
Type of prediction required. "link" gives the linear predictors(for "gaussian" models it gives the fitted values). "response" gives the fitted probabilities for "logit" and fitted mean for "poisson". "coefficients" gives the coefficients which is same as "coef" function. "class" (applies only to "logit") produces the class label corresponding to the maximum probability. |
... |
Not used argument. |
This function makes it easier to use the results to make a prediction or to see the fitted model.
The object returned depends the functions.
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
coef method and l0ara function.
Print the general information of the cross validated fit.
## S3 method for class 'cv.l0ara' print(x, ...)## S3 method for class 'cv.l0ara' print(x, ...)
x |
Fitted "cv.l0ara" object. |
... |
Not used argument. |
This function makes it easier to see the cross-validation results.
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
predict, coef methods and l0ara function.
Print the general information of the fit.
## S3 method for class 'l0ara' print(x, ...)## S3 method for class 'l0ara' print(x, ...)
x |
Fitted "l0ara" object. |
... |
Not used argument. |
This function makes it easier to see the fitted model.
Wenchuan Guo <[email protected]>, Shujie Ma <[email protected]>, Zhenqiu Liu <[email protected]>
predict, coef methods and l0ara function.