Package 'fastliu'

Extract coefficient estimates from a liureg object

Description

Prints coefficient estimates from a fitted liureg object.

Usage

## S3 method for class 'liureg'
coef(object, ...)

Arguments

object	A liureg object.
...	Not used in this implementation.

Value

The returned object is a data.frame containing the coefficients path.

Author(s)

Murat Genç

See Also

liureg(), predict(), summary(), pressliu(), residuals()

Examples

data("Hitters")
Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
coef(liu.mod)

---

Covariance matrix based on a fitted liureg object.

Description

For a scalar or vector tuning parameter lambda, the covliureg computes the covariance matrix for the estimates of a Liu regression model.

Usage

covliu(obj)

Arguments

obj	A liureg object.

Value

The returned object is a list of the matrix of estimated covariances.

Author(s)

Murat Genç and Ömer Özbilen

See Also

liureg(), coef(), predict(), summary(), pressliu(), residuals()

Examples

data("Hitters")
Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
# List of covariance matrices for 101 lambda values
cov.mat <- covliu(liu.mod)
print(cov.mat$lam1)

---

Diagonal Elements of the Hat Matrix

Description

For each value of the regularization parameter lambda, diagHliu returns the diagonal elements of the hat matrix. Unlike the hatliu function, only the diagonal elements of the hat matrix are calculated, thus the computation of diagonal elements is faster than hatliu.

Usage

diagHliu(obj)

Arguments

obj	A liureg object

Value

The returned object is a matrix whose columns are the diagonal elements of the hat matrix for each value of the lambda regularization parameter.

Author(s)

Murat Genç

See Also

liureg(), summary(), pressliu(), residuals()

Examples

data("Hitters")
Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
diagHliu(liu.mod)

---

Hat matrix of Liu Regression

Description

For each value of the regularization parameter lambda, hatliu returns the hat matrix of Liu regression. The hat matrix for Liu regression is computed using the formula $\mathbf{H}=\mathbf{X}\left(\mathbf{X}^{T}\mathbf{X}+\mathbf{I}_{p}\right)^{-1} \left(\mathbf{X}^{T}\mathbf{X}+\lambda\mathbf{I}_{p}\right)\left(\mathbf{X}^{T} \mathbf{X}\right)^{-1}\mathbf{X}^{T}.$

Usage

hatliu(obj)

Arguments

obj	A liureg object.

Value

The returned object is a list of matrices whose elements are the hat matrices for the values of the lambda regularization parameter.

Author(s)

Murat Genç

See Also

liureg(), summary(), pressliu(), residuals()

Examples

data("Hitters")
Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
# Hat matrix list
hatlist <- hatliu(liu.mod)
# Hat matrix for third regularization parameter
hatlist[[3]]

---

Hitters

Description

Major League Baseball Data from the 1986 and 1987 seasons.

Format

A data frame with 322 observations of major league players on the following 20 variables with explanations.

AtBat		Number of times at bat in 1986
Hits		Number of hits in 1986
HmRun		Number of home runs in 1986
Runs		Number of runs in 1986
RBI		Number of runs batted in 1986
Walks		Number of walks in 1986
Years		Number of years in the major leagues
CAtBat		Number of times at bat during his career
CHits		Number of hits during his career
CHmRun		Number of home runs during his career
CRuns		Number of runs during his career
CRBI		Number of runs batted in during his career
CWalks		Number of walks during his career
League		A factor with levels A and N indicating player's league at the end of 1986
Division		A factor with levels E and W indicating player's division at the end of 1986
PutOuts		Number of put outs in 1986
Assists		Number of assists in 1986
Errors		Number of errors in 1986
Salary		1987 annual salary on opening day in thousands of dollars
NewLeague		A factor with levels A and N indicating player's league at the beginning of 1987

Source

The dataset was retrieved from the StatLib library maintained at Carnegie Mellon University. This is part of the data used in the 1988 ASA Graphics Section Poster Session. The dataset is available in the R package ISLR2 (James et al., 2022). For more details, see the book, An Introduction to Statistical Learning with applications in R by James et al. (2013).

References

James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An Introduction to Statistical Learning with applications in R, https://www.statlearning.com, Springer-Verlag, New York.

James G, Witten D, Hastie T, Tibshirani R (2022). ISLR2: Introduction to Statistical Learning, Second Edition. R package version 1.3-2, https://CRAN.R-project.org/package=ISLR2.

---

Information Criteria for Liu Regression

Description

For each value of lambda, infoliu calculates the values of the AIC and BIC model selection criteria. Model selection criteria are based on the degrees of the freedom, $\texttt{df}=\mathrm{trace}\left(\mathbf{H}_\lambda\right)$ of the Liu regression model where $\mathbf{H}$ is the hat matrix of Liu regression model.

Usage

infoliu(obj)

Arguments

obj	A liureg object

Value

infoliu returns the matrix of information criteria for each value of the regularization parameter lambda.

Author(s)

Murat Genç

References

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transaction on Automatic Control, 9(6), 716-723. doi:10.1109/TAC.1974.1100705.

Liu, K. (1993). A new class of blased estimate in linear regression. Communications in Statistics-Theory and Methods, 22(2), 393-402. doi:10.1080/03610929308831027.

Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461-464. doi:10.1214/aos/1176344136.

See Also

predict(), summary()

Examples

data("Hitters")
Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
infoliu(liu.mod)

---

Computation of Liu Tuning Parameter

Description

lamest computes the Liu tuning parameters provided in the literature. The tuning parameter estimates are based on

• Liu (1993) doi:10.1080/03610929308831027,

• Ozkale and Kaciranlar (2007) doi:10.1080/03610920601126522,

• Liu (2011) doi:10.1016/j.jspi.2010.05.030.

Usage

lamest(obj, ...)

Arguments

obj	An object of class liureg.
...	Not used in this implemetation.

Details

The lamest function computes the following tuning parameter estimates available in the literature.

lam.mm (Liu, 1993)	$\displaystyle{1-\hat{\sigma}^{2}\left(\frac{\sum\limits _{j=1}^{p}\frac{1}{\lambda_{j}\left(1+\lambda_{j}\right)}}{\sum\limits _{j=1}^{p}\frac{\hat{\alpha}_{j}^{2}}{\left(1+\lambda_{j}\right)^{2}}}\right)}$
lam.CL (Liu, 1993)	$\displaystyle{1-\hat{\sigma}^{2}\left(\frac{\sum\limits _{j=1}^{p}\frac{1}{\left(1+\lambda_{j}\right)}}{\sum\limits _{j=1}^{p}\frac{\lambda_{j}\hat{\alpha}_{j}^{2}}{\left(1+\lambda_{j}\right)^{2}}}\right)}$
lam.opt (Liu, 1993)	$\displaystyle{\frac{\sum\limits _{j=1}^{p}\left(\frac{\alpha_{j}^{2}-\sigma^{2}}{\left(1+\lambda_{j}\right)^{2}}\right)}{\sum\limits _{j=1}^{p}\left(\frac{\sigma^{2}+\lambda_{j}\alpha_{j}^{2}}{\lambda_{j}\left(1+\lambda_{j}\right)^{2}}\right)}}$
lam.OK (Ozkale and Kaciranlar, 2007; Liu, 2011)	$\displaystyle{\frac{\sum\limits _{i=1}^{n}\frac{\tilde{e}_{i}}{1-g_{ii}}\left(\frac{\tilde{e}_{i}}{1-h_{1-ii}}-\frac{\hat{e}_{i}}{1-h_{ii}}\right)}{\sum\limits _{i=1}^{n}\left(\frac{\tilde{e}_{i}}{1-g_{ii}}-\frac{\hat{e}_{i}}{1-h_{ii}}\right)^{2}}}$ with $\hat{e}_{i}=y_{i}-\mathbf{x}_{i}^{T}\left(\mathbf{X}^{T}\mathbf{X}-\mathbf{x}_{i}\mathbf{x}_{i}^{T}\right)^{-1}\left(\mathbf{X}^{T}\mathbf{y}-\mathbf{x}_{i}y_{i}\right)$ and $\tilde{e}_{i}=y_{i}-\mathbf{x}_{i}^{T}\left(\mathbf{X}^{T}\mathbf{X}+\mathbf{I}-\mathbf{x}_{i}\mathbf{x}_{i}^{T}\right)^{-1}\left(\mathbf{X}^{T}\mathbf{y}-\mathbf{x}_{i}y_{i}\right)$ where $g_{ii}$ and $h_{ii}$ are the $i$th diagonal elements of $\mathbf{G}=\mathbf{X}\left(\mathbf{X}^{T}\mathbf{X}+\mathbf{I}\right)^{-1}\mathbf{X}^{T}$ and $\mathbf{H=}\mathbf{X}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{X}^{T}$, respectively.
lam.GCV	This is the $\lambda$ value corresponding to the minimum of the generalized cross-validition (GCV) values. The GCV is computed by $\frac{\mathrm{SSR}_{\lambda}}{n-1-\mathrm{trace}\left(\mathbf{H}_{\lambda}\right)}$ where $\mathrm{SSR}_{\lambda}$ is the residual sum of squares and $\mathrm{trace}\left(\mathbf{H}_{\lambda}\right)$ is the trace of the hat matrix at corresponding value of $\lambda$ from Liu regression.

Value

The return object is the Liu tuning parameter estimates based on the literature.

Author(s)

Murat Genç and Ömer Özbilen

References

Liu, K. (1993). A new class of blased estimate in linear regression. Communications in Statistics-Theory and Methods, 22(2), 393-402. doi:10.1080/03610929308831027.

Liu, X. Q. (2011). Improved Liu estimator in a linear regression model. Journal of Statistical Planning and Inference, 141(1), 189-196. doi:10.1016/j.jspi.2010.05.030.

Ozkale, M. R. and Kaciranlar, S. (2007). A prediction-oriented criterion for choosing the biasing parameter in Liu estimation. Communications in Statistics-Theory and Methods, 36(10), 1889-1903. doi:10.1080/03610920601126522. Imdadullah, M., Aslam, M., and Altaf, S., (2017). liureg: A Comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9(2), 232-247.

See Also

liureg(), predict(), summary(), pressliu(), residuals()

Examples

Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
lamest(liu.mod)

---

Fit a Liu Regression Coefficients Path

Description

liureg fits coefficients paths for Liu regression models over a grid of values for the regularization (biasing) parameter lambda. The returned object is of class liureg.

Usage

liureg(X, y, lambda = 1, scale = c("ulength", "unormal", "none"), ...)

Arguments

X	The design matrix of features. fastliu standardizes the data and includes an intercept term by default.
y	The response vector.
lambda	User-specified values of lambda. The default value is 1, which corresponds to the least squares estimator. A lambda sequence can be entered to generate multiple models.
scale	Scaling type of the design matrix. "ulength" corresponds to unit-length scaling. In this scaling the scaled design matrix is in the form of a correlation matrix. "unormal" scales the features to have unit variance (using $1/n$ rather than $1/(n-1)$ formula). "none" does not make scaling and computations are done on centered features.
...	Not used in this implementation.

Details

The sequence of Liu regression models indexed by the tuning parameter. $\lambda$ are obtained by

$\hat{\boldsymbol{\beta}}^{liu}\left(\lambda\right)= \left(\mathbf{X}^{T}\mathbf{X}+\mathbf{I}_{p}\right)^{-1} \left(\mathbf{X}^{T}\mathbf{y}+\lambda\hat{\boldsymbol{\beta}}^{ls}\right),$

where $\hat{\boldsymbol{\beta}}^{ls}$ is the ordinary least squares estimator.To obtain the models, the singular value decomposition (SVD) of the matrix $\mathbf{X}$ is used. This SVD is done once and is used to generate all models.

Explanatory variables in the design matrix are always centered before fitting a model in the fastliu package. For scaling, two options are possible: unit-length and unit-normal scaling. In unit-length scaling, the matrix of explanatory variables has correlation form. In unit-normal scaling, the explanatory variables have zero mean and unit variance. Both Coefficient estimates based on the scaled data and in original scale are presented. The intercept of the model is not penalized and computed by $\bar{y}-\bar{X}\boldsymbol{\hat{\beta}}_1$, where $\bar{X}$ is the row vector of the explanatory variables and $\boldsymbol{\hat{\beta}}_1$ is computed based on centered design matrix.

The returned liureg object is used for statistical testing of Liu coefficients, plotting method and computing the Liu regression related statistics.

Value

Fitted Liu regression object with the class of liureg

Author(s)

Murat Genç and Ömer Özbilen

See Also

coef(), predict(), summary(), pressliu(), residuals()

Examples

data("Hitters")
Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.05)
liu.mod <- liureg(X, y, lam)

---

Plot method for liureg objects

Description

Plot method for liureg objects

Usage

## S3 method for class 'liureg'
plot(x, type = c("coefpath", "biasvar", "info"), ...)

Arguments

x	A liureg object.
type	What to plot on the vertical axis. coefpath plots the coefficient path of the Liu regression; biasvar generates a bias-variaance plot, info plots the information criteria corresponding the regularization parameter values.
...	Other graphical parameters to plot.

Value

No return value.

Author(s)

Murat Genç

See Also

liureg(), predict(), summary()

Examples

Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
liu.mod <- liureg(X, y, seq(0, 1, 0.01))

# Liu coefficient paths
plot(liu.mod)

# Bias-variance trade-off
plot(liu.mod, type="biasvar")

---

Predict method for liureg objects

Description

Predict method for liureg objects

Usage

## S3 method for class 'liureg'
predict(object, newdata, ...)

Arguments

object	A liureg object.
newdata	A data frame of new values for X at which predictions are to be made. Can be a data.frame or a matrix.
...	Not used in this implementation.

Value

Depending on whether the lambda is a scalar or a vector, the predict.liureg function returns a vector or matrix of predictions, respectively.

Author(s)

Murat Genç

See Also

liureg(), predict(), summary(), pressliu(), residuals()

Examples

data("Hitters")
Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
# Predictions based on original X matrix.
predict(liu.mod)
# Predictions based on newdata. newdata can be a matrix or a data.frame.
predict(liu.mod, newdata=X[1:5, ])

---

Predicted Residual Sum of Squares (PRESS)

Description

pressliu computes the predicted residual sum of squares (PRESS) based on a Liu regression model.

Usage

pressliu(obj, digits = 5L, ...)

Arguments

obj	A liureg object.
digits	Decimal places in the columns of data frame of PRESS values. Can be an integer of vector of integers.
...	Not used in this implementation.

Details

The PRESS statistic is based on the predicted leave-one-out residual sum of squares. The statistic is computed as ${\sum\limits _{i=1}^{n}\left(\frac{\hat{e}_{\lambda i}}{1-h_{1-ii}}-\frac{e_{i}\left(h_{1-ii}-\tilde{\mathbf{H}}_{\lambda-ii}\right)}{\left(1-h_{1-ii}\right)\left(1-h_{ii}\right)}\right)^{2}}$ where $h_{ii}$ is the $i$th diagonal element of the hat matrix corresponding to the least squares estimator, $h_{1-ii}$ is the $i$th diagonal element of the hat matrix of the Liu estimator and $e_{\lambda i}$ is the residual at the specific value of $\lambda$.

Value

The returned object is a vector of PRESS values computed for each lambda..

Author(s)

Murat Genç, Ömer Özbilen

References

Liu, K. (1993). A new class of blased estimate in linear regression. Communications in Statistics-Theory and Methods, 22(2), 393-402. doi:10.1080/03610929308831027.

Ozkale, M. R. and Kaciranlar, S. (2007). A prediction-oriented criterion for choosing the biasing parameter in Liu estimation. Communications in Statistics-Theory and Methods, 36(10), 1889-1903. doi:10.1080/03610920601126522.

See Also

liureg(), pressliu(), residuals()

Examples

data("Hitters")
Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
pressliu(liu.mod)

---

Print method for liureg objects

Description

Prints coefficients paths for Liu regression models over a grid of values for the regularization (biasing) parameter lambda.

Usage

## S3 method for class 'liureg'
print(x, digits = max(3, getOption("digits") - 3), ...)

Arguments

x	An object of class liureg.
digits	Number of decimal places in the coefficients data.frame.
...	Not used in this implementation.

Value

The returned object is a data.frame showing the coefficients path.

Author(s)

Murat Genç

See Also

liureg(), summary(), pressliu(), residuals()

Examples

Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
print(liu.mod)

---

Print Method for Liu Regression Statistics

Description

statliu computes the statistics related to the Liu regression.

Usage

## S3 method for class 'statliu'
print(x, digits = 5, ...)

Arguments

x	A statliu object
digits	Number of decimal places in the data frame of Liu regression statistics.
...	Other parameters related to print.

Value

The return object is the statistics relatec to the Liu regression.

Author(s)

Murat Genç

See Also

liureg(), summary(), pressliu(), residuals()

Examples

Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
stats <- statliu(liu.mod)
print(stats)

---

Liu Regression Residuals

Description

Liu Regression Residuals

Usage

## S3 method for class 'liureg'
residuals(object, ...)

Arguments

object	An object of class liureg.
...	Not used in this implementation.

Value

The returned object is a vector or matrix whose columns are Liu residuals for each lambda.

Author(s)

Murat Genç

See Also

liureg(), pressliu(), residuals()

Examples

Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
residuals(liu.mod)

---

Liu Regression Statistics

Description

statliu computes the statistics related to the Liu regression.

Usage

statliu(obj)

Arguments

obj	An object of class liureg.

Details

EDF (Liu, 1993; Hastie et al., 2009)	Effective degrees of freedom, $n-\mathrm{trace}\left(2\mathbf{H}_\lambda\right)-\mathbf{H}_\lambda\mathbf{H}_\lambda^T$ for each $\lambda$ where $n$ is the number of the observations in the design matrix and $\mathbf{H}_\lambda$ is the hat matrix of Liu regression at $\lambda$.
sigma2	Computed $\hat{\sigma}^2$ from the Liu regression for each $\lambda$.
VAR	Variance from the Liu regression for each $\lambda$.
BIAS2	Squared-bias from the Liu regression for each $\lambda$.
MSE	Mean squared error (MSE) from the Liu regression for each $\lambda$.
FVal	F-statistics value from the Liu regression for each $\lambda$.
GCV	Generalized cross-validation (GCV) from the Liu regression for each $\lambda$. The GCV is computed by $\frac{\mathrm{SSR}_{\lambda}}{n-1-\mathrm{trace}\left(\mathbf{H}_{\lambda}\right)}$ where $\mathrm{SSR}_{\lambda}$ is the residual sum of squares and $\mathrm{trace}\left(\mathbf{H}_{\lambda}\right)$ is the trace of the hat matrix at corresponding value of $\lambda$ from Liu regression.
R2	R-squared from the Liu regression for each $\lambda$.
AdjR2	Adjusted R-squared from the Liu regression for each $\lambda$.

Value

The return object is the statistics related to the Liu regression.

Author(s)

Murat Genç

References

Liu, K. (1993). A new class of blased estimate in linear regression. Communications in Statistics-Theory and Methods, 22(2), 393-402. doi:10.1080/03610929308831027.

Hastie, T., Tibshirani, R., Friedman, J. H., Friedman, J. H. (2009). The elements of statistical learning: data mining, inference, and prediction (Vol. 2, pp. 1-758). New York: Springer.

See Also

liureg(), summary(), pressliu(), residuals()

Examples

Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
stats <- statliu(liu.mod)
print(stats)

---

Summarizing Liu Regression Fit

Description

summary method for liureg objects.

Usage

## S3 method for class 'liureg'
summary(object, digits, ...)

Arguments

object	An object of class liureg.
digits	Number of decimal places in the data frame of Liu regression statistics.
...	Not used in this implemetation.

Details

summary.liureg produces an object with S3 class summary.liureg. The function returns a list of summary statistics of the Liu regression fit for the grid of regularization parameter $\lambda$ values. Each element of the output list includes:

coefficients	A $p\times 5$ matrix with columns coefficient estimates, scaled coefficient estimates, scaled standard errors, scaled $t-$values with corresponding $p-$value.
Statistics	Liu related statistics $R^2$, $\textrm{adjusted}-R^2$, $F-$statistics, AIC, BIC and MSE values.

Value

The returned object is a list whose elements are Liu regression coefficient estimates and statistics related to Liu regression.

Author(s)

Murat Genç

See Also

liureg(), coef(), predict(), residuals()

Examples

Hitters <- na.omit(Hitters)
X <- model.matrix(Salary ~ ., Hitters)[, -1]
y <- Hitters$Salary
lam <- seq(0, 1, 0.01)
liu.mod <- liureg(X, y, lam)
summary(liu.mod)

---