| Title: | Regularization Paths of L1-Penalized SVMs |
|---|---|
| Description: | Computes regularization paths for L1-penalized support vector machine (SVM) models, including the standard SVM, receiver operating characteristic SVM (ROC-SVM), and fraud detection SVM (FD-SVM). The package implements efficient path algorithms to obtain solutions over a wide range of regularization parameter values. It also provides cross-validation for selecting an optimal regularization parameter, along with functions for prediction, visualization, and summarization of the fitted models. For more details, see Zhu et al. (2003) <https://proceedings.neurips.cc/paper_files/paper/2003/file/49d4b2faeb4b7b9e745775793141e2b2-Paper.pdf>, Kim et al. (2021) <doi:10.1002/sta4.400>, Park et al. (2023) <doi:10.1007/s42952-023-00207-6>. |
| Authors: | Hyungwoo Kim [aut, cre], Minhyoung Park [aut], Seung Jun Shin [aut] |
| Maintainer: | Hyungwoo Kim <[email protected]> |
| License: | GPL-3 |
| Version: | 0.1.0 |
| Built: | 2026-10-21 21:05:45 UTC |
| Source: | https://github.com/cran/l1penSVMs |
This function performs k-fold cross-validation to select the optimal for the L1-penalized Support Vector Machine (SVM), Receiver Operating Characteristic Support Vector Machine (ROC-SVM) and Fraud Detection Support Vector Machine (FD-SVM) based on the regularization path algorithm.
cv.l1svm( x, y, classification = c("SVM", "ROCSVM", "FDSVM"), nfold = NULL, sensitivity = NULL, init.t = NULL, max.iter = NULL, standardize = TRUE, incomp.size = NULL )cv.l1svm( x, y, classification = c("SVM", "ROCSVM", "FDSVM"), nfold = NULL, sensitivity = NULL, init.t = NULL, max.iter = NULL, standardize = TRUE, incomp.size = NULL )
x |
The data matrix (n x p) with n rows (observations) on p variables (columns); categorical variables are automatically one-hot encoded. |
y |
Response vector with class labels in {-1, 1}. Labels given as {0, 1} or as a two-level factor/character are automatically converted to this format. |
classification |
The classification method is to be applied. Either "SVM", "ROC-SVM" and "FD-SVM" |
nfold |
The number of folds in the cross-validation (CV), default is |
sensitivity |
The minimum proportion of actual fraud cases that are correctly classified as fraud for the "FD-SVM". |
init.t |
For "SVM" and "FD-SVM", the initial regularization parameter with sufficiently small t; default is 1e-5 |
max.iter |
The maximum number of iterations (total across the entire path algorithm); default is 10000 |
standardize |
Standardize each column of the data matrix to have zero mean and unit variance |
incomp.size |
Sample size for the incomplete U-statistic in ROC-SVM; default is the number of observations |
The cv.l1.svm determines the optimal regularization parameter and yields the cofficient estimates at the optimal
An object cv.l1svm returns a list with following components:
A vector of t values used in K-fold CV.
A vector of the averaged CV errors for different t values.
A vector of the averaged CV AUC-risks for different t values.
A vector of the averaged CV sensitivies for different t values.
A vector of the averaged CV specificies for different t values.
Optimal value of t for thee L1-penalized SVM models.
Optimal value of sensitivity at t=t.opt for the L1-penalized FD-SVM.
Optimal value of specificity at t=t.opt for the L1-penalized FD-SVM.
The estimated coefficients at the optimal parameter t for three L1-penalized SVM.
Hyungwoo Kim, MinHyung Park, Seung Jun Shin
Ji Zhu, Saharon Rosset, Robert Tibshirani and Trevor Hastie, "1-norm Support Vector Machines", Advances in neural information processing systems, 16.
Hyungwoo Kim, Insuk Sohn and Seung Jun Shin, "Regularization paths of L1-penalized ROC Curve-Optimizing Support Vector Machines", Stat, 10(1), e400.
Minhyoung Park, Hyungwoo Kim and Seung Jun Shin, "L1-penalized Fraud Detection Support Vector Machines", under
l1.svmpath, l1.rocsvmpath, l1.fdsvmpath, plot.l1svmpath, predict.l1svmpath
library(l1penSVMs) set.seed(2) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) # SVM cv.svm <- cv.l1svm(x, y, classification="SVM") # ROCSVM cv.rocsvm <- cv.l1svm(x, y, classification="ROCSVM") # FDSVM cv.fdsvm <- cv.l1svm(x, y, classification="FDSVM", sensitivity=0.9)library(l1penSVMs) set.seed(2) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) # SVM cv.svm <- cv.l1svm(x, y, classification="SVM") # ROCSVM cv.rocsvm <- cv.l1svm(x, y, classification="ROCSVM") # FDSVM cv.fdsvm <- cv.l1svm(x, y, classification="FDSVM", sensitivity=0.9)
This algorithm provides the initial solution of L1-penalized Fraud Detection Support Vector Machine (FD-SVM) for sufficiently small t.
l1.fdsvm.initialization(x, y, init.t)l1.fdsvm.initialization(x, y, init.t)
x |
The data matrix after dummy variable encoding |
y |
The |
init.t |
Initial regularization parameter with sufficiently small t; default is 1e-5 |
The intercept and coefficients at initial regularization parameter
This algorithm computes the entire solution paths for L1-penalized FDSVM with a relatively low cost compared to linear programming problem.
l1.fdsvmpath(x, y, init.t, max.iter)l1.fdsvmpath(x, y, init.t, max.iter)
x |
The data matrix (n x p) with n rows (observations) on p variables (columns) |
y |
The |
init.t |
Initial regularization parameter with sufficiently small t; default is 1e-5 |
max.iter |
The maximum number of iterations (total across the entire path algorithm); default is 3000 |
A 'l1.fdsvmpath' object is returned. It contains
The sequence of regularization parameter values in the path
The intercept sequence of length length(t)
A p*length(t) matrix of coefficients.
Seung Jun Shin, MinHyung Park, Hyungwoo Kim
Minhyoung Park, Hyungwoo Kim and Seung Jun Shin, "L1-penalized Fraud Detection Support Vector Machines", under
l1.svmpath, l1.fdsvm.initialization, plot.l1svmpath
library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) obj <- l1svmpath(x,y,classification="FDSVM")library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) obj <- l1svmpath(x,y,classification="FDSVM")
This algorithm computes the entire solution paths for L1-penalized Receiver Operating Characteristic Support Vector Machine (ROC-SVM)
l1.rocsvmpath(x, y, max.iter, incomp.size)l1.rocsvmpath(x, y, max.iter, incomp.size)
x |
The data matrix after dummy variable encoding |
y |
The |
max.iter |
The maximum number of iterations (total across the entire path algorithm); default is 3000. |
incomp.size |
Sample Size for the incomplete U-statistic in ROC-SVM; default is 2n. |
A 'l1.rocsvmpath' object is returned. It contains
A p*length(t) matrix of coefficients.
The sequence of regularization parameter values in the path
The data matrix after dummy variable encoding
The {-1, 1} valued response variable
Seung Jun Shin, MinHyung Park, Hyungwoo Kim
Hyungwoo Kim, Insuk Sohn and Seung Jun Shin, "Regularization paths of L1-penalized ROC Curve-Optimizing Support Vector Machines", Stat, 10(1), e400.
library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) obj <- l1svmpath(x,y,classification="ROCSVM")library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) obj <- l1svmpath(x,y,classification="ROCSVM")
This algorithm provides the initial solution of L1-penalized SVM for sufficiently small t.
l1.svm.initialization(x, y, init.t)l1.svm.initialization(x, y, init.t)
x |
The data matrix after dummy variable encoding |
y |
The |
init.t |
Initial regularization parameter with sufficiently small t; default is 1e-5 |
The intercept and coefficients at initial regularization parameter
This algorithm computes the entire solution paths for L1-penalized Support Vector Machine (SVM)
l1.svmpath(x, y, init.t, max.iter)l1.svmpath(x, y, init.t, max.iter)
x |
The data matrix after dummy variable encoding |
y |
The |
init.t |
The initial regularization parameter with sufficiently small t; default is 1e-5 |
max.iter |
The maximum number of iterations (total across the entire path algorithm); default is 3000. |
An object of class l1.svmpath. It contains
A vector of intercept estimates for different values of t
A p*length(t) matrix of coefficients
The sequence of regularization parameter values in the path
The data matrix after dummy variable encoding
The {-1, 1} valued response variable
Seung Jun Shin, MinHyung Park, Hyungwoo Kim
Hyungwoo Kim, Insuk Sohn and Seung Jun Shin, "Regularization paths of L1-penalized ROC Curve-Optimizing Support Vector Machines", Stat, 10(1), e400.
library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) obj <- l1svmpath(x,y,classification="SVM")library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) obj <- l1svmpath(x,y,classification="SVM")
This provides the entire solution paths for L1-penalized Support Vector Machine (SVM), Receiver Operating Characteristic Support Vector Machine (ROC-SVM) and Fraud Detection Support Vector Machine (FD-SVM) with a relatively low cost compared to linear programming problem.
l1svmpath( x, y, classification = c("SVM", "ROCSVM", "FDSVM"), init.t = NULL, max.iter = NULL, standardize = TRUE, incomp.size = NULL, fraud.label.fixed = FALSE )l1svmpath( x, y, classification = c("SVM", "ROCSVM", "FDSVM"), init.t = NULL, max.iter = NULL, standardize = TRUE, incomp.size = NULL, fraud.label.fixed = FALSE )
x |
The data matrix (n x p) with n rows (observations) on p variables (columns); categorical variables are automatically one-hot encoded. |
y |
Response vector with class labels in {-1, 1}. Labels given as {0, 1} or as a two-level factor/character are automatically converted to this format. |
classification |
The classification method is to be applied. Either "SVM", "ROCSVM" or "FDSVM" |
init.t |
For "SVM" and "FDSVM" models, the initial regularization parameter with sufficiently small t; default is 1e-5 |
max.iter |
The maximum number of iterations (total across the entire path algorithm); default is 10000 |
standardize |
Standardize each column of the data matrix to have zero mean and unit variance |
incomp.size |
Sample size for the incomplete U-statistic in ROC-SVM; default is the number of observations |
fraud.label.fixed |
If TRUE, preserves the existing {-1, +1} class encoding for FD-SVM. |
The algorithm used in l1.svmpath() for "SVM" is described in "1-norm Support Vector Machines" by Ji Zhu, Saharon Rosset, Robert Tibshirani and Trevor Hastie. "ROCSVM" is described in "Regularization paths of L1-penalized ROC Curve-Optimizing Support Vector Machines" by Hyungwoo Kim, Insuk Sohn and Seung Jun Shin. "FDSVM" is described in "L1-penalized Fraud Detection Support Vector Machines" by Minhyoung Park, Hyungwoo Kim and Seung Jun Shin. For more information, see references.
A 'l1.svmpath' object is returned. It contains
The sequence of regularization parameter values in the path
For "SVM" and "FDSVM" models, the intercept sequence of length length(t)
A numeric matrix with length(t) rows and p columns, containing coefficient estimates along the regularization path.
The data matrix after dummy variable encoding
The {-1, +1} valued response variable (For "FDSVM", Fraud +1 and Normal -1)
Seung Jun Shin, MinHyung Park, Hyungwoo Kim
Ji Zhu, Saharon Rosset, Robert Tibshirani and Trevor Hastie, "1-norm Support Vector Machines", Advances in neural information processing systems, 16.
Hyungwoo Kim, Insuk Sohn and Seung Jun Shin, "Regularization paths of L1-penalized ROC Curve-Optimizing Support Vector Machines", Stat, 10(1), e400.
Minhyoung Park, Hyungwoo Kim and Seung Jun Shin, "L1-penalized Fraud Detection Support Vector Machines", under
l1.svmpath, l1.rocsvmpath, l1.fdsvmpath, plot.l1svmpath, predict.l1svmpath
library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) # SVM obj <- l1svmpath(x,y,classification="SVM") # ROCSVM obj <- l1svmpath(x,y,classification="ROCSVM") # FDSVM obj <- l1svmpath(x,y,classification="FDSVM")library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) # SVM obj <- l1svmpath(x,y,classification="SVM") # ROCSVM obj <- l1svmpath(x,y,classification="ROCSVM") # FDSVM obj <- l1svmpath(x,y,classification="FDSVM")
lp functionIt provides the estimated coefficients for L1-penalized Support Vector Machine (SVM) and Receiver Operating Characteristic Support Vector Machine (ROC-SVM) using linear programming (LP).
lpfit.l1svm(x, y, t, classification = c("SVM", "ROCSVM"))lpfit.l1svm(x, y, t, classification = c("SVM", "ROCSVM"))
x |
The data matrix (n x p) with n rows (observations) on p variables (columns) |
y |
The |
t |
The value of regularization parameter |
classification |
The classification method is to be applied. Either "SVM" (the default) or "ROCSVM" |
A numeric vector containing the estimated coefficients
at the specified regularization parameter t.
For SVM, the first element is the intercept.
For ROC-SVM, only slope coefficients are returned.
set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) obj <- lpfit.l1svm(x, y, t=1.5, classification="SVM")set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) obj <- lpfit.l1svm(x, y, t=1.5, classification="SVM")
Plot the whole regularization paths for L1-penalized Support Vector Machine (SVM), Receiver Operating Characteristic Support Vector Machine (ROC-SVM) and Fraud Detection Support Vector Machine (FD-SVM)
## S3 method for class 'l1svmpath' plot(x, ...)## S3 method for class 'l1svmpath' plot(x, ...)
x |
An object of class |
... |
Additional graphical arguments passed to plotting functions. |
This function is a method for the generic plot() function,
Invisibly returns the input object of class l1svmpath. The function displays the regularization paths.
library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) # SVM obj <- l1svmpath(x, y, classification="SVM") plot(obj)library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) # SVM obj <- l1svmpath(x, y, classification="SVM") plot(obj)
Predict the class label or classification score
## S3 method for class 'l1svmpath' predict(object, x, t = NULL, ...)## S3 method for class 'l1svmpath' predict(object, x, t = NULL, ...)
object |
A fitted regularization path object of class |
x |
The test data matrix (n x p), with n rows (observations) on p variables (columns) |
t |
The value of regularization parameter |
... |
Additional arguments passed to the prediction method. |
This function returns the predicted class labels or the fitted classification scores for L1-penalized Support Vector Machine (SVM), Receiver Operating Characteristic Support Vector Machine (ROC-SVM) and Fraud Detection Support Vector Machine (FD-SVM)
The predicted class label for the L1-penalized SVM and FDSVM, or the fitted classification scores for the L1-penalized ROCSVM.
library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) # SVM obj <- l1svmpath(x,y,classification="SVM") predict(obj, x=x, t=1.5)library(l1penSVMs) set.seed(1) n <- 100; p <- 10 x <- matrix(rnorm(n*p),n,p) beta <- c(2,-1,0.5,rep(0,p-3)) fx <- -2 + crossprod(t(x),beta) y <- sign(fx + rnorm(n,0,1)) # SVM obj <- l1svmpath(x,y,classification="SVM") predict(obj, x=x, t=1.5)
Print Cross-Validation Results
## S3 method for class 'cv.l1svm' print(x, ...)## S3 method for class 'cv.l1svm' print(x, ...)
x |
An object of class |
... |
Additional arguments. |
Invisibly returns the input object of class "cv.l1svm". The function prints a summary of the cross-validation results.