| Title: | 'Tweedie' Compound Poisson Model in the Reproducing Kernel Hilbert Space |
|---|---|
| Description: | Kernel-based 'Tweedie' compound Poisson gamma model using high-dimensional predictors for the analyses of zero-inflated response variables. The package features built-in estimation, prediction and cross-validation tools and supports choice of different kernel functions. For more details, please see Yi Lian, Archer Yi Yang, Boxiang Wang, Peng Shi & Robert William Platt (2023) <doi:10.1080/00401706.2022.2156615>. |
| Authors: | Yi Lian [aut, cre], Archer Yi Yang [aut, cph], Boxiang Wang [aut], Peng Shi [aut], Robert W. Platt [aut] |
| Maintainer: | Yi Lian <[email protected]> |
| License: | GPL-3 |
| Version: | 1.0.3 |
| Built: | 2024-10-09 06:35:11 UTC |
| Source: | CRAN |
as.kernelMatrix in package KERE can be used
to coerce the kernelMatrix class to matrix objects representing a
kernel matrix. These matrices can then be used with the kernelMatrix
interfaces which most of the functions in KERE support.
## S4 method for signature 'matrix' as.kernelMatrix(x, center = FALSE)## S4 method for signature 'matrix' as.kernelMatrix(x, center = FALSE)
x |
matrix to be assigned the |
center |
center the kernel matrix in feature space (default: FALSE) |
Alexandros Karatzoglou
[email protected]
## Create toy data x <- rbind(matrix(rnorm(10),,2),matrix(rnorm(10,mean=3),,2)) y <- matrix(c(rep(1,5),rep(-1,5))) ### Use as.kernelMatrix to label the cov. matrix as a kernel matrix ### which is eq. to using a linear kernel K <- as.kernelMatrix(crossprod(t(x))) K## Create toy data x <- rbind(matrix(rnorm(10),,2),matrix(rnorm(10,mean=3),,2)) y <- matrix(c(rep(1,5),rep(-1,5))) ### Use as.kernelMatrix to label the cov. matrix as a kernel matrix ### which is eq. to using a linear kernel K <- as.kernelMatrix(crossprod(t(x))) K
A simulated dataset with covariate matrix x of size 30 x 5 and an outcome vector y of length 30.
data(dat)data(dat)
A list with 2 items:
Covariate matrix
Outcome vector
x is generated from standard normal distribution. y is generated from Tweedie distribution with mean equal to exp(sin(x) %*% (6, -4, 0, 0, 0)). Only the first two variables are associated with the outcome.
The kernel generating functions provided in KERE.
The Gaussian RBF kernel
The Polynomial kernel
The Linear kernel
The Hyperbolic tangent kernel
The Laplacian kernel
The Bessel kernel
The ANOVA RBF kernel where k(x,x) is a Gaussian
RBF kernel.
The Spline kernel \
rbfdot(sigma = 1) polydot(degree = 1, scale = 1, offset = 1) tanhdot(scale = 1, offset = 1) vanilladot() laplacedot(sigma = 1) besseldot(sigma = 1, order = 1, degree = 1) anovadot(sigma = 1, degree = 1) splinedot()rbfdot(sigma = 1) polydot(degree = 1, scale = 1, offset = 1) tanhdot(scale = 1, offset = 1) vanilladot() laplacedot(sigma = 1) besseldot(sigma = 1, order = 1, degree = 1) anovadot(sigma = 1, degree = 1) splinedot()
sigma |
The inverse kernel width used by the Gaussian the Laplacian, the Bessel and the ANOVA kernel |
degree |
The degree of the polynomial, bessel or ANOVA kernel function. This has to be an positive integer. |
scale |
The scaling parameter of the polynomial and tangent kernel is a convenient way of normalizing patterns without the need to modify the data itself |
offset |
The offset used in a polynomial or hyperbolic tangent kernel |
order |
The order of the Bessel function to be used as a kernel |
The kernel generating functions are used to initialize a kernel
function
which calculates the dot (inner) product between two feature vectors in a
Hilbert Space. These functions can be passed as a kernel argument on almost all
functions in KERE(e.g., ksvm, kpca etc).
Although using one of the existing kernel functions as a
kernel argument in various functions in KERE has the
advantage that optimized code is used to calculate various kernel expressions,
any other function implementing a dot product of class kernel can also be used as a kernel
argument. This allows the user to use, test and develop special kernels
for a given data set or algorithm.
Return an S4 object of class kernel which extents the
function class. The resulting function implements the given
kernel calculating the inner (dot) product between two vectors.
kpar |
a list containing the kernel parameters (hyperparameters) used. |
The kernel parameters can be accessed by the kpar function.
If the offset in the Polynomial kernel is set to 0, we obtain homogeneous polynomial
kernels, for positive values, we have inhomogeneous
kernels. Note that for negative values the kernel does not satisfy Mercer's
condition and thus the optimizers may fail.
In the Hyperbolic tangent kernel if the offset is negative the likelihood of obtaining a kernel matrix that is not positive definite is much higher (since then even some diagonal elements may be negative), hence if this kernel has to be used, the offset should always be positive. Note, however, that this is no guarantee that the kernel will be positive.
Alexandros Karatzoglou
[email protected]
kernelMatrix , kernelMult, kernelPol
rbfkernel <- rbfdot(sigma = 0.1) rbfkernel kpar(rbfkernel) ## create two vectors x <- rnorm(10) y <- rnorm(10) ## calculate dot product rbfkernel(x,y)rbfkernel <- rbfdot(sigma = 0.1) rbfkernel kpar(rbfkernel) ## create two vectors x <- rnorm(10) y <- rnorm(10) ## calculate dot product rbfkernel(x,y)
The built-in kernel classes in KERE
Objects can be created by calls of the form new("rbfkernel"),
new{"polykernel"}, new{"tanhkernel"},
new{"vanillakernel"}, new{"anovakernel"},
new{"besselkernel"}, new{"laplacekernel"},
new{"splinekernel"} or by calling the rbfdot, polydot, tanhdot,
vanilladot, anovadot, besseldot, laplacedot,
splinedot functions etc..
.Data:Object of class "function" containing
the kernel function
kpar:Object of class "list" containing the
kernel parameters
Class "kernel", directly.
Class "function", by class "kernel".
signature(kernel = "rbfkernel", x =
"matrix"): computes the kernel matrix
signature(kernel = "rbfkernel", x =
"matrix"): computes the quadratic kernel expression
signature(kernel = "rbfkernel", x =
"matrix"): computes the kernel expansion
signature(kernel = "rbfkernel", x =
"matrix"),,a: computes parts or the full kernel matrix, mainly
used in kernel algorithms where columns of the kernel matrix are
computed per invocation
Alexandros Karatzoglou
[email protected]
rbfkernel <- rbfdot(sigma = 0.1) rbfkernel is(rbfkernel) kpar(rbfkernel)rbfkernel <- rbfdot(sigma = 0.1) rbfkernel is(rbfkernel) kpar(rbfkernel)
kernelMatrix calculates the kernel matrix or .kernelPol computes the quadratic kernel expression , .kernelMult calculates the kernel expansion kernelFast computes the kernel matrix, identical
to kernelMatrix, except that it also requires the squared
norm of the first argument as additional input, useful in iterative
kernel matrix calculations.
## S4 method for signature 'kernel' kernelMatrix(kernel, x, y = NULL) ## S4 method for signature 'kernel' kernelPol(kernel, x, y = NULL, z, k = NULL) ## S4 method for signature 'kernel' kernelMult(kernel, x, y = NULL, z, blocksize = 256) ## S4 method for signature 'kernel' kernelFast(kernel, x, y, a)## S4 method for signature 'kernel' kernelMatrix(kernel, x, y = NULL) ## S4 method for signature 'kernel' kernelPol(kernel, x, y = NULL, z, k = NULL) ## S4 method for signature 'kernel' kernelMult(kernel, x, y = NULL, z, blocksize = 256) ## S4 method for signature 'kernel' kernelFast(kernel, x, y, a)
kernel |
the kernel function to be used to calculate the kernel
matrix.
This has to be a function of class |
x |
a data matrix to be used to calculate the kernel matrix. |
y |
second data matrix to calculate the kernel matrix. |
z |
a suitable vector or matrix |
k |
a suitable vector or matrix |
a |
the squared norm of |
blocksize |
the kernel expansion computations are done block wise
to avoid storing the kernel matrix into memory. |
Common functions used during kernel based computations.
The kernel parameter can be set to any function, of class
kernel, which computes the inner product in feature space between two
vector arguments. KERE provides the most popular kernel functions
which can be initialized by using the following
functions:
rbfdot Radial Basis kernel function
polydot Polynomial kernel function
vanilladot Linear kernel function
tanhdot Hyperbolic tangent kernel function
laplacedot Laplacian kernel function
besseldot Bessel kernel function
anovadot ANOVA RBF kernel function
splinedot the Spline kernel
(see example.)
kernelFast is mainly used in situations where columns of the
kernel matrix are computed per invocation. In these cases,
evaluating the norm of each row-entry over and over again would
cause significant computational overhead.
kernelMatrix returns a symmetric diagonal semi-definite matrix.kernelPol returns a matrix.kernelMult usually returns a one-column matrix.
Alexandros Karatzoglou
[email protected]
rbfdot, polydot,
tanhdot, vanilladot
## use the spam data x <- matrix(rnorm(10*10),10,10) ## initialize kernel function rbf <- rbfdot(sigma = 0.05) rbf ## calculate kernel matrix kernelMatrix(rbf, x) y <- matrix(rnorm(10*1),10,1) ## calculate the quadratic kernel expression kernelPol(rbf, x, ,y) ## calculate the kernel expansion kernelMult(rbf, x, ,y)## use the spam data x <- matrix(rnorm(10*10),10,10) ## initialize kernel function rbf <- rbfdot(sigma = 0.05) rbf ## calculate kernel matrix kernelMatrix(rbf, x) y <- matrix(rnorm(10*1),10,1) ## calculate the quadratic kernel expression kernelPol(rbf, x, ,y) ## calculate the kernel expansion kernelMult(rbf, x, ,y)
ktd_cv() performs cross-validation to determine the optimal regularization coefficient of the ktweedie model.
ktd_cv(x, y, kern, lambda, nfolds = 5, rho = 1.5, loss = "LL", ...)ktd_cv(x, y, kern, lambda, nfolds = 5, rho = 1.5, loss = "LL", ...)
x |
Covariate matrix. |
y |
Outcome vector (e.g. insurance cost). |
kern |
Choice of kernel. See |
lambda |
A vector of candidate regularization coefficients used in cross-validation. |
nfolds |
Number of folds in cross-validation. Default is 5. |
rho |
The power parameter of the Tweedie model. Default is 1.5 and can take any real value between 1 and 2. |
loss |
Criterion used in cross-validation. "LL" for log likelihood, "RMSE" for root mean squared error, "MAD" for mean absolute difference. Default is "LL". |
... |
Optional arguments to be passed to |
ktd_cv() is a built-in wrapper for cross-validation for the choice of regularization coefficient.
A list of two items.
LL or RMSE or MAD: a vector of validation error based on the user-specified loss, named by the corresponding lambda values;
Best_lambda: the lambda value in the pair that generates the best loss;
ktd_cv2d, ktd_estimate, ktd_predict
# Provide a sequence of candidate values to the argument lambda. # ktd_cv() will perform cross-validation to determine which is the best. ( cv1d <- ktd_cv(x = dat$x, y = dat$y, kern = rbfdot(sigma = 1e-8), lambda = 10^(-8:-1), nfolds = 5) )# Provide a sequence of candidate values to the argument lambda. # ktd_cv() will perform cross-validation to determine which is the best. ( cv1d <- ktd_cv(x = dat$x, y = dat$y, kern = rbfdot(sigma = 1e-8), lambda = 10^(-8:-1), nfolds = 5) )
ktd_cv2d() performs 2-dimensional random search from user-specified ranges to determine the optimal pair of regularization coefficient and kernel parameter of the ktweedie model.
ktd_cv2d( x, y, kernfunc, lambda, sigma, ncoefs, nfolds = 5, rho = 1.5, loss = "LL", ... )ktd_cv2d( x, y, kernfunc, lambda, sigma, ncoefs, nfolds = 5, rho = 1.5, loss = "LL", ... )
x |
Covariate matrix. |
y |
Outcome vector (e.g. insurance cost). |
kernfunc |
Choice of kernel function. See |
lambda |
A vector of length two indicating the lower and upper bound from which candidate regularization coefficient values are sampled uniformly on the log scale. |
sigma |
A vector of length two indicating the lower and upper bound from which candidate kernel parameter values are sampled uniformly on the log scale. |
ncoefs |
The number of candidate |
nfolds |
Number of folds in cross-validation. Default is 5. |
rho |
The power parameter of the Tweedie model. Default is 1.5 and can take any real value between 1 and 2. |
loss |
Criterion used in cross-validation. "LL" for log likelihood, "RMSE" for root mean squared error, "MAD" for mean absolute difference. Default is "LL". |
... |
Optional arguments to be passed to |
ktd_cv2d() is a built-in wrapper for 2D random search for the regularization coefficient and kernel parameter. For kernel functions with greater than one parameters, ktd_cv2d() supports the tuning of the first one.
A list of three items.
LL or RMSE or MAD: a vector of validation error based on the user-specified loss, named by the corresponding lambda and sigma values;
Best_lambda: the lambda value in the pair that generates the best loss;
Best_sigma: the sigma value in the pair that generates the best loss.
ktd_cv, ktd_estimate, ktd_predict
### Cross-validation # Provide the kernel function name (e.g. rbfdot) to the argument kernfunc, # NOT the kernel function object, e.g. rbfdot(sigma = 1). # Provide ranges where the candidate lambdas and sigmas are drawn from # to the arguments lambda and sigma. # The number of pairs of candidates to select from is specified by ncoefs. ( cv2d <- ktd_cv2d(x = dat$x, y = dat$y, kernfunc = rbfdot, lambda = c(1e-3, 1e0), sigma = c(1e-3, 1e0), ncoefs = 10) ) ### Followed by fitting fit <- ktd_estimate(x = dat$x, y = dat$y, kern = rbfdot(sigma = cv2d$Best_sigma), lam1 = cv2d$Best_lambda)### Cross-validation # Provide the kernel function name (e.g. rbfdot) to the argument kernfunc, # NOT the kernel function object, e.g. rbfdot(sigma = 1). # Provide ranges where the candidate lambdas and sigmas are drawn from # to the arguments lambda and sigma. # The number of pairs of candidates to select from is specified by ncoefs. ( cv2d <- ktd_cv2d(x = dat$x, y = dat$y, kernfunc = rbfdot, lambda = c(1e-3, 1e0), sigma = c(1e-3, 1e0), ncoefs = 10) ) ### Followed by fitting fit <- ktd_estimate(x = dat$x, y = dat$y, kern = rbfdot(sigma = cv2d$Best_sigma), lam1 = cv2d$Best_lambda)
ktd_estimate() estimates the coefficients of the kernel Tweedie model ktweedie and the sparse kernel Tweedie model sktweedie. The log of the expected Tweedie mean is modeled by a function in the reproducing kernel Hilbert space. The sktweedie has an integrated feature selection component that induces sparsity by applying weights on the features and penalizing the weights.
ktd_estimate( x, y, kern, lam1, rho = 1.5, ftol = 1e-08, partol = 1e-08, abstol = 0, maxit = 1e+06, sparsity = FALSE, lam2 = 0, innerpartol = 1e-06, innermaxit = 1e+06, verbose = FALSE )ktd_estimate( x, y, kern, lam1, rho = 1.5, ftol = 1e-08, partol = 1e-08, abstol = 0, maxit = 1e+06, sparsity = FALSE, lam2 = 0, innerpartol = 1e-06, innermaxit = 1e+06, verbose = FALSE )
x |
Covariate matrix. |
y |
Outcome vector (e.g. insurance cost). |
kern |
Choice of kernel. See |
lam1 |
A vector of regularization coefficients. |
rho |
The power parameter of the Tweedie model. Default is 1.5 and can take any real value between 1 and 2. |
ftol |
Stopping criterion based on objective function value. Default is 1e-8. See Details. |
partol |
Stopping criterion based on the coefficient values. Default is 1e-8. See Details. |
abstol |
Stopping criterion based on absolute value of the objective function. Default is 0. |
maxit |
Maximum number of iterations. |
sparsity |
Logical If true, the |
lam2 |
Regularization coefficient for the sparsity-inducing penalty in the |
innerpartol |
Stopping criterion for the inner loops that update kernel parameters and weights based on the coefficient values. See Details. |
innermaxit |
Maximum number of iterations for the inner loops that update kernel parameters and variable weights. See Details. |
verbose |
Logical indicating whether to show details of each update. |
ktd_estimate() stops when the absolute difference between the objective function values of the last two updates is smaller than ftol, or the sum of absolute differences between the coefficients of the last two updates is smaller than partol, or the objective function values is below abstol, before maxit is reached. For the sktweedie model, there are inner loops for the update of kernel regression coefficients and regularization weights. The innerpartol and innermaxit arguments are the counterparts of partol and maxit for the inner loops.
A list of three items.
estimates: a list containing the final objective function values and kernel Tweedie regression coefficients for each lam1.
data: stores the inputs, including the predictor matrix, the kernel function used in the fitting and lam1.
sparsity: a logical variable indicating whether the ktweedie or sktweedie is fitted.
ktd_cv, ktd_cv2d, ktd_predict, rbfdot
###### ktweedie ###### # Provide a sequence of candidate values to the argument lam1. # Provide a kernel object to the argument kern. lam1.seq <- c(1e-5, 1e-4, 1e-3, 1e-2, 1e-1, 1e0, 1e1) fit.ktd <- ktd_estimate(x = dat$x, y = dat$y, kern = rbfdot(sigma = 1e-8), lam1 = lam1.seq) ###### sktweedie ###### # Set sparsity to TRUE and a lam2 to control the level of sparsity # Decrease lam2 if "WARNING: All weights are zero..." fit.sktd <- ktd_estimate(x = dat$x, y = dat$y, kern = rbfdot(sigma = 0.1), lam1 = 5, sparsity = TRUE, lam2 = 1) # variables with fitted weight equal to 0 are not selected###### ktweedie ###### # Provide a sequence of candidate values to the argument lam1. # Provide a kernel object to the argument kern. lam1.seq <- c(1e-5, 1e-4, 1e-3, 1e-2, 1e-1, 1e0, 1e1) fit.ktd <- ktd_estimate(x = dat$x, y = dat$y, kern = rbfdot(sigma = 1e-8), lam1 = lam1.seq) ###### sktweedie ###### # Set sparsity to TRUE and a lam2 to control the level of sparsity # Decrease lam2 if "WARNING: All weights are zero..." fit.sktd <- ktd_estimate(x = dat$x, y = dat$y, kern = rbfdot(sigma = 0.1), lam1 = 5, sparsity = TRUE, lam2 = 1) # variables with fitted weight equal to 0 are not selected
ktd_predict() predicts the outcome with fitted ktweedie or sktweedie model at the user supplied new data.
ktd_predict(model, newdata, which.lam1 = 1, type = "link")ktd_predict(model, newdata, which.lam1 = 1, type = "link")
model |
Fitted model from |
newdata |
New x matrix for the prediction. If not provided, it will be the x matrix used to fit |
which.lam1 |
The index of the |
type |
The type of prediction to be made - " |
ktd_predict() uses the fitted model from ktd_estimate to estimate the mean outcome for new data points.
A list named prediction containing the vector of predicted outcomes.
ktd_estimate, ktd_cv, ktd_cv2d
# Fit a ktweedie model fit <- ktd_estimate(x = dat$x, y = dat$y, kern = rbfdot(sigma = 1e-6), lam1 = 10^(-5:1)) # Generate newx at which predictions are to be made. # The newdata should have the same dimension as the original trainig data. newx <- matrix(rnorm(10 * ncol(dat$x)), nrow = 10) pred <- ktd_predict(model = fit, newdata = newx, which.lam1 = 3, type = "link")# Fit a ktweedie model fit <- ktd_estimate(x = dat$x, y = dat$y, kern = rbfdot(sigma = 1e-6), lam1 = 10^(-5:1)) # Generate newx at which predictions are to be made. # The newdata should have the same dimension as the original trainig data. newx <- matrix(rnorm(10 * ncol(dat$x)), nrow = 10) pred <- ktd_predict(model = fit, newdata = newx, which.lam1 = 3, type = "link")