| Title: | B-Spline Kolmogorov-Arnold Networks in Native R 'torch' |
|---|---|
| Description: | Implements Kolmogorov-Arnold Networks (KAN) whose edge functions are B-splines plus a residual base activation, written in native R on top of the 'torch' package. Provides Cox-de Boor basis evaluation with explicit extrapolation policies, multi-layer networks, Adam and L-BFGS training with validation, early stopping, input and output normalization, gradient clipping and reproducible seeds, adaptive (sample-based) grid updates, grid refinement, KAN-specific sparsity regularization, pruning, edge-function visualization and model persistence. A dependency-free base R reference implementation with analytic gradients is included as a numerical oracle and as a training backend that does not need 'libtorch'. The design follows Liu et al. (2024) <doi:10.48550/arXiv.2404.19756>. |
| Authors: | Thang Le-Dinh [aut, cre, cph] |
| Maintainer: | Thang Le-Dinh <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.5.1 |
| Built: | 2026-11-18 00:14:23 UTC |
| Source: | https://github.com/cran/splineKANr |
Fraction of layer inputs outside the spline domain
kan_activation_check(state, x)kan_activation_check(state, x)
state |
A |
x |
Input matrix. |
A data frame with one row per layer: observed input range, spline domain and the fraction of input values outside the domain.
st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) kan_activation_check(st, matrix(runif(40, -1.5, 1.5), ncol = 2))st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) kan_activation_check(st, matrix(runif(40, -1.5, 1.5), ncol = 2))
For each problem and seed: simulates training and an independent test set, fits a KAN and reports out-of-sample RMSE, MAE and R^2 (against the noisy test response) plus the RMSE against the noise-free function.
kan_benchmark( problems = c("linear", "sin", "poly", "interaction", "noisy_sin"), seeds = 1:5, n_train = 500L, n_test = 1000L, hidden = NULL, ... )kan_benchmark( problems = c("linear", "sin", "poly", "interaction", "noisy_sin"), seeds = 1:5, n_train = 500L, n_test = 1000L, hidden = NULL, ... )
problems |
Problem names, see |
seeds |
Integer seeds (one replicate each). |
n_train, n_test
|
Sample sizes. |
|
Hidden widths; |
|
... |
Further arguments for |
With hidden = NULL, problems with one input use an additive
single-layer KAN and two-input problems use one hidden layer of width
4, which can approximate x1 * x2 through
(((x1 + x2)^2 - (x1 - x2)^2) / 4).
A data frame with one row per problem and seed.
kan_benchmark("linear", seeds = 1, epochs = 50, backend = "r")kan_benchmark("linear", seeds = 1, epochs = 50, backend = "r")
Differentiable with respect to x; used inside kan_layer().
kan_bspline_basis(x, grid, spline_order = 3L, right_closed = FALSE)kan_bspline_basis(x, grid, spline_order = 3L, right_closed = FALSE)
x |
Tensor of shape |
grid |
Tensor of shape |
spline_order |
Spline degree |
right_closed |
Logical, see |
Tensor of shape (batch, n_in, n_knots - k - 1).
if (torch::torch_is_installed()) { g <- torch::torch_tensor(kan_make_grid(2, 5, 3)) x <- torch::torch_rand(4, 2) * 2 - 1 kan_bspline_basis(x, g, 3)$sum(dim = 3) # ~1 on the domain }if (torch::torch_is_installed()) { g <- torch::torch_tensor(kan_make_grid(2, 5, 3)) x <- torch::torch_rand(4, 2) * 2 - 1 kan_bspline_basis(x, g, 3)$sum(dim = 3) # ~1 on the domain }
Evaluates all B-spline basis functions of degree spline_order defined by
the knot vector knots at the points x. This is the dependency-free
numerical oracle against which the torch backend is validated.
kan_bspline_basis_r(x, knots, spline_order = 3L, right_closed = FALSE)kan_bspline_basis_r(x, knots, spline_order = 3L, right_closed = FALSE)
x |
Numeric vector of evaluation points. |
knots |
Strictly increasing numeric knot vector. |
spline_order |
Spline degree |
right_closed |
Logical. If |
Intervals are half-open [t_j, t_{j+1}). Points outside
range(knots) get all-zero rows. Basis functions with zero-length
support denominators are treated as zero (0/0 := 0).
A matrix with length(x) rows and length(knots) - k - 1
columns.
g <- kan_make_grid(1, grid_size = 4, spline_order = 3)[1, ] B <- kan_bspline_basis_r(seq(-1, 1, by = 0.25), g, 3) rowSums(B) # partition of unity on [-1, 1]g <- kan_make_grid(1, grid_size = 4, spline_order = 3)[1, ] B <- kan_bspline_basis_r(seq(-1, 1, by = 0.25), g, 3) rowSums(B) # partition of unity on [-1, 1]
Uses the analytic identity
kan_bspline_deriv_r(x, knots, spline_order = 3L, right_closed = FALSE)kan_bspline_deriv_r(x, knots, spline_order = 3L, right_closed = FALSE)
x |
Numeric vector of evaluation points. |
knots |
Strictly increasing numeric knot vector. |
spline_order |
Spline degree |
right_closed |
Logical. If |
A matrix of the same shape as kan_bspline_basis_r().
g <- kan_make_grid(1, grid_size = 4, spline_order = 3)[1, ] kan_bspline_deriv_r(c(-0.3, 0.2), g, 3)g <- kan_make_grid(1, grid_size = 4, spline_order = 3)[1, ] kan_bspline_deriv_r(c(-0.3, 0.2), g, 3)
Evaluate one edge function on a grid of inputs
kan_edge_function(model, layer, i, j, from = NULL, to = NULL, n = 200L)kan_edge_function(model, layer, i, j, from = NULL, to = NULL, n = 200L)
model |
A |
layer, i, j
|
Layer index and edge |
from, to
|
Range of the (normalized) layer input. Defaults to the spline domain of the edge. |
n |
Number of evaluation points. |
A data frame with columns x, phi (full edge function),
spline and base (the weighted components).
st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) head(kan_edge_function(st, layer = 1, i = 1, j = 2))st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) head(kan_edge_function(st, layer = 1, i = 1, j = 2))
Edge importance (mean absolute edge activation)
kan_edge_importance(model, x)kan_edge_importance(model, x)
model |
A |
x |
Input sample. |
A list with one n_in x n_out matrix per layer holding
mean_n |phi_ij(x_n)|.
st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) kan_edge_importance(st, matrix(runif(40, -1, 1), ncol = 2))st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) kan_edge_importance(st, matrix(runif(40, -1, 1), ncol = 2))
Extend a domain grid by uniform knots on both sides
kan_extend_grid(grid, spline_order)kan_extend_grid(grid, spline_order)
grid |
Numeric matrix of domain knots (one row per input), with at least two strictly increasing columns. |
spline_order |
Number of knots |
A matrix with 2 * spline_order more columns than grid. The
spacing of the added knots is (max - min) / (ncol(grid) - 1), as in
pykan's extend_grid().
kan_extend_grid(matrix(c(-1, 0, 1), nrow = 1), spline_order = 2)kan_extend_grid(matrix(c(-1, 0, 1), nrow = 1), spline_order = 2)
Central finite-difference gradient of the MSE loss (base R)
kan_fd_gradient_r(state, x, y, layer, param, index, h = 1e-06)kan_fd_gradient_r(state, x, y, layer, param, index, h = 1e-06)
state |
A |
x |
Numeric input matrix. |
y |
Numeric response (vector or matrix with |
layer |
Layer index. |
param |
One of |
index |
Integer vector of linear indices into the parameter array. |
h |
Step size. |
Numeric vector of finite-difference derivatives
(L(p + h) - L(p - h)) / (2h).
st <- kan_init_state(kan_spec(c(1, 2, 1)), seed = 2) x <- matrix(seq(-0.9, 0.9, length.out = 7)) kan_fd_gradient_r(st, x, sin(3 * x), layer = 1, param = "coef", index = 1:3)st <- kan_init_state(kan_spec(c(1, 2, 1)), seed = 2) x <- matrix(seq(-0.9, 0.9, length.out = 7)) kan_fd_gradient_r(st, x, sin(3 * x), layer = 1, param = "coef", index = 1:3)
Trains a KAN of architecture c(ncol(x), hidden, ncol(y)) by minimizing
the mean squared error (plus optional KAN regularization) on normalized
data. Normalization constants are estimated on the training part only.
kan_fit( x, y, hidden = 5L, grid_size = 5L, spline_order = 3L, grid_range = c(-1, 1), extrapolate = c("linear", "none", "clamp"), base_fun = c("silu", "identity", "none"), bias = TRUE, epochs = 200L, batch_size = NULL, optimizer = c("adam", "lbfgs"), lr = NULL, weight_decay = 0, validation_split = 0.2, validation_data = NULL, patience = 30L, min_delta = 0, restore_best = TRUE, normalize_x = c("minmax", "standardize", "none"), normalize_y = TRUE, reg = kan_reg_control(), grid_update_every = 0L, grid_update_until = 50L, grid_eps = 0.02, grid_ridge = 1e-08, grad_clip = NULL, seed = NULL, backend = c("torch", "r"), dtype = "float32", init = NULL, init_control = list(), checkpoint_path = NULL, verbose = FALSE )kan_fit( x, y, hidden = 5L, grid_size = 5L, spline_order = 3L, grid_range = c(-1, 1), extrapolate = c("linear", "none", "clamp"), base_fun = c("silu", "identity", "none"), bias = TRUE, epochs = 200L, batch_size = NULL, optimizer = c("adam", "lbfgs"), lr = NULL, weight_decay = 0, validation_split = 0.2, validation_data = NULL, patience = 30L, min_delta = 0, restore_best = TRUE, normalize_x = c("minmax", "standardize", "none"), normalize_y = TRUE, reg = kan_reg_control(), grid_update_every = 0L, grid_update_until = 50L, grid_eps = 0.02, grid_ridge = 1e-08, grad_clip = NULL, seed = NULL, backend = c("torch", "r"), dtype = "float32", init = NULL, init_control = list(), checkpoint_path = NULL, verbose = FALSE )
x |
Numeric matrix or data frame of predictors. |
y |
Numeric response vector, or matrix for several outputs. |
|
Integer vector of hidden layer widths; |
|
grid_size, spline_order, grid_range, extrapolate, base_fun, bias
|
Architecture, see |
epochs |
Maximum number of passes over the training data. |
batch_size |
Mini-batch size; |
optimizer |
|
lr |
Learning rate; defaults to |
weight_decay |
L2 weight decay passed to Adam. |
validation_split |
Fraction of rows held out for validation (and
early stopping) when |
validation_data |
Optional |
patience |
Stop after this many epochs without an improvement of at
least |
min_delta |
Minimum improvement counted by early stopping. |
restore_best |
Return the parameters of the best epoch. |
normalize_x |
|
normalize_y |
Standardize the response(s) during training. |
reg |
Regularization settings from |
grid_update_every |
Update the grids from the training inputs every
this many epochs ( |
grid_update_until |
Last epoch at which grids may be updated. |
grid_eps |
Uniform/quantile blending for grid updates. |
grid_ridge |
Relative ridge penalty of the coefficient refit after a
grid update, see |
grad_clip |
Maximum gradient norm ( |
seed |
Integer seed for splitting, initialization and batching. |
backend |
|
dtype |
|
init |
Optional |
init_control |
List of initialization options passed to
|
checkpoint_path |
Optional file; the best model so far is written
there with |
verbose |
Print progress every |
An object of class kan_model with elements state, spec,
scalers, history (data frame of losses per epoch), best_epoch,
stopped_early, activation_check, settings and meta.
set.seed(1) x <- matrix(runif(300, -1, 1)) y <- sin(pi * x[, 1]) + rnorm(300, sd = 0.05) fit <- kan_fit(x, y, hidden = integer(), epochs = 30, backend = "r", seed = 1) kan_metrics(y, predict(fit, x, backend = "r"))set.seed(1) x <- matrix(runif(300, -1, 1)) y <- sin(pi * x[, 1]) + rnorm(300, sd = 0.05) fit <- kan_fit(x, y, hidden = integer(), epochs = 30, backend = "r", seed = 1) kan_metrics(y, predict(fit, x, backend = "r"))
Dependency-free reference implementation of the network defined by a
kan_state (see kan_init_state()). It is used as a numerical oracle
for the torch backend and
for analysis (edge plots, attribution, grid updates).
kan_forward_r(state, x, detail = FALSE)kan_forward_r(state, x, detail = FALSE)
state |
A |
x |
Numeric matrix with |
detail |
Logical; if |
If detail = FALSE, the output matrix. Otherwise a list with
output and layers; each layer entry has input, phi (array
n x n_in x n_out), spline, spline_deriv, base, base_deriv
and features (list of feature matrices per input).
st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) kan_forward_r(st, matrix(c(0.1, -0.4, 0.5, 0.9), ncol = 2))st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) kan_forward_r(st, matrix(c(0.1, -0.4, 0.5, 0.9), ncol = 2))
Computes L = mean((f(x) - y)^2) and its exact gradient with respect to
every trainable parameter by manual backpropagation. Used to validate
torch autograd and finite differences (Phase 2 of the validation plan).
kan_gradient_r(state, x, y)kan_gradient_r(state, x, y)
state |
A |
x |
Numeric input matrix. |
y |
Numeric response (vector or matrix with |
A list with loss, grads (per layer: coef, scale_base,
scale_sp, bias) and input_grad (gradient with respect to x).
st <- kan_init_state(kan_spec(c(1, 2, 1)), seed = 2) x <- matrix(seq(-0.9, 0.9, length.out = 7)) g <- kan_gradient_r(st, x, sin(3 * x)) g$lossst <- kan_init_state(kan_spec(c(1, 2, 1)), seed = 2) x <- matrix(seq(-0.9, 0.9, length.out = 7)) g <- kan_gradient_r(st, x, sin(3 * x)) g$loss
Draws initial parameters with base R's random number generator, so the
same seed gives the same network for every backend. The scheme follows
pykan's KANLayer: small random spline coefficients of size
noise_scale / G, base weights
scale_base_mu / sqrt(n_in) + scale_base_sigma * U(-1, 1) / sqrt(n_in)
and spline weights scale_sp / sqrt(n_in).
kan_init_state( spec, seed = NULL, noise_scale = 0.3, scale_base_mu = 0, scale_base_sigma = 1, scale_sp = 1 )kan_init_state( spec, seed = NULL, noise_scale = 0.3, scale_base_mu = 0, scale_base_sigma = 1, scale_sp = 1 )
spec |
A |
seed |
Optional integer seed passed to |
noise_scale |
Size of the initial coefficient noise. |
scale_base_mu, scale_base_sigma
|
Mean and spread of the base weights. |
scale_sp |
Initial spline weight. |
An object of class kan_state: a list with spec and layers.
Each layer holds coef (array n_in x n_out x (G + k)), scale_base,
scale_sp, mask (matrices n_in x n_out), bias (length n_out)
and grid (matrix n_in x (G + 2k + 1)).
st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) str(st$layers[[1]], max.level = 1)st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) str(st$layers[[1]], max.level = 1)
Maps (batch, n_in) to (batch, n_out). Every edge i -> j carries
phi_ij(x) = mask_ij * (scale_base_ij * b(x) + scale_sp_ij * s_ij(x))
with s_ij a B-spline with coefficients coef[i, j, ].
kan_layer( n_in, n_out, grid_size = 5L, spline_order = 3L, extrapolate = "linear", base_fun = "silu", bias = TRUE, dtype = "float32" )kan_layer( n_in, n_out, grid_size = 5L, spline_order = 3L, extrapolate = "linear", base_fun = "silu", bias = TRUE, dtype = "float32" )
n_in, n_out
|
Input and output widths. |
grid_size |
Number of grid intervals |
spline_order |
Spline degree |
extrapolate, base_fun, bias
|
See |
dtype |
|
An nn_module. Parameters: coef, scale_base, scale_sp,
bias; buffers: grid, mask. Its forward(x, detail = FALSE)
returns the output, or with detail = TRUE a list with output, phi
and spline.
if (torch::torch_is_installed()) { layer <- kan_layer(2, 3) layer(torch::torch_randn(4, 2)) }if (torch::torch_is_installed()) { layer <- kan_layer(2, 3) layer(torch::torch_randn(4, 2)) }
Creates the knot matrix used by every KAN layer: grid_size uniform
intervals on grid_range, extended by spline_order knots on each side.
kan_make_grid(n_in, grid_size = 5L, spline_order = 3L, grid_range = c(-1, 1))kan_make_grid(n_in, grid_size = 5L, spline_order = 3L, grid_range = c(-1, 1))
n_in |
Number of input dimensions (rows of the result). |
grid_size |
Number of intervals |
spline_order |
Spline degree |
grid_range |
Numeric vector |
A numeric matrix with n_in rows and 'grid_size + 2 * spline_order
1columns. Rowiholds the sorted knots for inputi'.
kan_make_grid(n_in = 2, grid_size = 5, spline_order = 3)kan_make_grid(n_in = 2, grid_size = 5, spline_order = 3)
Regression accuracy metrics
kan_metrics(y, yhat)kan_metrics(y, yhat)
y |
Observed values (vector or matrix). |
yhat |
Predicted values, same shape as |
A named numeric vector with rmse, mae, r2 (computed over
all elements) and n. r2 is NA (with a warning) when y is
constant.
kan_metrics(c(1, 2, 3), c(1.1, 1.9, 3.2))kan_metrics(c(1, 2, 3), c(1.1, 1.9, 3.2))
Build a torch module from a state
kan_module_from_state(state, dtype = "float32")kan_module_from_state(state, dtype = "float32")
state |
A |
dtype |
|
A kan_net() holding the state's numbers.
if (torch::torch_is_installed()) { st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) net <- kan_module_from_state(st) }if (torch::torch_is_installed()) { st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) net <- kan_module_from_state(st) }
Multi-layer B-spline KAN (torch module)
kan_net(spec, dtype = "float32")kan_net(spec, dtype = "float32")
spec |
A |
dtype |
|
An nn_module whose forward(x, detail = FALSE) returns the
output tensor, or with detail = TRUE a list with output and the
per-layer inputs, phi.
if (torch::torch_is_installed()) { net <- kan_net(kan_spec(c(2, 4, 1))) net(torch::torch_randn(3, 2)) }if (torch::torch_is_installed()) { net <- kan_net(kan_spec(c(2, 4, 1))) net(torch::torch_randn(3, 2)) }
Averages predictions over data while var is set to each value of a
grid: the "effect plot" used to interpret a fitted model.
kan_partial_dependence(model, data, var, n = 50L, output = 1L)kan_partial_dependence(model, data, var, n = 50L, output = 1L)
model |
A |
data |
Reference data (original scale). |
var |
Column name or index of the predictor. |
n |
Number of grid values. |
output |
Output column for multi-output models. |
A data frame of class kan_pd with columns value and yhat;
it has a plot() method.
set.seed(1) x <- matrix(runif(400, -1, 1), ncol = 2, dimnames = list(NULL, c("a", "b"))) fit <- kan_fit(x, x[, "a"]^2 + x[, "b"], hidden = integer(), epochs = 20, backend = "r", seed = 1) plot(kan_partial_dependence(fit, x, "a"))set.seed(1) x <- matrix(runif(400, -1, 1), ncol = 2, dimnames = list(NULL, c("a", "b"))) fit <- kan_fit(x, x[, "a"]^2 + x[, "b"], hidden = integer(), epochs = 20, backend = "r", seed = 1) plot(kan_partial_dependence(fit, x, "a"))
Plot the training history
kan_plot_history(model, log = TRUE)kan_plot_history(model, log = TRUE)
model |
A |
log |
Logical; log-scale the loss axis. |
model, invisibly.
set.seed(1) x <- matrix(runif(100, -1, 1)) fit <- kan_fit(x, x[, 1]^2, hidden = integer(), epochs = 10, backend = "r", seed = 1) kan_plot_history(fit)set.seed(1) x <- matrix(runif(100, -1, 1)) fit <- kan_fit(x, x[, 1]^2, hidden = integer(), epochs = 10, backend = "r", seed = 1) kan_plot_history(fit)
Nodes are drawn layer by layer; edge width and opacity are proportional
to edge importance on data (or 1 for active edges if data is
NULL).
kan_plot_network(model, data = NULL)kan_plot_network(model, data = NULL)
model |
A |
data |
Optional input sample on the original scale. |
The list of importance matrices, invisibly.
st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) m <- structure(list(state = st, spec = st$spec, scalers = list(x = kan_scaler(matrix(c(-1, 1, -1, 1), 2), "none"), y = kan_scaler(matrix(c(0, 1)), "none")), feature_names = c("x1", "x2"), response_names = "y"), class = "kan_model") kan_plot_network(m)st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) m <- structure(list(state = st, spec = st$spec, scalers = list(x = kan_scaler(matrix(c(-1, 1, -1, 1), 2), "none"), y = kan_scaler(matrix(c(0, 1)), "none")), feature_names = c("x1", "x2"), response_names = "y"), class = "kan_model") kan_plot_network(m)
Edges with importance below edge_threshold are masked. A hidden node is
removed when both its largest incoming and its largest outgoing edge
importance fall below node_threshold, mirroring pykan's
prune_node() / prune_edge() criteria. Retrain briefly afterwards
(kan_fit(..., init = pruned)) to recover accuracy.
kan_prune(model, x, node_threshold = 0.01, edge_threshold = 0.03)kan_prune(model, x, node_threshold = 0.01, edge_threshold = 0.03)
model |
A |
x |
Input sample on the original scale. |
node_threshold, edge_threshold
|
Importance thresholds. |
The pruned kan_model with element pruning describing the
removed nodes and masked edges.
set.seed(1) x <- matrix(runif(400, -1, 1), ncol = 2) fit <- kan_fit(x, sin(pi * x[, 1]), hidden = 3, epochs = 20, backend = "r", seed = 1) pruned <- kan_prune(fit, x, edge_threshold = 0.05) pruned$pruningset.seed(1) x <- matrix(runif(400, -1, 1), ncol = 2) fit <- kan_fit(x, sin(pi * x[, 1]), hidden = 3, epochs = 20, backend = "r", seed = 1) pruned <- kan_prune(fit, x, edge_threshold = 0.05) pruned$pruning
Convenience wrapper around kan_update_grid() that increases the number
of intervals while preserving the learned functions on x. Continue
training with kan_fit(..., init = refined_model).
kan_refine(model, x, grid_size, grid_eps = 0.02)kan_refine(model, x, grid_size, grid_eps = 0.02)
model |
A fitted |
x |
Input sample on the original scale. |
grid_size |
New (usually larger) grid size. |
grid_eps |
See |
The refined kan_model.
set.seed(1) x <- matrix(runif(200, -1, 1)) fit <- kan_fit(x, sin(3 * x), hidden = integer(), epochs = 20, backend = "r", verbose = FALSE) fit10 <- kan_refine(fit, x, grid_size = 10)set.seed(1) x <- matrix(runif(200, -1, 1)) fit <- kan_fit(x, sin(3 * x), hidden = integer(), epochs = 20, backend = "r", verbose = FALSE) fit10 <- kan_refine(fit, x, grid_size = 10)
Regularization settings
kan_reg_control( lambda = 0, lambda_l1 = 1, lambda_entropy = 2, lambda_coef = 0, lambda_coefdiff = 0 )kan_reg_control( lambda = 0, lambda_l1 = 1, lambda_entropy = 2, lambda_coef = 0, lambda_coefdiff = 0 )
lambda |
Overall strength; |
lambda_l1 |
Weight of the L1 norm of edge scales. |
lambda_entropy |
Weight of the edge-scale entropy. |
lambda_coef |
Weight of the L1 norm of spline coefficients. |
lambda_coefdiff |
Weight of the L1 norm of coefficient differences (a smoothness penalty). |
A list of class kan_reg.
kan_reg_control(lambda = 1e-3)kan_reg_control(lambda = 1e-3)
Regularization penalty in base R
kan_regularization_r(state, x, reg = kan_reg_control())kan_regularization_r(state, x, reg = kan_reg_control())
state |
A |
x |
Normalized input matrix. |
reg |
A |
The penalty before multiplication by reg$lambda.
st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) kan_regularization_r(st, matrix(runif(20, -1, 1), ncol = 2), kan_reg_control(lambda = 1))st <- kan_init_state(kan_spec(c(2, 3, 1)), seed = 1) kan_regularization_r(st, matrix(runif(20, -1, 1), ncol = 2), kan_reg_control(lambda = 1))
kan_save() writes the architecture, all parameters, knots,
normalization constants, training history and metadata to an .rds
file. kan_load() restores a kan_model that predicts identically.
kan_save(model, file) kan_load(file)kan_save(model, file) kan_load(file)
model |
A |
file |
Path of the |
kan_save() returns file invisibly; kan_load() returns the
kan_model.
set.seed(1) x <- matrix(runif(60, -1, 1)) fit <- kan_fit(x, x[, 1]^2, hidden = integer(), epochs = 5, backend = "r", seed = 1) f <- tempfile(fileext = ".rds") kan_save(fit, f) fit2 <- kan_load(f) all.equal(predict(fit, x, backend = "r"), predict(fit2, x, backend = "r"))set.seed(1) x <- matrix(runif(60, -1, 1)) fit <- kan_fit(x, x[, 1]^2, hidden = integer(), epochs = 5, backend = "r", seed = 1) f <- tempfile(fileext = ".rds") kan_save(fit, f) fit2 <- kan_load(f) all.equal(predict(fit, x, backend = "r"), predict(fit2, x, backend = "r"))
Apply or invert a scaler
kan_scale(x, scaler) kan_unscale(x, scaler)kan_scale(x, scaler) kan_unscale(x, scaler)
x |
Numeric matrix. |
scaler |
A |
The transformed matrix.
s <- kan_scaler(matrix(1:10, ncol = 2), "standardize") z <- kan_scale(matrix(1:10, ncol = 2), s) kan_unscale(z, s)s <- kan_scaler(matrix(1:10, ncol = 2), "standardize") z <- kan_scale(matrix(1:10, ncol = 2), s) kan_unscale(z, s)
Fit a column scaler
kan_scaler(x, method = c("minmax", "standardize", "none"), target = c(-1, 1))kan_scaler(x, method = c("minmax", "standardize", "none"), target = c(-1, 1))
x |
Numeric matrix. |
method |
|
target |
Target interval for |
A list of class kan_scaler with method, center, scale
and offset, so that z = (x - center) / scale + offset.
s <- kan_scaler(matrix(1:10, ncol = 2), "minmax") kan_scale(matrix(1:10, ncol = 2), s)s <- kan_scaler(matrix(1:10, ncol = 2), "minmax") kan_scale(matrix(1:10, ncol = 2), s)
Specify a B-spline KAN architecture
kan_spec( width, grid_size = 5L, spline_order = 3L, grid_range = c(-1, 1), extrapolate = c("linear", "none", "clamp"), base_fun = c("silu", "identity", "none"), bias = TRUE )kan_spec( width, grid_size = 5L, spline_order = 3L, grid_range = c(-1, 1), extrapolate = c("linear", "none", "clamp"), base_fun = c("silu", "identity", "none"), bias = TRUE )
width |
Integer vector of layer widths, e.g. |
grid_size |
Number of grid intervals |
spline_order |
Spline degree |
grid_range |
Initial domain |
extrapolate |
Behavior of an edge spline outside its domain:
|
base_fun |
Residual base activation |
bias |
Logical; add a trainable bias to every node of each layer. |
An object of class kan_spec.
kan_spec(c(2, 5, 1))kan_spec(c(2, 5, 1))
Synthetic regression problems
kan_synthetic( name = c("linear", "sin", "poly", "interaction", "noisy_sin", "additive"), n = 500L, seed = NULL )kan_synthetic( name = c("linear", "sin", "poly", "interaction", "noisy_sin", "additive"), n = 500L, seed = NULL )
name |
One of |
n |
Number of observations. |
seed |
Optional seed. |
A list with x (matrix), y (vector), f (noise-free target
function) and noise_sd.
d <- kan_synthetic("sin", n = 50, seed = 1) str(d)d <- kan_synthetic("sin", n = 50, seed = 1) str(d)
Re-places the knots of every edge spline at the empirical quantiles of the
current layer inputs, blended with a uniform grid:
grid = grid_eps * uniform + (1 - grid_eps) * quantile, as in pykan's
update_grid_from_samples(). The spline coefficients are then refitted
by (ridge-stabilized) least squares so that every edge keeps the same
spline values on the sample. Layers are processed in order, so each layer
sees the inputs produced by the already-updated layers before it.
kan_update_grid(state, x, grid_size = NULL, grid_eps = 0.02, ridge = 1e-08)kan_update_grid(state, x, grid_size = NULL, grid_eps = 0.02, ridge = 1e-08)
state |
A |
x |
Sample of network inputs (matrix with |
grid_size |
New number of intervals. |
grid_eps |
Weight of the uniform grid in |
ridge |
Relative ridge penalty for the coefficient refit ( |
An object of the same class as state with updated grid and
coef arrays.
st <- kan_init_state(kan_spec(c(1, 2, 1)), seed = 1) x <- matrix(rnorm(200, sd = 0.5)) st2 <- kan_update_grid(st, x) range(st2$layers[[1]]$grid)st <- kan_init_state(kan_spec(c(1, 2, 1)), seed = 1) x <- matrix(rnorm(200, sd = 0.5)) st2 <- kan_update_grid(st, x) range(st2$layers[[1]]$grid)
Run all numerical validations
kan_validate_all(backend = c("both", "r", "torch"))kan_validate_all(backend = c("both", "r", "torch"))
backend |
|
A list of data frames (basis, derivative, gradient) with an
attribute all_pass.
res <- kan_validate_all(backend = "r") attr(res, "all_pass")res <- kan_validate_all(backend = "r") attr(res, "all_pass")
Compares the basis with splines::splineDesign() on interior points and
checks non-negativity and the partition of unity on the domain.
kan_validate_basis( grid_sizes = c(3L, 5L, 10L), spline_orders = 1:3, n = 1000L, backend = c("both", "r", "torch"), tol = 1e-06 )kan_validate_basis( grid_sizes = c(3L, 5L, 10L), spline_orders = 1:3, n = 1000L, backend = c("both", "r", "torch"), tol = 1e-06 )
grid_sizes, spline_orders
|
Settings to test. |
n |
Number of evaluation points per setting. |
backend |
|
tol |
Acceptance threshold for maximum absolute errors. |
A data frame with one row per backend, grid size and order.
kan_validate_basis(backend = "r")kan_validate_basis(backend = "r")
Compares the analytic derivative (and, for torch, autograd) with
splines::splineDesign(derivs = 1) and with central finite differences at
points away from the knots. The relative error uses
|a - b| / max(|b|, atol) so values near zero are judged absolutely.
kan_validate_derivative( grid_sizes = c(3L, 5L, 10L), spline_orders = 1:3, n = 500L, backend = c("both", "r", "torch"), h = 1e-06, atol = 1e-06, tol = 1e-04 )kan_validate_derivative( grid_sizes = c(3L, 5L, 10L), spline_orders = 1:3, n = 500L, backend = c("both", "r", "torch"), h = 1e-06, atol = 1e-06, tol = 1e-04 )
grid_sizes, spline_orders
|
Settings to test. |
n |
Number of evaluation points per setting. |
backend |
|
h |
Finite-difference step. |
atol |
Absolute floor in the relative error. |
tol |
Acceptance threshold for the relative error. |
A data frame with one row per backend and setting.
kan_validate_derivative(backend = "r")kan_validate_derivative(backend = "r")
For random networks, compares the analytic base R gradient of the MSE loss with central finite differences, and (if available) with torch autograd in double precision. Inputs avoid the knots so the loss is smooth.
kan_validate_gradient( widths = list(c(1, 1), c(2, 3, 1), c(3, 4, 2, 1)), extrapolate = c("linear", "none", "clamp"), n = 40L, seed = 1L, backend = c("both", "r", "torch"), h = 1e-06, atol = 1e-06, tol = 0.001 )kan_validate_gradient( widths = list(c(1, 1), c(2, 3, 1), c(3, 4, 2, 1)), extrapolate = c("linear", "none", "clamp"), n = 40L, seed = 1L, backend = c("both", "r", "torch"), h = 1e-06, atol = 1e-06, tol = 0.001 )
widths |
List of architectures to test. |
extrapolate |
Extrapolation policies to test. |
n |
Number of samples. |
seed |
Seed. |
backend |
|
h |
Finite-difference step. |
atol |
Absolute floor in the relative error. |
tol |
Acceptance threshold for the relative error. |
A data frame with one row per architecture, policy and comparison.
kan_validate_gradient(widths = list(c(2, 3, 1)), backend = "r")kan_validate_gradient(widths = list(c(2, 3, 1)), backend = "r")
Draws a panel for every edge i -> j of layer, showing
phi_ij (solid) with its spline (dashed) and base (dotted) components,
on the normalized input scale of that layer. Masked edges are gray.
## S3 method for class 'kan_model' plot(x, layer = 1L, data = NULL, n = 200L, ...)## S3 method for class 'kan_model' plot(x, layer = 1L, data = NULL, n = 200L, ...)
x |
A |
layer |
Layer to draw. |
data |
Optional input sample (original scale) used to set the plotted range to the observed layer inputs and to add a rug. |
n |
Points per curve. |
... |
Further arguments passed to |
x, invisibly.
st <- kan_init_state(kan_spec(c(2, 2, 1)), seed = 1) m <- structure(list(state = st, spec = st$spec, scalers = list(x = kan_scaler(matrix(c(-1, 1, -1, 1), 2), "none"), y = kan_scaler(matrix(c(0, 1)), "none")), history = data.frame(), feature_names = c("x1", "x2"), response_names = "y"), class = "kan_model") plot(m, layer = 1)st <- kan_init_state(kan_spec(c(2, 2, 1)), seed = 1) m <- structure(list(state = st, spec = st$spec, scalers = list(x = kan_scaler(matrix(c(-1, 1, -1, 1), 2), "none"), y = kan_scaler(matrix(c(0, 1)), "none")), history = data.frame(), feature_names = c("x1", "x2"), response_names = "y"), class = "kan_model") plot(m, layer = 1)
Predict from a fitted KAN
## S3 method for class 'kan_model' predict(object, newdata, backend = c("auto", "torch", "r"), ...)## S3 method for class 'kan_model' predict(object, newdata, backend = c("auto", "torch", "r"), ...)
object |
A |
newdata |
Numeric matrix or data frame with the training predictors. If it has column names, they are matched to the training names. |
backend |
|
... |
Unused. |
A numeric vector (single output) or matrix of predictions on the original response scale.
set.seed(1) x <- matrix(runif(100, -1, 1)) fit <- kan_fit(x, x[, 1]^2, hidden = integer(), epochs = 10, backend = "r", seed = 1) head(predict(fit, x, backend = "r"))set.seed(1) x <- matrix(runif(100, -1, 1)) fit <- kan_fit(x, x[, 1]^2, hidden = integer(), epochs = 10, backend = "r", seed = 1) head(predict(fit, x, backend = "r"))