| Title: | Kolmogorov-Arnold Networks with Interchangeable Edge Bases Using 'torch' |
|---|---|
| Description: | Implements Kolmogorov-Arnold Networks (KAN) in R on top of the 'torch' package (no 'Python' needed) with one network engine and interchangeable edge bases: Chebyshev, Legendre and Jacobi polynomials, Gaussian radial basis functions (FastKAN), Mexican hat wavelets (Bozorgasl and Chen (2024) <doi:10.48550/arXiv.2405.12832>), safe Pade rational functions (Molina, Schramowski and Kersting (2020) <doi:10.48550/arXiv.1907.06732>), a hybrid of SiLU, Chebyshev and radial basis parts, and squared rectified linear unit bumps (Qiu et al. (2024) <doi:10.48550/arXiv.2406.02075>). Each layer may use a different basis, and further bases can be registered by other packages. Besides tabular regression, the same layers power graph KANs for node regression (neighbourhood aggregation with the normalized adjacency matrix, as in Kiamari, Kiamari and Krishnamachari (2024) <doi:10.48550/arXiv.2406.06470>) and lag-window KANs for one-step time series forecasting. Provides multi-layer networks with Adam training, training-only standardization, validation-based early stopping, reproducible seeds, edge-function evaluation and plots, and model persistence. A base R reference implementation with analytic gradients serves as a numerical oracle and as a training backend that does not need 'libtorch'. The network follows Liu et al. (2024) <doi:10.48550/arXiv.2404.19756>. |
| Authors: | Thang Le-Dinh [aut, cre, cph] |
| Maintainer: | Thang Le-Dinh <[email protected]> |
| License: | GPL-3 |
| Version: | 0.1.0 |
| Built: | 2026-11-20 21:36:06 UTC |
| Source: | https://github.com/cran/thangldKANr |
chebyshev_basis() evaluates by the
recurrence , ,
.
chebyshev_basis_deriv() returns with the
Chebyshev polynomials of the second kind , which has no
singularity at (unlike the form).
chebyshev_basis(z, degree) chebyshev_basis_deriv(z, degree)chebyshev_basis(z, degree) chebyshev_basis_deriv(z, degree)
z |
Numeric vector. |
degree |
Maximum degree |
A length(z) by degree + 1 matrix (column k + 1 = degree k).
chebyshev_basis(c(-1, 0, 0.5, 1), 3) chebyshev_basis_deriv(c(-1, 0, 1), 3)chebyshev_basis(c(-1, 0, 0.5, 1), 3) chebyshev_basis_deriv(c(-1, 0, 1), 3)
hybrid_basis() returns the features one edge combines: , the Chebyshev polynomials of
(as in ChebyKAN) and the Gaussian radial basis
functions on a fixed grid (as in FastKAN).
hybrid_basis_deriv() returns their derivatives with respect to
, using (no
singularity at ) and the chain rule through .
hybrid_basis( x, degree = 3L, num_grids = 8L, grid_min = -2, grid_max = 2, denominator = NULL ) hybrid_basis_deriv( x, degree = 3L, num_grids = 8L, grid_min = -2, grid_max = 2, denominator = NULL )hybrid_basis( x, degree = 3L, num_grids = 8L, grid_min = -2, grid_max = 2, denominator = NULL ) hybrid_basis_deriv( x, degree = 3L, num_grids = 8L, grid_min = -2, grid_max = 2, denominator = NULL )
x |
Numeric vector. |
degree |
Chebyshev degree |
num_grids |
Number of RBF centers |
grid_min, grid_max
|
Range of the RBF centers. |
denominator |
RBF width |
A matrix with columns silu, T0..TK and rbf1..rbfG.
hybrid_basis(c(-1, 0, 1), degree = 2, num_grids = 3)hybrid_basis(c(-1, 0, 1), degree = 2, num_grids = 3)
For every edge of one layer, evaluates the SiLU, Chebyshev and RBF parts
at the layer inputs produced by x and reports the standard deviation of
each part and its share of the summed standard deviations. This is a
description of the fitted coefficients (the parts are not orthogonal; see
thangld_kan()), not a variable-importance or causal measure.
hybrid_kan_components(model, x, layer = 1L)hybrid_kan_components(model, x, layer = 1L)
model |
A (fitted) |
x |
Predictor matrix or data frame on the original scale (columns as in the fit). |
layer |
Layer index. |
A data frame with one row per edge and part: input, output,
part, sd, share.
set.seed(1) x <- matrix(runif(200, -2, 2), 100, 2, dimnames = list(NULL, c("a", "b"))) fit <- thangld_kan_fit(thangld_kan(basis = "hybrid", c(2, 1), seed = 1), x, sin(x[, 1]) + x[, 2], epochs = 20, backend = "r", seed = 1) hybrid_kan_components(fit, x)set.seed(1) x <- matrix(runif(200, -2, 2), 100, 2, dimnames = list(NULL, c("a", "b"))) fit <- thangld_kan_fit(thangld_kan(basis = "hybrid", c(2, 1), seed = 1), x, sin(x[, 1]) + x[, 2], epochs = 20, backend = "r", seed = 1) hybrid_kan_components(fit, x)
jacobi_basis() evaluates by the three-term recurrence
with and
(standard normalization, ).
jacobi_basis_deriv() uses
. Special cases:
gives Legendre polynomials; gives Chebyshev
polynomials up to a constant factor per degree.
jacobi_basis(z, degree, alpha = 1, beta = 1) jacobi_basis_deriv(z, degree, alpha = 1, beta = 1)jacobi_basis(z, degree, alpha = 1, beta = 1) jacobi_basis_deriv(z, degree, alpha = 1, beta = 1)
z |
Numeric vector. |
degree |
Maximum degree |
alpha, beta
|
Jacobi parameters, both |
A length(z) by degree + 1 matrix.
jacobi_basis(c(-1, 0, 1), 3, alpha = 1, beta = 1)jacobi_basis(c(-1, 0, 1), 3, alpha = 1, beta = 1)
List the available edge bases
kan_bases()kan_bases()
A data frame with one row per basis: name (the value for the
basis argument of thangld_kan()), label, builtin and arguments
(the basis-specific arguments accepted by thangld_kan()).
kan_bases()kan_bases()
Bases used by the layers of a model
kan_layer_bases(model)kan_layer_bases(model)
model |
A |
Character vector with one basis name per layer.
kan_layer_bases(thangld_kan(c(2, 3, 1), basis = c("rbf", "jacobi"), seed = 1))kan_layer_bases(thangld_kan(c(2, 3, 1), basis = c("rbf", "jacobi"), seed = 1))
legendre_basis() evaluates by Bonnet's
recurrence , ,
.
legendre_basis_deriv() returns from
, , ,
which has no singularity at .
legendre_basis(z, degree) legendre_basis_deriv(z, degree)legendre_basis(z, degree) legendre_basis_deriv(z, degree)
z |
Numeric vector. |
degree |
Maximum degree |
Base R only; this is the numerical reference for the torch layer.
The polynomials are orthogonal on with unit weight; that
does not make the features orthogonal on a sample.
A length(z) by degree + 1 matrix (column k + 1 = degree k).
legendre_basis(c(-1, 0, 0.5, 1), 3) legendre_basis_deriv(c(-1, 0, 1), 3)legendre_basis(c(-1, 0, 0.5, 1), 3) legendre_basis_deriv(c(-1, 0, 1), 3)
mexican_hat() evaluates with
, so that and
; mexican_hat_deriv() returns
.
mexican_hat(u) mexican_hat_deriv(u)mexican_hat(u) mexican_hat_deriv(u)
u |
Numeric vector. |
On an edge the wavelet is used as without the
factor of the continuous wavelet transform (the same
convention as Wav-KAN); the learned weight absorbs the amplitude.
Wav-KAN codes the wavelet with the opposite sign,
, which corresponds to .
Numeric vector.
mexican_hat(c(-1, 0, 1)) stats::integrate(function(u) mexican_hat(u)^2, -Inf, Inf)$valuemexican_hat(c(-1, 0, 1)) stats::integrate(function(u) mexican_hat(u)^2, -Inf, Inf)$value
type = "edge" draws one learned edge function (see thangld_kan_edge());
type = "loss" draws the loss history of a fitted model.
## S3 method for class 'thangld_kan' plot( x, type = c("edge", "loss"), layer = 1L, input = 1L, output = 1L, scale = c("model", "original"), ... )## S3 method for class 'thangld_kan' plot( x, type = c("edge", "loss"), layer = 1L, input = 1L, output = 1L, scale = c("model", "original"), ... )
x |
A model. |
type |
|
layer, input, output, scale
|
Passed to |
... |
Graphical parameters passed to |
x, invisibly.
plot(thangld_kan(c(1, 1), seed = 1))plot(thangld_kan(c(1, 1), seed = 1))
Predict node targets
## S3 method for class 'thangld_graph_kan' predict(object, newdata = NULL, graph = NULL, backend = c("r", "torch"), ...)## S3 method for class 'thangld_graph_kan' predict(object, newdata = NULL, graph = NULL, backend = c("r", "torch"), ...)
object |
A fitted |
newdata |
|
graph |
Graph for |
backend |
|
... |
Unused. |
Numeric vector, one prediction per node, on the original scale.
Applies the stored training preprocessing, evaluates the network and
returns predictions on the original scale of the response. Columns of
newdata are matched by name when the model was trained with names
(order does not matter, extra columns are ignored, missing ones are an
error).
## S3 method for class 'thangld_kan' predict( object, newdata, backend = c("r", "torch"), dtype = c("float64", "float32"), ... )## S3 method for class 'thangld_kan' predict( object, newdata, backend = c("r", "torch"), dtype = c("float64", "float32"), ... )
object |
A fitted model. |
newdata |
Numeric matrix, data frame or vector of predictors. |
backend |
|
dtype |
Floating point type for |
... |
Unused. |
Numeric vector of predictions.
set.seed(2) x <- matrix(runif(100, -1, 1), ncol = 1, dimnames = list(NULL, "x1")) fit <- thangld_kan_fit(thangld_kan(c(1, 1), seed = 1), x, x[, 1]^3, epochs = 20, lr = 0.02, backend = "r") predict(fit, data.frame(x1 = c(-0.5, 0, 0.5)))set.seed(2) x <- matrix(runif(100, -1, 1), ncol = 1, dimnames = list(NULL, "x1")) fit <- thangld_kan_fit(thangld_kan(c(1, 1), seed = 1), x, x[, 1]^3, epochs = 20, lr = 0.02, backend = "r") predict(fit, data.frame(x1 = c(-0.5, 0, 0.5)))
Forecasts the value following the end of newdata (default: the series
used in fitting) with the fitted network and the stored training
standardization. Only one-step forecasts are supported.
## S3 method for class 'thangld_temporal_kan' predict(object, newdata = NULL, h = 1L, backend = c("r", "torch"), ...)## S3 method for class 'thangld_temporal_kan' predict(object, newdata = NULL, h = 1L, backend = c("r", "torch"), ...)
object |
A fitted |
newdata |
Optional series (history up to the forecast origin); at
least |
h |
Forecast horizon; must be 1. |
backend |
|
... |
Unused. |
A single forecast on the original scale of the series.
y <- as.numeric(datasets::lynx) fit <- thangld_temporal_kan_fit(thangld_temporal_kan(1:3, seed = 1), log(y[1:100]), epochs = 30, lr = 0.02, backend = "r", seed = 1) c(forecast = predict(fit, log(y[1:100])), actual = log(y[101]))y <- as.numeric(datasets::lynx) fit <- thangld_temporal_kan_fit(thangld_temporal_kan(1:3, seed = 1), log(y[1:100]), epochs = 30, lr = 0.02, backend = "r", seed = 1) c(forecast = predict(fit, log(y[1:100])), actual = log(y[101]))
Evaluates the "safe" Pade form used by rational activation functions (Molina, Schramowski and Kersting, 2020, version B) and by the group rational KAN of Yang and Wang (2025):
The denominator is at least 1, so has no poles. For
the function grows linearly in .
rational_deriv() returns (with ,
0 at , as in torch).
rational(x, num, den) rational_deriv(x, num, den)rational(x, num, den) rational_deriv(x, num, den)
x |
Numeric vector. |
num |
Numerator coefficients |
den |
Denominator coefficients |
Numeric vector of length(x).
rational(c(-2, 0, 2), num = c(0, 1, 0, 0.5), den = c(0, 0.5)) rational_deriv(c(-2, 0, 2), num = c(0, 1, 0, 0.5), den = c(0, 0.5))rational(c(-2, 0, 2), num = c(0, 1, 0, 0.5), den = c(0, 0.5)) rational_deriv(c(-2, 0, 2), num = c(0, 1, 0, 0.5), den = c(0, 0.5))
rbf_basis() evaluates for centers
centers and width (denominator) d; rbf_basis_deriv() returns
. rbf_grid() returns the
FastKAN default grid: num_grids equally spaced centers on
[grid_min, grid_max] and d = (grid_max - grid_min) / (num_grids - 1).
rbf_basis(u, centers, d) rbf_basis_deriv(u, centers, d) rbf_grid(grid_min = -2, grid_max = 2, num_grids = 8L)rbf_basis(u, centers, d) rbf_basis_deriv(u, centers, d) rbf_grid(grid_min = -2, grid_max = 2, num_grids = 8L)
u |
Numeric vector. |
centers |
Numeric vector of centers. |
d |
Positive width. |
grid_min, grid_max, num_grids
|
Grid definition. |
rbf_basis()/rbf_basis_deriv(): a length(u) by
length(centers) matrix. rbf_grid(): a list with centers and d.
g <- rbf_grid(-2, 2, 8) g round(rbf_basis(c(-2, 0, 2), g$centers, g$d), 3)g <- rbf_grid(-2, 2, 8) g round(rbf_basis(c(-2, 0, 2), g$centers, g$d), 3)
Adds an edge type to the registry so that thangld_kan(basis = name)
builds networks with it. Every other function of the package (fitting
with both backends, prediction, edge plots, save/load and
thangld_kan_validate()) then works with the new basis unchanged. This is
how other packages plug in their basis (for example a B-spline basis as
"bspline"), typically from their .onLoad().
register_kan_basis(name, impl, overwrite = FALSE) unregister_kan_basis(name)register_kan_basis(name, impl, overwrite = FALSE) unregister_kan_basis(name)
name |
Basis name: lower-case letters, digits and |
impl |
Named list of functions described in Details. |
overwrite |
Replace an earlier registration of |
impl is a named list of functions. Required:
layer_specs(widths, ...)Checks the basis arguments and returns
a list with one specification (a list) per layer. Its formal
arguments other than widths are the arguments thangld_kan()
accepts for this basis.
describe(spec)One-line description of a layer.
param_dims(p, q, spec)Named list of parameter array dimensions
for a layer with p inputs and q outputs.
init(p, q, spec)Named list of initial parameter arrays (uses
the R random number generator; the seed is set by thangld_kan()).
check_params(par, spec)Error if the parameters left their
domain; otherwise invisible TRUE.
forward_r(h, par, spec)Base R forward pass of an n x p
matrix: list(out = n x q matrix, cache = anything).
backward_r(cache, g, par, spec)Analytic backward pass given
the n x q gradient of the loss with respect to the layer output:
list(grads = named list like par, dh = n x p matrix).
forward_torch(h, par, spec)The same forward pass with 'torch'
tensors (par is a named list of tensors); must be differentiable by
autograd. Write tensor * scalar, not scalar * tensor, so that R
doubles are not rounded to float32.
edge_r(x, par, spec, i, j)Data frame describing edge
at inputs x, with the edge output in column phi.
Optional, used by thangld_kan_validate() (a missing one is reported as a
skipped check, never as passed): naive_forward(h, par, spec) (an
independent loop implementation), basis_checks() (list of
list(check, error, tol)), validation_fixture() (returns
list(model, x, y)), and label.
Models store only the basis name, so a model with a registered basis can be reloaded only after the basis is registered again.
register_kan_basis() returns name invisibly;
unregister_kan_basis() returns TRUE invisibly if a basis was
removed.
# a toy linear basis: phi(x) = w * x + b on every edge lin <- list( label = "linear (toy)", layer_specs = function(widths) rep(list(list()), length(widths) - 1L), describe = function(spec) "linear", param_dims = function(p, q, spec) list(w = c(p, q)), init = function(p, q, spec) list(w = matrix(rnorm(p * q, sd = 0.3), p, q)), check_params = function(par, spec) invisible(TRUE), forward_r = function(h, par, spec) list(out = h %*% par$w, cache = h), backward_r = function(cache, g, par, spec) list(grads = list(w = crossprod(cache, g)), dh = g %*% t(par$w)), forward_torch = function(h, par, spec) torch::torch_matmul(h, par$w), edge_r = function(x, par, spec, i, j) data.frame(phi = par$w[i, j] * x)) register_kan_basis("toy_linear", lin) m <- thangld_kan(c(2, 3, 1), basis = c("toy_linear", "chebyshev"), seed = 1) m unregister_kan_basis("toy_linear")# a toy linear basis: phi(x) = w * x + b on every edge lin <- list( label = "linear (toy)", layer_specs = function(widths) rep(list(list()), length(widths) - 1L), describe = function(spec) "linear", param_dims = function(p, q, spec) list(w = c(p, q)), init = function(p, q, spec) list(w = matrix(rnorm(p * q, sd = 0.3), p, q)), check_params = function(par, spec) invisible(TRUE), forward_r = function(h, par, spec) list(out = h %*% par$w, cache = h), backward_r = function(cache, g, par, spec) list(grads = list(w = crossprod(cache, g)), dh = g %*% t(par$w)), forward_torch = function(h, par, spec) torch::torch_matmul(h, par$w), edge_r = function(x, par, spec, i, j) data.frame(phi = par$w[i, j] * x)) register_kan_basis("toy_linear", lin) m <- thangld_kan(c(2, 3, 1), basis = c("toy_linear", "chebyshev"), seed = 1) m unregister_kan_basis("toy_linear")
relu_basis() evaluates the ReLU-KAN basis functions
on an equally spaced grid over grid_range = c(a, b) with :
, and
, so each function is a smooth bump with
support and peak value 1 at the center. With
grid_range = c(0, 1) the grid and are those of the original
ReLU-KAN layer (Qiu et al. 2024). relu_basis_deriv() returns
with ,
.
relu_basis(x, grid_size = 5L, k = 3L, grid_range = c(-2, 2)) relu_basis_deriv(x, grid_size = 5L, k = 3L, grid_range = c(-2, 2))relu_basis(x, grid_size = 5L, k = 3L, grid_range = c(-2, 2)) relu_basis_deriv(x, grid_size = 5L, k = 3L, grid_range = c(-2, 2))
x |
Numeric vector. |
grid_size |
Number of grid intervals |
k |
Support width in grid intervals minus one, |
grid_range |
Numeric |
A length(x) by grid_size + k matrix.
relu_basis(c(-1, 0, 1), grid_size = 5, k = 3)relu_basis(c(-1, 0, 1), grid_size = 5, k = 3)
A graph KAN layer first averages node states over neighborhoods with the
normalized adjacency matrix (thangld_graph_kan_adjacency()) and then
applies a Kolmogorov-Arnold layer to every node,
which is the "aggregate, then KAN" architecture of graph KANs (Kiamari,
Kiamari and Krishnamachari, 2024; the GraphKAN prototype of Zhang, 2024,
uses the same order with Fourier edges). By default edges are Chebyshev
expansions (layout c(in, out, K + 1), no bias, degree 3); any basis of
thangld_kan() can be used instead (basis, ..., options). With
L layers a node's prediction depends on its L-hop neighborhood. With
no graph (identity ) the model reduces to a Chebyshev KAN on
the node's own features, which is the natural baseline.
thangld_graph_kan( widths, basis = "chebyshev", ..., self_branch = FALSE, options = NULL, seed = NULL )thangld_graph_kan( widths, basis = "chebyshev", ..., self_branch = FALSE, options = NULL, seed = NULL )
widths |
Integer layer widths; the first is the number of node features, the last must be 1. |
basis |
Edge basis, one value or one per layer (see |
... |
Named basis-specific arguments, as in |
self_branch |
Logical; feed |
options |
|
seed |
Optional seed for the initialization. |
With self_branch = TRUE every layer receives the node's own state next
to its neighborhood mean,
so a layer has the inputs (the "concatenate self and
neighbors" update of GraphSAGE, Hamilton, Ying and Leskovec, 2017). The
network can then use the node's own features, its one-hop mean or both
in any layer, so a two-layer model can still fit a target that depends on
one hop only, or on the node's own features only. The default (FALSE)
is the plain "aggregate, then KAN" architecture above.
Task: transductive node regression (one numeric target per node, one graph, full-batch training).
An object of class thangld_graph_kan.
Kiamari, M., Kiamari, M. and Krishnamachari, B. (2024). GKAN: Graph Kolmogorov-Arnold Networks. doi:10.48550/arXiv.2406.06470
Hamilton, W. L., Ying, R. and Leskovec, J. (2017). Inductive representation learning on large graphs. Advances in Neural Information Processing Systems 30. doi:10.48550/arXiv.1706.02216
Kipf, T. N. and Welling, M. (2017). Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations. doi:10.48550/arXiv.1609.02907
thangld_graph_kan(c(2, 4, 1), seed = 1) thangld_graph_kan(c(2, 4, 1), self_branch = TRUE, seed = 1) thangld_graph_kan(c(2, 4, 1), basis = "rbf", seed = 1)thangld_graph_kan(c(2, 4, 1), seed = 1) thangld_graph_kan(c(2, 4, 1), self_branch = TRUE, seed = 1) thangld_graph_kan(c(2, 4, 1), basis = "rbf", seed = 1)
Returns (Kipf and Welling, 2017),
where is the degree matrix of ; with
self_loops = FALSE the identity is not added. This is the propagation
operator of every graph KAN layer.
thangld_graph_kan_adjacency(graph, n = NULL, self_loops = TRUE)thangld_graph_kan_adjacency(graph, n = NULL, self_loops = TRUE)
graph |
Either an |
n |
Number of nodes (needed for an edge list when the highest-index nodes are isolated; default: the largest index). |
self_loops |
Add |
A dense matrix.
edges <- cbind(c(1, 2, 3), c(2, 3, 4)) thangld_graph_kan_adjacency(edges)edges <- cbind(c(1, 2, 3), c(2, 3, 4)) thangld_graph_kan_adjacency(edges)
Transductive node regression on one graph: all nodes and their features are
propagated, the loss is the mean squared error over the train nodes, and
the val nodes are used only for early stopping (the epoch with the lowest
validation loss is restored). Labels of other nodes are never read; they may
be NA. Feature and target standardization constants are estimated on the
training nodes only. Training is full batch: one Adam step per epoch, after
which the training and validation losses are recorded.
thangld_graph_kan_fit( model, x, graph, y, train, val = NULL, epochs = 300L, lr = 0.01, betas = c(0.9, 0.999), eps = 1e-08, patience = NULL, min_delta = 0, self_loops = TRUE, backend = c("torch", "r"), dtype = c("float64", "float32"), verbose = FALSE )thangld_graph_kan_fit( model, x, graph, y, train, val = NULL, epochs = 300L, lr = 0.01, betas = c(0.9, 0.999), eps = 1e-08, patience = NULL, min_delta = 0, self_loops = TRUE, backend = c("torch", "r"), dtype = c("float64", "float32"), verbose = FALSE )
model |
A |
x |
Node feature matrix or data frame ( |
graph |
Adjacency matrix or edge list (see |
y |
Numeric target, one value per node ( |
train, val
|
Node indices (or logical vectors) of training and validation nodes; they must not overlap. |
epochs |
Maximum number of epochs (= Adam steps). |
lr, betas, eps
|
Adam settings (as |
patience |
Early-stopping patience in epochs ( |
min_delta |
Minimum validation improvement. |
self_loops |
Passed to |
backend |
|
dtype |
|
verbose |
Print progress. |
The fitted thangld_graph_kan with history, preprocess, a_hat, x
and train information.
g <- thangld_graph_kan_simulate(60, seed = 1) y <- sin(g$x[, 1]) + drop(thangld_graph_kan_adjacency(g$adjacency) %*% g$x[, 2]) fit <- thangld_graph_kan_fit(thangld_graph_kan(c(2, 3, 1), seed = 1), g$x, g$adjacency, y, train = 1:40, val = 41:50, epochs = 50, lr = 0.02, backend = "r") fitg <- thangld_graph_kan_simulate(60, seed = 1) y <- sin(g$x[, 1]) + drop(thangld_graph_kan_adjacency(g$adjacency) %*% g$x[, 2]) fit <- thangld_graph_kan_fit(thangld_graph_kan(c(2, 3, 1), seed = 1), g$x, g$adjacency, y, train = 1:40, val = 41:50, epochs = 50, lr = 0.02, backend = "r") fit
thangld_graph_kan_forward_r() evaluates the network on standardized features
x with propagation matrix a_hat. thangld_graph_kan_gradient_r() returns the
masked mean squared error over nodes and its analytic gradients with
respect to every coefficient and to x; thangld_graph_kan_fd_gradient_r() the
same by central differences (order 2 or 4), for checking.
thangld_graph_kan_forward_r(model, x, a_hat, layers = FALSE) thangld_graph_kan_gradient_r(model, x, a_hat, y, nodes) thangld_graph_kan_fd_gradient_r( model, x, a_hat, y, nodes, step = 1e-05, order = 2L )thangld_graph_kan_forward_r(model, x, a_hat, layers = FALSE) thangld_graph_kan_gradient_r(model, x, a_hat, y, nodes) thangld_graph_kan_fd_gradient_r( model, x, a_hat, y, nodes, step = 1e-05, order = 2L )
model |
A |
x |
Standardized node features. |
a_hat |
Propagation matrix ( |
layers |
Return every layer's output. |
y |
Target (model scale). |
nodes |
Indices of the nodes in the loss. |
step, order
|
Finite-difference step and order. |
See Description.
g <- thangld_graph_kan_simulate(8, seed = 2) m <- thangld_graph_kan(c(2, 2, 1), seed = 1) a <- thangld_graph_kan_adjacency(g$adjacency) thangld_graph_kan_forward_r(m, g$x, a) thangld_graph_kan_gradient_r(m, g$x, a, rnorm(8), 1:5)$lossg <- thangld_graph_kan_simulate(8, seed = 2) m <- thangld_graph_kan(c(2, 2, 1), seed = 1) a <- thangld_graph_kan_adjacency(g$adjacency) thangld_graph_kan_forward_r(m, g$x, a) thangld_graph_kan_gradient_r(m, g$x, a, rnorm(8), 1:5)$loss
Regression metrics
thangld_graph_kan_metrics(truth, estimate)thangld_graph_kan_metrics(truth, estimate)
truth, estimate
|
Numeric vectors of equal length. |
Named vector rmse, mae, r2.
thangld_graph_kan_metrics(c(1, 2, 3), c(1.1, 1.9, 3.2))thangld_graph_kan_metrics(c(1, 2, 3), c(1.1, 1.9, 3.2))
Models are plain R objects (no torch pointers), stored with saveRDS()
together with a format tag and the package and R versions.
thangld_graph_kan_save(model, file) thangld_graph_kan_load(file)thangld_graph_kan_save(model, file) thangld_graph_kan_load(file)
model |
A |
file |
Path. |
thangld_graph_kan_save(): file, invisibly. thangld_graph_kan_load(): the model.
f <- tempfile(fileext = ".rds") thangld_graph_kan_save(thangld_graph_kan(c(2, 1), seed = 1), f) thangld_graph_kan_load(f)f <- tempfile(fileext = ".rds") thangld_graph_kan_save(thangld_graph_kan(c(2, 1), seed = 1), f) thangld_graph_kan_load(f)
Nodes are split into blocks equal communities; an edge appears with
probability p_in inside a community and p_out between communities.
Features are independent plus a community mean shift of
shift. Used in examples, tests and benchmarks (no external data).
thangld_graph_kan_simulate( n = 150L, p = 2L, blocks = 3L, p_in = 0.1, p_out = 0.01, shift = 1, seed = NULL )thangld_graph_kan_simulate( n = 150L, p = 2L, blocks = 3L, p_in = 0.1, p_out = 0.01, shift = 1, seed = NULL )
n |
Number of nodes. |
p |
Number of features. |
blocks |
Number of communities. |
p_in, p_out
|
Edge probabilities. |
shift |
Size of the community mean shift of the features. |
seed |
Optional seed (the caller's random number stream is restored). |
A list with x (), adjacency (0/1 matrix),
edges (two-column matrix) and block (community labels).
g <- thangld_graph_kan_simulate(30, seed = 1) dim(g$adjacency)g <- thangld_graph_kan_simulate(30, seed = 1) dim(g$adjacency)
Checks the normalized adjacency against an explicit loop, the Chebyshev
forward pass against an independent loop implementation (explicit
neighbor sums and ), analytic gradients of the
masked loss against fourth-order central differences (), for both the plain and the self-branch architecture,
permutation equivariance, the identity-graph
reduction, save/load, and, with the torch runtime, torch forward, autograd
and full-batch Adam training against the base R oracle. These checks use
Chebyshev edges; for every other built-in basis the gradients (and, with
torch, the torch forward pass and autograd) of a self-branch graph model
are also checked. Torch checks that
cannot run are reported with pass = NA and counted in n_skipped; they
are never counted as passed.
thangld_graph_kan_validate(backend = c("both", "r", "torch"))thangld_graph_kan_validate(backend = c("both", "r", "torch"))
backend |
|
A data frame (group, check, error, tol, pass) with
attributes all_pass and n_skipped.
v <- thangld_graph_kan_validate("r") v[, c("check", "pass")]v <- thangld_graph_kan_validate("r") v[, c("check", "pass")]
Each layer computes , where
the edge functions are expansions in the chosen basis.
basis may differ between layers. The built-in bases follow published KAN
variants (see kan_bases()); other
bases are added with register_kan_basis().
thangld_kan(widths, basis = "chebyshev", ..., options = NULL, seed = NULL)thangld_kan(widths, basis = "chebyshev", ..., options = NULL, seed = NULL)
widths |
Integer layer widths; the last must be 1. |
basis |
Basis name, one value or one per layer (see |
... |
Named basis-specific arguments (see Details). |
options |
|
seed |
Optional seed for the initialization. |
Basis-specific arguments are passed by name in .... A per-layer
argument (marked "per layer") may be one value or one value per layer of
the network; in a mixed network the values for layers of another basis
are ignored. An argument that no selected basis uses is an error.
options gives basis-specific values that override ..., for example
options = list(jacobi = list(degree = 3)).
"chebyshev". degree (per
layer, default 4), init_sd (NULL: ).
Same layout and initialization as ChebyKAN.
"rbf"FastKAN: optional layer normalization, Gaussian radial
basis functions on a fixed grid and a SiLU base branch. num_grids
(8), grid_min (-2), grid_max (2), denominator (NULL: grid
spacing), layernorm (TRUE), base_update (TRUE), ln_eps
(1e-5), spline_init_sd (0.1).
"legendre". degree (per
layer, 4), init_sd (NULL).
"wavelet"Wav-KAN with the Mexican-hat wavelet, trainable
translation and scale per edge. scale_param ("log" or "raw"),
bias (TRUE).
"rational"Safe Pade form (no poles).
num_degree (per layer, 3), den_degree (per layer, 2, not
above num_degree), init_sd (NULL), den_init_sd (0.1).
"hybrid"Sum of SiLU, Chebyshev and RBF parts. bases (subset
of "silu", "chebyshev", "rbf"), degree (per layer, 3),
num_grids (per layer, 8), grid_min (-2), grid_max (2),
denominator (NULL), rbf_init_sd (0.1). See
hybrid_kan_components().
"jacobi".
degree (per layer, 4), alpha (1), beta (1), init_sd (NULL).
"relu"ReLU-KAN squared ReLU bumps with trainable phases.
grid_size (per layer, 5), k (per layer, 3), grid_range
(c(-2, 2)).
An object of class thangld_kan.
thangld_kan(c(2, 3, 1), basis = "chebyshev", degree = 4, seed = 1) thangld_kan(c(2, 3, 1), basis = "relu", grid_size = 4, seed = 1) # one basis per layer thangld_kan(c(2, 3, 1), basis = c("rbf", "legendre"), degree = 3, seed = 1)thangld_kan(c(2, 3, 1), basis = "chebyshev", degree = 4, seed = 1) thangld_kan(c(2, 3, 1), basis = "relu", grid_size = 4, seed = 1) # one basis per layer thangld_kan(c(2, 3, 1), basis = c("rbf", "legendre"), degree = 3, seed = 1)
Returns the edge function of layer layer from input node input to
output node output on a grid. An edge function is one component of the
fitted network: in a multi-layer network it is not the partial effect of
a predictor, and it is never a causal effect.
thangld_kan_edge( model, layer = 1L, input = 1L, output = 1L, x = NULL, n = 101L, scale = c("model", "original") )thangld_kan_edge( model, layer = 1L, input = 1L, output = 1L, x = NULL, n = 101L, scale = c("model", "original") )
model |
A model (fitted or not). |
layer, input, output
|
Edge indices (1-based). |
x |
Optional grid of edge inputs; by default |
n |
Number of grid points. |
scale |
|
A data frame with x, h (model-scale layer input), the
basis-specific components and phi (edge output on the model scale).
m <- thangld_kan(c(2, 3, 1), seed = 1) head(thangld_kan_edge(m, layer = 1, input = 2, output = 3, n = 5))m <- thangld_kan(c(2, 3, 1), seed = 1) head(thangld_kan_edge(m, layer = 1, input = 2, output = 3, n = 5))
Trains all parameters of a thangld_kan() network by mini-batch Adam on the
mean squared error. Inputs (and optionally the response) are
standardized with constants estimated on the training rows only;
predictions are returned on the original scale of y.
thangld_kan_fit( model, x, y, validation_data = NULL, validation_split = 0, epochs = 200L, batch_size = NULL, lr = 0.01, betas = c(0.9, 0.999), eps = 1e-08, patience = NULL, min_delta = 0, standardize_x = TRUE, standardize_y = TRUE, seed = NULL, backend = c("torch", "r"), dtype = c("float64", "float32"), verbose = FALSE )thangld_kan_fit( model, x, y, validation_data = NULL, validation_split = 0, epochs = 200L, batch_size = NULL, lr = 0.01, betas = c(0.9, 0.999), eps = 1e-08, patience = NULL, min_delta = 0, standardize_x = TRUE, standardize_y = TRUE, seed = NULL, backend = c("torch", "r"), dtype = c("float64", "float32"), verbose = FALSE )
model |
A |
x |
Numeric matrix or data frame of predictors. |
y |
Numeric response vector. |
validation_data |
Optional |
validation_split |
Fraction in |
epochs |
Maximum number of epochs. |
batch_size |
Mini-batch size; |
lr, betas, eps
|
Adam settings. |
patience |
|
min_delta |
Minimum improvement of the validation loss. |
standardize_x, standardize_y
|
Standardize with training mean and standard deviation. |
seed |
Optional seed for the batch order and the validation split; the caller's random number stream is restored afterwards. |
backend |
|
dtype |
|
verbose |
Print progress. |
Backends. backend = "torch" (default) uses R 'torch' autograd and
torch::optim_adam(). backend = "r" uses the base R analytic gradient
(thangld_kan_gradient_r()) and an Adam update that follows torch.optim.Adam;
it needs no libtorch. Both backends receive the same mini-batch schedule
(drawn in R from seed), so in float64 they follow the same trajectory
up to rounding.
Validation and early stopping. Supply validation_data (a list with
x and y) or validation_split. The validation loss is recorded every
epoch; with patience, training stops after patience epochs without an
improvement larger than min_delta. With validation data the parameters
of the epoch with the lowest validation loss are restored. Never use test
data here.
Constant columns in the training rows are fixed at their training value on the model scale during fitting and prediction (with a warning).
The fitted model with preprocess, history (epoch,
train_loss, val_loss on the standardized scale) and train.
set.seed(1) x <- matrix(runif(200, -2, 2), ncol = 2, dimnames = list(NULL, c("a", "b"))) y <- sin(x[, "a"]) + 0.5 * x[, "b"]^2 + rnorm(100, sd = 0.05) fit <- thangld_kan_fit(thangld_kan(c(2, 1), seed = 1), x, y, epochs = 50, lr = 0.02, backend = "r", validation_split = 0.2, seed = 1) fit head(predict(fit, x))set.seed(1) x <- matrix(runif(200, -2, 2), ncol = 2, dimnames = list(NULL, c("a", "b"))) y <- sin(x[, "a"]) + 0.5 * x[, "b"]^2 + rnorm(100, sd = 0.05) fit <- thangld_kan_fit(thangld_kan(c(2, 1), seed = 1), x, y, epochs = 50, lr = 0.02, backend = "r", validation_split = 0.2, seed = 1) fit head(predict(fit, x))
thangld_kan_forward_r() evaluates the network on model-scale inputs (already
standardized). thangld_kan_gradient_r() returns the mean squared error
and its analytic gradient with
respect to every parameter array and to the inputs.
thangld_kan_fd_gradient_r() approximates the same gradients by central
differences of order 2 (truncation error ) or 4
(); validation only. These functions use base R only and are
the reference for the torch backend.
thangld_kan_forward_r(model, x, layers = FALSE) thangld_kan_gradient_r(model, x, y) thangld_kan_fd_gradient_r(model, x, y, step = 1e-05, order = 2L)thangld_kan_forward_r(model, x, layers = FALSE) thangld_kan_gradient_r(model, x, y) thangld_kan_fd_gradient_r(model, x, y, step = 1e-05, order = 2L)
model |
A |
x |
Numeric matrix of model-scale inputs ( |
layers |
If |
y |
Numeric vector of length |
step |
Central-difference step. |
order |
Order of the central-difference stencil, 2 or 4. |
thangld_kan_forward_r(): numeric vector (or list of layer outputs).
The gradient functions: a list with loss, params (list per layer of
named gradient arrays shaped like model$params) and x.
m <- thangld_kan(c(2, 3, 1), seed = 1) x <- matrix(c(-1, 0.5, 2, -0.2), 2) thangld_kan_forward_r(m, x) g <- thangld_kan_gradient_r(m, x, c(0, 1)) fd <- thangld_kan_fd_gradient_r(m, x, c(0, 1)) max(abs(unlist(g$params) - unlist(fd$params)))m <- thangld_kan(c(2, 3, 1), seed = 1) x <- matrix(c(-1, 0.5, 2, -0.2), 2) thangld_kan_forward_r(m, x) g <- thangld_kan_gradient_r(m, x, c(0, 1)) fd <- thangld_kan_fd_gradient_r(m, x, c(0, 1)) max(abs(unlist(g$params) - unlist(fd$params)))
Regression metrics
thangld_kan_metrics(truth, estimate)thangld_kan_metrics(truth, estimate)
truth, estimate
|
Numeric vectors of equal length. |
Named vector rmse, mae, r2; r2 is NA when truth has
zero variance.
thangld_kan_metrics(c(1, 2, 3), c(1.1, 1.9, 3.2))thangld_kan_metrics(c(1, 2, 3), c(1.1, 1.9, 3.2))
A model is plain R data, saved with saveRDS() together with metadata
(format version, package version, R version, time); it can be reloaded
with or without libtorch.
thangld_kan_save(model, file) thangld_kan_load(file)thangld_kan_save(model, file) thangld_kan_load(file)
model |
A model. |
file |
Path of the |
thangld_kan_save() returns file invisibly; thangld_kan_load() returns the
model, with saving metadata in attr(, "saved").
m <- thangld_kan(c(2, 1), seed = 1) f <- tempfile(fileext = ".rds") thangld_kan_save(m, f) identical(m$params, thangld_kan_load(f)$params)m <- thangld_kan(c(2, 1), seed = 1) f <- tempfile(fileext = ".rds") thangld_kan_save(m, f) identical(m$params, thangld_kan_load(f)$params)
Runs the acceptance checks of thangldKANr on small fixtures in float64, for
each requested basis, and returns one row per check: basis-specific
checks, the forward pass against an independent loop implementation,
analytic gradients against fourth-order central differences (steps
1e-4, 1e-5, 1e-6) for a multi-layer network, save/load, and (when
the torch runtime is installed) the torch forward pass, autograd
gradients and a short Adam run against the base R backend. With
mixed = TRUE the same forward, gradient and torch checks also run on a
network whose layers use different bases. Missing checks (no torch
runtime, or a registered basis without a naive implementation) are
reported as skipped (pass = NA), never as passed. Errors are scaled
errors max(abs(a - b) / pmax(1, abs(a), abs(b))) unless stated
otherwise.
thangld_kan_validate( backend = c("both", "r", "torch"), basis = kan_bases()$name, mixed = TRUE )thangld_kan_validate( backend = c("both", "r", "torch"), basis = kan_bases()$name, mixed = TRUE )
backend |
|
basis |
Bases to validate (default: all available, see
|
mixed |
Also validate a network that mixes bases across layers. |
A data frame with basis, group, check, error, tol,
pass; attributes all_pass and n_skipped.
res <- thangld_kan_validate("r", basis = "chebyshev", mixed = FALSE) res[, c("check", "pass")]res <- thangld_kan_validate("r", basis = "chebyshev", mixed = FALSE) res[, c("check", "pass")]
A Temporal KAN here is a Kolmogorov-Arnold network that receives a window
of lagged values of a univariate series and predicts the next value
(one-step-ahead forecasting). The KAN part is a stack of edge layers of
any basis of thangld_kan() (default Chebyshev,
); with
hidden = integer() the forecast is an additive function of the lags (a
nonlinear additive autoregression), and hidden layers allow interactions
between lags. It is not recurrent or attention-based.
thangld_temporal_kan( lags = 1:12, hidden = integer(), basis = "chebyshev", ..., difference = FALSE, log = FALSE, options = NULL, seed = NULL )thangld_temporal_kan( lags = 1:12, hidden = integer(), basis = "chebyshev", ..., difference = FALSE, log = FALSE, options = NULL, seed = NULL )
lags |
Positive integer lags used as inputs, e.g. |
|
Integer widths of hidden layers ( |
|
basis |
Edge basis, one value or one per layer (see |
... |
Named basis-specific arguments, as in |
difference |
If |
log |
If |
options |
|
seed |
Optional seed for the initialization. |
An object of class thangld_temporal_kan.
thangld_temporal_kan(lags = 1:12, hidden = 4, seed = 1) thangld_temporal_kan(lags = 1:6, basis = "rbf", seed = 1)thangld_temporal_kan(lags = 1:12, hidden = 4, seed = 1) thangld_temporal_kan(lags = 1:6, basis = "rbf", seed = 1)
Builds, for every time index t with all lags available, the row
and the target , where z
is the (optionally log-transformed and differenced) series. Only values
before t enter row t.
thangld_temporal_kan_embed(y, lags, difference = FALSE, log = FALSE)thangld_temporal_kan_embed(y, lags, difference = FALSE, log = FALSE)
y |
Numeric vector or |
lags |
Positive integer lags. |
difference, log
|
Transformations as in |
A list with x (matrix, columns lag1, ...), target, index
(time index of each target in y) and z (transformed series aligned
with y, NA where undefined).
w <- thangld_temporal_kan_embed(1:10, lags = c(1, 3)) cbind(w$x, target = w$target, t = w$index)w <- thangld_temporal_kan_embed(1:10, lags = c(1, 3)) cbind(w$x, target = w$target, t = w$index)
Builds lag windows from y, estimates the standardization (mean and
standard deviation of the transformed series) from the training targets
only, holds out the last n_val windows (in time order) for validation
and early stopping, and trains by mini-batch Adam on the mean squared
one-step error. Mini-batches are drawn from training windows; every
window contains only values before its target, so shuffling windows does
not use future information. The validation windows come after all
training windows.
thangld_temporal_kan_fit( model, y, n_val = 0L, epochs = 200L, batch_size = 32L, lr = 0.01, betas = c(0.9, 0.999), eps = 1e-08, patience = NULL, min_delta = 0, seed = NULL, backend = c("torch", "r"), dtype = c("float64", "float32"), verbose = FALSE )thangld_temporal_kan_fit( model, y, n_val = 0L, epochs = 200L, batch_size = 32L, lr = 0.01, betas = c(0.9, 0.999), eps = 1e-08, patience = NULL, min_delta = 0, seed = NULL, backend = c("torch", "r"), dtype = c("float64", "float32"), verbose = FALSE )
model |
A |
y |
Numeric vector or |
n_val |
Number of final windows used for validation (0 for none). |
epochs, batch_size, lr, betas, eps, patience, min_delta
|
Training
settings (see |
seed |
Optional seed for the batch order. |
backend |
|
dtype |
|
verbose |
Print progress. |
The fitted thangld_temporal_kan with scaling, history, train and
the stored series y (used by predict.thangld_temporal_kan()).
y <- as.numeric(datasets::lynx) m <- thangld_temporal_kan(lags = 1:3, seed = 1) fit <- thangld_temporal_kan_fit(m, log(y), n_val = 10, epochs = 50, lr = 0.02, backend = "r", seed = 1) predict(fit)y <- as.numeric(datasets::lynx) m <- thangld_temporal_kan(lags = 1:3, seed = 1) fit <- thangld_temporal_kan_fit(m, log(y), n_val = 10, epochs = 50, lr = 0.02, backend = "r", seed = 1) predict(fit)
For every forecast origin from initial to length(y) - 1, the
value is forecast using only .
The Temporal KAN is (re)fitted on at the first origin and
then every refit_every origins (expanding window); between refits the
last fitted network and its standardization (estimated before that
refit's origin) are applied to the current history, so no value after an
origin is ever used for that origin's forecast. Validation windows for
early stopping are the last n_val windows before each refit origin.
thangld_temporal_kan_rolling( model, y, initial, refit_every = 1L, period = NULL, ... )thangld_temporal_kan_rolling( model, y, initial, refit_every = 1L, period = NULL, ... )
model |
An unfitted |
y |
Numeric vector or |
initial |
First forecast origin (number of observations available). |
refit_every |
Refit frequency in origins. |
period |
Seasonal period for the seasonal naive baseline and MASE
(default |
... |
Passed to |
Baselines computed at the same origins: naive () and seasonal naive () when
period = m > 1. MASE scales by the in-sample mean absolute one-step
seasonal-naive (or naive) error of the series up to the first origin.
A list with forecasts (data frame: origin, index, actual,
kan, naive, snaive, refit) and metrics (RMSE, MAE, MASE per
method).
y <- log(as.numeric(datasets::lynx)) r <- thangld_temporal_kan_rolling(thangld_temporal_kan(1:3, seed = 1), y, initial = 100, refit_every = 7, n_val = 10, epochs = 30, lr = 0.02, backend = "r", seed = 1) r$metricsy <- log(as.numeric(datasets::lynx)) r <- thangld_temporal_kan_rolling(thangld_temporal_kan(1:3, seed = 1), y, initial = 100, refit_every = 7, n_val = 10, epochs = 30, lr = 0.02, backend = "r", seed = 1) r$metrics
Save and load a Temporal KAN
thangld_temporal_kan_save(model, file) thangld_temporal_kan_load(file)thangld_temporal_kan_save(model, file) thangld_temporal_kan_load(file)
model |
A |
file |
Path of the |
thangld_temporal_kan_save() returns file invisibly;
thangld_temporal_kan_load() returns the model.
m <- thangld_temporal_kan(1:3, seed = 1) f <- tempfile(fileext = ".rds") thangld_temporal_kan_save(m, f) identical(thangld_temporal_kan_load(f)$params, m$params)m <- thangld_temporal_kan(1:3, seed = 1) f <- tempfile(fileext = ".rds") thangld_temporal_kan_save(m, f) identical(thangld_temporal_kan_load(f)$params, m$params)
Checks the lag-window model with Chebyshev edges (forward pass
against an independent cos(k acos z) loop implementation, analytic
gradients against fourth-order central differences for steps 1e-4,
1e-5, 1e-6), the lag-window construction against a direct loop, and
(with the torch runtime) the torch forward pass, autograd and a short Adam
run against the base R backend, and the gradients (and torch pass) of the
model with every other built-in basis. Skipped torch checks are reported with
pass = NA, never as passed.
thangld_temporal_kan_validate(backend = c("both", "r", "torch"))thangld_temporal_kan_validate(backend = c("both", "r", "torch"))
backend |
|
A data frame with group, check, error, tol, pass;
attributes all_pass and n_skipped.
thangld_temporal_kan_validate("r")[, c("check", "pass")]thangld_temporal_kan_validate("r")[, c("check", "pass")]