| Title: | Evidential Distance-Based Classification |
|---|---|
| Description: | Different evidential classifiers, which provide outputs in the form of Dempster-Shafer mass functions. The methods are: the evidential K-nearest neighbor rule, the evidential neural network, radial basis function neural networks, logistic regression, feed-forward neural networks. |
| Authors: | Thierry Denoeux |
| Maintainer: | Thierry Denoeux <[email protected]> |
| License: | GPL-3 |
| Version: | 2.0.2 |
| Built: | 2024-11-03 06:30:47 UTC |
| Source: | CRAN |
calcAB computes optimal coefficients alpha and beta needed to transform coefficients
from logistic regression (or connections weights between the last hidden layer and the output
layer of multilayer neural networks) into weights of evidence. These weights of evidence
can then be used to express the outputs of logistic regression or multilayer neural networks
as "latent" mass functions.
calcAB(W, mu = NULL)calcAB(W, mu = NULL)
W |
Vector of coefficients of length (d+1), where d is the number of features, in the case of M=2 classes, or (d+1,M) matrix of coefficients (or connection weights) in the case of M>2 classes. |
mu |
Optional vector containing the means of the d features. |
A list with two elements:
Vector of length d (M=2) or matrix of size (d,M) (for M>2) of coefficients alpha.
Vector of length d (M=2) or matrix of size (d,M) (for M>2) of coefficients beta.
Thierry Denoeux.
T. Denoeux. Logistic Regression, Neural Networks and Dempster-Shafer Theory: a New Perspective. Knowledge-Based Systems, Vol. 176, Pages 54–67, 2019.
## Example with 2 classes and logistic regression data(ionosphere) x<-ionosphere$x[,-2] y<-ionosphere$y-1 fit<-glm(y ~ x,family='binomial') AB<-calcAB(fit$coefficients,colMeans(x)) AB ## Example with K>2 classes and multilayer neural network library(nnet) data(glass) K<-max(glass$y) d<-ncol(glass$x) n<-nrow(x) x<-scale(glass$x) y<-as.factor(glass$y) p<-3 # number of hidden units fit<-nnet(y~x,size=p) # training a neural network with 3 hidden units W1<-matrix(fit$wts[1:(p*(d+1))],d+1,p) # Input-to-hidden weights W2<-matrix(fit$wts[(p*(d+1)+1):(p*(d+1) + K*(p+1))],p+1,K) # hidden-to-output weights a1<-cbind(rep(1,n),x)%*%W1 # hidden unit activations o1<-1/(1+exp(-a1)) # hidden unit outputs AB<-calcAB(W2,colMeans(o1)) AB## Example with 2 classes and logistic regression data(ionosphere) x<-ionosphere$x[,-2] y<-ionosphere$y-1 fit<-glm(y ~ x,family='binomial') AB<-calcAB(fit$coefficients,colMeans(x)) AB ## Example with K>2 classes and multilayer neural network library(nnet) data(glass) K<-max(glass$y) d<-ncol(glass$x) n<-nrow(x) x<-scale(glass$x) y<-as.factor(glass$y) p<-3 # number of hidden units fit<-nnet(y~x,size=p) # training a neural network with 3 hidden units W1<-matrix(fit$wts[1:(p*(d+1))],d+1,p) # Input-to-hidden weights W2<-matrix(fit$wts[(p*(d+1)+1):(p*(d+1) + K*(p+1))],p+1,K) # hidden-to-output weights a1<-cbind(rep(1,n),x)%*%W1 # hidden unit activations o1<-1/(1+exp(-a1)) # hidden unit outputs AB<-calcAB(W2,colMeans(o1)) AB
calcAB transforms coefficients alpha and beta computed by calcm into weights of
evidence, and then into mass and contour (plausibility) functions. These mass functions
can be used to express uncertainty about the prediction of logistic regression or multilayer
neural network classifiers (See Denoeux, 2019).
calcm(x, A, B)calcm(x, A, B)
x |
Matrix (n,d) of feature values, where d is the number of features, and n is the number of observations. Can be a vector if $d=1$. |
A |
Vector of length d (for M=2) or matrix of size (d,M) (for M>2) of coefficients alpha. |
B |
Vector of length d (for M=2) or matrix of size (d,M) (for M>2) of coefficients beta |
An error may occur if the absolute values of some coefficients are too high. It is then advised to recompute these coefficients by training the logistic regression or neural network classifier with L2 regularization. With M classes, the output mass functions have 2^M focal sets. Using this function with large M may cause memory issues.
A list with six elements:
Matrix (2^M,M) of focal sets.
Matrix (n,2^M) of mass functions (one in each row).
Matrix (n,M) containing the plausibilities of singletons.
Matrix (n,M) containing the degrees of belief of singletons.
Matrix (n,M) containing the normalized plausibilities of singletons.
Vector of length n containing the degrees of conflict.
Thierry Denoeux.
T. Denoeux. Logistic Regression, Neural Networks and Dempster-Shafer Theory: a New Perspective. Knowledge-Based Systems, Vol. 176, Pages 54–67, 2019.
## Example with 2 classes and logistic regression data(ionosphere) x<-ionosphere$x[,-2] y<-ionosphere$y-1 fit<-glm(y ~ x,family='binomial') AB<-calcAB(fit$coefficients,colMeans(x)) Bel<-calcm(x,AB$A,AB$B) Bel$focal Bel$mass[1:5,] Bel$pl[1:5,] Bel$conf[1:5] ## Example with K>2 classes and multilayer neural network library(nnet) data(glass) K<-max(glass$y) d<-ncol(glass$x) n<-nrow(x) x<-scale(glass$x) y<-as.factor(glass$y) p<-3 # number of hidden units fit<-nnet(y~x,size=p) # training a neural network with 3 hidden units W1<-matrix(fit$wts[1:(p*(d+1))],d+1,p) # Input-to-hidden weights W2<-matrix(fit$wts[(p*(d+1)+1):(p*(d+1) + K*(p+1))],p+1,K) # hidden-to-output weights a1<-cbind(rep(1,n),x)%*%W1 # hidden unit activations o1<-1/(1+exp(-a1)) # hidden unit outputs AB<-calcAB(W2,colMeans(o1)) Bel<-calcm(o1,AB$A,AB$B) Bel$focal Bel$mass[1:5,] Bel$pl[1:5,] Bel$conf[1:5]## Example with 2 classes and logistic regression data(ionosphere) x<-ionosphere$x[,-2] y<-ionosphere$y-1 fit<-glm(y ~ x,family='binomial') AB<-calcAB(fit$coefficients,colMeans(x)) Bel<-calcm(x,AB$A,AB$B) Bel$focal Bel$mass[1:5,] Bel$pl[1:5,] Bel$conf[1:5] ## Example with K>2 classes and multilayer neural network library(nnet) data(glass) K<-max(glass$y) d<-ncol(glass$x) n<-nrow(x) x<-scale(glass$x) y<-as.factor(glass$y) p<-3 # number of hidden units fit<-nnet(y~x,size=p) # training a neural network with 3 hidden units W1<-matrix(fit$wts[1:(p*(d+1))],d+1,p) # Input-to-hidden weights W2<-matrix(fit$wts[(p*(d+1)+1):(p*(d+1) + K*(p+1))],p+1,K) # hidden-to-output weights a1<-cbind(rep(1,n),x)%*%W1 # hidden unit activations o1<-1/(1+exp(-a1)) # hidden unit outputs AB<-calcAB(W2,colMeans(o1)) Bel<-calcm(o1,AB$A,AB$B) Bel$focal Bel$mass[1:5,] Bel$pl[1:5,] Bel$conf[1:5]
decision returns decisions from a loss matrix and mass functions computed
by an evidential classifier.
decision( m, L = 1 - diag(ncol(m) - 1), rule = c("upper", "lower", "pignistic", "hurwicz"), rho = 0.5 )decision( m, L = 1 - diag(ncol(m) - 1), rule = c("upper", "lower", "pignistic", "hurwicz"), rho = 0.5 )
m |
Matrix of masses for n test cases. Each row is a mass function. The first M columns correspond to the mass assigned to each of the M classes. The last column corresponds to the mass assigned to the whole set of classes. |
L |
The loss matrix of dimension (M,na) or (M+1,na), where na is the number
of actions. L[k,j] is the loss incurred if action j is chosen and the true class
is |
rule |
Decision rule to be used. Must be one of these: 'upper' (upper expectation), 'lower' (lower expectations), 'pignistic' (pignistic expectation), 'hurwicz' (weighted sum of the lower and upper expectations). |
rho |
Parameter between 0 and 1. Used only is rule='hurwicz'. |
This function implements the decision rules described in Denoeux (1997), with an arbitrary loss function. The decision rules are the minimization of the lower, upper or pignistic expectation, and Jaffray's decision rule based on minimizing a convex combination of the lower and upper expectations. The function also handles the case where there is an "unknown" class, in addition to the classes represented in the training set.
A n-vector with the decisions (integers between 1 and na).
Thierry Denoeux.
T. Denoeux. Analysis of evidence-theoretic decision rules for pattern classification. Pattern Recognition, 30(7):1095–1107, 1997.
## Example with M=2 classes m<-matrix(c(0.9,0.1,0,0.4,0.6,0,0.1,0.1,0.8),3,3,byrow=TRUE) ## Loss matrix with na=4 acts: assignment to class 1, assignment to class2, # rejection, and assignment to the unknown class. L<-matrix(c(0,1,1,1,0,1,0.2,0.2,0.2,0.25,0.25,0),3,4) d<-decision(m,L,'upper') ## instances 2 and 3 are rejected d<-decision(m,L,'lower') ## instance 2 is rejected, instance 3 is # assigned to the unknown class## Example with M=2 classes m<-matrix(c(0.9,0.1,0,0.4,0.6,0,0.1,0.1,0.8),3,3,byrow=TRUE) ## Loss matrix with na=4 acts: assignment to class 1, assignment to class2, # rejection, and assignment to the unknown class. L<-matrix(c(0,1,1,1,0,1,0.2,0.2,0.2,0.25,0.25,0),3,4) d<-decision(m,L,'upper') ## instances 2 and 3 are rejected d<-decision(m,L,'lower') ## instance 2 is rejected, instance 3 is # assigned to the unknown class
EkNNfit optimizes the parameters of the EkNN classifier.
EkNNfit( x, y, K, param = NULL, alpha = 0.95, lambda = 1/max(as.numeric(y)), optimize = TRUE, options = list(maxiter = 300, eta = 0.1, gain_min = 1e-06, disp = TRUE) )EkNNfit( x, y, K, param = NULL, alpha = 0.95, lambda = 1/max(as.numeric(y)), optimize = TRUE, options = list(maxiter = 300, eta = 0.1, gain_min = 1e-06, disp = TRUE) )
x |
Input matrix of size n x d, where n is the number of objects and d the number of attributes. |
y |
Vector of class labels (of length n). May be a factor, or a vector of integers from 1 to M (number of classes). |
K |
Number of neighbors. |
param |
Initial parameters (default: NULL). |
alpha |
Parameter |
lambda |
Parameter of the cost function. If |
optimize |
Boolean. If TRUE (default), the parameters are optimized. |
options |
A list of parameters for the optimization algorithm: maxiter (maximum number of iterations), eta (initial step of gradient variation), gain_min (minimum gain in the optimisation loop), disp (Boolean; if TRUE, intermediate results are displayed during the optimization). |
If the argument param is not supplied, the function EkNNinit is called.
A list with five elements:
The optimized parameters.
Final value of the cost function.
Leave-one-out error rate.
Leave-one-out predicted class labels (coded as integers from 1 to M).
Leave-one-out predicted mass functions. The first M columns correspond to the mass assigned to each class. The last column corresponds to the mass assigned to the whole set of classes.
Thierry Denoeux.
T. Denoeux. A k-nearest neighbor classification rule based on Dempster-Shafer theory. IEEE Transactions on Systems, Man and Cybernetics, 25(05):804–813, 1995.
L. M. Zouhal and T. Denoeux. An evidence-theoretic k-NN rule with parameter optimization. IEEE Transactions on Systems, Man and Cybernetics Part C, 28(2):263–271,1998.
## Iris dataset data(iris) x<-iris[,1:4] y<-iris[,5] fit<-EkNNfit(x,y,K=5)## Iris dataset data(iris) x<-iris[,1:4] y<-iris[,5] fit<-EkNNfit(x,y,K=5)
EkNNinit returns initial parameter values for the EkNN classifier.
EkNNinit(x, y, alpha = 0.95)EkNNinit(x, y, alpha = 0.95)
x |
Input matrix of size n x d, where n is the number of objects and d the number of attributes. |
y |
Vector of class lables (of length n). May be a factor, or a vector of integers from 1 to M (number of classes). |
alpha |
Parameter |
Each parameter is set ot the inverse of the square root of the mean
Euclidean distances wihin class k. Note that here is the square root
of the as defined in (Zouhal and Denoeux, 1998). By default, parameter alpha is set
to 0.95. This value normally does not have to be changed.
A list with two elements:
Vector of parameters , of length c, the number of classes.
Parameter , set to 0.95.
Thierry Denoeux.
T. Denoeux. A k-nearest neighbor classification rule based on Dempster-Shafer theory. IEEE Transactions on Systems, Man and Cybernetics, 25(05):804–813, 1995.
L. M. Zouhal and T. Denoeux. An evidence-theoretic k-NN rule with parameter optimization. IEEE Transactions on Systems, Man and Cybernetics Part C, 28(2):263–271,1998.
## Iris dataset data(iris) x<-iris[,1:4] y<-iris[,5] param<-EkNNinit(x,y) param## Iris dataset data(iris) x<-iris[,1:4] y<-iris[,5] param<-EkNNinit(x,y) param
EkNNval classifies instances in a test set using the EkNN classifier.
EkNNval(xtrain, ytrain, xtst, K, ytst = NULL, param = NULL)EkNNval(xtrain, ytrain, xtst, K, ytst = NULL, param = NULL)
xtrain |
Matrix of size ntrain x d, containing the values of the d attributes for the training data. |
ytrain |
Vector of class labels for the training data (of length ntrain). May be a factor, or a vector of integers from 1 to M (number of classes). |
xtst |
Matrix of size ntst x d, containing the values of the d attributes for the test data. |
K |
Number of neighbors. |
ytst |
Vector of class labels for the test data (optional). May be a factor, or a vector of integers from 1 to M (number of classes). |
param |
Parameters, as returned by |
If class labels for the test set are provided, the test error rate is also returned.
If parameters are not supplied, they are given default values by EkNNinit.
A list with three elements:
Predicted mass functions for the test data. The first M columns correspond to the mass assigned to each class. The last column corresponds to the mass assigned to the whole set of classes.
Predicted class labels for the test data (coded as integers from 1 to M).
Test error rate.
Thierry Denoeux.
T. Denoeux. A k-nearest neighbor classification rule based on Dempster-Shafer theory. IEEE Transactions on Systems, Man and Cybernetics, 25(05):804–813, 1995.
L. M. Zouhal and T. Denoeux. An evidence-theoretic k-NN rule with parameter optimization. IEEE Transactions on Systems, Man and Cybernetics Part C, 28(2):263–271,1998.
## Iris dataset data(iris) train<-sample(150,100) xtrain<-iris[train,1:4] ytrain<-iris[train,5] xtst<-iris[-train,1:4] ytst<-iris[-train,5] K<-5 fit<-EkNNfit(xtrain,ytrain,K) test<-EkNNval(xtrain,ytrain,xtst,K,ytst,fit$param)## Iris dataset data(iris) train<-sample(150,100) xtrain<-iris[train,1:4] ytrain<-iris[train,5] xtst<-iris[-train,1:4] ytst<-iris[-train,5] K<-5 fit<-EkNNfit(xtrain,ytrain,K) test<-EkNNval(xtrain,ytrain,xtst,K,ytst,fit$param)
The evclass package currently contains functions for three evidential classifiers: the evidential K-nearest neighbor (EK-NN) rule (Denoeux, 1995; Zouhal and Denoeux, 1998), the evidential neural network (Denoeux, 2000) and the RBF classifier with weight-of-evidence interpretation (Denoeux, 2019; Huang et al., 2022), as well as methods to compute output mass functions from trained logistic regression or multilayer classifiers as described in (Denoeux, 2019). In contrast with classical statistical classifiers, evidential classifiers quantify the uncertainty of the classification using Dempster-Shafer mass functions.
The main functions are: EkNNinit, EkNNfit and EkNNval
for the initialization, training and evaluation of the EK-NN classifier;
proDSinit, proDSfit and proDSval for the
evidential neural network classifier; decision for decision-making;
RBFinit, RBFfit and RBFval for the RBF classifier;
calcAB and calcm for computing output mass functions from trained
logistic regression or multilayer classifiers.
T. Denoeux. A k-nearest neighbor classification rule based on Dempster-Shafer theory. IEEE Transactions on Systems, Man and Cybernetics, 25(05):804–813, 1995.
T. Denoeux. Analysis of evidence-theoretic decision rules for pattern classification. Pattern Recognition, 30(7):1095–1107, 1997.
T. Denoeux. A neural network classifier based on Dempster-Shafer theory. IEEE Trans. on Systems, Man and Cybernetics A, 30(2):131–150, 2000.
L. M. Zouhal and T. Denoeux. An evidence-theoretic k-NN rule with parameter optimization. IEEE Transactions on Systems, Man and Cybernetics Part C, 28(2):263–271,1998.
T. Denoeux. Logistic Regression, Neural Networks and Dempster-Shafer Theory: a New Perspective. Knowledge-Based Systems, Vol. 176, Pages 54–67, 2019.
L., S. Ruan, P. Decazes and T. Denoeux. Lymphoma segmentation from 3D PET-CT images using a deep evidential network. International Journal of Approximate Reasoning, Vol. 149, Pages 39-60, 2022.
EkNNinit, EkNNfit,
EkNNval, proDSinit, proDSfit, proDSval,
RBFinit, RBFfit and RBFval, decision,
calcAB, calcm.
This data set contains the description of 214 fragments of glass originally collected for a study in the context of criminal investigation. Each fragment has a measured reflectivity index and chemical composition (weight percent of Na, Mg, Al, Si, K, Ca, Ba and Fe). As suggested by Ripley (1994), 29 instances were discarded, and the remaining 185 were re-grouped in four classes: window float glass (70), window non-float glass (76), vehicle window glass (17) and other (22). The data set was split randomly in a training set of size 89 and a test set of size 96.
data(glass)data(glass)
A list with two elements:
The 185 x 9 object-attribute matrix.
A 185-vector containing the class labels.
P. M. Murphy and D. W. Aha. UCI Reposition of machine learning databases. [Machine readable data repository]. University of California, Departement of Information and Computer Science, Irvine, CA.
B.D.Ripley, Flexible nonlinear approaches to classification, in "From Statistics to Neural Networks", V. Cherkassly, J. H. Friedman, and H. Wechsler, Eds., Berlin, Germany: Springer-Verlag, 1994, pp. 105–126.
T. Denoeux. A neural network classifier based on Dempster-Shafer theory. IEEE Trans. on Systems, Man and Cybernetics A, 30(2):131–150, 2000.
data(glass) table(glass$y)data(glass) table(glass$y)
This dataset was collected by a radar system and consists of phased array of 16 high-frequency antennas with a total transmitted power of the order of 6.4 kilowatts. The targets were free electrons in the ionosphere. "Good" radar returns are those showing evidence of some type of structure in the ionosphere. "Bad" returns are those that do not. There are 351 instances and 34 numeric attributes. The first 175 instances are training data, the rest are test data. This version of dataset was used by Zouhal and Denoeux (1998).
data(ionosphere)data(ionosphere)
A list with two elements:
The 351 x 34 object-attribute matrix.
A 351-vector containing the class labels.
P. M. Murphy and D. W. Aha. UCI Reposition of machine learning databases. [Machine readable data repository]. University of California, Departement of Information and Computer Science, Irvine, CA.
L. M. Zouhal and T. Denoeux. An evidence-theoretic k-NN rule with parameter optimization. IEEE Transactions on Systems, Man and Cybernetics Part C, 28(2):263–271,1998.
data(ionosphere) table(vehicles$y)data(ionosphere) table(vehicles$y)
proDSfit performs parameter optimization for the evidential neural network classifier.
proDSfit( x, y, param, lambda = 1/max(as.numeric(y)), mu = 0, optimProto = TRUE, options = list(maxiter = 500, eta = 0.1, gain_min = 1e-04, disp = 10) )proDSfit( x, y, param, lambda = 1/max(as.numeric(y)), mu = 0, optimProto = TRUE, options = list(maxiter = 500, eta = 0.1, gain_min = 1e-04, disp = 10) )
x |
Input matrix of size n x d, where n is the number of objects and d the number of attributes. |
y |
Vector of class lables (of length n). May be a factor, or a vector of integers from 1 to M (number of classes). |
param |
Initial parameters (see |
lambda |
Parameter of the cost function. If |
mu |
Regularization hyperparameter (default=0). |
optimProto |
Boolean. If TRUE, the prototypes are optimized (default). Otherwise, they are fixed. |
options |
A list of parameters for the optimization algorithm: maxiter (maximum number of iterations), eta (initial step of gradient variation), gain_min (minimum gain in the optimisation loop), disp (integer; if >0, intermediate results are displayed every disp iterations). |
If optimProto=TRUE (default), the prototypes are optimized. Otherwise, they are fixed to
their initial value.
A list with three elements:
Optimized network parameters.
Final value of the cost function.
Training error rate.
Thierry Denoeux.
T. Denoeux. A neural network classifier based on Dempster-Shafer theory. IEEE Trans. on Systems, Man and Cybernetics A, 30(2):131–150, 2000.
## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] xtst<-glass$x[90:185,] ytst<-glass$y[90:185] ## Initialization param0<-proDSinit(xapp,yapp,nproto=7) ## Training fit<-proDSfit(xapp,yapp,param0)## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] xtst<-glass$x[90:185,] ytst<-glass$y[90:185] ## Initialization param0<-proDSinit(xapp,yapp,nproto=7) ## Training fit<-proDSfit(xapp,yapp,param0)
proDSinit returns initial parameter values for the evidential neural network classifier.
proDSinit(x, y, nproto, nprotoPerClass = FALSE, crisp = FALSE)proDSinit(x, y, nproto, nprotoPerClass = FALSE, crisp = FALSE)
x |
Input matrix of size n x d, where n is the number of objects and d the number of attributes. |
y |
Vector of class labels (of length n). May be a factor, or a vector of integers from 1 to M (number of classes). |
nproto |
Number of prototypes. |
nprotoPerClass |
Boolean. If TRUE, there are |
crisp |
Boolean. If TRUE, the prototypes have full membership to only one class. (Available only if nprotoPerClass=TRUE). |
The prototypes are initialized by the k-means algorithms. The initial membership values of
each prototype to class are normally defined as the proportion of training samples
from class in the neighborhood of prototype . If arguments crisp and
nprotoPerClass are set to TRUE, the prototypes are assigned to one and only one class.
A list with four elements containing the initialized network parameters
Vector of length r, where r is the number of prototypes.
Vector of length r
Matrix of size (r,M), where M is the number of classes.
Matrix of size (r,d), containing the prototype coordinates.
Thierry Denoeux.
T. Denoeux. A neural network classifier based on Dempster-Shafer theory. IEEE Trans. on Systems, Man and Cybernetics A, 30(2):131–150, 2000.
## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] param0<-proDSinit(xapp,yapp,nproto=7) param0## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] param0<-proDSinit(xapp,yapp,nproto=7) param0
proDSval classifies instances in a test set using the evidential neural network classifier.
proDSval(x, param, y = NULL)proDSval(x, param, y = NULL)
x |
Matrix of size n x d, containing the values of the d attributes for the test data. |
param |
Neural network parameters, as provided by |
y |
Optional vector of class labels for the test data. May be a factor, or a vector of integers from 1 to M (number of classes). |
If class labels for the test set are provided, the test error rate is also returned.
A list with three elements:
Predicted mass functions for the test data. The first M columns correspond to the mass assigned to each class. The last column corresponds to the mass assigned to the whole set of classes.
Predicted class labels for the test data.
Test error rate (if the class label of test data has been provided).
Thierry Denoeux.
T. Denoeux. A neural network classifier based on Dempster-Shafer theory. IEEE Trans. on Systems, Man and Cybernetics A, 30(2):131–150, 2000.
## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] xtst<-glass$x[90:185,] ytst<-glass$y[90:185] ## Initialization param0<-proDSinit(xapp,yapp,nproto=7) ## Training fit<-proDSfit(xapp,yapp,param0) ## Test val<-proDSval(xtst,fit$param,ytst) ## Confusion matrix table(ytst,val$ypred)## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] xtst<-glass$x[90:185,] ytst<-glass$y[90:185] ## Initialization param0<-proDSinit(xapp,yapp,nproto=7) ## Training fit<-proDSfit(xapp,yapp,param0) ## Test val<-proDSval(xtst,fit$param,ytst) ## Confusion matrix table(ytst,val$ypred)
RBFfit performs parameter optimization for a radial basis function (RBF) classifier.
RBFfit( x, y, param, lambda = 0, control = list(fnscale = -1, trace = 2, maxit = 1000), optimProto = TRUE )RBFfit( x, y, param, lambda = 0, control = list(fnscale = -1, trace = 2, maxit = 1000), optimProto = TRUE )
x |
Input matrix of size n x d, where n is the number of objects and d the number of attributes. |
y |
Vector of class labels (of length n). May be a factor, or a vector of integers from 1 to M (number of classes). |
param |
Initial parameters (see |
lambda |
Regularization hyperparameter (default=0). |
control |
Parameters passed to function |
optimProto |
Boolean. If TRUE, the prototypes are optimized (default). Otherwise, they are fixed. |
The RBF neural network is trained by maximizing the conditional log-likelihood (or, equivalently,
by minimizing the cross-entropy loss function). The optimization procedure is the BFGS
algorithm implemented in function optim.
A list with three elements:
Optimized network parameters.
Final value of the log-likelihood objective function.
Training error rate.
Thierry Denoeux.
## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] ## Initialization param0<-RBFinit(xapp,yapp,nproto=7) ## Training fit<-RBFfit(xapp,yapp,param0,control=list(fnscale=-1,trace=2))## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] ## Initialization param0<-RBFinit(xapp,yapp,nproto=7) ## Training fit<-RBFfit(xapp,yapp,param0,control=list(fnscale=-1,trace=2))
RBFinit returns initial parameter values for a Radial Basis Function classifier.
RBFinit(x, y, nproto)RBFinit(x, y, nproto)
x |
Input matrix of size n x d, where n is the number of objects and d the number of attributes. |
y |
Vector of class labels (of length n). May be a factor, or a vector of integers from 1 to M (number of classes). |
nproto |
Number of prototypes |
The prototypes are initialized by the k-means algorithms. The hidden-to-output weights are initialized
by linear regression. The scale parameter for each prototype is computed as the inverse of the square
root of the mean squared distances to this prototype. The final number of prototypes may be different
from the desired number nproto depending on the result of the k-means clustering (clusters
composed of only one input vector are discarded).
A list with three elements containing the initialized network parameters
Matrix of size (R,d), containing the R prototype coordinates.
Vector of length R, containing the scale parameters.
Matrix of size (R,M), containing the hidden-to-output weights.
Thierry Denoeux.
## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] param0<-RBFinit(xapp,yapp,nproto=7) param0## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] param0<-RBFinit(xapp,yapp,nproto=7) param0
RBFval classifies instances in a test set using a radial basis function classifier. Function
calcm is called for computing output belief functions. It is recommended to set
calc.belief=FALSE when the number of classes is very large, to avoid memory problems.
RBFval(x, param, y = NULL, calc.belief = TRUE)RBFval(x, param, y = NULL, calc.belief = TRUE)
x |
Matrix of size n x d, containing the values of the d attributes for the test data. |
param |
Neural network parameters, as provided by |
y |
Optional vector of class labels for the test data. May be a factor, or a vector of integers from 1 to M (number of classes). |
calc.belief |
If TRUE (default), output belief functions are calculated. |
If class labels for the test set are provided, the test error rate is also returned.
A list with four elements:
Predicted class labels for the test data.
Test error rate (if the class label of test data has been provided).
Output probabilities.
If calc.belief=TRUE, output belief function, provided as a list
output by function calcm.
Thierry Denoeux.
T. Denoeux. Logistic Regression, Neural Networks and Dempster-Shafer Theory: a New Perspective. Knowledge-Based Systems, Vol. 176, Pages 54–67, 2019.
Ling Huang, Su Ruan, Pierre Decazes and Thierry Denoeux. Lymphoma segmentation from 3D PET-CT images using a deep evidential network. International Journal of Approximate Reasoning, Vol. 149, Pages 39-60, 2022.
## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] xtst<-glass$x[90:185,] ytst<-glass$y[90:185] ## Initialization param0<-RBFinit(xapp,yapp,nproto=7) ## Training fit<-RBFfit(xapp,yapp,param0) ## Test val<-RBFval(xtst,fit$param,ytst) ## Confusion matrix table(ytst,val$ypred)## Glass dataset data(glass) xapp<-glass$x[1:89,] yapp<-glass$y[1:89] xtst<-glass$x[90:185,] ytst<-glass$y[90:185] ## Initialization param0<-RBFinit(xapp,yapp,nproto=7) ## Training fit<-RBFfit(xapp,yapp,param0) ## Test val<-RBFval(xtst,fit$param,ytst) ## Confusion matrix table(ytst,val$ypred)
This dataset was collected from silhouettes by the HIPS (Hierarchical Image Processing System) extension BINATTS Four model vehicles were used for the experiment: bus, Chevrolet van, Saab 9000 and Opel Manta. The data were used to distinguish 3D objects within a 2-D silhouette of the objects. There are 846 instances and 18 numeric attributes. The first 564 objects are training data, the rest are test data. This version of dataset was used by Zouhal and Denoeux (1998).
data(vehicles)data(vehicles)
A list with two elements:
The 846 x 18 object-attribute matrix.
A 846-vector containing the class labels.
P. M. Murphy and D. W. Aha. UCI Reposition of machine learning databases. [Machine readable data repository]. University of California, Departement of Information and Computer Science, Irvine, CA.
L. M. Zouhal and T. Denoeux. An evidence-theoretic k-NN rule with parameter optimization. IEEE Transactions on Systems, Man and Cybernetics Part C, 28(2):263–271,1998.
data(vehicles) table(vehicles$y)data(vehicles) table(vehicles$y)