Package 'ConfMatrix'

Description

The ConfMatrix class works with confusion matrices, thus providing the possibility of calculating several indices with their corresponding variances and confidence intervals. A confusion matrix is constructed by comparing a sample of a set of common positions in the product and the ground truth. Appropriate sampling methods must be applied to generate the confusion matrix. It is considered that the classes of the ground truth correspond to the columns and that the classes of the product to be valued correspond to the rows. First, an object of this class of object must be created (instantiated) and then the methods that offer the index calculations will be invoked. Mnemonic method names are proposed and are therefore long, for example methods that provide averages start with "AV" and those that provide combinations start with "Comb". Methods related to a specific thematics class end with the ending "_i".

Mathematical elements

• $x_{ii}$: diagonal element of the matrix.

• $x_{ij}$: element $i,j$ of the matrix.

• $x_{i+}$: sum of all elements in rows $i$.

• $x_{+j}$: sum of all elements in column $j$.

• $M$: number of classes.

• $\overline{x}_{i+}$: sum of all elements of row $i$ except element $i$ of the diagonal.

• $\overline{x}_{+i}$: sum of all elements of column $i$ except element $i$ of the diagonal.

• $N_{Total}$:Total count of elements in the instance's Confusion Matrix.

  $N_{Total}=\sum_{i,j}^M x_{ij}$

• $N_i / N_j$: Total count of elements in row $i$ or column $j$.

  $N_{i}=x_{i+}$

  $N_{j}=x_{+j}$

• $N_{ij}$: Total count of elements in row $i$ and column $j$.

  $N_{ij}=x_{i+}+x_{+j}-x_{ii}$

Public fields

Methods

Public methods

• ConfMatrix$new()

• ConfMatrix$plot.index()

• ConfMatrix$plot.UserProdAcc()

• ConfMatrix$print()

• ConfMatrix$AllParameters()

• ConfMatrix$UserAcc()

• ConfMatrix$UserAcc_i()

• ConfMatrix$AvUserAcc()

• ConfMatrix$CombUserAcc()

• ConfMatrix$ProdAcc()

• ConfMatrix$ProdAcc_i()

• ConfMatrix$AvProdAcc()

• ConfMatrix$CombProdAcc()

• ConfMatrix$UserProdAcc()

• ConfMatrix$CombUserProdAcc()

• ConfMatrix$AvUserProdAcc()

• ConfMatrix$AvUserProdAcc_i()

• ConfMatrix$UserProdAcc_W()

• ConfMatrix$OverallAcc()

• ConfMatrix$Kappa()

• ConfMatrix$ModKappa()

• ConfMatrix$UserKappa_i()

• ConfMatrix$ModKappaUser_i()

• ConfMatrix$ProdKappa_i()

• ConfMatrix$ModKappaProd_i()

• ConfMatrix$DetailKappa()

• ConfMatrix$DetailCondKappa()

• ConfMatrix$DetailWKappa()

• ConfMatrix$Tau()

• ConfMatrix$DetailWTau()

• ConfMatrix$Ent()

• ConfMatrix$AvNormEnt()

• ConfMatrix$GeomAvNormEnt()

• ConfMatrix$AvMaxNormEnt()

• ConfMatrix$EntUser_i()

• ConfMatrix$NormEntUser()

• ConfMatrix$EntProd_i()

• ConfMatrix$NormEntProd()

• ConfMatrix$Sucess()

• ConfMatrix$Sucess_i()

• ConfMatrix$AvHellAcc()

• ConfMatrix$AvHellAcc_i()

• ConfMatrix$AvShortAcc()

• ConfMatrix$ShortAcc_i()

• ConfMatrix$GroundTruth()

• ConfMatrix$GroundTruth_i()

• ConfMatrix$HellingerDist()

• ConfMatrix$QES()

• ConfMatrix$MTypify()

• ConfMatrix$MBootStrap()

• ConfMatrix$MNormalize()

• ConfMatrix$MPseudoZeroes()

• ConfMatrix$OverallAcc.test()

• ConfMatrix$Kappa.test()

• ConfMatrix$Tau.test()

• ConfMatrix$TSCM.test()

• ConfMatrix$QIndep.test()

Method new()

Public method to create an instance of the ConfMatrix class. When creating it, values must be given to the matrix. The values of the matrix must be organized in such a way that the columns represent the classes in the reference and the rows represent the classes in the product being evaluated. The creation of a ConfMatrix instance includes a series of checks on the data. If checks are not met, the system generates coded error messages. The optional possibility of adding metadata to the matrix is offered.

Usage

ConfMatrix$new(
  Values,
  ID = NULL,
  Date = NULL,
  ClassNames = NULL,
  Source = NULL
)

Arguments

Returns

Object of the ConfMatrix class, or an error message.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
cm<-ConfMatrix$new (A,ID="5",Date="27-10-2023",Source="Congalton and Green,
2008")

Method plot.index()

Public method that provides a graph of the indices of the functions ConfMatrix$OverallAcc, ConfMatrix$Kappa, ConfMatrix$Tau, ConfMatrix$AvHellAcc and ConfMatrix$AvShortAcc with their corresponding standard deviations.

Usage

ConfMatrix$plot.index()

Returns

A graph of the indices of the functions OverallAcc, Kappa, Tau, AvHellAcc, AvShortAcc with their corresponding standard deviations.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90), nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$plot.index()

Method plot.UserProdAcc()

Public method that provides a graph for the user’s and producer’s accuracies and standard deviations.

Usage

ConfMatrix$plot.UserProdAcc()

Returns

The graph of the accuracy index of users and producers with their corresponding standard desviation.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90), nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$plot.UserProdAcc()

Method print()

Public method that shows all the data entered by the user for a instance.

Usage

ConfMatrix$print()

Returns

ConfMatrix object identifier, date, class name, data source and confusion matrix.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90), nrow=4,ncol=4)
p<-ConfMatrix$new(A,ClassNames=c("Deciduous","conifer","agriculture",
"shrub"),Source="Congalton and Green 2008")
p$print()

Method AllParameters()

Public method in which multiple parameters are calculated for the given confusion matrix. This method is equivalent to ConfMatrix$OverallAcc,ConfMatrix$UserAcc, ConfMatrix$ProdAcc,ConfMatrix$Kappa and ConfMatrix$MPseudoZeroes.

Usage

ConfMatrix$AllParameters()

Returns

The following list of elements: the confusion matrix, dimension, total sum of cell values, overall accuracy, overall accuracy variance, global kappa index, global kappa simplified variance, producer accuracy by class, user accuracy by class, and pseudoceros matrix.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90), nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$AllParameters()

Method UserAcc()

Public method for deriving the index called user’s accuracy for all the classes in a ConfMatrix object instance. The user's accuracy for the class $i$ of a thematic product is calculated by dividing the value in the diagonal of class $i$ by the sum of all values in the row of the class $i$ (row marginal). The method also offers the variance and confidence interval. The reference Congalton and Green (2008) is followed for the computations.

$UserAcc=\dfrac{x_{ii}}{x_{i+}}$

$\sigma^2_{UserAcc}=\dfrac{UserAcc \cdot (1-UserAcc)}{N_{i}}$

Usage

ConfMatrix$UserAcc(a = NULL)

Arguments

Returns

A list of vectors, containing the user’s accuracy real values for all classes, their variances and confidence intervals for each class, respectively.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90), nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$UserAcc()

Method UserAcc_i()

Public method for deriving the index called user’s accuracy for a specific class $i$ in a ConfMatrix object instance. The user’s accuracy for the class $i$ of a thematic product is calculated by dividing the value in the diagonal of class $i$ by the sum of all values in the row of the class i (row marginal). The method also offers the variance and confidence interval. The reference Congalton and Green (2008) is followed for the computations.

$UserAcc_{i}=\dfrac{x_{ii}}{x_{i+}}$

$\sigma^2_{UserAcc_i}=\dfrac{UserAcc_i \cdot (1-UserAcc_i)}{N_{i}}$

Usage

ConfMatrix$UserAcc_i(i, a = NULL)

Arguments

Returns

A list of real values containing the user’s accuracy for class i, its variance, and its confidence interval.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90), nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$UserAcc_i(2)

Method AvUserAcc()

Public method that provides the arithmetic average, without weighing, of all user’s accuracies of a ConfMatrix object instance. The method also offers the variance and confidence interval. The reference Tung and LeDrew (1988) is followed for the calculations.

$AvUserAcc=\dfrac{1}{M} \sum^M_{i=1} \dfrac{x_{ii}} { x_{i+}}$

$\sigma^2_{AvUserAcc}=\dfrac{AvUserAcc \cdot (1-AvUserAcc)}{N_{Total}}$

Usage

ConfMatrix$AvUserAcc(a = NULL)

Arguments

Returns

A list of real values containing the average user’s accuracy, its variance, and its confidence interval.

Examples

A<-matrix(c(352,43,89,203),nrow=2,ncol=2)
p<-ConfMatrix$new(A,Source="Tung and LeDrew 1988")
p$AvUserAcc()

Method CombUserAcc()

Public method that provides the combined user's accuracy. Which is the average of the overall accuracy and the average user's accuracy. The method also offers the variance and confidence interval. The reference Tung and LeDrew (1988) is followed for the calculations.

$CombUserAcc=\dfrac{OverallAcc+AvUserAcc}{2}$

$\sigma^2_{CombUserAcc}=\dfrac{CombUserAcc \cdot (1-CombUserAcc)}{N_{Total}}$

where:

1. $OverallAcc$: overall accuracy.

Usage

ConfMatrix$CombUserAcc(a = NULL)

Arguments

Returns

A list of real values containing the combined accuracy from the user's perspective, its variation and confidence interval.

Examples

A<-matrix(c(352,43,89,203),nrow=2,ncol=2)
p<-ConfMatrix$new(A,Source="Tung and LeDrew 1988")
p$CombUserAcc()

Method ProdAcc()

Public method for deriving the index called producer’s accuracy for all the classes in a ConfMatrix object instance. The producer’s accuracy for the class i of a thematic product is calculated by dividing the value in the diagonal of class $i$ by the sum of all values in the row of the class $i$ (column marginal). The method also offers the variance and confidence interval. The reference Congalton and Green (2008) if followed for the computations.

$ProdAcc=\dfrac{x_{ii}}{x_{+j}}$

$\sigma^2_{ProdAcc}=\dfrac{ProdAcc \cdot (1-ProdAcc)}{N_{j}}$

Usage

ConfMatrix$ProdAcc(a = NULL)

Arguments

Returns

A list of vectors each one containing the producer’s accuracy real values for all classes, their variances and confidence intervals for each class, respectively.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$ProdAcc()

Method ProdAcc_i()

Public method for deriving the index called producer’s accuracy for a specific class $i$ in a ConfMatrix object instance. The user’s accuracy for the class $i$ of a thematic product is calculated by dividing the value in the diagonal of class $i$ by the sum of all values in the column of the class $i$ (column marginal). The method also offers the variance and confidence interval. The reference Congalton and Green (2008) is followed for the calculations.

$ProdAcc_{i}=\dfrac{x_{ii}}{x_{+j}}$

$\sigma^2_{ProdAcc_i}=\dfrac{ProdAcc_i \cdot (1-ProdAcc_i)}{N_{j}}$

Usage

ConfMatrix$ProdAcc_i(i, a = NULL)

Arguments

Returns

A list of real values containing the producer’s accuracy for class i, its variance, and its confidence interval.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$ProdAcc_i(1)

Method AvProdAcc()

Public method that provides the arithmetic average of all producer’s accuracies of a ConfMatrix object instance. The method also offers the variance and confidence interval. The reference Tung and LeDrew (1988) is followed for the calculations.

$AvProdAcc=\dfrac{1}{M} \sum^M_{i=1} \dfrac{x_{ii}} { x_{+j}}$

$\sigma^2_{AvProdAcc}=\dfrac{AvProdAcc \cdot (1-AvProdAcc)}{N_{Total}}$

Usage

ConfMatrix$AvProdAcc(a = NULL)

Arguments

Returns

A list of real values containing the average producer’s accuracy, its variance, and its confidence interval.

Examples

A<-matrix(c(352,43,89,203),nrow=2,ncol=2)
p<-ConfMatrix$new(A,Source="Tung and LeDrew 1988")
p$AvProdAcc()

Method CombProdAcc()

Public method that provides the combined producer's accuracy. Which is the average of the overall accuracy and the average producer accuracy. The method also offers the variance and confidence interval. The reference Tung and LeDrew (1988) is followed for the calculations.

$CombProdAcc=\dfrac{OverallAcc+AvProdAcc}{2}$

$\sigma^2_{CombProdAcc}=\dfrac{CombProdAcc \cdot (1-CombProdAcc)}{N_{Total}}$

where:

1. $OverallAcc$: overall accuracy.

2. $AvProdAcc$: average accuracy from producer's perspective.

Usage

ConfMatrix$CombProdAcc(a = NULL)

Arguments

Returns

A list of real values containing the combined accuracy from producer's perspective, its variance and confidence interval.

Examples

A<-matrix(c(352,43,89,203),nrow=2,ncol=2)
p<-ConfMatrix$new(A,Source="Tung and LeDrew 1988")
p$CombProdAcc()

Method UserProdAcc()

Public method that calculates the user’s and the producer’s indexes jointly. This method is equivalent to the methods ConfMatrix$UserAcc and ConfMatrix$ProdAcc.

Usage

ConfMatrix$UserProdAcc()

Returns

A list containing the producer's and user's accuracies and their standard deviations, respectively.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$UserProdAcc()

Method CombUserProdAcc()

Public method that provides the combined accuracy, defined by the average of the overall accuracy and the Hellden's average accuracy, which refers to the average user's and producer's accuracies. The method also offers the variance and confidence interval. The reference Liu et al. (2007) is followed for the calculations.

$CombUserProdAcc=\dfrac{OverallAcc+AvHellAcc}{2}$

$\sigma^2_{CombUserProdAcc}=\dfrac{CombUserProdAcc \cdot (1-CombUserProdAcc)}{N_{Total}}$

where:

1. $OverallAcc$: overall accuracy.

2. $AvHellAcc$: average of Hellden's mean accuracy index.

Usage

ConfMatrix$CombUserProdAcc(a = NULL)

Arguments

Returns

A list of real values containing the combined accuracy from both user's and producer's perspectives, its variance and confidence interval.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$CombUserProdAcc()

Method AvUserProdAcc()

Public method that provides the arithmetic average of all user’s and producer’s accuracy indexes of a ConfMatrix object instance. The method also offers the variance and confidence interval. The reference Liu et al. (2007) is followed for the calculations.

$AvUserProdAcc=\dfrac{AvUserAcc+AvProdAcc}{2}$

$\sigma^2_{AvUserProdAcc}=\dfrac{AvUserProdAcc \cdot (1-AvUserProdAcc)}{N_{Total}}$

where:

1. $AvUserAcc$: average accuracy from user's perspective.

2. $AvProdAcc$: average accuracy from producer's perspective.

Usage

ConfMatrix$AvUserProdAcc(a = NULL)

Arguments

Returns

A list of real values containing the average mean precision values from the user's and producer's perspective, their variance and confidence interval.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$AvUserProdAcc()

Method AvUserProdAcc_i()

Public method that provides the average of user’s and producer’s accuracies for a specific class i. The method also offers the variance and confidence interval. The reference Liu et al. (2007) is followed for the calculations.

$AvUserProdAcc_i=\dfrac{UserAcc_i+ProdAcc_i}{2}$

$\sigma^2_{AvUserProdAcc_i}=\dfrac{AvUserProdAcc_i \cdot (1-AvUserProdAcc_i)}{N_{ij}}$

where:

1. $UserAcc_i$: user accuracy index for class i.

2. $ProdAcc_i$: producer accuracy index for class i.

Usage

ConfMatrix$AvUserProdAcc_i(i, a = NULL)

Arguments

Returns

A list of real values containing the average of user’s and producer’s accuracies, its variance and confidence interval for class i.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$AvUserProdAcc_i(2)

Method UserProdAcc_W()

Public method that calculates the weighted user's, producer’s and overall accuracies and their standard deviations. The reference Congalton and Green (2008) is followed for the computations.

Be

$Overall_W=\dfrac{\sum^M_{i=1} p_{ii} }{\sum^M_{i,j=1} p_{ij}}$

where $p_{ij}=\dfrac{x_{ij}}{\sum^M_{i,j=1} x_{ij}}$

$UserAcc_W=\dfrac{p_{o_{i+}}}{p_{i+}}$

$ProdAcc_W=\dfrac{p_{o_{+j}}}{p_{+j}}$

$\sigma^2_{UserAcc_W}=\dfrac{UserAcc_W \cdot (1-UserAcc_W)}{N_i}$

$\sigma^2_{ProdAcc_W}=\dfrac{ProdAcc_W \cdot (1-ProdAcc_W)}{N_j}$

where $p_o=\sum^M_{i,j=1} w_{ij}p_{ij}$ and $0 \leq w_{ij} \leq 1$ for $i \neq j$ and $w_{ii}=1$ for $i = j$.

Usage

ConfMatrix$UserProdAcc_W(WM)

Arguments

Returns

A list with the weight matrix, the product of the confusion matrix and the weight matrix, overall, user and producer weighted accuracies and their standard deviations.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
WM<- t(matrix(c(1,0,0.67,1,0,1,0,0,1,0,1,1,0.91,0,0.61,1),nrow=4,ncol=4))
p$UserProdAcc_W(WM)

Method OverallAcc()

Public method to calculate the global index called Overall Accuracy. The Overall Accuracy is calculated by dividing the sum of the entries that form the major diagonal (i.e., the number of correct classifications) by the total number of cases. The method also offers the variance and confidence interval. The reference Congalton and Green (2008) is followed for the computations.

$OverallAcc = \dfrac{\sum_{i=1}^{M} x_{ii}}{\sum_{i, j=1}^{M} x_{ij}}$

$\sigma^2_{OverallAcc}=\dfrac{OverallAcc \cdot (1-OverallAcc)}{N_{Total}}$

Usage

ConfMatrix$OverallAcc(a = NULL)

Arguments

Returns

A list of real values containing the overall accuracy, its variance, and its confidence interval.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A)
p$OverallAcc()

Method Kappa()

Public method that provides Kappa coefficient, which measures the relationship between the observed proportion of agreement and the proportion expected to occur by chance. The method also offers the variance and confidence interval. The reference Cohen (1960) is followed for the calculations.

$Kappa=\dfrac{OverallAcc-ExpAcc}{1-ExpAcc}$

$ExpAcc= \dfrac{x_{+ i}x_{i +}}{ ( \sum_{i,j=1}^M x_{ij} )^{2} }$

$\sigma^2_{Kappa}=\dfrac{OverallAcc-ExpAcc}{(1-ExpAcc)^2 \cdot N_{Total}}$

where:

1. $OverallAcc$: overall accuracy.

2. $ExpAcc$: expected accuracy of agreement if agreement were purely random.

Usage

ConfMatrix$Kappa(a = NULL)

Arguments

Returns

A list of real values containing with kappa coefficient, its variance and confidence interval.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$Kappa()

Method ModKappa()

Public method that provides the overall modified kappa coefficient. The method also offers the variance and confidence interval. The references Stehman (1997) and Foody (1992) are followed for the calculations.

$ModKappa=\dfrac{OverallAcc-\dfrac{1}{M}}{1-\dfrac{1}{M}}$

$\sigma^2_{ModKappa}=\dfrac{OverallAcc \cdot (1- OverallAcc)} { \left(1-\dfrac{1}{M} \right)^2 \cdot N_{Total}}$

where:

1. $OverallAcc$: overall accuracy.

Usage

ConfMatrix$ModKappa(a = NULL)

Arguments

Returns

A list of real values containing modified coefficient kappa, its variance and its confidence interval.

Examples

A <- matrix(c(317,61,2,35,23,120,4,29,0,0,60,0,0,0,0,8),nrow=4,ncol=4)
p <- ConfMatrix$new(A,Source="Foody 1992")
p$ModKappa()

Method UserKappa_i()

Public method derived by the kappa coefficient evaluated from the user's perspective, for a specific class i. The method also offers the variance and confidence interval. The reference Rosenfield and Fitzpatrick-Lins (1986) is followed for the calculations.

$UserKappa_i=\dfrac{UserAcc_i-\dfrac{ x_{i + }} {\sum^M_{i,j=1} x_{ij}}}{1-\dfrac{ x_{i + }} {\sum^M_{i,j=1} x_{ij}}}$

$\sigma^2_{UserKappa_i}=\dfrac{UserAcc_i \cdot (1-UserAcc_i)} { \left(1-\dfrac{ x_{i + }} {\sum^M_{i,j=1} x_{ij}}\right)^2 \cdot N_{i}}$

where:

1. $UserAcc_i$: user accuracy index for class i.

Usage

ConfMatrix$UserKappa_i(i, a = NULL)

Arguments

Returns

A list of real values containing the kappa coefficient (user’s perspective), its variance and its confidence interval.

Examples

A<-matrix(c(73,13,5,1,0,21,32,13,3,0,16,39,35, 29,13,3,5,7,28,48,1,0,2,3,17),
nrow=5,ncol=5)
p<-ConfMatrix$new(A,Source="Næsset 1996")
p$UserKappa_i(2)

Method ModKappaUser_i()

Public method, derived from the general modified kappa coefficient, which provides the modified kappa coefficient from the user's perspective and for a specific class i. Equitable probabilities of belonging to each class are assumed. The method also offers the variance and confidence interval. The references Stehman (1997) and Foody (1992) are followed for the calculations.

$ModKappaUser_i=\dfrac{UserAcc_i-\dfrac{1}{M}} {1-\dfrac{1}{M}}$

$\sigma^2_{ModKappaUser_i}=\dfrac{UserAcc_i \cdot (1- UserAcc_i)}{ \left(1- \dfrac{1}{M} \right)^2 \cdot N_{i}}$

where:

1. $UserAcc_i$: user accuracy index for class i.

Usage

ConfMatrix$ModKappaUser_i(i, a = NULL)

Arguments

Returns

A list of real values containing the modified kappa coefficient from the user's perspective, its variance and confidence interval.

Examples

A<-matrix(c(0,12,0,0,12,0,0,0,0,0,0,12,0,0,12,0),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Liu et al. 2007")
p$ModKappaUser_i(2)

Method ProdKappa_i()

Public method derived by the kappa coefficient evaluated from the producer's perspective, for a specific class i. The method also offers the variance and confidence interval. The reference Rosenfield and Fitzpatrick-Lins (1986) is followed for the calculations.

$ProdKappa_i=\dfrac{ProdAcc_i-\dfrac{ x_{ + i }} {\sum^M_{i,j=1} x_{ij}}}{1-\dfrac{ x_{+ i }} {\sum^M_{i,j=1} x_{ij}}}$

$\sigma^2_{ProdKappa_i}=\dfrac{ProdAcc_i \cdot (1- ProdAcc_i)} {\left(1-\dfrac{ x_{+ i }} {\sum^M_{i,j=1} x_{ij}} \right)^2 \cdot N_{j}}$

where:

1. $ProdAcc_i$: producer accuracy index for class i.

Usage

ConfMatrix$ProdKappa_i(i, a = NULL)

Arguments

Returns

A list of real values containing the coefficient kappa (producer’s), its variance and its confidence interval.

Examples

A <- matrix(c(73,13,5,1,0,21,32,13,3,0,16,39,35,29,13,3,5,7,28,48,1,0,2,3,17),
nrow=5,ncol=5)
p<-ConfMatrix$new(A,Source="Næsset 1996")
p$ProdKappa_i(2)

Method ModKappaProd_i()

Public method, derived from the general modified kappa coefficient, which provides the modified kappa coefficient from the producer's perspective and for a specific class i. Equitable probabilities of belonging to each class are assumed. The method also offers the variance and confidence interval. The references Stehman (1997) and Foody (1992) are followed for the calculations.

$ModKappaProd_i=\dfrac{ProdAcc_i-\dfrac{1}{M}} {1-\dfrac{1}{M}}$

$\sigma^2_{ModKappaProd_i}=\dfrac{ProdAcc_i \cdot (1- ProdAcc_i)}{ \left( 1-\dfrac{1}{M} \right)^2 \cdot N_{j}}$

where:

1. $ProdAcc_i$: producer accuracy index for class i.

Usage

ConfMatrix$ModKappaProd_i(i, a = NULL)

Arguments

Returns

A list of real values containing the modified kappa coefficient from the producer's perspective, its variance and confidence interval.

Examples

A<-matrix(c(317,61,2,35,23,120,4,29,0,0,60,0,0,0,0,8),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Foody 1992")
p$ModKappaProd_i(2)

Method DetailKappa()

Public method that calculates the general Kappa agreement index, its standard deviation and the test statistic to test its significance. The delta method has been used to calculate the sample variance. The reference Congalton and Green (2008) is followed for the computations.

$Kappa=\dfrac{OverallAcc-ExpAcc}{1-ExpAcc}$

$ExpAcc= \dfrac{x_{+ j} \cdot x_{i +}}{\sum_{(i,j=1}^M x_{ij})^{2}}$

$\sigma^2_{Kappa} = \dfrac{1}{N_{Total}} \left( \dfrac{\theta_1 (1-\theta_1) }{(1-\theta_2)^2} + \dfrac{2(1-\theta_1)(2\theta_1\theta_2-\theta_3)}{(1-\theta_2)^3} + \dfrac{(1-\theta_1)^2(\theta_4-4\theta_2^2)}{(1-\theta_2)^4} \right)$

where

$\theta_1=OverallAcc= \sum_{i, j=1}^{M} \dfrac{ x_{ii}}{ x_{ij}}$

$\theta_2=ExpAcc=\sum^M_{i=1} \left( \dfrac{x_{+ i}}{\sum_{j=1}^M x_{ij}} \cdot \dfrac{x_{i +}}{\sum_{j=1}^M x_{ij}} \right)$

$\theta_3=\sum^M_{i=1} \left( \dfrac{x_{ii} x_{+ i}}{\sum_{j=1}^M x_{ij}} \cdot \dfrac{x_{ii} x_{i +}}{\sum_{j=1}^M x_{ij}} \right)$

$\theta_4=\dfrac{1}{ ( \sum_{i,j=1}^M x_{ij})^3} \sum_{i,j=1}^M x_{ij} (x_{j+}+x_{+i})^2$

$Z=\dfrac{Kappa}{\sqrt{\sigma^2_{Kappa}}}$

Where:

1. $ExpAcc$: expected accuracy of agreement if agreement were purely random.

2. $OverallAcc$: overall accuracy.

3. $\theta_1, \theta_2, \theta_3, \theta_4$: real values.

4. $Z$: the test statistic.

Usage

ConfMatrix$DetailKappa()

Returns

A list of real values containing the kappa coefficient, its standard deviation, and the value of its test statistic.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$DetailKappa()

Method DetailCondKappa()

Public method that calculates the Kappa class agreement index (conditional Kappa) from the perspective of user (i) and producer (j) and its standard deviations. The reference Congalton and Green (2008) is followed for the computations.

$CondKappa_{user}=\dfrac{\dfrac{x_{ii}}{x_{i+}}-x_{+j}}{1-x_{+j}}$

$CondKappa_{producer}=\dfrac{\dfrac{x_{ii}}{x_{+j}}-x_{i+}}{1-x_{i+}}$

$\sigma^2_{CondKappa_{producer}}=\dfrac{1}{N_{Total}} \cdot \dfrac{x_{+j}-x_{ii}}{x_{+j}^3 (1-x_{i+})^3} \cdot ((x_{+j}-x_{ii})\cdot (x_{+j}x_{i+}-x_{ii}) + x_{ii} (1-x_{+j}-x_{i+}+x_{ii}) )$

$\sigma^2_{CondKappa_{user}}$ is done in an analogous way by exchanging $x_{i+}$ to $x_{+j}$.

Usage

ConfMatrix$DetailCondKappa()

Returns

A list of real values containing conditional Kappa index of the user's and the producer's, and its corresponding standard deviation.

Examples

A<-matrix(c(0.2361,0.0694,0.1389,0.0556,0.1667,0.0417,0.1111,0,0.1806),
ncol=3,nrow=3)
p<-ConfMatrix$new(A,Source="Czaplewski 1994")
p$DetailCondKappa ()

Method DetailWKappa()

Public method that calculates the general Kappa agreement index (weighted) and its standard deviation. The reference Fleiss et al. (1969); Næsset (1996) and Congalton and Green (2008) are followed for the computations.

Be $p_{ij}=\dfrac{x_{ij}}{\sum^M_{i,j} x_{ij}}$ for each element $i,j$ for the matrix and $0 \leq w_{ij} \leq 1$ for $i \neq j$ and $w_{ii}=1$ for $i = j$. If the elements of the weight are greater than 1, their value must be given as a percentage.

Therefore, let:

$p_o=\sum^M_{i,j=1} w_{ij}p_{ij}$

be the weighted agreement, and

$p_c=\sum^M_{i,j=1} w_{ij}p_{i+}p_{+j}$

with $p_{i+}, p_{+j}$ analogous to $x_{i+}, x_{+j}$.

Then, the weighted Kappa is defined by

$Kappa_w=\dfrac{p_o-p_c}{1-p_c}$

The variance may be estimated by

$\sigma^2_{Kappa_w}=\dfrac{1}{N_{Total} (1-p_c)^4} \left( \sum^M_{i,j=1} p_{ij} [ w_{ij} (1-p_c)-(\overline{w}_{i+}+\overline{w}_{+j}) (1-p_o)]^2 -(p_op_c-2p_c+p_o)^2 \right)$

where $\overline{w}_{i+}=\sum^M_{j=1} w_{ij}p_{+j}$ and $\overline{w}_{+j}=\sum^M_{i=1} w_{ij}p_{i+}$

Its statistic is given by:

$Z=\dfrac{Kappa_W}{\sqrt{\sigma^2_{Kappa_W}}}$

Usage

ConfMatrix$DetailWKappa(WM)

Arguments

Returns

A list with the weight matrix, kappa index obtained from the original matrix and the weight matrix, its standard deviations and the value of its test statistic.

Examples

A <- matrix(c(1,1,0,0,0,5,55,27,23,0,3,30,68,74,4,0,8,8,39,26,0,0,2,4,26),
nrow=5)
WM <- matrix(c(1,0.75,0.5,0.25,0,0.75,1,0.75,0.5,0.25,0.5,0.75,1,0.75,0.5,
0.25,0.5,0.75,1,0.75,0,0.25,0.5,0.75,1),nrow=5)
p<-ConfMatrix$new(A, Source="Næsset 1996")
p$DetailWKappa(WM)

Method Tau()

Public method that calculates the Tau index and its variance. Its value indicates how much the classification has improved compared to a random classification of the N elements into M groups. The method also offers the variance and confidence interval. The reference Ma and Redmond (1995) is followed for the computations.

$Tau = \dfrac{OverallAcc-PrAgCoef}{1-PrAgCoef}$

$PrAgCoef=\dfrac{1}{M}$

$\sigma^2_{Tau}=\dfrac{OverallAcc \cdot (1-OverallAcc)} {N_{Total} \cdot (1-PrAgCoef)^2}$

Where:

1. $OverallAcc$: overall accuracy.

2. $PrAgCoef$: a priori random agreement coefficient.

Usage

ConfMatrix$Tau(a = NULL)

Arguments

Returns

A list of real values containing the Tau index, its variance and confidence interval.

Examples

A<-matrix(c(238051,7,132,0,0,24,9,2,189,1,4086,188,0,4,16,45,1,0,939,5082,
51817,0,34,500,1867,325,17,0,0,5,11148,1618,78,0,0,0,0,48,4,834,2853,340,
32,0,197,5,151,119,135,726,6774,75,1,553,0,105,601,110,174,155,8257,8,0,
29,36,280,0,0,6,5,2993,0,115,2,0,4,124,595,0,0,4374),nrow=9,ncol=9)
p<-ConfMatrix$new(A,Source="Muñoz 2016")
p$Tau()

Method DetailWTau()

Public method that calculates the general Tau concordance index (weighted) and its standard deviation.

Be $p_{ij}=\dfrac{x_{ij}}{\sum^M_{i,j} x_{ij}}$ for each element $i,j$ for the matrix and $0 \leq w_{ij} \leq 1$ for $i \neq j$ and $w_{ii}=1$ for $i = j$. If the elements of the weight are greater than 1, their value must be given as a percentage. The following real values are defined:

$\theta_1=\sum_{i}^{M} p_{ii}$

$\theta_2=\sum^M_{i=1} w_{ij}p_{i+}$

$\theta_3=\sum^M_{i=1} \left( p_{ii} (w_{ij}+p_{+j}) \right)$

$\theta_4=\sum^M_{i,j=1} p_{ij} m_{ij}$

where $m_{ij}$ are the elements of a matrix, which are given by $(w_{ij}+p_{+j})^2$

Therefore,

$Tau_W=\dfrac{\theta_1-\theta_2}{1-\theta_2}$

$\sigma^2_{Tau_W}=\dfrac{1}{N_{Total}} \left( \dfrac{\theta_1 (1-\theta_1)}{(1-\theta_2)^2} + 2 \dfrac{1-\theta_1}{(1-\theta_2)^3} (2 \theta_1 \theta_2-\theta_3) + \dfrac{(1-\theta_1)^2}{(1-\theta_2)^4} (\theta_4 - 4 \theta_2^2) \right)$

The statistic is given by

$Z=\dfrac{Tau_W}{\sqrt{\sigma^2_{Tau_W}}}$

Where:

1. $\theta_1, \theta_2, \theta_3, \theta_4$: real values.

2. $Z$: the test statistic.

Usage

ConfMatrix$DetailWTau(WV)

Arguments

Returns

A list with the weighted Tau index, the weight matrix, its standard deviation and its statistics.

Examples

A<-matrix(c(65,6,0,4,4,81,11,7,22,5,85,3,24,8,19,90),nrow=4,ncol=4)
WV <-matrix(c(0.4, 0.1, 0.4, 0.1),ncol=4)
p<-ConfMatrix$new(A,Source="Congalton and Green 2008")
p$DetailWTau(WV)

Method Ent()

Public method for calculating product entropy,which refers to the lack of orden and predictability that the product presents. The method also offers the variance and confidence interval. The reference Finn (1993) is followed for the calculations.

$Ent=\sum^M_{i,j=1} \left(\dfrac{x_{ij}}{\sum^M_{i,j=1} x_{ij}} \cdot \log \left(\dfrac{x_{ij}}{\dfrac{ x_{i+} \cdot x_{+j}}{\sum^M_{i,j=1} x_{ij}}} \right) \right)$

$\sigma^2_{Ent}=\dfrac{Ent \cdot (1-Ent)}{N_{Total}}$

Usage

ConfMatrix$Ent(a = NULL, v = NULL)

Arguments

Returns

A list of real values containing the entropy, its variance and confidence interval.

Examples

A<-matrix(c(35,4,12,2,14,11,9,5,11,3,38,12,1,0,4,2),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Finn 1993")
p$Ent(v=2)

Method AvNormEnt()

Public method that calculates normalized entropy using the arithmetic mean of the entropies on the product and the reference. The method also offers the variance and confidence interval. The reference Strehl and Ghosh (2002) is followed for the calculations.

$AvNormEnt=\dfrac{2Ent}{Ent_i(A)+Ent_i(B)}$

$Ent_i(A)=-\sum^M_{j=1} \left( \left(\dfrac{ x_{+j}} {\sum^M_{i,j=1} x_{ij} }\right) \cdot \log \left(\dfrac{ x_{+j}} {\sum^M_{i,j=1} x_{ij} }\right) \right)$

$Ent_i(B)=-\sum^M_{i=1}\left( \left(\dfrac{ x_{i+}} {\sum^M_{i,j=1} x_{ij} }\right) \cdot \log \left(\dfrac{x_{i+}} {\sum^M_{i,j=1} x_{ij} }\right) \right)$

$\sigma^2_{AvNormEnt}=\dfrac{AvNormEnt \cdot (1-AvNormEnt)}{N_{Total}}$

where:

1. $Ent$: product entropy.

2. $Ent_i(A)$: entropy with respect to the classes i of the product. A is a matrix.

3. $Ent_i(B)$: entropy with respect to the class i on the reference. B is a matrix.

Usage

ConfMatrix$AvNormEnt(a = NULL, v = NULL)

Arguments

Returns

A list of real values containing the normalized entropy (arithmetic mean of the entropies on the product and reference), its variance and confidence interval.

Examples

A<-matrix(c(0,12,0,0,12,0,0,0,0,0,0,12,0,0,12,0),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Liu et al. 2007")
p$AvNormEnt(v=2)

Method GeomAvNormEnt()

Public method that calculates the normalized entropy using the geometric mean of the product and reference entropies. The method also offers the variance and confidence interval. The reference Ghosh et al. (2002) is followed for the calculations.

$GeomAvNormEnt=\dfrac{Ent}{\sqrt{Ent_i(A) \cdot Ent_i(B)}}$

$Ent_i(A)=-\sum^M_{j=1} \left( \left(\dfrac{ x_{+j}} {\sum^M_{i,j=1} x_{ij} }\right) \cdot \log \left(\dfrac{ x_{+j}} {\sum^M_{i,j=1} x_{ij} }\right) \right)$

$Ent_i(B)=-\sum^M_{i=1}\left( \left(\dfrac{ x_{i+}} {\sum^M_{i,j=1} x_{ij} }\right) \cdot \log \left(\dfrac{ x_{i+}} {\sum^M_{i,j=1} x_{ij} }\right) \right)$

$\sigma^2_{GeomAvNormEnt}= \dfrac{GeomAvNormEnt \cdot (1-GeomAvNormEnt)}{N_{Total}}$

where:

1. $Ent$: product entropy.

2. $Ent_i(A)$: entropy with respect to the classes i of the product. A is a matrix.

3. $Ent_i(B)$: entropy with respect to the class i of the reference. B is a matrix.

Usage

ConfMatrix$GeomAvNormEnt(a = NULL, v = NULL)

Arguments

Returns

A list of real values containing the normalized entropy (geometric mean of the entropies on the product and reference), its variance and confidence interval.

Examples

A<-matrix(c(0,12,0,0,12,0,0,0,0,0,0,12,0,0,12,0),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Liu et al. 2007")
p$GeomAvNormEnt(v=2)

Method AvMaxNormEnt()

Public method that provides normalized entropy using the arithmetic mean of the maximum entropies of the product and reference. The method also offers the variance and confidence interval. The reference Strehl (2002) is followed for the calculations.

$AvMaxNormEnt=\dfrac{2 Ent}{max(Ent_i(A))+max(Ent_i(B))}= \dfrac{Ent}{\log M}$

$Ent_i(A)=-\sum^M_{j=1} \left( \left(\dfrac{ x_{+j}} {\sum^M_{i,j=1} x_{ij} }\right) \cdot \log \left(\dfrac{ x_{+j}} {\sum^M_{i,j=1} x_{ij} }\right) \right)$

$Ent_i(B)=-\sum^M_{i=1}\left( \left(\dfrac{ x_{i+}} {\sum^M_{i,j=1} x_{ij} }\right) \cdot \log \left(\dfrac{ x_{i+}} {\sum^M_{i,j=1} x_{ij} }\right) \right)$

$\sigma^2_{AvMaxNormEnt}= \dfrac{AvMaxNormEnt \cdot (1-AvMaxNormEnt)}{N_{Total}}$

where:

1. $Ent$: product entropy.

2. $Ent_i(A)$: entropy with respect to the classes i of the product. A is a matrix.

3. $Ent_i(B)$: entropy with respect to the class i on the reference. B is a matrix.

Usage

ConfMatrix$AvMaxNormEnt(a = NULL, v = NULL)

Arguments

Returns

A list of real values containing the normalized entropy (arithmetic mean of the maximum entropies of the product and of reference), its variance, and its confidence interval.

Examples

A<-matrix(c(8,0,0,0,0,16,0,0,0,0,8,0,0,0,0,16),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Liu et al. 2007")
p$AvMaxNormEnt(v=2)

Method EntUser_i()

Public method that calculates relative change of entropy for a given class $i$ of the product. The method also offers the variance and confidence interval. The reference Finn (1993) is followed for the calculations.

$EntUser_i= \dfrac{Ent_i(A)-Ent_i(A|b_i)}{Ent_i(A)}$

$Ent_i(A)=-\sum^M_{j=1} \left( \left(\dfrac{ x_{+j}} {\sum^M_{i,j=1} x_{ij} }\right) \cdot \log \left(\dfrac{ x_{+j}} {\sum^M_{i,j=1} x_{ij} }\right) \right)$

$Ent_i(A|b_i)=-\sum^M_{j=1} \left( \left(\dfrac{ x_{ij}} { x_{i+} }\right) \cdot \log \left(\dfrac{x_{ij}} { x_{i+}}\right) \right)$

$\sigma^2_{EntUser_i}= \dfrac{EntUser_i \cdot (1-EntUser_i)}{N_{Total}}$

where:

1. $Ent_i(A)$: entropy with respect to the classes i of the product. A is a matrix.

2. $Ent_i(A|b_i)$: Producer entropy knowing that the location corresponding to reference B is in class $b_i$. B is a matrix.

Usage

ConfMatrix$EntUser_i(i, a = NULL, v = NULL)

Arguments

Returns

A list of real values containing the relative change of entropy for given class i, its variance, its confidence interval, producer's entropy, and producer's entropy knowing that the location corresponding to reference B is in class $b_i$.

Examples

A<-matrix(c(35,4,12,2,14,11,9,5,11,3,38,12,1,0,4,2),nrow=4,ncol=4)
p<-ConfMatrix$new(A,Source="Finn 1993")
p$EntUser_i(1,v=2)

Method NormEntUser()

Public method that calculates normalized entropy of the product. The method also offers the variance and confidence interval. The reference Finn (1993) is followed for the calculations.

$NormEntUser=\dfrac{Ent}{Ent_i(B)}$

$Ent_i(B)=-\sum^M_{i=1} \left( \left( \dfrac {x_{i+}} {\sum^M_{i,j=1} x_{ij} }\right) \cdot \log \left( \dfrac{ x_{i+}} {\sum^M_{i,j=1} x_{ij} }\right) \right)$

$\sigma^2_{NormEntUser}=\dfrac{NormEntUser \cdot (1-NormEntUser)}{N_{Total}}$