Title: | Clustering and Classification with the Contaminated Normal |
---|---|
Description: | Fits mixtures of multivariate contaminated normal distributions (with eigen-decomposed scale matrices) via the expectation conditional- maximization algorithm under a clustering or classification paradigm Methods are described in Antonio Punzo, Angelo Mazza, and Paul D McNicholas (2018) <doi:10.18637/jss.v085.i10>. |
Authors: | Antonio Punzo, Angelo Mazza, Paul D. McNicholas |
Maintainer: | Angelo Mazza <[email protected]> |
License: | GPL-2 |
Version: | 1.3.8 |
Built: | 2024-11-27 06:46:40 UTC |
Source: | CRAN |
This package allows to fit, according to the expectation-conditional maximization algorithm, the 14 parsimonious mixtures of multivariate contaminated normal distributions, with eigen-decomposed scale matrices, introduced by Punzo and McNicholas (2016). Model-based clustering and classification scenarios are implemented. Likelihood-based model selection criteria can be adopted to select the parsimonious model and the number of groups.
Package: | ContaminatedMixt |
Type: | Package |
Version: | 1.1 |
Date: | 2016-09-02 |
License: | GNU-2 |
Antonio Punzo, Angelo Mazza, Paul D. McNicholas
Maintainer: Angelo Mazza <[email protected]>
Punzo A., Mazza A. and McNicholas P. D. (2018). ContaminatedMixt: An R Package for Fitting Parsimonious Mixtures of Multivariate Contaminated Normal Distributions. Journal of Statistical Software, 85(10), 1–25.
Punzo A. and McNicholas P. D. (2016). Parsimonious mixtures of multivariate contaminated normal distributions. Biometrical Journal, 58(6), 1506–1537.
Evaluates the agreement of a given partition with respect to the partition arising from the mixture of multivariate contaminated normal distributions. If the mixture has been fitted for classification purposes, the agreement will be based on the unlabeled observations only.
agree(object, givgroup, criterion = "BIC")
agree(object, givgroup, criterion = "BIC")
object |
An object of class |
givgroup |
vector, of the same dimension of the number of observations used to fit the model in |
criterion |
an optional character string with the information criterion to consider; supported values are: |
A contingency table.
Antonio Punzo, Angelo Mazza, Paul D. McNicholas
Punzo A., Mazza A. and McNicholas P. D. (2018). ContaminatedMixt: An R Package for Fitting Parsimonious Mixtures of Multivariate Contaminated Normal Distributions. Journal of Statistical Software, 85(10), 1–25.
Punzo A. and McNicholas P. D. (2016). Parsimonious mixtures of multivariate contaminated normal distributions. Biometrical Journal, 58(6), 1506–1537.
ContaminatedMixt-package
,
CNmixt
Fits, by using the expectation conditional-maximization (ECM) algorithm, parsimonious mixtures of multivariate contaminated normal distributions (with eigen-decomposed scale matrices) to the given data within a clustering paradigm (default) or classification paradigm. Can be run in parallel. Likelihood-based model selection criteria are used to select the parsimonious model and the number of groups.
CNmixt(X, G, contamination = NULL, model = NULL, initialization = "mixt", alphafix = NULL, alphamin = 0.5, seed = NULL, start.z = NULL, start.v = NULL, start = 0, label = NULL, AICcond = FALSE, iter.max = 1000, threshold = 1.0e-10, parallel = FALSE, eps = 1e-100,verbose = TRUE) CNmixtCV(X, G, contamination = NULL, model = NULL, initialization = "mixt", k = 10,alphafix = NULL, alphamin = 0.5, seed = NULL, start.z = NULL, start.v = NULL, start = 0, label = NULL, iter.max = 1000, threshold = 1.0e-10, parallel = FALSE, eps = 1e-100, verbose = TRUE)
CNmixt(X, G, contamination = NULL, model = NULL, initialization = "mixt", alphafix = NULL, alphamin = 0.5, seed = NULL, start.z = NULL, start.v = NULL, start = 0, label = NULL, AICcond = FALSE, iter.max = 1000, threshold = 1.0e-10, parallel = FALSE, eps = 1e-100,verbose = TRUE) CNmixtCV(X, G, contamination = NULL, model = NULL, initialization = "mixt", k = 10,alphafix = NULL, alphamin = 0.5, seed = NULL, start.z = NULL, start.v = NULL, start = 0, label = NULL, iter.max = 1000, threshold = 1.0e-10, parallel = FALSE, eps = 1e-100, verbose = TRUE)
X |
a |
G |
a vector containing the numbers of groups to be tried. |
contamination |
an optional boolean indicating if the model(s) to be fitted have to be contaminated or not.
If |
model |
a vector indicating the model(s) to be fitted.
In the multivariate case ( |
initialization |
initialization strategy for the ECM algorithm. It can be:
|
alphafix |
a vector of length |
alphamin |
a vector of length |
seed |
the seed for the random number generator, when random initializations are used; if |
start.z |
initial |
start.v |
initial |
start |
when |
label |
a vector of integers of length equal to the number of rows of |
AICcond |
When |
iter.max |
maximum number of iterations in the ECM algorithm.
Default value is |
threshold |
threshold for Aitken's acceleration procedure.
Default value is |
parallel |
When |
eps |
an optional scalar.
It sets the smallest value for the eigenvalues of the component scale matrices.
Default value is |
k |
number of equal sized subsamples used in |
verbose |
write text to the console |
The multivariate data contained in X
are either clustered or classified using parsimonious mixtures of multivariate contaminated normal distributions with some or all of the 14 parsimonious models described in Punzo and McNicholas (2016).
Model specification (via the model
argument) follows the nomenclature popularized in other packages such as mixture and mclust.
Such a nomenclature refers to the decomposition and constraints on the scale matrix (see Banfield and Raftery, 1993, Celeux and Govaert, 1995 and Punzo and McNicholas, 2016 for details):
The nomenclature describes (in order) the volume (), shape (
), and orientation (
), in terms of
"V"
ariable, "E"
qual, or the "I"
dentity matrix.
As an example, the string "VEI"
would refer to the model where .
Note that for
, several models are equivalent (for example,
"EEE"
and "VVV"
).
Thus, for only one model from each set of equivalent models will be run.
The algorithms detailed in Celeux and Govaert (1995) are considered in the first CM-step of the ECM algorithm to update for all the models apart from
"EVE"
and "VVE"
.
For "EVE"
and "VVE"
, majorization-minimization (MM) algorithms (Hunter and Lange, 2000) and accelerated line search algorithms on the Stiefel manifold (Absil, Mahony and Sepulchre, 2009 and Browne and McNicholas, 2014), which are especially preferable in higher dimensions (Browne and McNicholas, 2014), are used to update ; the same approach is also adopted in the mixture package for those models.
Starting values are very important to the successful operation of these algorithms and so care must be taken in the interpretation of results.
All the initializations considered here provide initial quantities for the first CM-step of the ECM algorithm.
The predictive ability of a model for classification may be estimated using the cross-validated error rate, returned by CNmixtCV
or trough the the AICcond criterion (Vandewalle et al., 2013).
CNmixt
returns an object of class ContaminatedMixt
.
CNmixtCV
returns a list with the cross-validated error rate estimated for each model.
Antonio Punzo, Angelo Mazza, Paul D. McNicholas
Absil P. A., Mahony R. and Sepulchre R. (2009). Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ.
Banfield J. D. and Raftery A. E. (1993). Model-Based Gaussian and Non-Gaussian Clustering. Biometrics, 49(3), 803–821.
Browne R. P. and McNicholas P. D. (2013). Estimating Common Principal Components in High Dimensions. Advances in Data Analysis and Classification, 8(2), 217–226.
Browne, R. P. and McNicholas P. D. (2014). Orthogonal Stiefel manifold optimization for eigen-decomposed covariance parameter estimation in mixture models. Statistics and Computing, 24(2), 203–210.
Browne R. P. and McNicholas P. D. (2015). mixture: Mixture Models for Clustering and Classification. R package version 1.4.
Celeux G. and Govaert G. (1995). Gaussian Parsimonious Clustering Models. Pattern Recognition. 28(5), 781–793.
Hunter D. R. and Lange K. (2000). Rejoinder to Discussion of “Optimization Transfer Using Surrogate Objective Functions”. Journal of Computational and Graphical Statistics, 9(1), 52–59.
Punzo A., Mazza A. and McNicholas P. D. (2018). ContaminatedMixt: An R Package for Fitting Parsimonious Mixtures of Multivariate Contaminated Normal Distributions. Journal of Statistical Software, 85(10), 1–25.
Punzo A. and McNicholas P. D. (2016). Parsimonious mixtures of multivariate contaminated normal distributions. Biometrical Journal, 58(6), 1506–1537.
Vandewalle V., Biernacki C., Celeux G. and Govaert G. (2013). A predictive deviance criterion for selecting a generative model in semi-supervised classification. Computational Statistics and Data Analysis, 64, 220–236.
## Note that the example is extremely simplified ## in order to reduce computation time # Artificial data from an EEI Gaussian mixture with G = 2 components library("mnormt") p <- 2 set.seed(12345) X1 <- rmnorm(n = 200, mean = rep(2, p), varcov = diag(c(5, 0.5))) X2 <- rmnorm(n = 200, mean = rep(-2, p), varcov = diag(c(5, 0.5))) noise <- matrix(runif(n = 40, min = -20, max = 20), nrow = 20, ncol = 2) X <- rbind(X1, X2, noise) group <- rep(c(1, 2, 3), times = c(200, 200, 20)) plot(X, col = group, pch = c(3, 4, 16)[group], asp = 1, xlab = expression(X[1]), ylab = expression(X[2])) # ---------------------- # # Model-based clustering # # ---------------------- # res1 <- CNmixt(X, model = c("EEI", "VVV"), G = 2, parallel = FALSE) summary(res1) agree(res1, givgroup = group) plot(res1, contours = TRUE, asp = 1, xlab = expression(X[1]), ylab = expression(X[2])) # -------------------------- # # Model-based classification # # -------------------------- # indlab <- sample(1:400, 20) lab <- rep(0,nrow(X)) lab[indlab] <- group[indlab] res2 <- CNmixt(X, G = 2, model = "EEI", label = lab) agree(res2, givgroup = group)
## Note that the example is extremely simplified ## in order to reduce computation time # Artificial data from an EEI Gaussian mixture with G = 2 components library("mnormt") p <- 2 set.seed(12345) X1 <- rmnorm(n = 200, mean = rep(2, p), varcov = diag(c(5, 0.5))) X2 <- rmnorm(n = 200, mean = rep(-2, p), varcov = diag(c(5, 0.5))) noise <- matrix(runif(n = 40, min = -20, max = 20), nrow = 20, ncol = 2) X <- rbind(X1, X2, noise) group <- rep(c(1, 2, 3), times = c(200, 200, 20)) plot(X, col = group, pch = c(3, 4, 16)[group], asp = 1, xlab = expression(X[1]), ylab = expression(X[2])) # ---------------------- # # Model-based clustering # # ---------------------- # res1 <- CNmixt(X, model = c("EEI", "VVV"), G = 2, parallel = FALSE) summary(res1) agree(res1, givgroup = group) plot(res1, contours = TRUE, asp = 1, xlab = expression(X[1]), ylab = expression(X[2])) # -------------------------- # # Model-based classification # # -------------------------- # indlab <- sample(1:400, 20) lab <- rep(0,nrow(X)) lab[indlab] <- group[indlab] res2 <- CNmixt(X, G = 2, model = "EEI", label = lab) agree(res2, givgroup = group)
Cluster prediction for multivariate observations based on uncontaminated/contaminated normal mixture models
CNpredict(newdata, prior, mu, invSigma, eta=NULL, alpha=NULL) ## S3 method for class 'ContaminatedMixt' predict(object, newdata, ...)
CNpredict(newdata, prior, mu, invSigma, eta=NULL, alpha=NULL) ## S3 method for class 'ContaminatedMixt' predict(object, newdata, ...)
newdata |
a |
object |
an object of class |
... |
Options to be passed to |
prior |
a vector with |
mu |
a |
invSigma |
an array with |
alpha |
a vector of |
eta |
a vector of |
a vector with group membership
Antonio Punzo, Angelo Mazza, Paul D. McNicholas
Punzo A., Mazza A. and McNicholas P. D. (2018). ContaminatedMixt: An R Package for Fitting Parsimonious Mixtures of Multivariate Contaminated Normal Distributions. Journal of Statistical Software, 85(10), 1–25.
Punzo A. and McNicholas P. D. (2016). Parsimonious mixtures of multivariate contaminated normal distributions. Biometrical Journal, 58(6), 1506–1537.
point <- c(0,0,0) mu <- c(1,-2,3) Sigma <- diag(3) alpha <- 0.8 eta <- 5 f <- dCN(point, mu, Sigma, alpha, eta) x <- rCN(10, mu, Sigma, alpha, eta)
point <- c(0,0,0) mu <- c(1,-2,3) Sigma <- diag(3) alpha <- 0.8 eta <- 5 f <- dCN(point, mu, Sigma, alpha, eta) x <- rCN(10, mu, Sigma, alpha, eta)
Probability density function and random number generation for the multivariate contaminated normal distribution.
dCN(x, mu = rep(0,p), Sigma, alpha = 0.99, eta = 1.01) rCN(n, mu = rep(0,p), Sigma, alpha = 0.99, eta = 1.01)
dCN(x, mu = rep(0,p), Sigma, alpha = 0.99, eta = 1.01) rCN(n, mu = rep(0,p), Sigma, alpha = 0.99, eta = 1.01)
x |
either a vector of length |
mu |
either a vector of length |
Sigma |
a symmetric positive-definite matrix representing the scale matrix of the distribution; a vector of length 1 is also allowed (in this case, |
alpha |
proportion of good observations; it must be a number between 0 and 1 |
eta |
degree of contamination; it should be a number greater than 1 |
n |
the number of random vectors to be generated |
dCN
returns a vector of density values; rCN
returns a matrix of n
rows of random vectors
Antonio Punzo, Angelo Mazza, Paul D. McNicholas
Punzo A., Mazza A. and McNicholas P. D. (2018). ContaminatedMixt: An R Package for Fitting Parsimonious Mixtures of Multivariate Contaminated Normal Distributions. Journal of Statistical Software, 85(10), 1–25.
Punzo A. and McNicholas P. D. (2016). Parsimonious mixtures of multivariate contaminated normal distributions. Biometrical Journal, 58(6), 1506–1537.
point <- c(0,0,0) mu <- c(1,-2,3) Sigma <- diag(3) alpha <- 0.8 eta <- 5 f <- dCN(point, mu, Sigma, alpha, eta) x <- rCN(10, mu, Sigma, alpha, eta)
point <- c(0,0,0) mu <- c(1,-2,3) Sigma <- diag(3) alpha <- 0.8 eta <- 5 f <- dCN(point, mu, Sigma, alpha, eta) x <- rCN(10, mu, Sigma, alpha, eta)
ContaminatedMixt
Class Objects.
These functions extract values from ContaminatedMixt
class objects.
getBestModel(object, criterion = "BIC", G = NULL, model = NULL, contamination = NULL) getPosterior(object, ...) getSize(object, ...) getCluster(object, ...) getPar(object, ...) getCV(object) getIC(object,criteria) getDetection(object,...) whichBest(object, criteria = NULL, G = NULL, model = NULL, contamination = NULL) whichBestCV (object, G=NULL, model=NULL, contamination=NULL) ## S3 method for class 'ContaminatedMixt' summary(object, criterion = "BIC", digits = getOption("digits")-2, ...) ## S3 method for class 'ContaminatedMixt' print(x, ...)
getBestModel(object, criterion = "BIC", G = NULL, model = NULL, contamination = NULL) getPosterior(object, ...) getSize(object, ...) getCluster(object, ...) getPar(object, ...) getCV(object) getIC(object,criteria) getDetection(object,...) whichBest(object, criteria = NULL, G = NULL, model = NULL, contamination = NULL) whichBestCV (object, G=NULL, model=NULL, contamination=NULL) ## S3 method for class 'ContaminatedMixt' summary(object, criterion = "BIC", digits = getOption("digits")-2, ...) ## S3 method for class 'ContaminatedMixt' print(x, ...)
object , x
|
a class |
criterion |
a string with the information criterion to consider; supported values are: |
criteria |
a vector of strings with the names of information criteria to consider.
If |
G |
an optional vector containing the numbers of groups to consider. If not specified, all the estimated models are considered. |
model |
an optional vector of character strings indicating the parsimonious models to consider. If not specified, all the estimated models are considered. |
contamination |
an optional boolean indicating if the model(s) to be considered have to be contaminated or not. If |
digits |
integer used for number formatting. |
... |
additional arguments to be passed to |
When several models have been estimated, these functions consider the best model according to the information criterion in criterion
, among the estimated models having a number of components among those in G
and a parsimonious model among those in model
.
whichBestCV
considers the best model according to the cross-validated error rates computed by CNmixtCV
.
getIC
provides values for the information criteria in criteria
.
The getBestModel
method returns a ContaminatedMixt
object containing the best model only, selected as described above.
Carries out the M-step for EM algorithm
m.step(X, modelname, z, mtol=1e-10, mmax=10)
m.step(X, modelname, z, mtol=1e-10, mmax=10)
X |
a matrix such that |
modelname |
A three letter sequence indicating the covariance structure.
Possible values are: |
z |
A matrix of weights such that |
mtol |
The convergence criteria for the M-step if an iterative procedure is necessary. |
mmax |
The maximum number of iterations for an iterative procedure. |
A list of the model parameters with the mu
, Sigma
, invSigma
and px
for each group.
Antonio Punzo, Angelo Mazza, Paul D. McNicholas
Punzo A., Mazza A. and McNicholas P. D. (2018). ContaminatedMixt: An R Package for Fitting Parsimonious Mixtures of Multivariate Contaminated Normal Distributions. Journal of Statistical Software, 85(10), 1–25.
Punzo A. and McNicholas P. D. (2016). Parsimonious mixtures of multivariate contaminated normal distributions. Biometrical Journal, 58(6), 1506–1537.
point <- c(0,0,0) mu <- c(1,-2,3) Sigma <- diag(3) alpha <- 0.8 eta <- 5 f <- dCN(point, mu, Sigma, alpha, eta) x <- rCN(10, mu, Sigma, alpha, eta)
point <- c(0,0,0) mu <- c(1,-2,3) Sigma <- diag(3) alpha <- 0.8 eta <- 5 f <- dCN(point, mu, Sigma, alpha, eta) x <- rCN(10, mu, Sigma, alpha, eta)
A matrix of scatterplots, for objects of class ContaminatedMixt
, is produced.
## S3 method for class 'ContaminatedMixt' pairs(x, criterion = "BIC", ...)
## S3 method for class 'ContaminatedMixt' pairs(x, criterion = "BIC", ...)
x |
an object of class |
criterion |
an optional character string with the information criterion to consider; supported values are: |
... |
Options to be passed to |
Antonio Punzo, Angelo Mazza, Paul D. McNicholas
Scatterplot, with optionally superimposed contours, for objects of class ContaminatedMixt
.
## S3 method for class 'ContaminatedMixt' plot(x, criterion = "BIC", contours = FALSE, xmarg = 1, ymarg = 2, res = 200, levels = seq(.0001,1,by=0.01), ...)
## S3 method for class 'ContaminatedMixt' plot(x, criterion = "BIC", contours = FALSE, xmarg = 1, ymarg = 2, res = 200, levels = seq(.0001,1,by=0.01), ...)
x |
an object of class |
criterion |
a string with the information criterion to consider; supported values are: |
contours |
if |
xmarg |
scalar argument giving the position of the variable to be used on the |
ymarg |
scalar argument giving the position of the variable to be used on the |
res |
scalar argument giving the resolution for the calculation grid required for the contour plot.
Default is 200, which results in a |
levels |
Numeric vector giving the levels at which contours should be drawn. Default is to draw a contour in 0.01 steps, starting from the contour of height .0001. This may result in more/less contours than desired depending on the resulting density. |
... |
Options to be passed to |
Antonio Punzo, Angelo Mazza, Paul D. McNicholas
These data are the results of a chemical analysis of wines grown in the same region in Italy but derived from three different cultivars.
The analysis determined the quantities of 13 constituents found in each of the three types of wine: Barolo, Grignolino, Barbera.
The data set is used to evaluate the ability of the CNmixt()
function in clustering the data assuming unknown their cultivars.
data(wine)
data(wine)
This data frame contains 178 rows, each corresponding to a different cultivar of wine produced in Piedmont (Italy), and 14 columns.
The first column is the type of wine (Type
), a factor variable with the following levels: Barolo
, Grignolino
, Barbera
.
The variables measured on the three types of wines are the following:
Alcohol
, Malic acid
, Ash
, Alcalinity
, Magnesium
, Phenols
, Flavanoids
, Nonflavanoids
, Proanthocyanins
, Color intensity
, Hue
, OD280.OD315Dilution
, Proline
.
All variables but the label class are continuous.
The original data set comprises 27 variables. Here a subset of 14 variables only has been included.
This dataset is from the UCI machine learning repository and it is available at http://archive.ics.uci.edu/ml/datasets/Wine.
Forina M., Lanteri S. Armanino C., Casolino C., Casale M., Oliveri, P. (2008). V-PARVUS. An Extendible Pachage of programs for esplorative data analysis, classification and regression analysis. Dip. Chimica e Tecnologie Farmaceutiche ed Alimentari, Universita' di Genova.
ContaminatedMixt-package
, CNmixt
data("wine") group <- wine[, 1] pairs(wine[, -1], cex = 0.6, pch = c(2, 3, 1)[group], col = c(3, 4, 2)[group], gap = 0, cex.labels = 0.6) res3 <- CNmixt(wine[, -1], G = 3, model = "EEE", initialization = "random.post", seed = 5, parallel = FALSE) agree(res3, givgroup = group) pairs(res3, cex = 0.6, gap = 0, cex.labels = 0.6)
data("wine") group <- wine[, 1] pairs(wine[, -1], cex = 0.6, pch = c(2, 3, 1)[group], col = c(3, 4, 2)[group], gap = 0, cex.labels = 0.6) res3 <- CNmixt(wine[, -1], G = 3, model = "EEE", initialization = "random.post", seed = 5, parallel = FALSE) agree(res3, givgroup = group) pairs(res3, cex = 0.6, gap = 0, cex.labels = 0.6)