Package 'Rdimtools'

Description

It generates samples from predefined shapes, set by dname parameter. Also incorporated a functionality to add white noise with degree noise.

Usage

aux.gensamples(
  n = 496,
  noise = 0.01,
  dname = c("swiss", "crown", "helix", "saddle", "ribbon", "bswiss", "cswiss",
    "twinpeaks", "sinusoid", "mobius", "R12in72"),
  ...
)

Arguments

Value

an $(n\times p)$ matrix of generated data by row. For all methods other than "R12in72", it returns a matrix with $p=3$.

Author(s)

Kisung You

References

Hein M, Audibert J (2005). “Intrinsic Dimensionality Estimation of Submanifolds in $R^ d$.” In Proceedings of the 22nd International Conference on Machine Learning, 289–296.

van der Maaten L (2009). “Learning a Parametric Embedding by Preserving Local Structure.” Proceedings of AI-STATS.

Examples

## generating toy example datasets
set.seed(100)
dat.swiss = aux.gensamples(50, dname="swiss")
dat.crown = aux.gensamples(50, dname="crown")
dat.helix = aux.gensamples(50, dname="helix")

Description

Given data, it first computes pairwise distance (method) using one of measures defined from dist function. Then, type controls how nearest neighborhood graph should be constructed. Finally, symmetric parameter controls how nearest neighborhood graph should be symmetrized.

Usage

aux.graphnbd(
  data,
  method = "euclidean",
  type = c("proportion", 0.1),
  symmetric = "union",
  pval = 2
)

Arguments

Value

a named list containing

mask

a binary matrix of indicating existence of an edge for each element.

dist

corresponding distance matrix. -Inf is returned for non-connecting edges.

Nearest Neighbor(NN) search

Our package supports three ways of defining nearest neighborhood. First is knn, which finds k nearest points and flag them as neighbors. Second is enn - epsilon nearest neighbor - that connects all the data poinst within a certain radius. Finally, proportion flag is to connect proportion-amount of data points sequentially from the nearest to farthest.

Symmetrization

In many graph setting, it starts from dealing with undirected graphs. NN search, however, does not necessarily guarantee if symmetric connectivity would appear or not. There are two easy options for symmetrization; intersect for connecting two nodes if both of them are nearest neighbors of each other and union for only either of them to be present.

Author(s)

Kisung You

Examples

## Generate data
set.seed(100)
X = aux.gensamples(n=100)

## Test three different types of neighborhood connectivity
nn1 = aux.graphnbd(X,type=c("knn",20))         # knn with k=20
nn2 = aux.graphnbd(X,type=c("enn",1))          # enn with radius = 1
nn3 = aux.graphnbd(X,type=c("proportion",0.4)) # connecting 40% of edges

## Visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3), pty="s")
image(nn1$mask); title("knn with k=20")
image(nn2$mask); title("enn with radius=1")
image(nn3$mask); title("proportion of ratio=0.4")
par(opar)

Description

From the celebrated Mercer's Theorem, we know that for a mapping $\phi$, there exists a kernel function - or, symmetric bilinear form, $K$ such that

$K(x,y) = <\phi(x),\phi(y)>$

where $<,>$ is standard inner product. aux.kernelcov is a collection of 20 such positive definite kernel functions, as well as centering of such kernel since covariance requires a mean to be subtracted and a set of transformed values $\phi(x_i),i=1,2,\dots,n$ are not centered after transformation. Since some kernels require parameters - up to 2, its usage will be listed in arguments section.

Usage

aux.kernelcov(X, ktype)

Arguments

Details

There are 20 kernels supported. Belows are the kernels when given two vectors $x,y$, $K(x,y)$

linear

$=<x,y>+c$

polynomial

$=(<x,y>+c)^d$

gaussian

$=exp(-c\|x-y\|^2)$, $c>0$

laplacian

$=exp(-c\|x-y\|)$, $c>0$

anova

$=\sum_k exp(-c(x_k-y_k)^2)^d$, $c>0,d\ge 1$

sigmoid

$=tanh(a<x,y>+b)$

rational quadratic

$=1-(\|x-y\|^2)/(\|x-y\|^2+c)$

multiquadric

$=\sqrt{\|x-y\|^2 + c^2}$

inverse quadric

$=1/(\|x-y\|^2+c^2)$

inverse multiquadric

$=1/\sqrt{\|x-y\|^2+c^2}$

circular

$= \frac{2}{\pi} arccos(-\frac{\|x-y\|}{c}) - \frac{2}{\pi} \frac{\|x-y\|}{c}\sqrt{1-(\|x-y\|/c)^2}$, $c>0$

spherical

$= 1-1.5\frac{\|x-y\|}{c}+0.5(\|x-y\|/c)^3$, $c>0$

power/triangular

$=-\|x-y\|^d$, $d\ge 1$

log

$=-\log (\|x-y\|^d+1)$

spline

$= \prod_i ( 1+x_i y_i(1+min(x_i,y_i)) - \frac{x_i + y_i}{2} min(x_i,y_i)^2 + \frac{min(x_i,y_i)^3}{3} )$

Cauchy

$=\frac{c^2}{c^2+\|x-y\|^2}$

Chi-squared

$=\sum_i \frac{2x_i y_i}{x_i+y_i}$

histogram intersection

$=\sum_i min(x_i,y_i)$

generalized histogram intersection

$=sum_i min( |x_i|^c,|y_i|^d )$

generalized Student-t

$=1/(1+\|x-y\|^d)$, $d\ge 1$

Value

a named list containing

K

a $(p\times p)$ kernelizd gram matrix.

Kcenter

a $(p\times p)$ centered version of K.

Author(s)

Kisung You

References

Hofmann, T., Scholkopf, B., and Smola, A.J. (2008) Kernel methods in machine learning. arXiv:math/0701907.

Examples

## generate a toy data
set.seed(100)
X = aux.gensamples(n=100)

## compute a few kernels
Klin = aux.kernelcov(X, ktype=c("linear",0))
Kgau = aux.kernelcov(X, ktype=c("gaussian",1))
Klap = aux.kernelcov(X, ktype=c("laplacian",1))

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3), pty="s")
image(Klin$K, main="kernel=linear")
image(Kgau$K, main="kernel=gaussian")
image(Klap$K, main="kernel=laplacian")
par(opar)

Description

This function is mainly used for tracking progress for this package.

Usage

aux.pkgstat()

Examples

## run with following command
aux.pkgstat()

Description

aux.preprocess can perform one of following operations; "center", "scale", "cscale", "decorrelate" and "whiten". See below for more details.

Usage

aux.preprocess(
  data,
  type = c("center", "scale", "cscale", "decorrelate", "whiten")
)

Arguments

Value

named list containing:

pX

an $(n\times p)$ matrix after preprocessing in accordance with type parameter

info

a list containing

• type: name of preprocessing procedure.

• mean: a mean vector of length $p$.

• multiplier: a $(p\times p)$ matrix or 1 for "center".

Operations

we have following operations,

"center"

subtracts mean of each column so that every variable has mean $0$.

"scale"

turns each column corresponding to variable have variance $1$.

"cscale"

combines "center" and "scale".

"decorrelate"

"center" and sets its covariance term having diagonal entries only.

"whiten"

"decorrelate" and sets all diagonal elements be $1$.

Author(s)

Kisung You

Examples

## Generate data
set.seed(100)
X = aux.gensamples(n=200)

## 5 types of preprocessing
X_center = aux.preprocess(X)
X_scale  = aux.preprocess(X,type="scale")
X_cscale = aux.preprocess(X,type="cscale")
X_decorr = aux.preprocess(X,type="decorrelate")
X_whiten = aux.preprocess(X,type="whiten")

Description

This is a fast implementation of Floyd-Warshall algorithm to find the shortest path in a pairwise sense using 'RcppArmadillo'. A logical input is also accepted.

Usage

aux.shortestpath(dist)

Arguments

Value

an $(n\times n)$ matrix containing pairwise shortest path.

Author(s)

Kisung You

References

Floyd, R.W. (1962) Algorithm 97: Shortest Path. Commincations of the ACMS, Vol.5(6):345.

Examples

## generate a toy data
X = aux.gensamples(n=10)

## Find knn graph with k=5
Xgraph = aux.graphnbd(X,type=c("knn",5))

## Separately use binarized and real distance matrices
W1 = aux.shortestpath(Xgraph$mask)
W2 = aux.shortestpath(Xgraph$dist)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,2), pty="s")
image(W1, main="from binarized")
image(W2, main="from Euclidean distance")
par(opar)

Description

Adaptive Dimension Reduction (Ding et al. 2002) iteratively finds the best subspace to perform data clustering. It can be regarded as one of remedies for clustering in high dimensional space. Eigenvectors of a between-cluster scatter matrix are used as basis of projection.

Usage

do.adr(X, ndim = 2, ...)

Arguments

Value

a named Rdimtools S3 object containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

projection

a $(p\times ndim)$ whose columns are basis for projection.

trfinfo

a list containing information for out-of-sample prediction.

algorithm

name of the algorithm.

References

Ding C, Xiaofeng He, Hongyuan Zha, Simon HD (2002). “Adaptive Dimension Reduction for Clustering High Dimensional Data.” In Proceedings 2002 IEEE International Conference on Data Mining, 147–154.

See Also

do.ldakm

Examples

## load iris data
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## compare ADR with other methods
outADR = do.adr(X)
outPCA = do.pca(X)
outLDA = do.lda(X, label)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(outADR$Y, col=label, pch=19, main="ADR")
plot(outPCA$Y, col=label, pch=19, main="PCA")
plot(outLDA$Y, col=label, pch=19, main="LDA")
par(opar)

Description

Adaptive Maximum Margin Criterion (AMMC) is a supervised linear dimension reduction method. The method uses different weights to characterize the different contributions of the training samples embedded in MMC framework. With the choice of a=0, b=0, and lambda=1, it is identical to standard MMC method.

Usage

do.ammc(
  X,
  label,
  ndim = 2,
  preprocess = c("center", "scale", "cscale", "decorrelate", "whiten"),
  a = 1,
  b = 1,
  lambda = 1
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Lu J, Tan Y (2011). “Adaptive Maximum Margin Criterion for Image Classification.” In 2011 IEEE International Conference on Multimedia and Expo, 1–6.

See Also

do.mmc

Examples

## load iris data
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## try different lambda values
out1 = do.ammc(X, label, lambda=0.1)
out2 = do.ammc(X, label, lambda=1)
out3 = do.ammc(X, label, lambda=10)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, main="AMMC::lambda=0.1", pch=19, cex=0.5, col=label)
plot(out2$Y, main="AMMC::lambda=1",   pch=19, cex=0.5, col=label)
plot(out3$Y, main="AMMC::lambda=10",  pch=19, cex=0.5, col=label)
par(opar)

Description

Average Neighborhood Margin Maximization (ANMM) is a supervised method for feature extraction. It aims to find a projection mapping in the following manner; for each data point, the algorithm tries to pull the neighboring points in the same class while pushing neighboring points of different classes far away. It is known that ANMM does suffer less from small sample size problem, which is bottleneck for LDA.

Usage

do.anmm(
  X,
  label,
  ndim = 2,
  preprocess = c("null", "center", "scale", "cscale", "decorrelate", "whiten"),
  No = ceiling(nrow(X)/10),
  Ne = ceiling(nrow(X)/10)
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Wang F, Zhang C (2007). “Feature Extraction by Maximizing the Average Neighborhood Margin.” In 2007 IEEE Conference on Computer Vision and Pattern Recognition, 1–8.

Examples

## load iris data
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## perform ANMM on different choices of neighborhood size
out1 = do.anmm(X, label, No=6, Ne=6)
out2 = do.anmm(X, label, No=2, Ne=10)
out3 = do.anmm(X, label, No=10,Ne=2)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, main="(No,Ne)=(6,6)",  pch=19, cex=0.5, col=label)
plot(out2$Y, main="(No,Ne)=(2,10)", pch=19, cex=0.5, col=label)
plot(out3$Y, main="(No,Ne)=(10,2)", pch=19, cex=0.5, col=label)
par(opar)

Description

Adaptive Subspace Iteration (ASI) iteratively finds the best subspace to perform data clustering. It can be regarded as one of remedies for clustering in high dimensional space. Eigenvectors of a within-cluster scatter matrix are used as basis of projection.

Usage

do.asi(X, ndim = 2, ...)

Arguments

Value

a named Rdimtools S3 object containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

projection

a $(p\times ndim)$ whose columns are basis for projection.

algorithm

name of the algorithm.

Author(s)

Kisung You

References

Li T, Ma S, Ogihara M (2004). “Document Clustering via Adaptive Subspace Iteration.” In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 218.

See Also

do.ldakm

Examples

## use iris data
data(iris, package="Rdimtools")
set.seed(100)
subid = sample(1:150, 50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## compare ASI with other methods
outASI = do.asi(X)
outPCA = do.pca(X)
outLDA = do.lda(X, label)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(outASI$Y, pch=19, col=label, main="ASI")
plot(outPCA$Y, pch=19, col=label, main="PCA")
plot(outLDA$Y, pch=19, col=label, main="LDA")
par(opar)

Description

A Bayesian formulation of classical Multidimensional Scaling is presented. Even though this method is based on MCMC sampling, we only return maximum a posterior (MAP) estimate that maximizes the posterior distribution. Due to its nature without any special tuning, increasing mc.iter requires much computation. A note on the method is that this algorithm does not return an explicit form of projection matrix so it's classified in our package as a nonlinear method. Also, automatic dimension selection is not supported for simplicity as well as consistency with other methods in the package.

Usage

do.bmds(
  X,
  ndim = 2,
  par.a = 5,
  par.alpha = 0.5,
  par.step = 1,
  mc.iter = 50,
  print.progress = FALSE
)

Arguments

Value

a named Rdimtools S3 object containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

algorithm

name of the algorithm.

Author(s)

Kisung You

References

Oh M, Raftery AE (2001). “Bayesian Multidimensional Scaling and Choice of Dimension.” Journal of the American Statistical Association, 96(455), 1031–1044.

Examples

## load iris data
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## compare with other methods
outBMD <- do.bmds(X, ndim=2)
outPCA <- do.pca(X, ndim=2)
outLDA <- do.lda(X, label, ndim=2)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(outBMD$Y, pch=19, col=label, main="Bayesian MDS")
plot(outPCA$Y, pch=19, col=label, main="PCA")
plot(outLDA$Y, pch=19, col=label, main="LDA")
par(opar)

Description

Bayesian PCA (BPCA) is a further variant of PCA in that it imposes prior and encodes basis selection mechanism. Even though the model is fully Bayesian, do.bpca faithfully follows the original paper by Bishop in that it only returns the mode value of posterior as an estimate, in conjunction with ARD-motivated prior as well as consideration of variance to be estimated. Unlike PPCA, it uses full basis and returns relative weight for each base in that the smaller $\alpha$ value is, the more likely corresponding column vector of mp.W to be selected as potential basis.

Usage

do.bpca(X, ndim = 2, ...)

Arguments

Value

a named Rdimtools S3 object containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

projection

a $(p\times ndim)$ whose columns are basis for projection.

mp.itercount

the number of iterations taken for EM algorithm to converge.

mp.sigma2

estimated $\sigma^2$ value via EM algorithm.

mp.alpha

length-ndim-1 vector of relative weight for each base in mp.W.

mp.W

an $(ndim\times ndim-1)$ matrix from EM update.

algorithm

name of the algorithm.

Author(s)

Kisung You

References

Bishop C (1999). “Bayesian PCA.” In Advances in Neural Information Processing Systems, volume 11, 382–388.

See Also

do.pca, do.ppca

Examples

## Not run: 
## use iris dataset
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
lab   = as.factor(iris[subid,5])

## compare BPCA with others
out1  <- do.bpca(X, ndim=2)
out2  <- do.pca(X,  ndim=2)
out3  <- do.lda(X, lab, ndim=2)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, col=lab, pch=19, cex=0.8, main="Bayesian PCA")
plot(out2$Y, col=lab, pch=19, cex=0.8, main="PCA")
plot(out3$Y, col=lab, pch=19, cex=0.8, main="LDA")
par(opar)

## End(Not run)

Description

Canonical Correlation Analysis (CCA) is similar to Partial Least Squares (PLS), except for one objective; while PLS focuses on maximizing covariance, CCA maximizes the correlation. This difference sometimes incurs quite distinct results compared to PLS. For algorithm aspects, we used recursive gram-schmidt orthogonalization in conjunction with extracting projection vectors under eigen-decomposition formulation, as the problem dimension matters only up to original dimensionality.

Usage

do.cca(data1, data2, ndim = 2)

Arguments

Value

a named list containing

Y1

an $(n\times ndim)$ matrix of projected observations from data1.

Y2

an $(n\times ndim)$ matrix of projected observations from data2.

projection1

a $(N\times ndim)$ whose columns are loadings for data1.

projection2

a $(M\times ndim)$ whose columns are loadings for data2.

trfinfo1

a list containing information for out-of-sample prediction for data1.

trfinfo2

a list containing information for out-of-sample prediction for data2.

eigvals

a vector of eigenvalues for iterative decomposition.

Author(s)

Kisung You

References

Hotelling H (1936). “RELATIONS BETWEEN TWO SETS OF VARIATES.” Biometrika, 28(3-4), 321–377.

See Also

do.pls

Examples

## generate 2 normal data matrices
set.seed(100)
mat1 = matrix(rnorm(100*12),nrow=100)+10 # 12-dim normal
mat2 = matrix(rnorm(100*6), nrow=100)-10 # 6-dim normal

## project onto 2 dimensional space for each data
output = do.cca(mat1, mat2, ndim=2)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,2))
plot(output$Y1, main="proj(mat1)")
plot(output$Y2, main="proj(mat2)")
par(opar)

Description

Constrained Graph Embedding (CGE) is a semi-supervised embedding method that incorporates partially available label information into the graph structure that find embeddings consistent with the labels.

Usage

do.cge(
  X,
  label,
  ndim = 2,
  type = c("proportion", 0.1),
  preprocess = c("null", "center", "scale", "cscale", "whiten", "decorrelate")
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

Author(s)

Kisung You

References

He X, Ji M, Bao H (2009). “Graph Embedding with Constraints.” In IJCAI.

Examples

## use iris data
data(iris)
X     = as.matrix(iris[,2:4])
label = as.integer(iris[,5])
lcols = as.factor(label)

## copy a label and let 10% of elements be missing
nlabel = length(label)
nmissing = round(nlabel*0.10)
label_missing = label
label_missing[sample(1:nlabel, nmissing)]=NA

## try different neighborhood sizes
out1 = do.cge(X, label_missing, type=c("proportion",0.10))
out2 = do.cge(X, label_missing, type=c("proportion",0.25))
out3 = do.cge(X, label_missing, type=c("proportion",0.50))

## visualize
opar = par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, main="10% connected",  pch=19, col=lcols)
plot(out2$Y, main="25% connected", pch=19, col=lcols)
plot(out3$Y, main="50% connected", pch=19, col=lcols)
par(opar)

Description

Conformal Isomap(C-Isomap) is a variant of a celebrated method of Isomap. It aims at, rather than preserving full isometry, maintaining infinitestimal angles - conformality - in that it alters geodesic distance to reflect scale information.

Usage

do.cisomap(
  X,
  ndim = 2,
  type = c("proportion", 0.1),
  symmetric = c("union", "intersect", "asymmetric"),
  weight = TRUE,
  preprocess = c("center", "scale", "cscale", "whiten", "decorrelate")
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

Author(s)

Kisung You

References

Silva VD, Tenenbaum JB (2003). “Global Versus Local Methods in Nonlinear Dimensionality Reduction.” In Becker S, Thrun S, Obermayer K (eds.), Advances in Neural Information Processing Systems 15, 721–728. MIT Press.

Examples

## generate data
set.seed(100)
X <- aux.gensamples(dname="cswiss",n=100)

## 1. original Isomap
output1 <- do.isomap(X,ndim=2)

## 2. C-Isomap
output2 <- do.cisomap(X,ndim=2)

## 3. C-Isomap on a binarized graph
output3 <- do.cisomap(X,ndim=2,weight=FALSE)

## Visualize three different projections
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(output1$Y, main="Isomap")
plot(output2$Y, main="C-Isomap")
plot(output3$Y, main="Binarized C-Isomap")
par(opar)

Description

One of drawbacks of Neighborhood Preserving Embedding (NPE) is the small-sample-size problem under high-dimensionality of original data, where singular matrices to be decomposed suffer from rank deficiency. Instead of applying PCA as a preprocessing step, Complete NPE (CNPE) transforms the singular generalized eigensystem computation of NPE into two eigenvalue decomposition problems.

Usage

do.cnpe(
  X,
  ndim = 2,
  type = c("proportion", 0.1),
  preprocess = c("center", "scale", "cscale", "decorrelate", "whiten")
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Wang Y, Wu Y (2010). “Complete Neighborhood Preserving Embedding for Face Recognition.” Pattern Recognition, 43(3), 1008–1015.

Examples

## generate data of 3 types with clear difference
dt1  = aux.gensamples(n=20)-50
dt2  = aux.gensamples(n=20)
dt3  = aux.gensamples(n=20)+50
lab  = rep(1:3, each=20)

## merge the data
X      = rbind(dt1,dt2,dt3)

## try different numbers for neighborhood size
out1 = do.cnpe(X, type=c("proportion",0.10))
out2 = do.cnpe(X, type=c("proportion",0.25))
out3 = do.cnpe(X, type=c("proportion",0.50))

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, col=lab, pch=19, main="CNPE::10% connected")
plot(out2$Y, col=lab, pch=19, main="CNPE::25% connected")
plot(out3$Y, col=lab, pch=19, main="CNPE::50% connected")
par(opar)

Description

Curvilinear Component Analysis (CRCA) is a type of self-organizing algorithms for manifold learning. Like MDS, it aims at minimizing a cost function (Stress) based on pairwise proximity. Parameter lambda is a heaviside function for penalizing distance pair of embedded data, and alpha controls learning rate similar to that of subgradient method in that at each iteration $t$ the gradient is weighted by $\alpha /t$.

Usage

do.crca(X, ndim = 2, lambda = 1, alpha = 1, maxiter = 1000, tolerance = 1e-06)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

niter

the number of iterations until convergence.

trfinfo

a list containing information for out-of-sample prediction.

Author(s)

Kisung You

References

Demartines P, Herault J (1997). “Curvilinear Component Analysis: A Self-Organizing Neural Network for Nonlinear Mapping of Data Sets.” IEEE Transactions on Neural Networks, 8(1), 148–154.

Hérault J, Jausions-Picaud C, Guérin-Dugué A (1999). “Curvilinear Component Analysis for High-Dimensional Data Representation: I. Theoretical Aspects and Practical Use in the Presence of Noise.” In Goos G, Hartmanis J, van Leeuwen J, Mira J, Sánchez-Andrés JV (eds.), Engineering Applications of Bio-Inspired Artificial Neural Networks, volume 1607, 625–634. Springer Berlin Heidelberg, Berlin, Heidelberg. ISBN 978-3-540-66068-2 978-3-540-48772-2.

See Also

do.crda

Examples

## use iris data
data(iris)
set.seed(100)
subid = sample(1:150, 50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## different initial learning rates
out1 <- do.crca(X,alpha=1)
out2 <- do.crca(X,alpha=5)
out3 <- do.crca(X,alpha=10)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, col=label, pch=19, main="alpha=1.0")
plot(out2$Y, col=label, pch=19, main="alpha=5.0")
plot(out3$Y, col=label, pch=19, main="alpha=10.0")
par(opar)

Description

Curvilinear Distance Analysis (CRDA) is a variant of Curvilinear Component Analysis in that the input pairwise distance is altered by curvilinear distance on a data manifold. Like in Isomap, it first generates neighborhood graph and finds shortest path on a constructed graph so that the shortest-path length plays as an approximate geodesic distance on nonlinear manifolds.

Usage

do.crda(
  X,
  ndim = 2,
  type = c("proportion", 0.1),
  symmetric = "union",
  weight = TRUE,
  lambda = 1,
  alpha = 1,
  maxiter = 1000,
  tolerance = 1e-06
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

niter

the number of iterations until convergence.

trfinfo

a list containing information for out-of-sample prediction.

Author(s)

Kisung You

References

Lee JA, Lendasse A, Verleysen M (2002). “Curvilinear Distance Analysis versus Isomap.” In ESANN.

Lee JA, Lendasse A, Verleysen M (2004). “Nonlinear Projection with Curvilinear Distances: Isomap versus Curvilinear Distance Analysis.” Neurocomputing, 57, 49–76.

See Also

do.isomap, do.crca

Examples

## load iris data
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## different settings of connectivity
out1 <- do.crda(X, type=c("proportion",0.10))
out2 <- do.crda(X, type=c("proportion",0.25))
out3 <- do.crda(X, type=c("proportion",0.50))

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, col=label, pch=19, main="10% connected")
plot(out2$Y, col=label, pch=19, main="25% connected")
plot(out3$Y, col=label, pch=19, main="50% connected")
par(opar)

Description

Collaborative Representation-based Projection (CRP) is an unsupervised linear dimension reduction method. Its embedding is based on $\ell$_2 graph construction, similar to that of SPP where sparsity constraint is imposed via $\ell_1$ optimization problem. Note that though it may be way faster, rank deficiency can pose a great deal of problems, especially when the dataset is large.

Usage

do.crp(
  X,
  ndim = 2,
  preprocess = c("center", "scale", "cscale", "decorrelate", "whiten"),
  lambda = 1
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Yang W, Wang Z, Sun C (2015). “A Collaborative Representation Based Projections Method for Feature Extraction.” Pattern Recognition, 48(1), 20–27.

See Also

do.spp

Examples

## use iris dataset
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
lab   = as.factor(iris[subid,5])

## test different regularization parameters
out1 <- do.crp(X,ndim=2,lambda=0.1)
out2 <- do.crp(X,ndim=2,lambda=1)
out3 <- do.crp(X,ndim=2,lambda=10)

# visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, col=lab, pch=19, main="CRP::lambda=0.1")
plot(out2$Y, col=lab, pch=19, main="CRP::lambda=1")
plot(out3$Y, col=lab, pch=19, main="CRP::lambda=10")
par(opar)

Description

Constraint Score (Zhang et al. 2008) is a filter-type algorithm for feature selection using pairwise constraints. It first marks all pairwise constraints as same- and different-cluster and construct a feature score for both constraints. It takes ratio or difference of feature score vectors and selects the indices with smallest values.

Usage

do.cscore(X, label, ndim = 2, ...)

Arguments

Value

a named Rdimtools S3 object containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

cscore

a length-$p$ vector of constraint scores. Indices with smallest values are selected.

featidx

a length-$ndim$ vector of indices with highest scores.

projection

a $(p\times ndim)$ whose columns are basis for projection.

trfinfo

a list containing information for out-of-sample prediction.

algorithm

name of the algorithm.

References

Zhang D, Chen S, Zhou Z (2008). “Constraint Score: A New Filter Method for Feature Selection with Pairwise Constraints.” Pattern Recognition, 41(5), 1440–1451.

See Also

do.cscoreg

Examples

## use iris data
## it is known that feature 3 and 4 are more important.
data(iris)
iris.dat = as.matrix(iris[,1:4])
iris.lab = as.factor(iris[,5])

## try different strategy
out1 = do.cscore(iris.dat, iris.lab, score="ratio")
out2 = do.cscore(iris.dat, iris.lab, score="difference", lambda=0)
out3 = do.cscore(iris.dat, iris.lab, score="difference", lambda=0.5)
out4 = do.cscore(iris.dat, iris.lab, score="difference", lambda=1)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(2,2))
plot(out1$Y, col=iris.lab, main="ratio")
plot(out2$Y, col=iris.lab, main="diff/lambda=0")
plot(out3$Y, col=iris.lab, main="diff/lambda=0.5")
plot(out4$Y, col=iris.lab, main="diff/lambda=1")
par(opar)

Description

Constraint Score is a filter-type algorithm for feature selection using pairwise constraints. It first marks all pairwise constraints as same- and different-cluster and construct a feature score for both constraints. It takes ratio or difference of feature score vectors and selects the indices with smallest values. Graph laplacian is constructed for approximated nonlinear manifold structure.

Usage

do.cscoreg(X, label, ndim = 2, score = c("ratio", "difference"), lambda = 0.5)

Arguments

Value

a named Rdimtools S3 object containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

cscore

a length-$p$ vector of constraint scores. Indices with smallest values are selected.

featidx

a length-$ndim$ vector of indices with highest scores.

projection

a $(p\times ndim)$ whose columns are basis for projection.

algorithm

name of the algorithm.

Author(s)

Kisung You

References

Zhang D, Chen S, Zhou Z (2008). “Constraint Score: A New Filter Method for Feature Selection with Pairwise Constraints.” Pattern Recognition, 41(5), 1440–1451.

See Also

do.cscore

Examples

## use iris data
## it is known that feature 3 and 4 are more important.
data(iris)
set.seed(100)
subid    = sample(1:150,50)
iris.dat = as.matrix(iris[subid,1:4])
iris.lab = as.factor(iris[subid,5])

## try different strategy
out1 = do.cscoreg(iris.dat, iris.lab, score="ratio")
out2 = do.cscoreg(iris.dat, iris.lab, score="difference", lambda=0)
out3 = do.cscoreg(iris.dat, iris.lab, score="difference", lambda=0.5)
out4 = do.cscoreg(iris.dat, iris.lab, score="difference", lambda=1)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(2,2))
plot(out1$Y, pch=19, col=iris.lab, main="ratio")
plot(out2$Y, pch=19, col=iris.lab, main="diff/lambda=0")
plot(out3$Y, pch=19, col=iris.lab, main="diff/lambda=0.5")
plot(out4$Y, pch=19, col=iris.lab, main="diff/lambda=1")
par(opar)

Description

Doublue Adjacency Graphs-based Discriminant Neighborhood Embedding (DAG-DNE) is a variant of DNE. As its name suggests, it introduces two adjacency graphs for homogeneous and heterogeneous samples accordaing to their labels.

Usage

do.dagdne(
  X,
  label,
  ndim = 2,
  numk = max(ceiling(nrow(X)/10), 2),
  preprocess = c("center", "scale", "cscale", "decorrelate", "whiten")
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Ding C, Zhang L (2015). “Double Adjacency Graphs-Based Discriminant Neighborhood Embedding.” Pattern Recognition, 48(5), 1734–1742.

See Also

do.dne

Examples

## load iris data
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## try different numbers for neighborhood size
out1 = do.dagdne(X, label, numk=5)
out2 = do.dagdne(X, label, numk=10)
out3 = do.dagdne(X, label, numk=20)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, main="nbd size=5", col=label, pch=19)
plot(out2$Y, main="nbd size=10",col=label, pch=19)
plot(out3$Y, main="nbd size=20",col=label, pch=19)
par(opar)

Description

Diversity-Induced Self-Representation (DISR) is a feature selection method that aims at ranking features by both representativeness and diversity. Self-representation controlled by lbd1 lets the most representative features to be selected, while lbd2 penalizes the degree of inter-feature similarity to enhance diversity from the chosen features.

Usage

do.disr(
  X,
  ndim = 2,
  preprocess = c("null", "center", "scale", "cscale", "whiten", "decorrelate"),
  lbd1 = 1,
  lbd2 = 1
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

featidx

a length-$ndim$ vector of indices with highest scores.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Liu Y, Liu K, Zhang C, Wang J, Wang X (2017). “Unsupervised Feature Selection via Diversity-Induced Self-Representation.” Neurocomputing, 219, 350–363.

See Also

do.rsr

Examples

## use iris data
data(iris)
set.seed(100)
subid = sample(1:150, 50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

#### try different lbd combinations
out1 = do.disr(X, lbd1=1, lbd2=1)
out2 = do.disr(X, lbd1=1, lbd2=5)
out3 = do.disr(X, lbd1=5, lbd2=1)
out4 = do.disr(X, lbd1=5, lbd2=5)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(2,2))
plot(out1$Y, main="(lbd1,lbd2)=(1,1)", col=label, pch=19)
plot(out2$Y, main="(lbd1,lbd2)=(1,5)", col=label, pch=19)
plot(out3$Y, main="(lbd1,lbd2)=(5,1)", col=label, pch=19)
plot(out4$Y, main="(lbd1,lbd2)=(5,5)", col=label, pch=19)
par(opar)

Description

do.dm discovers low-dimensional manifold structure embedded in high-dimensional data space using Diffusion Maps (DM). It exploits diffusion process and distances in data space to find equivalent representations in low-dimensional space.

Usage

do.dm(
  X,
  ndim = 2,
  preprocess = c("null", "center", "scale", "cscale", "decorrelate", "whiten"),
  bandwidth = 1,
  timescale = 1,
  multiscale = FALSE
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

eigvals

a vector of eigenvalues for Markov transition matrix.

Author(s)

Kisung You

References

Nadler B, Lafon S, Coifman RR, Kevrekidis IG (2005). “Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators.” In Proceedings of the 18th International Conference on Neural Information Processing Systems, NIPS'05, 955–962.

Coifman RR, Lafon S (2006). “Diffusion Maps.” Applied and Computational Harmonic Analysis, 21(1), 5–30.

Examples

## load iris data
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## compare different bandwidths
out1 <- do.dm(X,bandwidth=10)
out2 <- do.dm(X,bandwidth=100)
out3 <- do.dm(X,bandwidth=1000)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, pch=19, col=label, main="DM::bandwidth=10")
plot(out2$Y, pch=19, col=label, main="DM::bandwidth=100")
plot(out3$Y, pch=19, col=label, main="DM::bandwidth=1000")
par(opar)

Description

Discriminant Neighborhood Embedding (DNE) is a supervised subspace learning method. DNE tries to move multi-class data points in high-dimensional space in accordance with local intra-class attraction and inter-class repulsion.

Usage

do.dne(
  X,
  label,
  ndim = 2,
  numk = max(ceiling(nrow(X)/10), 2),
  preprocess = c("center", "scale", "cscale", "decorrelate", "whiten")
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Zhang W, Xue X, Lu H, Guo Y (2006). “Discriminant Neighborhood Embedding for Classification.” Pattern Recognition, 39(11), 2240–2243.

Examples

## load iris data
data(iris)
set.seed(100)
subid = sample(1:150,50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## try different numbers for neighborhood size
out1 = do.dne(X, label, numk=5)
out2 = do.dne(X, label, numk=10)
out3 = do.dne(X, label, numk=20)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, main="DNE::nbd size=5",  col=label, pch=19)
plot(out2$Y, main="DNE::nbd size=10", col=label, pch=19)
plot(out3$Y, main="DNE::nbd size=20", col=label, pch=19)
par(opar)

Description

Dual view of PPCA optimizes the latent variables directly from a simple Bayesian approach to model the noise using the multivariate Gaussian distribution of zero mean and spherical covariance $\beta^{-1} I$. When $\beta$ is too small, the algorithm automatically returns an error and provides a guideline for minimal value that enables successful computation.

Usage

do.dppca(X, ndim = 2, beta = 1)

Arguments

Value

a named Rdimtools S3 object containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

algorithm

name of the algorithm.

References

Lawrence N (2005). “Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models.” Journal of Machine Learning Research, 6(60), 1783-1816.

See Also

do.ppca

Examples

## load iris data
data(iris)
X     = as.matrix(iris[,1:4])
lab   = as.factor(iris[,5])

## compare difference choices of 'beta'
embed1 <- do.dppca(X, beta=0.2)
embed2 <- do.dppca(X, beta=1)
embed3 <- do.dppca(X, beta=5)

## Visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3), pty="s")
plot(embed1$Y , col=lab, pch=19, main="beta=0.2")
plot(embed2$Y , col=lab, pch=19, main="beta=1")
plot(embed3$Y , col=lab, pch=19, main="beta=5")
par(opar)

Description

Discriminative Sparsity Preserving Projection (DSPP) is a supervised dimension reduction method that employs sparse representation model to adaptively build both intrinsic adjacency graph and penalty graph. It follows an integration of global within-class structure into manifold learning under exploiting discriminative nature provided from label information.

Usage

do.dspp(
  X,
  label,
  ndim = 2,
  preprocess = c("center", "scale", "cscale", "decorrelate", "whiten"),
  lambda = 1,
  rho = 1
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Gao Q, Huang Y, Zhang H, Hong X, Li K, Wang Y (2015). “Discriminative Sparsity Preserving Projections for Image Recognition.” Pattern Recognition, 48(8), 2543–2553.

Examples

## Not run: 
## use iris data
data(iris)
set.seed(100)
subid = sample(1:150, 50)
X     = as.matrix(iris[subid,1:4])
label = as.factor(iris[subid,5])

## try different rho values
out1 <- do.dspp(X, label, ndim=2, rho=0.01)
out2 <- do.dspp(X, label, ndim=2, rho=0.1)
out3 <- do.dspp(X, label, ndim=2, rho=1)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, main="rho=0.01", col=label, pch=19)
plot(out2$Y, main="rho=0.1",  col=label, pch=19)
plot(out3$Y, main="rho=1",    col=label, pch=19)
par(opar)

## End(Not run)

Description

Distinguishing Variance Embedding (DVE) is an unsupervised nonlinear manifold learning method. It can be considered as a balancing method between Maximum Variance Unfolding and Laplacian Eigenmaps. The algorithm unfolds the data by maximizing the global variance subject to the locality-preserving constraint. Instead of defining certain kernel, it applies local scaling scheme in that it automatically computes adaptive neighborhood-based kernel bandwidth.

Usage

do.dve(
  X,
  ndim = 2,
  type = c("proportion", 0.1),
  preprocess = c("null", "center", "scale", "cscale", "decorrelate", "whiten")
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

Author(s)

Kisung You

References

Wang Q, Li J (2009). “Combining Local and Global Information for Nonlinear Dimensionality Reduction.” Neurocomputing, 72(10-12), 2235–2241.

Qinggang W, Jianwei L, Xuchu W (2010). “Distinguishing Variance Embedding.” Image and Vision Computing, 28(6), 872–880.

Examples

## generate swiss-roll dataset of size 100
set.seed(100)
X <- aux.gensamples(dname="crown", n=100)

## try different nbd size
out1 <- do.dve(X, type=c("proportion",0.5))
out2 <- do.dve(X, type=c("proportion",0.7))
out3 <- do.dve(X, type=c("proportion",0.9))

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, main="50% connected")
plot(out2$Y, main="70% connected")
plot(out3$Y, main="90% connected")
par(opar)

Description

Local Discriminant Embedding (LDE) suffers from a small-sample-size problem where scatter matrix may suffer from rank deficiency. Exponential LDE (ELDE) provides not only a remedy for the problem using matrix exponential, but also a flexible framework to transform original data into a new space via distance diffusion mapping similar to kernel-based nonlinear mapping.

Usage

do.elde(
  X,
  label,
  ndim = 2,
  t = 1,
  preprocess = c("center", "scale", "cscale", "decorrelate", "whiten"),
  k1 = max(ceiling(nrow(X)/10), 2),
  k2 = max(ceiling(nrow(X)/10), 2)
)

Arguments

Value

a named list containing

Y

an $(n\times ndim)$ matrix whose rows are embedded observations.

trfinfo

a list containing information for out-of-sample prediction.

projection

a $(p\times ndim)$ whose columns are basis for projection.

Author(s)

Kisung You

References

Dornaika F, Bosaghzadeh A (2013). “Exponential Local Discriminant Embedding and Its Application to Face Recognition.” IEEE Transactions on Cybernetics, 43(3), 921–934.

See Also

do.lde

Examples

## generate data of 3 types with difference
set.seed(100)
dt1  = aux.gensamples(n=20)-50
dt2  = aux.gensamples(n=20)
dt3  = aux.gensamples(n=20)+50

## merge the data and create a label correspondingly
X      = rbind(dt1,dt2,dt3)
label  = rep(1:3, each=20)

## try different kernel bandwidth
out1 = do.elde(X, label, t=1)
out2 = do.elde(X, label, t=10)
out3 = do.elde(X, label, t=100)

## visualize
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(out1$Y, pch=19, col=label, main="ELDE::bandwidth=1")
plot(out2$Y, pch=19, col=label, main="ELDE::bandwidth=10")
plot(out3$Y, pch=19, col=label, main="ELDE::bandwidth=100")
par(opar)

Description

Enhanced Locality Preserving Projection proposed in 2013 (ELPP2) is built upon a parameter-free philosophy from PFLPP. It further aims to exclude its projection to be uncorrelated in the sense that the scatter matrix is placed in a generalized eigenvalue problem.

Usage

do.elpp2(
  X,
  ndim = 2,
  preprocess = c("center", "scale", "cscale", "decorrelate", "whiten")
)

Arguments