Title: | Accelerated line search algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form. |
---|---|
Description: | Using of the accelerated line search algorithm for simultaneously diagonalize a set of symmetric positive definite matrices. |
Authors: | Dariush Najarzadeh |
Maintainer: | Dariush Najarzadeh <[email protected]> |
License: | GPL (>= 2) |
Version: | 1.0 |
Built: | 2024-12-18 06:40:59 UTC |
Source: | CRAN |
Let
where is a positive integer and called as the number of groups,
,
, ...,
are positive integers and called as the sample sizes,
is an orthonormal matrix, and
,
, ...,
are positive-definite and are usually sample covariance matrices.
The minimization of the objective function
that depends on
a orthonormal matrix
is required for a potpourri of statistical problems.
means that
,
, ...,
are simultaneously simultaneously diagonalizable.
This situation is encountered when looking for common principal components, for example, and the
Flury and Gautschi (1986) method is a popular approach.
Lefkomtch (2004), Boik (2007), and Browne and McNicholas (2012) report that the Flury and Gautschi method is not effective for higher dimensional problems.
Browne and McNicholas (2013)
obtain several simple majorization-minizmation (MM) algorithms that provide solutions
to this problem and are effective in higher dimensions. They compare these solutions
with each others in terms of convergence and computational time.
They found that the accelerated line search (ALS) algorithm is a computationally efficient procedure to this problem.
Extensive review of the this algorithm and similar algorithms can be found in Absil et al. (2008).
In the following, we briefly describe the ALS algorithm used to minimize the objective function
.
ALS algorithm is based on the update formula
where
, where
in the sense of the
QR decomposition of a matrix
;
The
decomposition of a matrix
is the decomposition of
as
, where
belongs to the orthogonal group and
belongs to the set of all upper triangular matrices with strictly
positive diagonal elements,
where
and for and
,
is the smallest nonnegative integer
such that
where is the Frobenius inner product.
Starting from initial iterate
, for a given
,
we stop the algorithm when
Package: | ALSCPC |
Version: | 1.0 |
Date: | 2013-09-05 |
License: | GPL (>= 2) |
Dariush Najarzadeh
Maintainer: Dariush Najarzadeh <[email protected]>
Absil, P. A., Mahony, R., & Sepulchre, R. (2009). Optimization algorithms on matrix manifolds. Princeton University Press.
Boik, R. J. (2007). Spectral models for covariance matrics. Biometrika, 89, 159-182.
Browne, R. P., and McNicholas, P. D. (2012). Orthogonal Stiefel manifold optimization for eigen-decomposed covariance parameter estimation in mixture models. Statistics and Computing, 1-8.
Browne, R. P., and McNicholas, P. D. (2013). Estimating common principal components in high dimensions. arXiv preprint arXiv:1302.2102.
Flury, B. N., and Gautschi, W. (1986). An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form. SIAM Journal on Scientific and Statistical Computing, 7(1), 169-184.
Lefkomtch, L. P. (2004). Consensus principal components. Biometrical Journal, 35, 567-580.
by using of the accelerated line search algorithmThe ALS.CPC
function implement
ALS algorithm based on the update formula
until convergence (i.e. )
and return the orthogonal matrix
,
is the smallest nonnegative integer
such that
.
ALS.CPC(alpha,beta,sigma,epsilon,G,nval,D,S)
ALS.CPC(alpha,beta,sigma,epsilon,G,nval,D,S)
alpha |
positive real number. |
beta |
real number belong to (0,1). |
sigma |
real number belong to (0,1). |
epsilon |
small positive constant controlling error term. |
G |
number of groups in common principal components analysis. |
nval |
a numeric vector containing the positive integers of sample sizes minus one in each group. |
D |
an initial square orthogonal matrix of order |
S |
a list of length |
An orthogonal matrix such that minimize .
Dariush Najarzadeh
Absil, P. A., Mahony, R., & Sepulchre, R. (2009). Optimization algorithms on matrix manifolds. Princeton University Press.
nval<-numeric(3) nval[[1]]<-49 nval[[2]]<-49 nval[[3]]<-49 S<-vector("list",length=3) setosa<-iris[1:50,1:4]; names(setosa)<-NULL versicolor<-iris[51:100,1:4]; names(versicolor)<-NULL virginica<-iris[101:150,1:4]; names(virginica)<-NULL S[[1]]<-as.matrix(var(versicolor)) S[[2]]<-as.matrix(var(virginica)) S[[3]]<-as.matrix(var(setosa)) D<-diag(4) ALS.CPC(10,0.5,0.4,1e-5,G=3,nval,D,S)
nval<-numeric(3) nval[[1]]<-49 nval[[2]]<-49 nval[[3]]<-49 S<-vector("list",length=3) setosa<-iris[1:50,1:4]; names(setosa)<-NULL versicolor<-iris[51:100,1:4]; names(versicolor)<-NULL virginica<-iris[101:150,1:4]; names(virginica)<-NULL S[[1]]<-as.matrix(var(versicolor)) S[[2]]<-as.matrix(var(virginica)) S[[3]]<-as.matrix(var(setosa)) D<-diag(4) ALS.CPC(10,0.5,0.4,1e-5,G=3,nval,D,S)