2024-12-11 @Atsushi Kawaguchi

The msma package provides functions for a matrix decomposition method incorporating sparse and supervised modeling for a multiblock multivariable data analysis.

Preparation

Install package (as necessary)

if(!require("msma")) install.packages("msma")

Load package

library(msma)

Getting started

Simulated multiblock data (list data) by using the function simdata.

Sample size is 50. The correlation coeficient is 0.8. The numbers of columns for response and predictor can be specified by the argument Yps and Xps, respectively. The length of vecor represents the number of blocks. That is, response has three blocks with the numbers of columns being 3, 4, and 5 and predictor has one block with the number of columns being 3.

dataset0 = simdata(n = 50, rho = 0.8, Yps = c(3, 4, 5), Xps = 3, seed=1)
X0 = dataset0$X; Y0 = dataset0$Y

The data generated here is applied to the msma function.

One Component

The argument comp can specify the number of components. The arguments lambdaX and lambdaY can specify the regularization parameters for X and Y, respectively.

First, we set comp=1, which will perform an analysis with 1 component.

fit01 = msma(X0, Y0, comp=1, lambdaX=0.05, lambdaY=1:3)
fit01

## Call:
## msma.default(X = X0, Y = Y0, comp = 1, lambdaX = 0.05, lambdaY = 1:3)
## 
## Numbers of non-zeros for X block: 
##        comp1
## block1     3
## 
## Numbers of non-zeros for X super: 
##         comp1
## comp1-1     1
## 
## Numbers of non-zeros for Y block: 
##        comp1
## block1     1
## block2     1
## block3     1
## 
## Numbers of non-zeros for Y super: 
##         comp1
## comp1-1     3

The plot function is available. In default setting, the block weights are displayed as a barplot.

plot(fit01)

Two Component

Next, we set comp=2, which will perform an analysis with 2 components.

fit02 = msma(X0, Y0, comp=2, lambdaX=0.03, lambdaY=0.01*(1:3))
fit02

## Call:
## msma.default(X = X0, Y = Y0, comp = 2, lambdaX = 0.03, lambdaY = 0.01 * 
##     (1:3))
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2
## block1     3     3
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2
## comp1-1     1     1
## 
## Numbers of non-zeros for Y block: 
##        comp1 comp2
## block1     3     3
## block2     4     4
## block3     5     5
## 
## Numbers of non-zeros for Y super: 
##         comp1 comp2
## comp1-1     3     3

Single Block

Two matrics are prepared by specifying arguments Yps and Xps.

dataset1 = simdata(n = 50, rho = 0.8, Yps = 5, Xps = 5, seed=1)
X1 = dataset1$X[[1]]; Y1 = dataset1$Y

Principal Component Analysis (PCA)

If input is a matrix, a principal component analysis is implemented.

(fit111 = msma(X1, comp=5))

## Call:
## msma.default(X = X1, comp = 5)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     5     5     5     5
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

The weight (loading) vectors can be obtained as follows.

fit111$wbX

## $block1
##           comp1       comp2       comp3         comp4       comp5
## X.1.1 0.4309622 -0.74170223 -0.03672379  0.1325580413 -0.49520613
## X.1.2 0.4483196  0.31188303  0.63228246  0.5490205405  0.02310504
## X.1.3 0.4601597 -0.19547078 -0.38567734  0.1474129336  0.76129277
## X.1.4 0.4392794  0.55811865 -0.57117598 -0.0006449093 -0.41145448
## X.1.5 0.4566923  0.05386584  0.35196769 -0.8119567864  0.07331836

The bar plots of weight vectors are provided by the function plot. The component number is specified by the argument axes. The plot type is selected by the argument plottype. Furthermore, since this function uses the barplot function originally built into R, its arguments are also available. In the following example, on the horizontal axis, the magnification of the variable names is set to 0.7 by setting cex.names=0.7, and the variable names are oriented as las=2.

par(mfrow=c(1,2))
plot(fit111, axes = 1, plottype="bar", cex.names=0.7, las=2)
plot(fit111, axes = 2, plottype="bar", cex.names=0.7, las=2)

The score vectors for first six subjects.

lapply(fit111$sbX, head)

## $block1
##            [,1]          [,2]        [,3]        [,4]        [,5]
## [1,]  0.7097369  0.0487564120  0.10746733 -0.02462727 -0.00598565
## [2,] -0.6976955 -0.5423072581 -0.98211121 -0.23652145 -0.16120137
## [3,]  2.4367362 -0.0238218850 -0.32403419 -0.44206969 -0.47004393
## [4,] -2.4460385 -0.0007036966  0.08112764  0.14263545 -0.45584684
## [5,]  1.7708133  0.9741849574 -0.64716134  0.09377875 -0.78085072
## [6,] -0.8043943 -0.9469205017 -0.34705994 -0.62641753  0.26617649

The scatter plots for the score vectors specified by the argument v. The argument axes is specified by the two length vector represents which components are displayed.

par(mfrow=c(1,2))
plot(fit111, v="score", axes = 1:2, plottype="scatter")
plot(fit111, v="score", axes = 2:3, plottype="scatter")

When the argument v was specified as “cpev”, the cummulative eigenvalues are plotted.

par(mfrow=c(1,2))
plot(fit111, v="cpev", ylim=c(0.7, 1))

Other functions in R for PCA

There is the R function prcomp to implement PCA.

(fit1112 = prcomp(X1, scale=TRUE))

## Standard deviations (1, .., p=5):
## [1] 2.0446732 0.5899513 0.4458638 0.3926788 0.3439156
## 
## Rotation (n x k) = (5 x 5):
##             PC1         PC2         PC3           PC4         PC5
## [1,] -0.4309746  0.74172462 -0.03722419 -1.351296e-01  0.49442882
## [2,] -0.4483141 -0.31171881  0.63044575 -5.510830e-01 -0.02629751
## [3,] -0.4601629  0.19547669 -0.38616901 -1.416349e-01 -0.76213651
## [4,] -0.4392701 -0.55816074 -0.57114566  4.296727e-05  0.41144993
## [5,] -0.4566918 -0.05405032  0.35470924  8.111640e-01 -0.06859643

summary(fit1112)

## Importance of components:
##                           PC1     PC2     PC3     PC4     PC5
## Standard deviation     2.0447 0.58995 0.44586 0.39268 0.34392
## Proportion of Variance 0.8361 0.06961 0.03976 0.03084 0.02366
## Cumulative Proportion  0.8361 0.90575 0.94551 0.97634 1.00000

This Rotation is almost the same as the output of msma, but it can be made closer by setting the argument ceps as follows.

fit1113 = msma(X1, comp=5, ceps=0.0000001)
fit1113$wbX

## $block1
##           comp1       comp2       comp3       comp4       comp5
## X.1.1 0.4309745 -0.74172365 -0.03694568  0.13514153 -0.49444798
## X.1.2 0.4483141  0.31172789  0.63155896  0.54980542  0.02621947
## X.1.3 0.4601628 -0.19547781 -0.38588003  0.14252697  0.76211630
## X.1.4 0.4392701  0.55815708 -0.57114810  0.00105051 -0.41145010
## X.1.5 0.4566918  0.05404487  0.35306479 -0.81187175  0.06871165

Plotting the scores with the signs turned over, we see that similar scores are calculated.

par(mfrow=c(1,2))
biplot(fit1112)
plot(-fit1113$sbX[[1]][,1:2],xlab="Component 1",ylab="Component 2")

The ggfortify package is also available for the PCA plot.

Sparse PCA

If lambdaX (>0) is specified, a sparse principal component analysis is implemented.

(fit112 = msma(X1, comp=5, lambdaX=0.1))

## Call:
## msma.default(X = X1, comp = 5, lambdaX = 0.1)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     4     4     5     4
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

par(mfrow=c(1,2))
plot(fit112, axes = 1, plottype="bar", las=2)
plot(fit112, axes = 2, plottype="bar", las=2)

Supervised Sparse PCA

The outcome Z is generated.

set.seed(1); Z = rbinom(50, 1, 0.5)

If the outcome Z is specified, a supervised sparse principal component analysis is implemented.

(fit113 = msma(X1, Z=Z, comp=5, lambdaX=0.02))

## Call:
## msma.default(X = X1, Z = Z, comp = 5, lambdaX = 0.02)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     5     5     4     5
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

par(mfrow=c(1,2))
plot(fit113, axes = 1, plottype="bar", las=2)
plot(fit113, axes = 2, plottype="bar", las=2)

Partial Least Squres (PLS)

If the another input Y1 is specified, a partial least squres is implemented.

(fit121 = msma(X1, Y1, comp=2))

## Call:
## msma.default(X = X1, Y = Y1, comp = 2)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2
## block1     5     5
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2
## comp1-1     1     1
## 
## Numbers of non-zeros for Y block: 
##        comp1 comp2
## block1     5     5
## 
## Numbers of non-zeros for Y super: 
##         comp1 comp2
## comp1-1     1     1

The component number is specified by the argument axes. When the argument XY was specified as “XY”, the scatter plots for Y score against X score are plotted.

par(mfrow=c(1,2))
plot(fit121, axes = 1, XY="XY")
plot(fit121, axes = 2, XY="XY")

Sparse PLS

If lambdaX and lambdaY are specified, a sparse PLS is implemented.

(fit122 = msma(X1, Y1, comp=2, lambdaX=0.5, lambdaY=0.5))

## Call:
## msma.default(X = X1, Y = Y1, comp = 2, lambdaX = 0.5, lambdaY = 0.5)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2
## block1     2     2
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2
## comp1-1     1     1
## 
## Numbers of non-zeros for Y block: 
##        comp1 comp2
## block1     2     2
## 
## Numbers of non-zeros for Y super: 
##         comp1 comp2
## comp1-1     1     1

par(mfrow=c(1,2))
plot(fit122, axes = 1, XY="XY")
plot(fit122, axes = 2, XY="XY")

Supervised Sparse PLS

If the outcome Z is specified, a supervised sparse PLS is implemented.

(fit123 = msma(X1, Y1, Z, comp=2, lambdaX=0.5, lambdaY=0.5))

## Call:
## msma.default(X = X1, Y = Y1, Z = Z, comp = 2, lambdaX = 0.5, 
##     lambdaY = 0.5)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2
## block1     2     2
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2
## comp1-1     1     1
## 
## Numbers of non-zeros for Y block: 
##        comp1 comp2
## block1     2     2
## 
## Numbers of non-zeros for Y super: 
##         comp1 comp2
## comp1-1     1     1

par(mfrow=c(1,2))
plot(fit123, axes = 1, XY="XY")
plot(fit123, axes = 2, XY="XY")

Multi Block

Multiblock data is a list of data matrix.

dataset2 = simdata(n = 50, rho = 0.8, Yps = c(2, 3), Xps = c(3, 4), seed=1)
X2 = dataset2$X; Y2 = dataset2$Y

PCA

The input class is list.

class(X2)

## [1] "list"

The list length is 2 for 2 blocks.

length(X2)

## [1] 2

list of data matrix structure.

lapply(X2, dim)

## [[1]]
## [1] 50  3
## 
## [[2]]
## [1] 50  4

The function msma is applied to this list X2 as follows.

(fit211 = msma(X2, comp=1))

## Call:
## msma.default(X = X2, comp = 1)
## 
## Numbers of non-zeros for X block: 
##        comp1
## block1     3
## block2     4
## 
## Numbers of non-zeros for X super: 
##         comp1
## comp1-1     2

The bar plots for the block and super weights (loadings) specified the argument block.

par(mfrow=c(1,2))
plot(fit211, axes = 1, plottype="bar", block="block", las=2)
plot(fit211, axes = 1, plottype="bar", block="super")

Sparse PCA

If lambdaX with the length of 2 (same as the length of blocks) are specified, a multiblock sparse PCA is implemented.

(fit212 = msma(X2, comp=1, lambdaX=c(0.5, 0.5)))

## Call:
## msma.default(X = X2, comp = 1, lambdaX = c(0.5, 0.5))
## 
## Numbers of non-zeros for X block: 
##        comp1
## block1     3
## block2     2
## 
## Numbers of non-zeros for X super: 
##         comp1
## comp1-1     2

The bar plots for the block and super weights (loadings).

par(mfrow=c(1,2))
plot(fit212, axes = 1, plottype="bar", block="block", las=2)
plot(fit212, axes = 1, plottype="bar", block="super")

Supervised Sparse PCA

If the outcome Z is specified, a supervised analysis is implemented.

(fit213 = msma(X2, Z=Z, comp=1, lambdaX=c(0.5, 0.5)))

## Call:
## msma.default(X = X2, Z = Z, comp = 1, lambdaX = c(0.5, 0.5))
## 
## Numbers of non-zeros for X block: 
##        comp1
## block1     3
## block2     2
## 
## Numbers of non-zeros for X super: 
##         comp1
## comp1-1     2

par(mfrow=c(1,2))
plot(fit213, axes = 1, plottype="bar", block="block", las=2)
plot(fit213, axes = 1, plottype="bar", block="super")

Nested Component

A vector of length 2 can be given to the comp argument to perform the nested component analysis, which is a method to consider multiple components even in the super component. The first element of the vector corresponds to the number of block components and the second element corresponds to the number of (nested) super components.

(fit214 = msma(X2, comp=c(2,3)))

## Call:
## msma.default(X = X2, comp = c(2, 3))
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2
## block1     3     3
## block2     4     4
## 
## Numbers of non-zeros for X super: 
## $comp1
## comp1-1 comp1-2 comp1-3 
##       2       2       2 
## 
## $comp2
## comp2-1 comp2-2 comp2-3 
##       2       2       2

In this example, there are 2 block components and 3 super components.

fit214$wbX

## $block1
##            comp1       comp2
## X.1.1 -0.5307011 -0.75618454
## X.1.2 -0.6006433  0.01688668
## X.1.3 -0.5979833  0.65414049
## 
## $block2
##           comp1         comp2
## X.2.1 0.4841242 -6.914840e-01
## X.2.2 0.5172474  5.175554e-05
## X.2.3 0.5241242  7.171312e-01
## X.2.4 0.4726233 -8.702155e-02

For the block weights, the number of blocks is 2 since there are two data matrices as shown as follows, and the number of rows is 3 and 4, the number of variables in each.

The number of components is 2 for the first element of the vector specified by the comp argument, which is the number of columns in each matrix.

par(mfrow=c(1,2))
plot(fit214, axes = 1, axes2 = 1, plottype="bar", block="block", las=2)
plot(fit214, axes = 2, axes2 = 1, plottype="bar", block="block", las=2)

fit214$wsX

## $comp1
##             comp1      comp2      comp3
## block1 -0.5198025 -0.8542864  0.4915551
## block2  0.8542864 -0.5198025 -0.8708465
## 
## $comp2
##             comp1      comp2      comp3
## block1 -0.3887274 -0.9213528  0.9340406
## block2  0.9213528 -0.3887274 -0.3571669

par(mfrow=c(2,3))
for(j in 1:2) for(i in 1:3) plot(fit214, axes = j, axes2 = i, plottype="bar", block="super")

PLS

If the another input (list) Y2 is specified, the partial least squared is implemented.

(fit221 = msma(X2, Y2, comp=1))

## Call:
## msma.default(X = X2, Y = Y2, comp = 1)
## 
## Numbers of non-zeros for X block: 
##        comp1
## block1     3
## block2     4
## 
## Numbers of non-zeros for X super: 
##         comp1
## comp1-1     2
## 
## Numbers of non-zeros for Y block: 
##        comp1
## block1     2
## block2     3
## 
## Numbers of non-zeros for Y super: 
##         comp1
## comp1-1     2

par(mfrow=c(1,2))
plot(fit221, axes = 1, plottype="bar", block="block", XY="X", las=2)
plot(fit221, axes = 1, plottype="bar", block="super", XY="X")

par(mfrow=c(1,2))
plot(fit221, axes = 1, plottype="bar", block="block", XY="Y", las=2)
plot(fit221, axes = 1, plottype="bar", block="super", XY="Y")

Sparse PLS

The regularized parameters lambdaX and lambdaY are specified vectors with same length with the length of lists X2 and Y2, respectively.

(fit222 = msma(X2, Y2, comp=1, lambdaX=c(0.5, 0.5), lambdaY=c(0.5, 0.5)))

## Call:
## msma.default(X = X2, Y = Y2, comp = 1, lambdaX = c(0.5, 0.5), 
##     lambdaY = c(0.5, 0.5))
## 
## Numbers of non-zeros for X block: 
##        comp1
## block1     2
## block2     2
## 
## Numbers of non-zeros for X super: 
##         comp1
## comp1-1     2
## 
## Numbers of non-zeros for Y block: 
##        comp1
## block1     1
## block2     3
## 
## Numbers of non-zeros for Y super: 
##         comp1
## comp1-1     2

par(mfrow=c(1,2))
plot(fit222, axes = 1, plottype="bar", block="block", XY="X", las=2)
plot(fit222, axes = 1, plottype="bar", block="super", XY="X")

par(mfrow=c(1,2))
plot(fit222, axes = 1, plottype="bar", block="block", XY="Y", las=2)
plot(fit222, axes = 1, plottype="bar", block="super", XY="Y")

Supervised Sparse PLS

(fit223 = msma(X2, Y2, Z, comp=1, lambdaX=c(0.5, 0.5), lambdaY=c(0.5, 0.5)))

## Call:
## msma.default(X = X2, Y = Y2, Z = Z, comp = 1, lambdaX = c(0.5, 
##     0.5), lambdaY = c(0.5, 0.5))
## 
## Numbers of non-zeros for X block: 
##        comp1
## block1     2
## block2     2
## 
## Numbers of non-zeros for X super: 
##         comp1
## comp1-1     2
## 
## Numbers of non-zeros for Y block: 
##        comp1
## block1     1
## block2     3
## 
## Numbers of non-zeros for Y super: 
##         comp1
## comp1-1     2

par(mfrow=c(1,2))
plot(fit223, axes = 1, plottype="bar", block="block", XY="X", las=2)
plot(fit223, axes = 1, plottype="bar", block="super", XY="X")

par(mfrow=c(1,2))
plot(fit223, axes = 1, plottype="bar", block="block", XY="Y", las=2)
plot(fit223, axes = 1, plottype="bar", block="super", XY="Y")

Parameter Selection

Number of Components

number of components search

Single Block

PCA

(ncomp11 = ncompsearch(X1, comps = c(1, 5, 10*(1:2)), nfold=5))

## $criterion
## [1] "CV"
## 
## $comps
## $comps[[1]]
## [1]  1  5 10 20
## 
## $comps[[2]]
## [1] 1
## 
## 
## $mincriterion
## [1] 6.89466e-32 1.00000e+00
## 
## $criterions
## $criterions[[1]]
## [1] 1.583224e-01 6.894660e-32 1.335149e-04 1.676873e-04
## 
## $criterions[[2]]
## [1] 1 1 1 1
## 
## 
## $optncomp
## [1] 5 1
## 
## $optlambdaX
## NULL
## 
## $optlambdaY
## NULL
## 
## $optlambdaXsup
## NULL
## 
## $optlambdaYsup
## NULL
## 
## attr(,"class")
## [1] "ncompsearch"

plot(ncomp11)

(ncomp12 = ncompsearch(X1, comps = 20, criterion="BIC"))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
##  [1]  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20
## 
## $comps[[2]]
## [1] 1
## 
## 
## $mincriterion
##       comp5 comp1.comp1 
##   -71.33402        -Inf 
## 
## $criterions
## $criterions$bic
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##  -1.718502  -2.161108  -2.598561  -3.322634 -71.334021 -65.464450 -65.345615 
##      comp8      comp9     comp10     comp11     comp12     comp13     comp14 
## -65.226304 -65.116775 -65.006274 -64.906509 -64.779055 -64.652276 -64.570783 
##     comp15     comp16     comp17     comp18     comp19     comp20 
## -64.466593 -64.351462 -64.250782 -64.136179 -64.012210 -63.916326 
## 
## $criterions$bic2
##  comp1.comp1  comp2.comp1  comp3.comp1  comp4.comp1  comp5.comp1  comp6.comp1 
##         -Inf    -71.59729         -Inf    -73.74255         -Inf    -70.23735 
##  comp7.comp1  comp8.comp1  comp9.comp1 comp10.comp1 comp11.comp1 comp12.comp1 
##    -65.17860    -65.12805    -65.01969    -64.95044    -64.84591    -64.78461 
## comp13.comp1 comp14.comp1 comp15.comp1 comp16.comp1 comp17.comp1 comp18.comp1 
##    -64.67248    -64.58989    -64.50975    -64.48484    -64.38352    -64.31288 
## comp19.comp1 comp20.comp1 
##    -64.21075    -64.13996 
## 
## 
## $optncomp
## [1] 5 1
## 
## $optlambdaX
## NULL
## 
## $optlambdaY
## NULL
## 
## $optlambdaXsup
## NULL
## 
## $optlambdaYsup
## NULL
## 
## attr(,"class")
## [1] "ncompsearch"

plot(ncomp12)

PLS

ncomp21 = ncompsearch(X2, Y2, comps = c(1, 5, 10*(1:2)), nfold=5)
plot(ncomp21)

Multi Block

The multi block structure has

dataset3 = simdata(n = 50, rho = 0.8, Yps = rep(4, 5), Xps = rep(4, 5), seed=1)
X3 = dataset3$X; Y3 = dataset3$Y

PCA

ncomp31 = ncompsearch(X3, comps = 20, criterion="BIC")
plot(ncomp31)

Nested

(ncomp32 = ncompsearch(X3, comps = list(20, 20), criterion="BIC"))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
##  [1]  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20
## 
## $comps[[2]]
##  [1]  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20
## 
## 
## $mincriterion
## $mincriterion$comp4
## [1] -70.91364
## 
## $mincriterion$comp5
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -61.41185 -61.30142 -61.19099 -61.08056 -60.97013 -60.85971 -60.74928 -60.63885 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -60.52842 -60.41799 -60.30756 -60.19713 -60.08670 -59.97627 -59.86584 -59.75541 
##    comp17    comp18    comp19    comp20 
## -59.64498 -59.53455 -59.42413 -59.31370 
## 
## 
## $criterions
## $criterions$bic
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##  -1.825223  -2.241347  -2.958985 -70.913638 -65.437056 -65.295837 -65.167319 
##      comp8      comp9     comp10     comp11     comp12     comp13     comp14 
## -65.024228 -64.880958 -64.756298 -64.620058 -64.471492 -64.336200 -64.189436 
##     comp15     comp16     comp17     comp18     comp19     comp20 
## -64.059047 -63.938738 -63.796081 -63.658972 -63.518790 -63.381507 
## 
## $criterions$bic2
## $criterions$bic2$comp1
##       comp1       comp2       comp3       comp4       comp5       comp6 
##   0.9714158   0.7048689   0.3285174  -0.4043394 -69.4380900 -67.7265723 
##       comp7       comp8       comp9      comp10      comp11      comp12 
## -67.6384809 -67.5823767 -67.5434884 -67.5286939 -67.2700548 -67.1623796 
##      comp13      comp14      comp15      comp16      comp17      comp18 
## -67.0396123 -66.8838681 -66.7511621 -66.6109674 -66.5486936 -66.5247680 
##      comp19      comp20 
## -66.4782760 -66.2661651 
## 
## $criterions$bic2$comp2
##       comp1       comp2       comp3       comp4       comp5       comp6 
##   0.3896461   0.0920536  -0.2677686  -0.9695676 -70.4341693 -67.4695615 
##       comp7       comp8       comp9      comp10      comp11      comp12 
## -67.3803202 -67.2205497 -67.0813932 -67.0202994 -66.8557241 -66.6491064 
##      comp13      comp14      comp15      comp16      comp17      comp18 
## -66.5337401 -66.7972115 -66.5820747 -66.5653334 -66.2197623 -66.2342068 
##      comp19      comp20 
## -66.1349637 -66.0416309 
## 
## $criterions$bic2$comp3
##       comp1       comp2       comp3       comp4       comp5       comp6 
##   2.4279355   2.1068041   1.6379884   0.8785787 -68.4196269 -65.3979563 
##       comp7       comp8       comp9      comp10      comp11      comp12 
## -65.3435182 -65.2375780 -65.1403246 -64.9847675 -64.8125188 -64.7246089 
##      comp13      comp14      comp15      comp16      comp17      comp18 
## -64.6887399 -64.5006564 -64.4994906 -64.4321271 -64.2073816 -64.1108483 
##      comp19      comp20 
## -64.0341786 -63.9969333 
## 
## $criterions$bic2$comp4
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##   4.356252   4.011048   3.618331   2.814239 -66.585146 -62.622443 -62.580169 
##      comp8      comp9     comp10     comp11     comp12     comp13     comp14 
## -62.466523 -62.267025 -62.195052 -62.061747 -61.990197 -61.924409 -61.807630 
##     comp15     comp16     comp17     comp18     comp19     comp20 
## -61.691383 -61.536162 -61.423361 -61.351672 -61.220166 -60.997759 
## 
## $criterions$bic2$comp5
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -61.41185 -61.30142 -61.19099 -61.08056 -60.97013 -60.85971 -60.74928 -60.63885 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -60.52842 -60.41799 -60.30756 -60.19713 -60.08670 -59.97627 -59.86584 -59.75541 
##    comp17    comp18    comp19    comp20 
## -59.64498 -59.53455 -59.42413 -59.31370 
## 
## $criterions$bic2$comp6
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -54.47951 -54.36908 -54.25865 -54.14822 -54.03779 -53.92736 -53.81693 -53.70650 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -53.59607 -53.48564 -53.37522 -53.26479 -53.15436 -53.04393 -52.93350 -52.82307 
##    comp17    comp18    comp19    comp20 
## -52.71264 -52.60221 -52.49178 -52.38135 
## 
## $criterions$bic2$comp7
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -52.31442 -52.20399 -52.09356 -51.98313 -51.87270 -51.76228 -51.65185 -51.54142 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -51.43099 -51.32056 -51.21013 -51.09970 -50.98927 -50.87884 -50.76841 -50.65798 
##    comp17    comp18    comp19    comp20 
## -50.54755 -50.43712 -50.32670 -50.21627 
## 
## $criterions$bic2$comp8
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -50.08316 -49.97273 -49.86230 -49.75188 -49.64145 -49.53102 -49.42059 -49.31016 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -49.19973 -49.08930 -48.97887 -48.86844 -48.75801 -48.64758 -48.53715 -48.42672 
##    comp17    comp18    comp19    comp20 
## -48.31630 -48.20587 -48.09544 -47.98501 
## 
## $criterions$bic2$comp9
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -47.85359 -47.74316 -47.63273 -47.52230 -47.41187 -47.30145 -47.19102 -47.08059 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -46.97016 -46.85973 -46.74930 -46.63887 -46.52844 -46.41801 -46.30758 -46.19715 
##    comp17    comp18    comp19    comp20 
## -46.08672 -45.97630 -45.86587 -45.75544 
## 
## $criterions$bic2$comp10
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -45.65215 -45.54172 -45.43129 -45.32086 -45.21043 -45.10000 -44.98957 -44.87914 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -44.76871 -44.65828 -44.54785 -44.43742 -44.32699 -44.21657 -44.10614 -43.99571 
##    comp17    comp18    comp19    comp20 
## -43.88528 -43.77485 -43.66442 -43.55399 
## 
## $criterions$bic2$comp11
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -43.48931 -43.37888 -43.26845 -43.15802 -43.04760 -42.93717 -42.82674 -42.71631 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -42.60588 -42.49545 -42.38502 -42.27459 -42.16416 -42.05373 -41.94330 -41.83287 
##    comp17    comp18    comp19    comp20 
## -41.72244 -41.61202 -41.50159 -41.39116 
## 
## $criterions$bic2$comp12
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -41.24013 -41.12970 -41.01927 -40.90884 -40.79841 -40.68798 -40.57755 -40.46712 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -40.35669 -40.24626 -40.13583 -40.02541 -39.91498 -39.80455 -39.69412 -39.58369 
##    comp17    comp18    comp19    comp20 
## -39.47326 -39.36283 -39.25240 -39.14197 
## 
## $criterions$bic2$comp13
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -39.02971 -38.91928 -38.80885 -38.69842 -38.58799 -38.47756 -38.36713 -38.25671 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -38.14628 -38.03585 -37.92542 -37.81499 -37.70456 -37.59413 -37.48370 -37.37327 
##    comp17    comp18    comp19    comp20 
## -37.26284 -37.15241 -37.04198 -36.93155 
## 
## $criterions$bic2$comp14
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -36.83400 -36.72357 -36.61314 -36.50271 -36.39228 -36.28185 -36.17142 -36.06099 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -35.95056 -35.84013 -35.72970 -35.61928 -35.50885 -35.39842 -35.28799 -35.17756 
##    comp17    comp18    comp19    comp20 
## -35.06713 -34.95670 -34.84627 -34.73584 
## 
## $criterions$bic2$comp15
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -34.61410 -34.50367 -34.39324 -34.28281 -34.17238 -34.06195 -33.95152 -33.84109 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -33.73066 -33.62024 -33.50981 -33.39938 -33.28895 -33.17852 -33.06809 -32.95766 
##    comp17    comp18    comp19    comp20 
## -32.84723 -32.73680 -32.62637 -32.51594 
## 
## $criterions$bic2$comp16
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -32.37908 -32.26865 -32.15822 -32.04779 -31.93736 -31.82694 -31.71651 -31.60608 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -31.49565 -31.38522 -31.27479 -31.16436 -31.05393 -30.94350 -30.83307 -30.72264 
##    comp17    comp18    comp19    comp20 
## -30.61221 -30.50178 -30.39136 -30.28093 
## 
## $criterions$bic2$comp17
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -30.23846 -30.12803 -30.01760 -29.90717 -29.79674 -29.68631 -29.57588 -29.46545 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -29.35502 -29.24460 -29.13417 -29.02374 -28.91331 -28.80288 -28.69245 -28.58202 
##    comp17    comp18    comp19    comp20 
## -28.47159 -28.36116 -28.25073 -28.14030 
## 
## $criterions$bic2$comp18
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -28.00617 -27.89574 -27.78531 -27.67489 -27.56446 -27.45403 -27.34360 -27.23317 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -27.12274 -27.01231 -26.90188 -26.79145 -26.68102 -26.57059 -26.46016 -26.34973 
##    comp17    comp18    comp19    comp20 
## -26.23931 -26.12888 -26.01845 -25.90802 
## 
## $criterions$bic2$comp19
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -25.79950 -25.68907 -25.57864 -25.46821 -25.35779 -25.24736 -25.13693 -25.02650 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -24.91607 -24.80564 -24.69521 -24.58478 -24.47435 -24.36392 -24.25349 -24.14306 
##    comp17    comp18    comp19    comp20 
## -24.03263 -23.92221 -23.81178 -23.70135 
## 
## $criterions$bic2$comp20
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -23.59531 -23.48488 -23.37445 -23.26402 -23.15359 -23.04316 -22.93273 -22.82230 
##     comp9    comp10    comp11    comp12    comp13    comp14    comp15    comp16 
## -22.71187 -22.60144 -22.49101 -22.38059 -22.27016 -22.15973 -22.04930 -21.93887 
##    comp17    comp18    comp19    comp20 
## -21.82844 -21.71801 -21.60758 -21.49715 
## 
## 
## 
## $optncomp
## [1] 4 5
## 
## $optlambdaX
## NULL
## 
## $optlambdaY
## NULL
## 
## $optlambdaXsup
## NULL
## 
## $optlambdaYsup
## NULL
## 
## attr(,"class")
## [1] "ncompsearch"

par(mfrow=c(1,2))
plot(ncomp32,1)
plot(ncomp32,2)

PLS

ncomp41 = ncompsearch(X3, Y3, comps = c(1, 5, 10*(1:2)), criterion="BIC")
plot(ncomp41)

Combined Search

The number of components and regularized parameters can be selected by the function optparasearch. The following options are available.

criteria = c("BIC", "CV")
search.methods = c("regparaonly", "regpara1st", "ncomp1st", "simultaneous")

Single Block

PCA

The regparaonly method searches for the regularized parameters with a fixed number of components.

(opt11 = optparasearch(X1, search.method = "regparaonly", criterion="BIC", comp=ncomp11$optncomp))

## $optncomp
## [1] 5 1
## 
## $optlambdaX
## [1] 0.1719097
## 
## $search.method
## [1] "regparaonly"
## 
## $criterion
## [1] "BIC"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit311 = msma(X1, comp=opt11$optncomp, lambdaX=opt11$optlambdaX))

## Call:
## msma.default(X = X1, comp = opt11$optncomp, lambdaX = opt11$optlambdaX)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     3     3     4     4
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

The ncomp1st method identifies the number of components with a regularized parameter of 0, then searches for the regularized parameters with the selected number of components.

(opt12 = optparasearch(X1, search.method = "ncomp1st", criterion="BIC"))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
##  [1]  1  2  3  4  5  6  7  8  9 10
## 
## $comps[[2]]
## [1] 1
## 
## 
## $mincriterion
##       comp5 comp1.comp1 
##   -71.33402        -Inf 
## 
## $criterions
## $criterions$bic
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##  -1.718502  -2.161108  -2.598561  -3.322634 -71.334021 -65.464450 -65.345615 
##      comp8      comp9     comp10 
## -65.226304 -65.116775 -65.006274 
## 
## $criterions$bic2
##  comp1.comp1  comp2.comp1  comp3.comp1  comp4.comp1  comp5.comp1  comp6.comp1 
##         -Inf    -71.59729         -Inf    -73.74255         -Inf    -70.23735 
##  comp7.comp1  comp8.comp1  comp9.comp1 comp10.comp1 
##    -65.17860    -65.12805    -65.01969    -64.95044 
## 
## 
## $optncomp
## [1] 5 1
## 
## $optlambdaX
## [1] 0.1719097
## 
## $search.method
## [1] "ncomp1st"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit312 = msma(X1, comp=opt12$optncomp, lambdaX=opt12$optlambdaX))

## Call:
## msma.default(X = X1, comp = opt12$optncomp, lambdaX = opt12$optlambdaX)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     3     3     4     4
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

The regpara1st identifies the regularized parameters by fixing the number of components, then searching for the number of components with the selected regularized parameters.

(opt13 = optparasearch(X1, search.method = "regpara1st", criterion="BIC"))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
##  [1]  1  2  3  4  5  6  7  8  9 10
## 
## $comps[[2]]
## [1] 1
## 
## 
## $mincriterion
##       comp5 comp1.comp1 
##   -70.52345   -70.49841 
## 
## $criterions
## $criterions$bic
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##  -1.718325  -2.202584  -2.657239  -3.466906 -70.523451 -65.561012 -65.520186 
##      comp8      comp9     comp10 
## -65.470343 -65.437522 -65.383752 
## 
## $criterions$bic2
##  comp1.comp1  comp2.comp1  comp3.comp1  comp4.comp1  comp5.comp1  comp6.comp1 
##    -70.49841    -73.16344    -74.32756         -Inf    -73.78251    -69.14916 
##  comp7.comp1  comp8.comp1  comp9.comp1 comp10.comp1 
##    -65.21102    -65.12684    -65.05441    -64.99683 
## 
## 
## $optncomp
## [1] 5 1
## 
## $optlambdaX
## [1] 0.2578646
## 
## $optlambdaY
## NULL
## 
## $optlambdaXsup
## NULL
## 
## $optlambdaYsup
## NULL
## 
## $search.method
## [1] "regpara1st"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit313 = msma(X1, comp=opt13$optncomp, lambdaX=opt13$optlambdaX))

## Call:
## msma.default(X = X1, comp = opt13$optncomp, lambdaX = opt13$optlambdaX)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     2     3     2     3
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

The simultaneous method identifies the number of components by searching the regularized parameters in each component.

(opt14 = optparasearch(X1, search.method = "simultaneous", criterion="BIC"))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
##  [1]  1  2  3  4  5  6  7  8  9 10
## 
## $comps[[2]]
## [1] 1
## 
## 
## $mincriterion
## cve.criterion       lambdaX 
##  -72.09696114    0.02865162 
## 
## $criterions
## $criterions[[1]]
## cve.criterion cve.criterion cve.criterion cve.criterion cve.criterion 
##     -1.718502     -2.202584     -2.677000     -3.497959    -72.096961 
## cve.criterion cve.criterion cve.criterion cve.criterion cve.criterion 
##    -65.591708    -65.520186    -65.470343    -65.437522    -65.383752 
## 
## $criterions[[2]]
##    lambdaX    lambdaX    lambdaX    lambdaX    lambdaX    lambdaX    lambdaX 
## 0.02865162 0.25786460 0.34381947 0.34381947 0.17190973 0.17190973 0.25786460 
##    lambdaX    lambdaX    lambdaX 
## 0.25786460 0.25786460 0.25786460 
## 
## 
## $optncomp
## [1] 5 1
## 
## $optlambdaX
## [1] 0.1719097
## 
## $optlambdaY
## NULL
## 
## $optlambdaXsup
## NULL
## 
## $optlambdaYsup
## NULL
## 
## $search.method
## [1] "simultaneous"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit314 = msma(X1, comp=opt14$optncomp, lambdaX=opt14$optlambdaX))

## Call:
## msma.default(X = X1, comp = opt14$optncomp, lambdaX = opt14$optlambdaX)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     3     3     4     4
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

The argument maxpct4ncomp=0.5 means that 0.5λ is used as the regularized parameter when the number of components is searched and where λ is the maximum of the regularized parameters among the possible candidates.

(opt132 = optparasearch(X1, search.method = "ncomp1st", criterion="BIC", maxpct4ncomp=0.5))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
##  [1]  1  2  3  4  5  6  7  8  9 10
## 
## $comps[[2]]
## [1] 1
## 
## 
## $mincriterion
## cve.criterion       lambdaX 
##  -72.09696114    0.01432581 
## 
## $criterions
## $criterions[[1]]
## cve.criterion cve.criterion cve.criterion cve.criterion cve.criterion 
##     -1.718502     -2.190527     -2.649988     -3.450188    -72.096961 
## cve.criterion cve.criterion cve.criterion cve.criterion cve.criterion 
##    -65.591708    -65.511970    -65.464676    -65.395245    -65.327174 
## 
## $criterions[[2]]
##    lambdaX    lambdaX    lambdaX    lambdaX    lambdaX    lambdaX    lambdaX 
## 0.01432581 0.17190973 0.17190973 0.18623555 0.17190973 0.17190973 0.21488717 
##    lambdaX    lambdaX    lambdaX 
## 0.21488717 0.21488717 0.21488717 
## 
## 
## $optncomp
## [1] 5 1
## 
## $optlambdaX
## [1] 0.1719097
## 
## $search.method
## [1] "ncomp1st"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit3132 = msma(X1, comp=opt132$optncomp, lambdaX=opt132$optlambdaX))

## Call:
## msma.default(X = X1, comp = opt132$optncomp, lambdaX = opt132$optlambdaX)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     3     3     4     4
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

The result with the argument regpara1st depends on the number of components and the default value is 10. The number of components is set as follows.

(opt133 = optparasearch(X1, search.method = "regpara1st", criterion="BIC", comp=5))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
## [1] 1 2 3 4 5
## 
## $comps[[2]]
## [1] 1
## 
## 
## $mincriterion
##       comp5 comp1.comp1 
##   -72.09696        -Inf 
## 
## $criterions
## $criterions$bic
##      comp1      comp2      comp3      comp4      comp5 
##  -1.718465  -2.190527  -2.649988  -3.406731 -72.096961 
## 
## $criterions$bic2
## comp1.comp1 comp2.comp1 comp3.comp1 comp4.comp1 comp5.comp1 
##        -Inf   -73.11697   -73.72111        -Inf   -74.54420 
## 
## 
## $optncomp
## [1] 5 1
## 
## $optlambdaX
## [1] 0.1719097
## 
## $optlambdaY
## NULL
## 
## $optlambdaXsup
## NULL
## 
## $optlambdaYsup
## NULL
## 
## $search.method
## [1] "regpara1st"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit3133 = msma(X1, comp=opt133$optncomp, lambdaX=opt133$optlambdaX))

## Call:
## msma.default(X = X1, comp = opt133$optncomp, lambdaX = opt133$optlambdaX)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5
## block1     5     3     3     4     4
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5
## comp1-1     1     1     1     1     1

PLS

For PLS, two parameters λ_X and λ_Y are used in arguments lambdaX and lambdaY to control sparseness for data X and Y, respectively.

(opt21 = optparasearch(X2, Y2, search.method = "regparaonly", criterion="BIC"))

## $optncomp
## [1] 10  1
## 
## $optlambdaX
##   lambdaX1   lambdaX2 
## 0.04122716 0.45340561 
## 
## $optlambdaY
##  lambdaY1  lambdaY2 
## 0.4750089 0.1880227 
## 
## $optlambdaXsup
## lambdaXsup1 
##   0.4700392 
## 
## $optlambdaYsup
## lambdaYsup1 
##    0.574523 
## 
## $search.method
## [1] "regparaonly"
## 
## $criterion
## [1] "BIC"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit321 = msma(X2, Y2, comp=opt21$optncomp, lambdaX=opt21$optlambdaX, lambdaY=opt21$optlambdaY))

## Call:
## msma.default(X = X2, Y = Y2, comp = opt21$optncomp, lambdaX = opt21$optlambdaX, 
##     lambdaY = opt21$optlambdaY)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     3     3     3     3     3     3     3     3     3      3
## block2     3     2     1     2     2     2     2     2     2      2
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     2     2     2     2     2     2     2     2     2      2
## 
## Numbers of non-zeros for Y block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     1     1     2     2     1     1     1     1     1      1
## block2     3     2     3     2     2     3     3     3     3      3
## 
## Numbers of non-zeros for Y super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     2     2     2     2     2     2     2     2     2      2

Multi Block

PCA

(opt31 = optparasearch(X3, search.method = "regparaonly", criterion="BIC"))

## $optncomp
## [1] 10  1
## 
## $optlambdaX
##  lambdaX1  lambdaX2  lambdaX3  lambdaX4  lambdaX5 
## 0.1310995 0.1513608 0.1590729 0.1300171 0.3984990 
## 
## $optlambdaXsup
## lambdaXsup1 
##   0.1493833 
## 
## $search.method
## [1] "regparaonly"
## 
## $criterion
## [1] "BIC"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit331 = msma(X3, comp=opt31$optncomp, lambdaX=opt31$optlambdaX, lambdaXsup=opt31$optlambdaXsup))

## Call:
## msma.default(X = X3, comp = opt31$optncomp, lambdaX = opt31$optlambdaX, 
##     lambdaXsup = opt31$optlambdaXsup)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     0     4     0     2     2     0     0     3     2      2
## block2     0     0     2     2     0     2     2     0     0      0
## block3     4     0     2     0     0     4     4     0     3      3
## block4     0     2     2     3     0     0     2     0     0      0
## block5     0     0     2     2     1     2     0     0     1      1
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     1     2     4     4     2     3     3     1     3      3

(opt32 = optparasearch(X3, search.method = "regparaonly", criterion="BIC", whichselect="X"))

## $optncomp
## [1] 10  1
## 
## $optlambdaX
##  lambdaX1  lambdaX2  lambdaX3  lambdaX4  lambdaX5 
## 0.3932986 0.4540824 0.1590729 0.3900513 0.1328330 
## 
## $search.method
## [1] "regparaonly"
## 
## $criterion
## [1] "BIC"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit332 = msma(X3, comp=opt32$optncomp, lambdaX=opt32$optlambdaX, lambdaXsup=opt32$optlambdaXsup))

## Call:
## msma.default(X = X3, comp = opt32$optncomp, lambdaX = opt32$optlambdaX, 
##     lambdaXsup = opt32$optlambdaXsup)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     2     2     1     2     2     1     1     1     1      1
## block2     2     1     2     1     2     1     1     1     1      1
## block3     4     4     3     4     3     3     3     3     3      3
## block4     1     2     1     2     2     3     3     3     3      3
## block5     4     4     2     3     3     4     3     3     4      3
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     5     5     5     5     5     5     5     5     5      5

(opt33 = optparasearch(X3, search.method = "regparaonly", criterion="BIC", whichselect="Xsup"))

## $optncomp
## [1] 10  1
## 
## $optlambdaXsup
## [1] 0.1493833
## 
## $search.method
## [1] "regparaonly"
## 
## $criterion
## [1] "BIC"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit333 = msma(X3, comp=opt33$optncomp, lambdaX=opt33$optlambdaX, lambdaXsup=opt33$optlambdaXsup))

## Call:
## msma.default(X = X3, comp = opt33$optncomp, lambdaX = opt33$optlambdaX, 
##     lambdaXsup = opt33$optlambdaXsup)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     0     0     3     0     2     0     3     1     0      0
## block2     0     0     4     3     3     0     0     0     2      0
## block3     4     0     0     4     0     2     3     0     0      3
## block4     0     4     3     3     4     0     0     0     0      4
## block5     4     0     4     2     2     0     0     0     0      4
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     2     1     4     4     4     1     2     1     1      3

Nested

ncomp1st

(opt341 = optparasearch(X3, search.method = "ncomp1st", criterion="BIC", comp=c(8, 8)))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
## [1] 1 2 3 4 5 6 7 8
## 
## $comps[[2]]
## [1] 1 2 3 4 5 6 7 8
## 
## 
## $mincriterion
## $mincriterion$comp4
## [1] -70.91364
## 
## $mincriterion$comp5
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -66.71245 -66.60202 -66.49160 -66.38117 -66.27074 -66.16031 -66.04988 -65.93945 
## 
## 
## $criterions
## $criterions$bic
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##  -1.825223  -2.241347  -2.958985 -70.913638 -65.437056 -65.295837 -65.167319 
##      comp8 
## -65.024228 
## 
## $criterions$bic2
## $criterions$bic2$comp1
##       comp1       comp2       comp3       comp4       comp5       comp6 
##   0.9714158   0.7048689   0.3285174  -0.4043394 -69.4380900 -67.7265723 
##       comp7       comp8 
## -67.6384809 -67.5823767 
## 
## $criterions$bic2$comp2
##       comp1       comp2       comp3       comp4       comp5       comp6 
##  -0.9355045  -1.2330970  -1.5929192  -2.2947183 -71.7593199 -68.7947121 
##       comp7       comp8 
## -68.7054709 -68.5457003 
## 
## $criterions$bic2$comp3
##       comp1       comp2       comp3       comp4       comp5       comp6 
##  -0.2223657  -0.5434971  -1.0123128  -1.7717225 -71.0699281 -68.0482575 
##       comp7       comp8 
## -67.9938194 -67.8878793 
## 
## $criterions$bic2$comp4
##        comp1        comp2        comp3        comp4        comp5        comp6 
##   0.38080030   0.03559579  -0.35712052  -1.16121248 -70.56059815 -66.59789445 
##        comp7        comp8 
## -66.55562062 -66.44197444 
## 
## $criterions$bic2$comp5
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -66.71245 -66.60202 -66.49160 -66.38117 -66.27074 -66.16031 -66.04988 -65.93945 
## 
## $criterions$bic2$comp6
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -61.10526 -60.99483 -60.88440 -60.77397 -60.66354 -60.55311 -60.44269 -60.33226 
## 
## $criterions$bic2$comp7
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -60.26533 -60.15490 -60.04447 -59.93404 -59.82361 -59.71318 -59.60275 -59.49232 
## 
## $criterions$bic2$comp8
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -59.35922 -59.24879 -59.13836 -59.02793 -58.91750 -58.80707 -58.69664 -58.58621 
## 
## 
## 
## $optncomp
## [1] 4 5
## 
## $optlambdaX
##  lambdaX1  lambdaX2  lambdaX3  lambdaX4  lambdaX5 
## 0.4916233 0.0378402 0.1193047 0.4875641 0.2324578 
## 
## $optlambdaXsup
## lambdaXsup1 
##  0.03734583 
## 
## $search.method
## [1] "ncomp1st"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit341 = msma(X3, comp=opt341$optncomp, lambdaX=opt341$optlambdaX, lambdaXsup=opt341$optlambdaXsup))

## Call:
## msma.default(X = X3, comp = opt341$optncomp, lambdaX = opt341$optlambdaX, 
##     lambdaXsup = opt341$optlambdaXsup)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4
## block1     2     2     1     1
## block2     4     4     4     2
## block3     4     3     3     4
## block4     1     2     1     2
## block5     4     3     2     3
## 
## Numbers of non-zeros for X super: 
## $comp1
## comp1-1 comp1-2 comp1-3 comp1-4 comp1-5 
##       5       3       4       3       4 
## 
## $comp2
## comp2-1 comp2-2 comp2-3 comp2-4 comp2-5 
##       5       5       5       4       5 
## 
## $comp3
## comp3-1 comp3-2 comp3-3 comp3-4 comp3-5 
##       5       5       4       5       5 
## 
## $comp4
## comp4-1 comp4-2 comp4-3 comp4-4 comp4-5 
##       5       4       4       5       5

regparaonly

(opt342 = optparasearch(X3, search.method = "regparaonly", criterion="BIC", comp=c(4, 5)))

## $optncomp
## [1] 4 5
## 
## $optlambdaX
##  lambdaX1  lambdaX2  lambdaX3  lambdaX4  lambdaX5 
## 0.4916233 0.0378402 0.1193047 0.4875641 0.2324578 
## 
## $optlambdaXsup
## lambdaXsup1 
##  0.03734583 
## 
## $search.method
## [1] "regparaonly"
## 
## $criterion
## [1] "BIC"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit342 = msma(X3, comp=opt342$optncomp, lambdaX=opt342$optlambdaX, lambdaXsup=opt342$optlambdaXsup))

## Call:
## msma.default(X = X3, comp = opt342$optncomp, lambdaX = opt342$optlambdaX, 
##     lambdaXsup = opt342$optlambdaXsup)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4
## block1     2     2     1     1
## block2     4     4     4     2
## block3     4     3     3     4
## block4     1     2     1     2
## block5     4     3     2     3
## 
## Numbers of non-zeros for X super: 
## $comp1
## comp1-1 comp1-2 comp1-3 comp1-4 comp1-5 
##       5       3       4       3       4 
## 
## $comp2
## comp2-1 comp2-2 comp2-3 comp2-4 comp2-5 
##       5       5       5       4       5 
## 
## $comp3
## comp3-1 comp3-2 comp3-3 comp3-4 comp3-5 
##       5       5       4       5       5 
## 
## $comp4
## comp4-1 comp4-2 comp4-3 comp4-4 comp4-5 
##       5       4       4       5       5

regpara1st

(opt344 = optparasearch(X3, search.method = "regpara1st", criterion="BIC", comp=c(8, 8)))

## $criterion
## [1] "BIC"
## 
## $comps
## $comps[[1]]
## [1] 1 2 3 4 5 6 7 8
## 
## $comps[[2]]
## [1] 1 2 3 4 5 6 7 8
## 
## 
## $mincriterion
## $mincriterion$comp8
## [1] -66.21239
## 
## $mincriterion$comp1
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -72.94766 -70.49607 -72.33764 -71.01866 -70.42981 -71.30597 -70.95240 -72.22721 
## 
## 
## $criterions
## $criterions$bic
##       comp1       comp2       comp3       comp4       comp5       comp6 
##  -0.1836786  -0.5779916  -1.7049413  -2.2299697  -2.4415729  -3.0043768 
##       comp7       comp8 
##  -4.5907311 -66.2123921 
## 
## $criterions$bic2
## $criterions$bic2$comp1
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
## -72.94766 -70.49607 -72.33764 -71.01866 -70.42981 -71.30597 -70.95240 -72.22721 
## 
## $criterions$bic2$comp2
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##  -1.675412 -71.958098 -68.637860 -68.768105 -68.674988 -68.579625 -68.632507 
##      comp8 
## -68.610421 
## 
## $criterions$bic2$comp3
##       comp1       comp2       comp3       comp4       comp5       comp6 
##  -0.5020861  -1.1361134  -1.9949863 -72.0947400 -66.2946417 -66.2913120 
##       comp7       comp8 
## -66.2428068 -66.0847116 
## 
## $criterions$bic2$comp4
##       comp1       comp2       comp3       comp4       comp5       comp6 
##  -0.8056794  -1.4272522  -1.9331660 -73.0341196 -69.3215754 -69.2068339 
##       comp7       comp8 
## -69.0520420 -69.2894776 
## 
## $criterions$bic2$comp5
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##  -1.975579 -73.980345 -69.389158 -69.407574 -69.198641 -69.416905 -69.123203 
##      comp8 
## -69.170870 
## 
## $criterions$bic2$comp6
##      comp1      comp2      comp3      comp4      comp5      comp6      comp7 
##  -0.904339  -1.865053 -71.893451 -68.907627 -68.848114 -68.800760 -68.743626 
##      comp8 
## -68.799567 
## 
## $criterions$bic2$comp7
##       comp1       comp2       comp3       comp4       comp5       comp6 
##  -0.6752677  -1.8879290 -71.6237651 -67.9334613 -67.9519149 -67.9633865 
##       comp7       comp8 
## -68.0538679 -67.9845799 
## 
## $criterions$bic2$comp8
##     comp1     comp2     comp3     comp4     comp5     comp6     comp7     comp8 
##      -Inf -69.00500 -69.28636 -69.62148 -69.38213 -69.36005 -68.77962 -68.87249 
## 
## 
## 
## $optncomp
## [1] 8 1
## 
## $optlambdaX
##  lambdaX1  lambdaX2  lambdaX3  lambdaX4  lambdaX5 
## 0.1310995 0.1513608 0.1590729 0.1300171 0.3984990 
## 
## $optlambdaY
## NULL
## 
## $optlambdaXsup
## lambdaXsup1 
##   0.1493833 
## 
## $optlambdaYsup
## NULL
## 
## $search.method
## [1] "regpara1st"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit344 = msma(X3, comp=opt344$optncomp, lambdaX=opt344$optlambdaX, lambdaXsup=opt344$optlambdaXsup))

## Call:
## msma.default(X = X3, comp = opt344$optncomp, lambdaX = opt344$optlambdaX, 
##     lambdaXsup = opt344$optlambdaXsup)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8
## block1     0     4     0     2     2     0     0     3
## block2     0     0     2     2     0     2     2     0
## block3     4     0     2     0     0     4     4     0
## block4     0     2     2     3     0     0     2     0
## block5     0     0     2     2     1     2     0     0
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8
## comp1-1     1     2     4     4     2     3     3     1

This is computationally expensive and takes much longer to execute.

(opt345 = optparasearch(X3, search.method = "simultaneous", criterion="BIC", comp=c(8, 8)))
(fit345 = msma(X3, comp=opt345$optncomp, lambdaX=opt345$optlambdaX))

PLS

This is computationally expensive and takes much longer to execute due to the large number of blocks.

(opt41 = optparasearch(X3, Y3, search.method = "regparaonly", criterion="BIC"))
(fit341 = msma(X3, Y3, comp=opt41$optncomp, lambdaX=opt41$optlambdaX, lambdaY=opt41$optlambdaY, lambdaXsup=opt41$optlambdaXsup, lambdaYsup=opt41$optlambdaYsup))

In this example, it works by narrowing down the parameters as follows.

(opt42 = optparasearch(X3, Y3, search.method = "regparaonly", criterion="BIC", whichselect=c("Xsup","Ysup")))

## $optncomp
## [1] 10  1
## 
## $optlambdaXsup
## lambdaXsup1 
##    0.614961 
## 
## $optlambdaYsup
## lambdaYsup1 
##   0.2417596 
## 
## $search.method
## [1] "regparaonly"
## 
## $criterion
## [1] "BIC"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit342 = msma(X3, Y3, comp=opt42$optncomp, lambdaX=opt42$optlambdaX, lambdaY=opt42$optlambdaY, lambdaXsup=opt42$optlambdaXsup, lambdaYsup=opt42$optlambdaYsup))

## Call:
## msma.default(X = X3, Y = Y3, comp = opt42$optncomp, lambdaX = opt42$optlambdaX, 
##     lambdaY = opt42$optlambdaY, lambdaXsup = opt42$optlambdaXsup, 
##     lambdaYsup = opt42$optlambdaYsup)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     1     0     1     0     0     0     1     0     0      0
## block2     0     0     0     0     0     0     0     0     0      0
## block3     0     1     0     0     0     0     0     0     0      1
## block4     0     0     0     1     0     1     0     1     1      0
## block5     0     0     0     0     1     0     0     0     0      0
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     1     1     1     1     1     1     1     1     1      1
## 
## Numbers of non-zeros for Y block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     0     0     4     2     2     4     0     0     3      3
## block2     0     4     2     2     3     0     0     0     2      3
## block3     3     0     0     2     2     0     0     3     0      3
## block4     4     2     0     3     3     0     0     0     4      4
## block5     4     3     0     0     3     0     2     0     2      2
## 
## Numbers of non-zeros for Y super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     3     3     2     4     5     1     1     1     4      5

Another example dataset is generated.

dataset4 = simdata(n = 50, rho = 0.8, Yps = rep(4, 2), Xps = rep(4, 3), seed=1)
X4 = dataset4$X; Y4 = dataset4$Y

With this number of blocks, the calculation can be performed in a relatively short time.

(opt43 = optparasearch(X4, Y4, search.method = "regparaonly", criterion="BIC"))

## $optncomp
## [1] 10  1
## 
## $optlambdaX
##  lambdaX1  lambdaX2  lambdaX3 
## 0.1659782 0.4483126 0.1461557 
## 
## $optlambdaY
##  lambdaY1  lambdaY2 
## 0.5138342 0.4073084 
## 
## $optlambdaXsup
## lambdaXsup1 
##   0.1571942 
## 
## $optlambdaYsup
## lambdaYsup1 
##   0.1728372 
## 
## $search.method
## [1] "regparaonly"
## 
## $criterion
## [1] "BIC"
## 
## $criterion4ncomp
## [1] "BIC"
## 
## attr(,"class")
## [1] "optparasearch"

(fit343 = msma(X4, Y4, comp=opt43$optncomp, lambdaX=opt43$optlambdaX, lambdaY=opt43$optlambdaY, lambdaXsup=opt43$optlambdaXsup, lambdaYsup=opt43$optlambdaYsup))

## Call:
## msma.default(X = X4, Y = Y4, comp = opt43$optncomp, lambdaX = opt43$optlambdaX, 
##     lambdaY = opt43$optlambdaY, lambdaXsup = opt43$optlambdaXsup, 
##     lambdaYsup = opt43$optlambdaYsup)
## 
## Numbers of non-zeros for X block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     2     3     3     2     0     2     3     2     2      2
## block2     2     1     1     2     0     2     2     3     2      2
## block3     3     3     3     0     3     2     2     2     2      2
## 
## Numbers of non-zeros for X super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     3     3     3     2     1     3     3     3     3      3
## 
## Numbers of non-zeros for Y block: 
##        comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## block1     2     1     1     1     0     0     0     0     0      0
## block2     3     2     2     0     2     1     1     1     1      1
## 
## Numbers of non-zeros for Y super: 
##         comp1 comp2 comp3 comp4 comp5 comp6 comp7 comp8 comp9 comp10
## comp1-1     2     2     2     1     1     1     1     1     1      1

Example for msma

Preparation

Getting started

One Component

Two Component

Single Block

Principal Component Analysis (PCA)

Other functions in R for PCA

Sparse PCA

Supervised Sparse PCA

Partial Least Squres (PLS)

Sparse PLS

Supervised Sparse PLS

Multi Block

PCA

Sparse PCA

Supervised Sparse PCA

Nested Component

PLS

Sparse PLS

Supervised Sparse PLS

Parameter Selection

Number of Components

Single Block

PCA

PLS

Multi Block

PCA

Nested

PLS

Combined Search

Single Block

PCA

PLS

Multi Block

PCA

Nested

PLS