Title: | Iterative Max-Min Entropy Margin-Maximization with Interaction Terms for Feature Selection |
---|---|

Description: | Based on large margin principle, this package performs feature selection methods: "IM4E"(Iterative Margin-Maximization under Max-Min Entropy Algorithm); "Immigrate"(Iterative Max-Min Entropy Margin-Maximization with Interaction Terms Algorithm); "BIM"(Boosted version of IMMIGRATE algorithm); "Simba"(Iterative Search Margin Based Algorithm); "LFE"(Local Feature Extraction Algorithm). This package also performs prediction for the above feature selection methods. |

Authors: | Ruzhang Zhao, Pengyu Hong, Jun S. Liu |

Maintainer: | Ruzhang Zhao<ruzhangzhao@gmail.com> |

License: | GPL (>= 2) |

Version: | 0.2.1 |

Built: | 2024-02-06 07:44:03 UTC |

Source: | CRAN |

- BIM
- IM4E
- Immigrate
- LFE
- one.IM4E
- one.Immigrate
- Parkinsons Dataset
- pred.values
- predict.BIM
- predict.IM4E
- predict.Immigrate
- predict.LFE
- Simba

This function performs BIM algorithm (Boosted version of IMMIGRATE).

```
BIM(
xx,
yy,
nBoost = 3,
max_iter = 5,
removesmall = FALSE,
sigstart = 0.02,
sigend = 4
)
```

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`nBoost` |
number of classifiers in BIM, default to be 3 |

`max_iter` |
maximum number of iteration for IMMIRGATE classifier, default to be 5 |

`removesmall` |
whether remove features with small weights, default to be FALSE |

`sigstart` |
start of sigma used in algorithm, default to be 0.02 |

`sigend` |
end of sigma used in algorithm, default to be 4 |

`matrix` |
list of weight matrices |

`weights` |
coefficient vectors for classifiers |

`sample_wt` |
sample weights, refer to cost function in link below for more details |

Zhao, Ruzhang, Pengyu Hong, and Jun S. Liu. "IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms." Entropy 22.3 (2020): 291.

Please refer to https://www.mdpi.com/1099-4300/22/3/291/htm for more details.

```
data(park)
xx<-park$xx
yy<-park$yy
re<-BIM(xx,yy)
```

This function performs IM4E(Iterative Margin-Maximization under Max-Min Entropy) algorithm.

```
IM4E(
xx,
yy,
epsilon = 0.01,
sig = 1,
lambda = 1,
max_iter = 10,
removesmall = FALSE
)
```

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`epsilon` |
criterion for stopping iteration, default to be 0.01 |

`sig` |
sigma used in algorithm, default to be 1 |

`lambda` |
lambda used in algorithm, default to be 1 |

`max_iter` |
maximum number of iteration |

`removesmall` |
whether remove features with small weights, default to be FALSE |

`w` |
weight vector obtained by IM4E algorithm |

`iter_num` |
number of iteration for convergence |

`final_c` |
final cost value. Refer to the cost function in reference below for more details |

Bei Y, Hong P. Maximizing margin quality and quantity[C]//Machine Learning for Signal Processing (MLSP), 2015 IEEE 25th International Workshop on. IEEE, 2015: 1-6.

```
data(park)
xx<-park$xx
yy<-park$yy
re<-IM4E(xx,yy)
print(re)
```

This function performs IMMIGRATE(Iterative Max-Min Entropy Margin-Maximization with Interaction Terms ) algorithm. IMMIGRATE is a hypothesis-margin based feature selection method with interaction terms. Its weight matrix reflects the relative importance of features and their iteractions, which can be used for feature selection.

```
Immigrate(
xx,
yy,
w0,
epsilon = 0.01,
sig = 1,
max_iter = 10,
removesmall = FALSE,
randomw0 = FALSE
)
```

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`w0` |
initial weight matrix, default to be diagonal matrix when missing |

`epsilon` |
criterion for stopping iteration |

`sig` |
sigma used in algorithm, default to be 1. Refer to the cost function in the link below for more details |

`max_iter` |
maximum number of iteration |

`removesmall` |
whether to remove features with small weights, default to be FALSE |

`randomw0` |
whether to use randomly initial weights, default to be FALSE |

`w` |
weight matrix obtained by IMMIGRATE algorithm |

`iter_num` |
number of iteration for convergence |

`final_c` |
final cost value. Refer to the cost function in link below for more details |

Zhao, Ruzhang, Pengyu Hong, and Jun S. Liu. "IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms." Entropy 22.3 (2020): 291.

Please refer to https://www.mdpi.com/1099-4300/22/3/291/htm for more details.

Please refer to https://github.com/RuzhangZhao/Immigrate/ for implementation demo.

```
data(park)
xx<-park$xx
yy<-park$yy
re<-Immigrate(xx,yy)
print(re)
```

This function performs LFE(Local Feature Extraction) algorithm.

`LFE(xx, yy, T = 5)`

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`T` |
number of instance used to update weights, default to be 5 |

`w` |
new weight matrix after LFE algorithm |

Sun Y, Wu D. A relief based feature extraction algorithm[C]//Proceedings of the 2008 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, 2008: 188-195.

```
data(park)
xx<-park$xx
yy<-park$yy
re<-LFE(xx,yy)
print(re)
```

This function performs (IM4E)Iterative Margin-Maximization under Max-Min Entropy algorithm for one loop.

`one.IM4E(train_xx, train_yy, w, sig = 1, lambda = 1)`

`train_xx` |
model matrix of explanatory variables |

`train_yy` |
label vector |

`w` |
initial weight |

`sig` |
sigma used in algorithm, default to be 1 |

`lambda` |
lambda used in algorithm, default to be 1 |

`w` |
new weight vector after one loop |

`C` |
cost after one loop |

This function performs Immigrate(Iterative Max-Min Entropy Margin-Maximization with Interaction Terms) algorithm for one loop.

`one.Immigrate(train_xx, train_yy, W, sig = 1)`

`train_xx` |
model matrix of explanatory variables |

`train_yy` |
label vector |

`W` |
initial weight matrix |

`sig` |
sigma used in algorithm, default to be 1 |

`W` |
new weight matrix after one loop |

`C` |
cost after one loop |

Please refer to https://github.com/RuzhangZhao/Immigrate/ for implementation demo.

```
data(park)
xx<-park$xx
yy<-park$yy
W0 <- diag(rep(1,ncol(xx)),ncol(xx))/sqrt(ncol(xx))
re<-one.Immigrate(xx,yy,W0)
print(re$w)
```

Parkinsons Dataset

`data(park)`

An object of class

Frank, A. and A. Asuncion. UCI Machine Learning Repository. 2010.

```
data(park)
xx <- park$xx
yy <- park$yy
```

This function performs some statistical value prediction

`pred.values(y_train, y_test, pred_train, pred_test)`

`y_train` |
label vector for training data |

`y_test` |
label vector for test data |

`pred_train` |
predicted probabilities for training data |

`pred_test` |
predicted probabilities for test data |

`AUC_train` |
AUC for training data |

`AUC_test` |
AUC for test data |

`accuracy_test` |
accuracy for test data |

`precision_test` |
precision for test data |

`recall_test` |
recall for test data |

`F1_test` |
F1 score for test data |

`thre` |
threshold to separate two labels, obtained from training data |

```
y_train<-c(0,1,0,1,0,1)
y_test<-c(0,1,0,1)
pred_train<-c(0.77,0.89,0.32,0.96,0.10,0.67)
pred_test<-c(0.68,0.75,0.50,0.81)
re<-pred.values(y_train,y_test,pred_train,pred_test)
print(re)
```

This function performs the predition for BIM algorithm (Boosted version of IMMIGRATE).

```
## S3 method for class 'BIM'
predict(object, xx, yy, newx, type = "both", ...)
```

`object` |
result of BIM algorithm |

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`newx` |
new model matrix to be predicted |

`type` |
the form of final output |

`...` |
further arguments passed to or from other methods |

`response` |
predicted probabilities for for new data (newx) |

`class` |
predicted class for for new data (newx) |

Zhao, Ruzhang, Pengyu Hong, and Jun S. Liu. "IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms." Entropy 22.3 (2020): 291.

Please refer to https://www.mdpi.com/1099-4300/22/3/291/htm for more details.

```
data(park)
xx<-park$xx
yy<-park$yy
index<-c(1:floor(nrow(xx)*0.3))
train_xx<-xx[-index,]
test_xx<-xx[index,]
train_yy<-yy[-index]
test_yy<-yy[index]
re<-BIM(train_xx,train_yy)
res<-predict(re,train_xx,train_yy,test_xx,type="class")
print(res)
```

This function performs the predition for IM4E(Iterative Margin-Maximization under Max-Min Entropy) algorithm.

```
## S3 method for class 'IM4E'
predict(object, xx, yy, newx, sig = 1, type = "both", ...)
```

`object` |
weight or result of IM4E algorithm |

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`newx` |
new model matrix to be predicted |

`sig` |
sigma used in algorithm, default to be 1 |

`type` |
the form of final output, default to be "both". One can also choose "response"(predicted probabilities) or "class"(predicted labels). |

`...` |
further arguments passed to or from other methods |

`response` |
predicted probabilities for new data (newx) |

`class` |
predicted class labels for new data (newx) |

Bei Y, Hong P. Maximizing margin quality and quantity[C]//Machine Learning for Signal Processing (MLSP), 2015 IEEE 25th International Workshop on. IEEE, 2015: 1-6.

```
data(park)
xx<-park$xx
yy<-park$yy
index<-c(1:floor(nrow(xx)*0.3))
train_xx<-xx[-index,]
test_xx<-xx[index,]
train_yy<-yy[-index]
test_yy<-yy[index]
re<-IM4E(train_xx,train_yy)
res<-predict(re,train_xx,train_yy,test_xx,type="class")
print(res)
```

This function performs the predition for Immigrate(Iterative Max-Min Entropy Margin-Maximization with Interaction Terms) algorithm.

```
## S3 method for class 'Immigrate'
predict(object, xx, yy, newx, sig = 1, type = "both", ...)
```

`object` |
result of Immigrate algorithm |

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`newx` |
new model matrix to be predicted |

`sig` |
sigma used in prediction function, default to be 1. Refer to the prediction function in the link below for more details |

`type` |
the form of final output, default to be "both". One can also choose "response"(predicted probabilities) or "class"(predicted labels). |

`...` |
further arguments passed to or from other methods |

`response` |
predicted probabilities for new data (newx) |

`class` |
predicted class labels for new data (newx) |

Please refer to https://www.mdpi.com/1099-4300/22/3/291/htm for more details.

Please refer to https://github.com/RuzhangZhao/Immigrate/ for implementation demo.

```
data(park)
xx<-park$xx
yy<-park$yy
index<-c(1:floor(nrow(xx)*0.3))
train_xx<-xx[-index,]
test_xx<-xx[index,]
train_yy<-yy[-index]
test_yy<-yy[index]
re<-Immigrate(train_xx,train_yy)
res<-predict(re,train_xx,train_yy,test_xx,type="class")
print(res)
```

This function performs predition for LFE(Local Feature Extraction) algorithm.

```
## S3 method for class 'LFE'
predict(object, xx, yy, newx, ...)
```

`object` |
weights obtained from LFE |

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`newx` |
new model matrix to be predicted |

`...` |
further arguments passed to or from other methods |

predicted labels for new data (newx)

Sun Y, Wu D. A relief based feature extraction algorithm[C]//Proceedings of the 2008 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, 2008: 188-195.

```
data(park)
xx<-park$xx
yy<-park$yy
w<-LFE(xx,yy)
pred<-predict(w,xx,yy,xx)
print(pred)
```

This function performs Simba(Iterative Search Margin Based Algorithm).

`Simba(xx, yy, T = 5)`

`xx` |
model matrix of explanatory variables |

`yy` |
label vector |

`T` |
number of instance used to update weights, default to be 5 |

`w` |
new weight after Simba algorithm |

Gilad-Bachrach R, Navot A, Tishby N. Margin based feature selection-theory and algorithms[C]//Proceedings of the twenty-first international conference on Machine learning. ACM, 2004: 43.

```
data(park)
xx<-park$xx
yy<-park$yy
re<-Simba(xx,yy)
print(re)
```