Vignette of R package kko

This package provides a kernel knockoffs selection procedure, dubbed KKO, for the nonparametric additive model. The procedure integrates three key components: the knockoffs, the subsampling for stability, and the random feature mapping for nonparametric function approximation. Finite-sample false discovery rate (FDR) control guarantee is established for KKO, see Dai et al. (2021).

Generate data

Let us begin by creating some synthetic data. The data is generated from additive polynomial function.

library(ggplot2)
library(kko)
library(knockoff)
set.seed(12345)

### generate regression coefficent
p=20 # number of predictors
sig_mag=10 # signal strength
s=5  # sparsity, number of nonzero component functions
reg_coef=c(rep(1,s),rep(0,p-s))  # regression coefficient
reg_coef=reg_coef*(2*(rnorm(p)>0)-1)*sig_mag

### generate response and design
model="poly"
n= 600 # sample size
X=matrix(rnorm(n*p),n,p)   # generate design
X_k = create.second_order(X) # generate knockoff
y=generate_data(X,reg_coef,model) # response

Kernel knockoffs selection

We then apply KKO method to generate importance scores of variables.

rkernel="laplacian" # kernel choice
rk_scale=1  # scaling paramtere of kernel
rfn_range=c(2,3,4)  # number of random features
cv_folds=15  # folds of cross-validation in group lasso
n_stb=200 # number of subsampling for importance scores 
n_stb_tune=100 # number of subsampling for tuning random feature number
frac_stb=1/2 # fraction of subsample
nCores_para=2 # number of cores for parallelization

### KKO selection 
kko_fit=kko(X,y,X_k,rfn_range,n_stb_tune,n_stb,cv_folds,frac_stb,nCores_para,rkernel,rk_scale)

The importance scores by KKO are the difference of selection frequencies between variables and knockoffs, ranging from −1 to 1. The active variables are expected to have high positive scores (close to one). Those of null variables are expcted to stay centered at zero.

reg_coef  # true regression coefficient 
##  [1]  10  10 -10 -10  10   0   0   0   0   0   0   0   0   0   0   0   0   0   0
## [20]   0
W=kko_fit$importance_score # knockoff importance scores generated by KKO 
W 
##  [1]  0.703333333  0.160000000  0.870000000  0.886666667  0.776666667
##  [6] -0.006666667  0.023333333 -0.040000000 -0.006666667  0.000000000
## [11] -0.003333333 -0.003333333 -0.003333333  0.000000000 -0.043333333
## [16] -0.016666667 -0.030000000  0.003333333  0.000000000 -0.003333333
mydata=data.frame(W=W,var_group=ifelse(reg_coef!=0,"Active","NUll"))
myplot = ggplot(mydata, aes(W, fill = var_group)) +  
  geom_histogram(color = "gray2",binwidth=1/p) + theme_bw()+
  xlab("Importance scores")+ylab("Number of variables")+
  xlim(-1,1)

print(myplot)
## Warning: Removed 4 rows containing missing values or values outside the scale range
## (`geom_bar()`).

Knockoff filtering

We apply knockoff filter on KKO importance scores. The filter computes a threshold on scores, and pick significant variables above the threshold.

fdr=0.2 #FDR control level 
thres = knockoff.threshold(W, fdr=fdr) # thresholding on scores by knockoff filter
selected = which(W >= thres) 
selected  # indices of selected variables 
## [1] 1 2 3 4 5

Reference

  1. Xiaowu Dai, Xiang Lyu, and Lexin Li. Kernel Knockoffs Selection for Nonparametric Additive Models. arXiv preprint arXiv:2105.11659 (2021).