Package 'FRESA.CAD'

Description

Contains a set of utilities for building and testing formula-based models for Computer Aided Diagnosis/prognosis applications via feature selection. Bootstrapped Stage Wise Model Selection (B:SWiMS) controls the false selection (FS) for linear, logistic, or Cox proportional hazards regression models. Utilities include functions for: univariate/longitudinal analysis, data conditioning (i.e. covariate adjustment and normalization), model validation and visualization.

Details

Purpose: The design of diagnostic or prognostic multivariate models via the selection of significantly discriminant features. The models are selected via the bootstrapped step-wise selection of model features that offer a significant improvement in subject classification/error. The false selection control is achieved by train-test partitions, where train sets are used to select variables and test sets used to evaluate model performance. Variables that do not improve subject classification/error on the blind test are not included in the models. The main function of this package is the selection and cross-validation of diagnostic/prognostic linear, logistic, or Cox proportional hazards regression model constructed from a large set of candidate features. The variable selection may start by conditioning all variables via a covariate-adjustment and a z-inverse-rank-transformation. In order to integrate features with partial discriminant power, the package can be used to categorize the continuous variables and rank their discriminant power. Once ranked, each feature is bootstrap-tested in a multivariate model, and its blind performance is evaluated. Variables with a statistical significant improvement in classification/error are stored and finally inserted into the final model according to their relative store frequency. A cross-validation procedure may be used to diagnose the amount of model shrinkage produced by the selection scheme.

Author(s)

Jose Gerardo Tamez-Pena, Antonio Martinez-Torteya, Israel Alanis and Jorge Orozco Maintainer: <[email protected]>

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in medicine 27(2), 157-172.

Examples

## Not run: 
    ### Fresa Package Examples ####
    library("epiR")
    library("FRESA.CAD")
    library(network)
    library(GGally)
    library("e1071")


    # Start the graphics device driver to save all plots in a pdf format
    pdf(file = "Fresa.Package.Example.pdf",width = 8, height = 6)


    # Get the stage C prostate cancer data from the rpart package

    data(stagec,package = "rpart")
    options(na.action = 'na.pass')
    dataCancer <- cbind(pgstat = stagec$pgstat,
                        pgtime = stagec$pgtime,
                        as.data.frame(model.matrix(Surv(pgtime,pgstat) ~ .,stagec))[-1])

    #Impute missing values
    dataCancerImputed <- nearestNeighborImpute(dataCancer)

    # Remove the incomplete cases
    dataCancer <- dataCancer[complete.cases(dataCancer),]


    # Load a pre-stablished data frame with the names and descriptions of all variables
    data(cancerVarNames)
    # the Heat Map
    hm <- heatMaps(cancerVarNames,varRank=NULL,Outcome="pgstat",
                   data=dataCancer,title="Heat Map",hCluster=FALSE
                   ,prediction=NULL,Scale=TRUE,
                   theFiveColors=c("blue","cyan","black","yellow","red"),
                   outcomeColors = 
                     c("blue","lightgreen","yellow","orangered","red"),
                   transpose=FALSE,cexRow=0.50,cexCol=0.80,srtCol=35)

    # The univariate analysis
    UniRankFeaturesRaw <- univariateRankVariables(variableList = cancerVarNames,
                                                  formula = "pgstat ~ 1+pgtime",
                                                  Outcome = "pgstat",
                                                  data = dataCancer, 
                                                  categorizationType = "Raw", 
                                                  type = "LOGIT", 
                                                  rankingTest = "zIDI",
                                                  description = "Description",
                                                  uniType="Binary")

    print(UniRankFeaturesRaw)

    # A simple BSIWMS Model

    BSWiMSModel <- BSWiMS.model(formula = Surv(pgtime, pgstat) ~ 1, dataCancerImputed)

    # The Log-Rank Analysis using survdiff

    lrsurvdiff <- survdiff(Surv(pgtime,pgstat)~
                  BSWiMSModel$BSWiMS.model$back.model$linear.predictors > 0,
                  data=dataCancerImputed)

    # The Log-Rank Analysis EmpiricalSurvDiff and permutations of the null Chi distribution
    lrp <- EmpiricalSurvDiff(dataCancerImputed$pgtime,dataCancerImputed$pgstat,
                             BSWiMSModel$BSWiMS.model$back.model$linear.predictors > 0,
                             type="Chi",plots=TRUE,samples=10000)

    # The Log-Rank Analysis EmpiricalSurvDiff and permutations of the null SLR distribution
    lrp <- EmpiricalSurvDiff(dataCancerImputed$pgtime,dataCancerImputed$pgstat,
                             BSWiMSModel$BSWiMS.model$back.model$linear.predictors > 0,
                             type="SLR",plots=TRUE,samples=10000)

    # The Log-Rank Analysis EmpiricalSurvDiff and bootstrapping the SLR distribution
    lrp <- EmpiricalSurvDiff(dataCancerImputed$pgtime,dataCancerImputed$pgstat,
                             BSWiMSModel$BSWiMS.model$back.model$linear.predictors > 0,
                             computeDist=TRUE,plots=TRUE)

    #The performance of the final model using the summary function
    sm <- summary(BSWiMSModel$BSWiMS.model$back.model)
    print(sm$coefficients)
    pv <- plot(sm$bootstrap)

    # The equivalent model
    eq <- reportEquivalentVariables(BSWiMSModel$BSWiMS.model$back.model,data=dataCancer,
                                    variableList=cancerVarNames,Outcome = "pgstat",
                                    timeOutcome="pgtime",
                                    type = "COX");

    print(eq$equivalentMatrix)

    #The list of all models of the bootstrap forward selection 
    print(BSWiMSModel$forward.selection.list)

    #With FRESA.CAD we can do a leave-one-out using the list of models
    pm <- ensemblePredict(BSWiMSModel$forward.selection.list,
                          dataCancer,predictType = "linear",type="LOGIT",Outcome="pgstat")

    #Ploting the ROC with 95
    pm <- plotModels.ROC(cbind(dataCancer$pgstat,
                               pm$ensemblePredict),main=("LOO Forward Selection Median Predict"))

    #The plotModels.ROC provides the diagnosis confusion matrix.
    summary(epi.tests(pm$predictionTable))



    #FRESA.CAD can be used to create a bagged model using the forward selection formulas
    bagging <- baggedModel(BSWiMSModel$forward.selection.list,dataCancer,useFreq=32)
    pm <- predict(bagging$bagged.model)
    pm <- plotModels.ROC(cbind(dataCancer$pgstat,pm),main=("Bagged"))

    #Let's check the performance of the model 
    sm <- summary(bagging$bagged.model)
    print(sm$coefficients)

    #Using bootstrapping object I can check the Jaccard Index
    print(bagging$Jaccard.SM)

    #Ploting the evolution of the coefficient value
    plot(bagging$coefEvolution$grade,main="Evolution of grade")


    gplots::heatmap.2(bagging$formulaNetwork,trace="none",
                      mar=c(10,10),main="eB:SWIMS Formula Network")
    barplot(bagging$frequencyTable,las = 2,cex.axis=1.0,
            cex.names=0.75,main="Feature Frequency")
    n <- network::network(bagging$formulaNetwork, directed = FALSE,
                          ignore.eval = FALSE,names.eval = "weights")
    ggnet2(n, label = TRUE, size = "degree",size.cut = 3,size.min = 1, 
           mode = "circle",edge.label = "weights",edge.label.size=4)


    # Get a Cox proportional hazards model using:
    # - The default parameters

    mdCOXs <- FRESA.Model(formula = Surv(pgtime, pgstat) ~ 1,data = dataCancer)
    sm <- summary(mdCOXs$BSWiMS.model)
    print(sm$coefficients)

    # The model with singificant improvement in the residual error
    mdCOXs <- FRESA.Model(formula = Surv(pgtime, pgstat) ~ 1,
                          data = dataCancer,OptType = "Residual" )
    sm <- summary(mdCOXs$BSWiMS.model)
    print(sm$coefficients)

    # Get a Cox proportional hazards model using second order models:
    mdCOX <- FRESA.Model(formula = Surv(pgtime, pgstat) ~ 1,
                         data = dataCancer,categorizationType="RawRaw")
    sm <- summary(mdCOX$BSWiMS.model)
    print(sm$coefficients)

    namesc <- names(mdCOX$BSWiMS.model$coefficients)[-1]
    hm <- heatMaps(mdCOX$univariateAnalysis[namesc,],varRank=NULL,
                   Outcome="pgstat",data=dataCancer,
                   title="Heat Map",hCluster=FALSE,prediction=NULL,Scale=TRUE,
                   theFiveColors=c("blue","cyan","black","yellow","red"),
                   outcomeColors = c("blue","lightgreen","yellow","orangered","red"),
                   transpose=FALSE,cexRow=0.50,cexCol=0.80,srtCol=35)

    # The LOO estimation
    pm <- ensemblePredict(mdCOX$BSWiMS.models$formula.list,dataCancer,
                          predictType = "linear",type="LOGIT",Outcome="pgstat")
    pm <- plotModels.ROC(cbind(dataCancer$pgstat,pm$ensemblePredict),main=("LOO Median Predict"))
    #Let us check the diagnosis performance
    summary(epi.tests(pm$predictionTable))

    # Get a Logistic model using FRESA.Model
    # - The default parameters
    dataCancer2 <-dataCancer 
    dataCancer2$pgtime <-NULL
    mdLOGIT <- FRESA.Model(formula = pgstat ~ 1,data = dataCancer2)
    if (!is.null(mdLOGIT$bootstrappedModel)) pv <- plot(mdLOGIT$bootstrappedModel)
    sm <- summary(mdLOGIT$BSWiMS.model)
    print(sm$coefficients)


    ## FRESA.Model with Cross Validation and Recursive Partitioning and Regression Trees


    md <- FRESA.Model(formula = Surv(pgtime, pgstat) ~ 1,data = dataCancer,
                      CVfolds = 10,repeats = 5,equivalent = TRUE,usrFitFun=rpart::rpart)

    colnames(md$cvObject$Models.testPrediction)

    pm <- plotModels.ROC(md$cvObject$LASSO.testPredictions,theCVfolds=10,main="CV LASSO",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$KNN.testPrediction,theCVfolds=10,main="KNN",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=10,
                         predictor="Prediction",main="B:SWiMS Bagging",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=10,
                         predictor="Ensemble.B.SWiMS"
                         ,main="Forward Selection Median Ensemble",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=10,
                         predictor="Ensemble.Forward",main="Forward Selection Bagging",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=10,
                         predictor="eB.SWiMS",main="Equivalent Model",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=10,
                         predictor="Forward.Selection.Bagged",main="The Forward Bagging",cex=0.90)

    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=20,
                         predictor="usrFitFunction",
                         main="Recursive Partitioning and Regression Trees",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=20,
                         predictor="usrFitFunction_Sel",
                         main="Recursive Partitioning and Regression Trees with FS",cex=0.90)


    ## FRESA.Model with Cross Validation, LOGISTIC and Support Vector Machine


    md <- FRESA.Model(formula = pgstat ~ 1,data = dataCancer2,
                      CVfolds = 10,repeats = 5,equivalent = TRUE,usrFitFun=svm)

    pm <- plotModels.ROC(md$cvObject$LASSO.testPredictions,theCVfolds=10,main="CV LASSO",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$KNN.testPrediction,theCVfolds=10,main="KNN",cex=0.90)
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=10,
                         predictor="Prediction",main="B:SWiMS Bagging",cex=0.90)

    md$cvObject$Models.testPrediction[,"usrFitFunction"] <- 
                      md$cvObject$Models.testPrediction[,"usrFitFunction"] - 0.5
    md$cvObject$Models.testPrediction[,"usrFitFunction_Sel"] <- 
                      md$cvObject$Models.testPrediction[,"usrFitFunction_Sel"] - 0.5
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=10,
                         predictor="usrFitFunction",
                         main="SVM",cex = 0.90)
    pm <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds=10,
                         predictor="usrFitFunction_Sel",
                         main="SVM with FS",cex = 0.90)


    # Shut down the graphics device driver
    dev.off()

   
## End(Not run)

Description

This function removes model terms that do not significantly affect the integrated discrimination improvement (IDI) or the net reclassification improvement (NRI) of the model.

Usage

backVarElimination_Bin(object,
	                   pvalue = 0.05,
	                   Outcome = "Class",
	                   data,
	                   startOffset = 0,
	                   type = c("LOGIT", "LM", "COX"),
	                   selectionType = c("zIDI", "zNRI")
					   )

Arguments

Details

For each model term $x_i$, the IDI or NRI is computed for the Full model and the reduced model( where the term $x_i$ removed). The term whose removal results in the smallest drop in improvement is selected. The hypothesis: the term adds classification improvement is tested by checking the pvalue of improvement. If $p(IDI or NRI)>pvalue$, then the term is removed. In other words, only model terms that significantly aid in subject classification are kept. The procedure is repeated until no term fulfils the removal criterion.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in medicine 27(2), 157-172.

See Also

backVarElimination_Res, bootstrapVarElimination_Bin, bootstrapVarElimination_Res

Description

This function removes model terms that do not significantly improve the "net residual" (NeRI)

Usage

backVarElimination_Res(object,
	                       pvalue = 0.05,
	                       Outcome = "Class",
	                       data,
	                       startOffset = 0, 
	                       type = c("LOGIT", "LM", "COX"),
	                       testType = c("Binomial", "Wilcox", "tStudent", "Ftest"),
	                       setIntersect = 1
						   )

Arguments

Details

For each model term $x_i$, the residuals are computed for the Full model and the reduced model( where the term $x_i$ removed). The term whose removal results in the smallest drop in residuals improvement is selected. The hypothesis: the term improves residuals is tested by checking the pvalue of improvement. If $p(residuals better than reduced residuals)>pvalue$, then the term is removed. In other words, only model terms that significantly aid in improving residuals are kept. The procedure is repeated until no term fulfils the removal criterion. The p-values of improvement can be computed via a sign-test (Binomial) a paired Wilcoxon test, paired t-test or f-test. The first three tests compare the absolute values of the residuals, while the f-test test if the variance of the residuals is improved significantly.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

See Also

backVarElimination_Bin, bootstrapVarElimination_Bin bootstrapVarElimination_Res

Description

This function will take the frequency-ranked of variables and the list of models to create a single bagged model

Usage

baggedModel(modelFormulas,
	            data,
	            type=c("LM","LOGIT","COX"),
	            Outcome=NULL,
	            timeOutcome=NULL,
	            frequencyThreshold=0.025,
	            univariate=NULL,
				useFreq=TRUE,
				n_bootstrap=1,
				equifreqCorrection=0
	            )
	baggedModelS(modelFormulas,
                 data,
                 type=c("LM","LOGIT","COX"),
                 Outcome=NULL,
                 timeOutcome=NULL)

Arguments

Value

Author(s)

Jose G. Tamez-Pena

See Also

ensemblePredict

Description

Ranked Plot a set of measurements with error bars or confidence intervals (CI)

Usage

barPlotCiError(ciTable, 
               metricname, 
			   thesets, 
			   themethod, 
			   main, 
			   angle = 0, 
			   offsets = c(0.1,0.1),
               scoreDirection = ">",
               ho=NULL,		   
			   ...)

Arguments

Value

Author(s)

Jose G. Tamez-Pena

Description

Evaluates a data set with a set of fitting/filtering methods and returns the observed cross-validation performance

Usage

BinaryBenchmark(theData = NULL, theOutcome = "Class", reps = 100, trainFraction = 0.5,
                referenceCV = NULL,referenceName = "Reference"
                ,referenceFilterName="Reference")
RegresionBenchmark(theData = NULL, theOutcome = "Class", reps = 100, trainFraction = 0.5,
                referenceCV = NULL,referenceName = "Reference"
                ,referenceFilterName="Reference")
OrdinalBenchmark(theData = NULL, theOutcome = "Class", reps = 100, trainFraction = 0.5,
                referenceCV = NULL,referenceName = "Reference"
                ,referenceFilterName="Reference")

CoxBenchmark(theData = NULL, theOutcome = "Class", reps = 100, trainFraction = 0.5,
                referenceCV = NULL,referenceName = "Reference"
                ,referenceFilterName="COX.BSWiMS")

Arguments

Details

The benchmark functions provide the performance of different classification algorithms (BinaryBenchmark), registration algorithms (RegresionBenchmark) or ordinal regression algorithms (OrdinalBenchmark) The evaluation method is based on applying the random cross-validation method (randomCV) that randomly splits the data into train and test sets. The user can provide a Cross validated object that will define the train-test partitions.

The BinaryBenchmark compares: BSWiMS,Random Forest ,RPART,LASSO,SVM/mRMR,KNN and the ensemble of them in their ability to correctly classify the test data. Furthermore, it evaluates the ability of the following feature selection algorithms: BSWiMS or ReferenceCV, LASSO, RPART, RF/BSWiMS, IDI, NRI, t-test, Wilcoxon, Kendall, and mRMR in their ability to select the best set of features for the following classification methods: SVM, KNN, Naive Bayes, Random Forest Nearest Centroid (NC) with root sum square (RSS) , and NC with Spearman correlation

The RegresionBenchmark compares: BSWiMS,Random Forest ,RPART,LASSO,SVM/mRMR and the ensemble of them in their ability to correctly predict the test data. Furthermore, it evaluates the ability of the following feature selection algorithms: BSWiMS or referenceCV, LASSO, RPART, RF/BSWiMS, F-Test, W-Test, Pearson Kendall, and mRMR in their ability to select the best set of features for the following regression methods: Linear Regression, Robust Regression, Ridge Regression, LASSO, SVM, and Random Forest.

The OrdinalBenchmark compares: BSWiMS,Random Forest ,RPART,LASSO,KNN ,SVM and the ensemble of them in their ability to correctly predict the test data. Furthermore, it evaluates the ability of the following feature selection algorithms: BSWiMS or referenceCV, LASSO, RPART, RF/BSWiMS, F-Test, Kendall, and mRMR in their ability to select the best set of features for the following regression methods: Ordinal, KNN, SVM, Random Forest, and Naive Bayes.

The CoxBenchmark compares: BSWiMS, LASSO, BeSS and Univariate Cox analysis in their ability to correctly predict the risk of event happening. It uses cox regression with the four alternatives, but BSWiMS, LASSO are also compared as Wrapper methods.

Value

cpuElapsedTimes

Author(s)

Jose G. Tamez-Pena

See Also

randomCV

Examples

## Not run: 

		### Binary Classification Example ####
		# Start the graphics device driver to save all plots in a pdf format
		pdf(file = "BinaryClassificationExample.pdf",width = 8, height = 6)
		# Get the stage C prostate cancer data from the rpart package

		data(stagec,package = "rpart")

		# Prepare the data. Create a model matrix without the event time
		stagec$pgtime <- NULL
		stagec$eet <- as.factor(stagec$eet)
		options(na.action = 'na.pass')
		stagec_mat <- cbind(pgstat = stagec$pgstat,
		as.data.frame(model.matrix(pgstat ~ .,stagec))[-1])

		# Impute the missing data
        dataCancerImputed <- nearestNeighborImpute(stagec_mat)
        dataCancerImputed[,1:ncol(dataCancerImputed)] <- sapply(dataCancerImputed,as.numeric)	

		# Cross validating a LDA classifier.
		# 80
		cv <- randomCV(dataCancerImputed,"pgstat",MASS::lda,trainFraction = 0.8, 
		repetitions = 10,featureSelectionFunction = univariate_tstudent,
		featureSelection.control = list(limit = 0.5,thr = 0.975));

		# Compare the LDA classifier with other methods
		cp <- BinaryBenchmark(referenceCV = cv,referenceName = "LDA",
		                      referenceFilterName="t.Student")
		pl <- plot(cp,prefix = "StageC: ")

		# Default Benchmark classifiers method (BSWiMS) and filter methods. 
		# 80
		cp <- BinaryBenchmark(theData = dataCancerImputed,
		theOutcome = "pgstat", reps = 10, fraction = 0.8)

		# plot the Cross Validation Metrics
		pl <- plot(cp,prefix = "Stagec:");

		# Shut down the graphics device driver
		dev.off()

		#### Regression Example ######
		# Start the graphics device driver to save all plots in a pdf format
		pdf(file = "RegressionExample.pdf",width=8, height=6)

		# Get the body fat data from the TH package

		data("bodyfat", package = "TH.data")

		# Benchmark regression methods and filter methods. 
		#80
		cp <- RegresionBenchmark(theData = bodyfat, 
		theOutcome = "DEXfat", reps = 10, fraction = 0.8)

		# plot the Cross Validation Metrics
		pl <- plot(cp,prefix = "Body Fat:");
		# Shut down the graphics device driver
		dev.off()

		#### Ordinal Regression Example #####
		# Start the graphics device driver to save all plots in a pdf format
		pdf(file = "OrdinalRegressionExample.pdf",width=8, height=6)


		# Get the GBSG2 data
		data("GBSG2", package = "TH.data")

		# Prepare the model frame for benchmarking
		GBSG2$time <- NULL;
		GBSG2$cens <- NULL;
		GBSG2_mat <- cbind(tgrade = as.numeric(GBSG2$tgrade),
		as.data.frame(model.matrix(tgrade~.,GBSG2))[-1])

		# Benchmark regression methods and filter methods. 
		#30
		cp <- OrdinalBenchmark(theData = GBSG2_mat, 
		theOutcome = "tgrade", reps = 10, fraction = 0.3)

		# plot the Cross Validation Metrics
		pl <- plot(cp,prefix = "GBSG:");

		# Shut down the graphics device driver
		dev.off()

	
## End(Not run)

Description

Fits a BeSS::bess object to the data, and return the selected features

Usage

BESS(formula = formula, data=NULL, method="sequential", ic.type="BIC",...)
    BESS_GSECTION(formula = formula, data=NULL, method="gsection", ic.type="NULL",...)
    BESS_EBIC(formula = formula, data=NULL, ic.type="EBIC",...)

Arguments

Value

Author(s)

Jorge Orozco

See Also

BeSS::bess

Description

This function bootstraps the model n times to estimate for each variable the empirical distribution of model coefficients, area under ROC curve (AUC), integrated discrimination improvement (IDI) and net reclassification improvement (NRI). At each bootstrap the non-observed data is predicted by the trained model, and statistics of the test prediction are stored and reported. The method keeps track of predictions and plots the bootstrap-validated ROC. It may plots the blind test accuracy, sensitivity, and specificity, contrasted with the bootstrapped trained distributions.

Usage

bootstrapValidation_Bin(fraction = 1,
	                    loops = 200,
	                    model.formula,
	                    Outcome,
	                    data,
	                    type = c("LM", "LOGIT", "COX"),
	                    plots = FALSE,
						best.model.formula=NULL)

Arguments

Details

The bootstrap validation will estimate the confidence interval of the model coefficients and the NRI and IDI. The non-sampled values will be used to estimate the blind accuracy, sensitivity, and specificity. A plot to monitor the evolution of the bootstrap procedure will be displayed if plots is set to TRUE. The plot shows the train and blind test ROC. The density distribution of the train accuracy, sensitivity, and specificity are also shown, with the blind test results drawn along the y-axis.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

See Also

bootstrapValidation_Res, plot.bootstrapValidation_Bin, summary.bootstrapValidation_Bin

Description

This function bootstraps the model n times to estimate for each variable the empirical bootstrapped distribution of model coefficients, and net residual improvement (NeRI). At each bootstrap the non-observed data is predicted by the trained model, and statistics of the test prediction are stores and reported.

Usage

bootstrapValidation_Res(fraction = 1,
	                        loops = 200,
	                        model.formula,
	                        Outcome,
	                        data,
	                        type = c("LM", "LOGIT", "COX"),
	                        plots = FALSE,
							bestmodel.formula=NULL)

Arguments

Details

The bootstrap validation will estimate the confidence interval of the model coefficients and the NeRI. It will also compute the train and blind test root-mean-square error (RMSE), as well as the distribution of the NeRI p-values.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

See Also

bootstrapValidation_Bin, plot.bootstrapValidation_Res

Description

This function removes model terms that do not improve the bootstrapped integrated discrimination improvement (IDI) or net reclassification improvement (NRI) significantly.

Usage

bootstrapVarElimination_Bin(object,
	                        pvalue = 0.05,
	                        Outcome = "Class",
	                        data,
	                        startOffset = 0, 
	                        type = c("LOGIT", "LM", "COX"),
	                        selectionType = c("zIDI", "zNRI"),
	                        loops = 64,
	                        print=TRUE,
	                        plots=TRUE
	                        )

Arguments

Details

For each model term $x_i$, the IDI or NRI is computed for the Full model and the reduced model( where the term $x_i$ removed). The term whose removal results in the smallest drop in bootstrapped improvement is selected. The hypothesis: the term adds classification improvement is tested by checking the p value of average improvement. If $p(IDI or NRI)>pvalue$, then the term is removed. In other words, only model terms that significantly aid in subject classification are kept. The procedure is repeated until no term fulfils the removal criterion.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in medicine 27(2), 157-172.

See Also

bootstrapVarElimination_Res, backVarElimination_Bin, backVarElimination_Res

Description

This function removes model terms that do not improve the bootstrapped net residual improvement (NeRI) significantly.

Usage

bootstrapVarElimination_Res(object,
	                            pvalue = 0.05,
	                            Outcome = "Class",
	                            data,
	                            startOffset = 0, 
	                            type = c("LOGIT", "LM", "COX"),
	                            testType = c("Binomial",
	                                         "Wilcox",
	                                         "tStudent",
	                                         "Ftest"),
	                            loops = 64,
	                            setIntersect = 1,
	                            print=TRUE,
	                            plots=TRUE
                                )

Arguments

Details

For each model term $x_i$, the residuals are computed for the Full model and the reduced model( where the term $x_i$ removed). The term whose removal results in the smallest drop in bootstrapped test residuals improvement is selected. The hypothesis: the term improves residuals is tested by checking the p-value of average improvement. If $p(residuals better than reduced residuals)>pvalue$, then the term is removed. In other words, only model terms that significantly aid in improving residuals are kept. The procedure is repeated until no term fulfils the removal criterion. The p-values of improvement can be computed via a sign-test (Binomial) a paired Wilcoxon test, paired t-test or f-test. The first three tests compare the absolute values of the residuals, while the f-test test if the variance of the residuals is improved significantly.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

See Also

bootstrapVarElimination_Bin, backVarElimination_Res, bootstrapValidation_Res

Description

This function returns a set of models that best predict the outcome. Based on a Bootstrap Stage Wise Model Selection algorithm.

Usage

BSWiMS.model(formula,
	            data,
	            type = c("Auto","LM","LOGIT","COX"),
	            testType = c("Auto","zIDI",
	                         "zNRI",
	                         "Binomial",
	                         "Wilcox",
	                         "tStudent",
	                         "Ftest"),
	            pvalue=0.05,
	            variableList=NULL,
	            size=0,
	            loops=20,
	            elimination.bootstrap.steps = 200,
	            fraction=1.0,
	            maxTrainModelSize=20,
	            maxCycles=20,
	            print=FALSE,
	            plots=FALSE,
	            featureSize=0,
	            NumberofRepeats=1,
	            bagPredictType=c("Bag","wNN","Ens")
	            )

Arguments

Details

This is a core function of FRESA.CAD. The function will generate a set of B:SWiMS models from the data based on the provided baseline formula. The function will loop extracting a models whose all terms are statistical significant. After each loop it will remove the significant terms, and it will repeat the model generation until no mode significant models are found or the maximum number of cycles is reached.

Value

Author(s)

Jose G. Tamez-Pena

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in medicine 27(2), 157-172.

Examples

## Not run: 

		# Start the graphics device driver to save all plots in a pdf format
		pdf(file = "BSWiMS.model.Example.pdf",width = 8, height = 6)

		# Get the stage C prostate cancer data from the rpart package
		data(stagec,package = "rpart")
		options(na.action = 'na.pass')
		stagec_mat <- cbind(pgstat = stagec$pgstat,
             pgtime = stagec$pgtime,
             as.data.frame(model.matrix(Surv(pgtime,pgstat) ~ .*.,stagec))[-1])
		fnames <- colnames(stagec_mat)
		fnames <- str_replace_all(fnames,":","__")
		colnames(stagec_mat) <- fnames

		dataCancerImputed <- nearestNeighborImpute(stagec_mat)

		# Get a Cox proportional hazards model using:
		# - The default parameters
		md <- BSWiMS.model(formula = Surv(pgtime, pgstat) ~ 1,
						  data = dataCancerImputed)

		#Plot the bootstrap validation
		pt <- plot(md$BSWiMS.model$bootCV)

		#Get the coefficients summary
		sm <- summary(md)
		print(sm$coefficients)

		#Plot the bagged model 
		pl <- plotModels.ROC(cbind(dataCancerImputed$pgstat,
							  predict(md,dataCancerImputed)),
							 main = "Bagging Predictions")


		# Get a Cox proportional hazards model using:
		# - The default parameters but repeated 10 times
		md <- BSWiMS.model(formula = Surv(pgtime, pgstat) ~ 1,
						   data = dataCancerImputed,
						   NumberofRepeats = 10)

		#Get the coefficients summary
		sm <- summary(md)
		print(sm$coefficients)

		#Check all the formulas
		print(md$formula.list)

		#Plot the bagged model 
		pl <- plotModels.ROC(cbind(dataCancerImputed$pgstat,
								   predict(md,dataCancerImputed)),
							 main = "Bagging Predictions")


		# Get a  regression of the survival time

		timeSubjects <- dataCancerImputed
		timeSubjects$pgtime <- log(timeSubjects$pgtime)

		md <- BSWiMS.model(formula = pgtime ~ 1,
						  data = timeSubjects,
						  )
		pt <- plot(md$BSWiMS.model$bootCV)
		sm <- summary(md)
		print(sm$coefficients)

		# Get a logistic regression model using
		# - The default parameters and removing time as possible predictor
		data(stagec,package = "rpart")
		stagec$pgtime <- NULL
		stagec_mat <- cbind(pgstat = stagec$pgstat,
                     as.data.frame(model.matrix(pgstat ~ .*.,stagec))[-1])
		fnames <- colnames(stagec_mat)
		fnames <- str_replace_all(fnames,":","__")
		colnames(stagec_mat) <- fnames
		dataCancerImputed <- nearestNeighborImpute(stagec_mat)


		md <- BSWiMS.model(formula = pgstat ~ 1,
						  data = dataCancerImputed)

		pt <- plot(md$BSWiMS.model$bootCV)
		sm <- summary(md)
		print(sm$coefficients)


		# Get a ordinal regression of grade model using GBSG2 data
		# - The default parameters and removing the 
		# time and status as possible predictor

		data("GBSG2", package = "TH.data")

		# Prepare the model frame for prediction
		GBSG2$time <- NULL;
		GBSG2$cens <- NULL;
		GBSG2_mat <- cbind(tgrade = as.numeric(GBSG2$tgrade),
                       as.data.frame(model.matrix(tgrade~.*.,GBSG2))[-1])

		fnames <- colnames(GBSG2_mat)
		fnames <- str_replace_all(fnames,":","__")
		colnames(GBSG2_mat) <- fnames

		md <- BSWiMS.model(formula = tgrade ~ 1,
						   data = GBSG2_mat)

		sm <- summary(md$oridinalModels$theBaggedModels[[1]]$bagged.model)
		print(sm$coefficients)
		sm <- summary(md$oridinalModels$theBaggedModels[[2]]$bagged.model)
		print(sm$coefficients)

		print(table(GBSG2_mat$tgrade,predict(md,GBSG2_mat)))

		# Shut down the graphics device driver
		dev.off()

	
## End(Not run)

Description

The predicted binary probabilities are calibrated to match the observed event rate. A logistic model is used to calibrate the predicted probability to the actual event rate.

Usage

calBinProb(BinaryOutcome=NULL, 
	            OutcomeProbability=NULL
				)

Arguments

Value

The logistic model calibrated to the observed outcome rate

Author(s)

Jose G. Tamez-Pena

Description

It will estimate the baseline hazard (ho) and the time interval that best describes a estimations of the probabilities of time-to-event Poisson events

Usage

CalibrationProbPoissonRisk(Riskdata,trim=0.10)
	CoxRiskCalibration(ml,data,outcome,time,trim=0.10,timeInterval=NULL)

Arguments

Details

The function will estimate the baseline hazard of Poisson events and its corresponding time interval from a list of predicted probability that the event will occur for censored (Outome=0) of the actual event happened (Outcome=1). If the timeInterval is not provided, the funtion will estimete the initial time interval to be used to get the best time interval that models the rate of events.

Value

Author(s)

Jose G. Tamez-Pena

See Also

RRPlot

Examples

#TBD

Description

This data frame contains two columns, one with names of variables, and the other with descriptions of such variables. It is used in several examples of this package. Specifically, it is used in examples working with the stage C prostate cancer data from the rpart package

Usage

data(cancerVarNames)

Format

A data frame with names and descriptions of the variables used in several examples

Var

A column with the names of the variables

Description

A column with a short description of the variables

Examples

data(cancerVarNames)

Description

This function returns the outcome associated features and the supervised-classifier present at each one of the unsupervised data clusters

Usage

ClustClass(formula = formula,
	            data=NULL,
	            filtermethod=univariate_KS,
	            clustermethod=GMVECluster,
	            classmethod=LASSO_1SE,
	            filtermethod.control=list(pvalue=0.1,limit=21),
	            clustermethod.control= list(p.threshold = 0.95, 
	                                   p.samplingthreshold = 0.5),
	            classmethod.control=list(family = "binomial"),
	            pca=TRUE,
	            normalize=TRUE
	            )

Arguments

Details

This function will first call the filter function that should return the relevant a named vector with the p-value of the features associated with the outcome. Then it will call user-supplied clustering algorithm that must return a relevant data partition based on the discovered features. The returned object of the clustering function must contain a $classification object indicates the class label of each data point. Finally, the function will call the classification function on each cluster returned by the clustering function.

Value

Author(s)

Jose G. Tamez-Pena

Examples

## Not run: 	
      library(mlbench) # Location of the Sonar data set
	  library(mclust) # The cluster library
      data(Sonar)
      Sonar$Class <- 1*(Sonar$Class == "M")
	  #Train hierachical classifier
      mc <- ClustClass(Class~.,Sonar,clustermethod=Mclust,clustermethod.control=list(G = 1:4))
	  #report the classification
      pb <- predict(mc,Sonar)
      print(table(1*(pb>0.0),Sonar$Class))
	
## End(Not run)

Description

Returns the set of Gaussian Ellipsoids that best model the data

Usage

clusterISODATA(dataset,
                 clusteringMethod=GMVECluster,
                 trainFraction=0.99,
                 randomTests=10,
                 jaccardThreshold=0.45,
                 isoDataThreshold=0.75,
                 plot=TRUE,
                 ...)

Arguments

Details

The data will be clustered N times as defined by a number of randomTests. After clustering, the Jaccard Index map will be generated and ordered from high to low. The mean clusters parameters (Covariance sets) associated with the point with the highest Jaccard index will define the first cluster. A cluster will be added if the Mahalanobis distance between clusters is greater than the given acceptance p.value (isoDataThreshold) Only clusters associated with points with a Jaccard index greater than jaccardThreshold will be considered.

Value

Author(s)

Jose G. Tamez-Pena

Description

This function performs a cross-validation analysis of a feature selection algorithm based on the integrated discrimination improvement (IDI) or the net reclassification improvement (NRI) to return a predictive model. It is composed of an IDI/NRI-based feature selection followed by an update procedure, ending with a bootstrapping backwards feature elimination. The user can control how many train and blind test sets will be evaluated.

Usage

crossValidationFeatureSelection_Bin(size = 10,
	                                fraction = 1.0,
	                                pvalue = 0.05,
	                                loops = 100,
	                                covariates = "1",
	                                Outcome,
	                                timeOutcome = "Time",
	                                variableList,
	                                data,
	                                maxTrainModelSize = 20,
	                                type = c("LM", "LOGIT", "COX"),
	                                selectionType = c("zIDI", "zNRI"),
	                                startOffset = 0,
	                                elimination.bootstrap.steps = 100,
	                                trainFraction = 0.67,
	                                trainRepetition = 9,
	                                bootstrap.steps = 100,
	                                nk = 0,
	                                unirank = NULL,
	                                print=TRUE,
	                                plots=TRUE,
	                                lambda="lambda.1se",
	                                equivalent=FALSE,
	                                bswimsCycles=10,
	                                usrFitFun=NULL,
	                                featureSize=0)

Arguments

Details

This function produces a set of data and plots that can be used to inspect the degree of over-fitting or shrinkage of a model. It uses bootstrapped data, cross-validation data, and, if possible, retrain data. During each cycle, a train and a test ROC will be generated using bootstrapped data. At the end of the cross-validation feature selection procedure, a set of three plots may be produced depending on the specifications of the analysis. The first plot shows the ROC for each cross-validation blind test. The second plot, if enough samples are given, shows the ROC of each model trained and tested in the blind test partition. The final plot shows ROC curves generated with the train, the bootstrapped blind test, and the cross-validation test data. Additionally, this plot will also contain the ROC of the cross-validation mean test data, and of the cross-validation coherence. These set of plots may be used to get an overall perspective of the expected model shrinkage. Along with the plots, the function provides the overall performance of the system (accuracy, sensitivity, and specificity). The function also produces a report of the expected performance of a KNN algorithm trained with the selected features of the model, and an elastic net algorithm. The test predictions obtained with these algorithms can then be compared to the predictions generated by the logistic, linear, or Cox proportional hazards regression model.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in medicine 27(2), 157-172.

See Also

crossValidationFeatureSelection_Res, ForwardSelection.Model.Bin, ForwardSelection.Model.Res

Description

This function performs a cross-validation analysis of a feature selection algorithm based on net residual improvement (NeRI) to return a predictive model. It is composed of a NeRI-based feature selection followed by an update procedure, ending with a bootstrapping backwards feature elimination. The user can control how many train and blind test sets will be evaluated.

Usage

crossValidationFeatureSelection_Res(size = 10,
	                                    fraction = 1.0,
	                                    pvalue = 0.05,
	                                    loops = 100,
	                                    covariates = "1",
	                                    Outcome,
	                                    timeOutcome = "Time",
	                                    variableList,
	                                    data,
	                                    maxTrainModelSize = 20,
	                                    type = c("LM", "LOGIT", "COX"),
	                                    testType = c("Binomial",
	                                                 "Wilcox",
	                                                 "tStudent",
	                                                 "Ftest"),
	                                    startOffset = 0,
	                                    elimination.bootstrap.steps = 100,
	                                    trainFraction = 0.67,
	                                    trainRepetition = 9,
	                                    setIntersect = 1,
	                                    unirank = NULL,
	                                    print=TRUE,
	                                    plots=TRUE,
	                                    lambda="lambda.1se",
	                                    equivalent=FALSE,
	                                    bswimsCycles=10,
	                                    usrFitFun=NULL,
	                                    featureSize=0)

Arguments

Details

This function produces a set of data and plots that can be used to inspect the degree of over-fitting or shrinkage of a model. It uses bootstrapped data, cross-validation data, and, if possible, retrain data.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

See Also

crossValidationFeatureSelection_Bin, improvedResiduals, bootstrapVarElimination_Res

Description

A formula based wrapper of the getSignature function

Usage

CVsignature(formula = formula,data=NULL,...)

Arguments

Value

Author(s)

Jose G. Tamez-Pena

See Also

getSignature,signatureDistance

Description

Permutations or Bootstrapping computation of the standardized log-rank (SLR) or the Chi=SLR^2 p-values for differences in survival times

Usage

EmpiricalSurvDiff(times=times,
	                  status=status,
	                  groups=groups,
	                  samples=1000,
	                  type=c("SLR","Chi"),
	                  plots=FALSE,
	                  minAproxSamples=100,
	                  computeDist=FALSE,
	                  ...
	                  )

Arguments

Details

It will compute the null distribution of the SRL or the square SLR (Chi) via permutations, and it will return the p-value of differences between survival times between two groups. It may also be used to compute the empirical distribution of the difference in SLR using bootstrapping. (computeDist=TRUE) The p-values will be estimated based on the sampled distribution, or normal-approximated along the tails.

Value

Author(s)

Jose G. Tamez-Pena

Examples

## Not run: 

      library(rpart)
      data(stagec)

      # The Log-Rank Analysis using survdiff

      lrsurvdiff <- survdiff(Surv(pgtime,pgstat)~grade>2,data=stagec)
      print(lrsurvdiff)

      # The Log-Rank Analysis: permutations of the null Chi distribution
      lrp <- EmpiricalSurvDiff(stagec$pgtime,stagec$pgstat,stagec$grade>2,
                         type="Chi",plots=TRUE,samples=10000,
                         main="Chi Null Distribution")
      print(list(unlist(c(lrp$LR,lrp$pvalue))))

      # The Log-Rank Analysis: permutations of the null SLR distribution
      lrp <- EmpiricalSurvDiff(stagec$pgtime,stagec$pgstat,stagec$grade>2,
                         type="SLR",plots=TRUE,samples=10000,
                         main="SLR Null Distribution")
      print(list(unlist(c(lrp$LR,lrp$pvalue))))

      # The Log-Rank Analysis: Bootstraping the SLR distribution
      lrp <- EmpiricalSurvDiff(stagec$pgtime,stagec$pgstat,stagec$grade>2,
                         computeDist=TRUE,plots=TRUE,samples=100000,
                         main="SLR Null and SLR bootrapped")
      print(list(unlist(c(lrp$LR,lrp$pvalue))))
	
	
## End(Not run)

Description

Given a list of model formulas, this function will train such models and return the a single(ensemble) prediction from the list of formulas on a test data set. It may also provides a k-nearest neighbors (KNN) prediction using the features listed in such models.

Usage

ensemblePredict(formulaList,
	              trainData,
	              testData = NULL, 
	              predictType = c("prob", "linear"),
	              type = c("LOGIT", "LM", "COX","SVM"),
	              Outcome = NULL,
	              nk = 0
				  )

Arguments

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

Description

This function fits the candidate variables to the provided model formula,for each strata, on a control population. If the variance of the residual (the fitted observation minus the real observation) is reduced significantly, then, such residual is used in the resulting data frame. Otherwise, the control mean is subtracted to the observation.

Usage

featureAdjustment(variableList,
	                  baseFormula,
	                  strata = NA,
	                  data,
	                  referenceframe,
	                  type = c("LM", "GLS", "RLM","NZLM","SPLINE","MARS","LOESS"),
	                  pvalue = 0.05,
	                  correlationGroup = "ID",
	                  ...
	                  )

Arguments

Value

A data frame, where each input observation has been adjusted from data at each strata

Note

This function prints the residuals and the F-statistic for all candidate variables

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

Description

Sequential application of feature selection, linear transformation, data scaling then fit

Usage

filteredFit(formula = formula, 
	            data=NULL,
	            filtermethod=univariate_KS,
	            filtermethod.control=list(limit=0),
				Transf=c("none","PCA","CCA","ILAA"),
				Transf.control=list(thr=0.8),
	            Scale="none",
				Scale.control=list(strata=NA),
				refNormIDs=NULL,
				trainIDs=NULL,
	            fitmethod=e1071::svm,
	            ...
	            )

Arguments

Value

Author(s)

Jose G. Tamez-Pena

Description

Returns the top set of features that are statistically associated with the outcome.

Usage

univariate_Logit(data=NULL, Outcome=NULL, pvalue=0.2, adjustMethod="BH", 
                 uniTest=c("zIDI","zNRI"),limit=0,...,n=0)
univariate_residual(data=NULL, Outcome=NULL, pvalue=0.2, adjustMethod="BH",
                    uniTest=c("Ftest","Binomial","Wilcox","tStudent"),
                    type=c("LM","LOGIT"),limit=0,...,n=0)
univariate_tstudent(data=NULL, Outcome=NULL, pvalue=0.2, adjustMethod="BH",
                    limit=0,...,n=0)
univariate_Wilcoxon(data=NULL, Outcome=NULL, pvalue=0.2, adjustMethod="BH",
                     limit=0,...,n=0)
univariate_KS(data=NULL, Outcome=NULL, pvalue=0.2, adjustMethod="BH",
                     limit=0,...,n=0)
univariate_DTS(data=NULL, Outcome=NULL, pvalue=0.2, adjustMethod="BH",
                     limit=0,...,n=0)
univariate_correlation(data=NULL, Outcome=NULL, pvalue=0.2, adjustMethod="BH",
                       method = "kendall",limit=0,...,n=0)
univariate_cox(data=NULL, Outcome=NULL, pvalue=0.2, adjustMethod="BH",
                     limit=0,...,n=0)
univariate_BinEnsemble(data,Outcome, pvalue=0.2,limit=0,adjustMethod="BH",...)
univariate_Strata(data,Outcome,pvalue=0.2,limit=0,
                   adjustMethod="BH",
                   unifilter=univariate_BinEnsemble,strata="Gender",...)
correlated_Remove(data=NULL,fnames=NULL,thr=0.999,isDataCorMatrix=FALSE)

Arguments

Value

Named vector with the adjusted p-values or the list of no-correlated features for the correlated_Remove

Author(s)

Jose G. Tamez-Pena

Examples

## Not run: 

        library("FRESA.CAD")

        ### Univariate Filter Examples ####

        # Get the stage C prostate cancer data from the rpart package
        data(stagec,package = "rpart")

        # Prepare the data. Create a model matrix without the event time and interactions
        stagec$pgtime <- NULL
        stagec$eet <- as.factor(stagec$eet)
        options(na.action = 'na.pass')
        stagec_mat <- cbind(pgstat = stagec$pgstat,
                            as.data.frame(model.matrix(pgstat ~ .*.,stagec))[-1])
        fnames <- colnames(stagec_mat)
        fnames <- str_replace_all(fnames,":","__")
        colnames(stagec_mat) <- fnames

        # Impute the missing data
        dataCancerImputed <- nearestNeighborImpute(stagec_mat)
        dataCancerImputed[,1:ncol(dataCancerImputed)] <- sapply(dataCancerImputed,as.numeric)

        # Get the top Features associated to pgstat

        q_values <- univariate_Logit(data=dataCancerImputed, 
                                    Outcome="pgstat",
                                    pvalue = 0.05)

        qValueMatrix <- q_values
        idiqValueMatrix <- q_values
        barplot(-log(q_values),las=2,cex.names=0.4,ylab="-log(Q)",
        main="Association with PGStat: IDI Test")

        q_values <- univariate_Logit(data=dataCancerImputed, 
                                    Outcome="pgstat", 
                                    uniTest="zNRI",pvalue = 0.05)
        qValueMatrix <- cbind(idiqValueMatrix,q_values[names(idiqValueMatrix)])

        q_values <- univariate_residual(data=dataCancerImputed, 
                                    Outcome="pgstat", 
                                    pvalue = 0.05,type="LOGIT")
        qValueMatrix <- cbind(qValueMatrix,q_values[names(idiqValueMatrix)])

        q_values <- univariate_tstudent(data=dataCancerImputed, 
                                       Outcome="pgstat", 
                                       pvalue = 0.05)
        qValueMatrix <- cbind(qValueMatrix,q_values[names(idiqValueMatrix)])

        q_values <- univariate_Wilcoxon(data=dataCancerImputed, 
                                       Outcome="pgstat", 
                                       pvalue = 0.05)
        qValueMatrix <- cbind(qValueMatrix,q_values[names(idiqValueMatrix)])

        q_values <- univariate_correlation(data=dataCancerImputed, 
                                       Outcome="pgstat", 
                                       pvalue = 0.05)
        qValueMatrix <- cbind(qValueMatrix,q_values[names(idiqValueMatrix)])

        q_values <- univariate_correlation(data=dataCancerImputed, 
                                          Outcome="pgstat", 
                                          pvalue = 0.05,
                                          method = "pearson")

        #The qValueMatrix has the qValues of all filter methods.  
        qValueMatrix <- cbind(qValueMatrix,q_values[names(idiqValueMatrix)])
        colnames(qValueMatrix) <- c("IDI","NRI","F","t","W","K","P")
        #Do the log transform to display the heatmap
        qValueMatrix <- -log10(qValueMatrix)
        #the Heatmap of the q-values
        gplots::heatmap.2(qValueMatrix,Rowv = FALSE,dendrogram = "col",
        main = "Method q.values",cexRow = 0.4)

    
## End(Not run)

Description

This function performs a bootstrap sampling to rank the variables that statistically improve prediction. After the frequency rank, the function uses a forward selection procedure to create a final model, whose terms all have a significant contribution to the integrated discrimination improvement (IDI) or the net reclassification improvement (NRI). For each bootstrap, the IDI/NRI is computed and the variable with the largest statically significant IDI/NRI is added to the model. The procedure is repeated at each bootstrap until no more variables can be inserted. The variables that enter the model are then counted, and the same procedure is repeated for the rest of the bootstrap loops. The frequency of variable-inclusion in the model is returned as well as a model that uses the frequency of inclusion.

Usage

ForwardSelection.Model.Bin(size = 100,
	                            fraction = 1,
	                            pvalue = 0.05, 
	                            loops = 100,
	                            covariates = "1",
	                            Outcome,
	                            variableList,
	                            data, 
	                            maxTrainModelSize = 20,
	                            type = c("LM", "LOGIT", "COX"),
	                            timeOutcome = "Time",
	                            selectionType=c("zIDI", "zNRI"),
	                            cores = 6,
	                            randsize = 0,
	                            featureSize=0)

Arguments

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in medicine 27(2), 157-172.

See Also

ForwardSelection.Model.Res

Description

This function performs a bootstrap sampling to rank the most frequent variables that statistically aid the models by minimizing the residuals. After the frequency rank, the function uses a forward selection procedure to create a final model, whose terms all have a significant contribution to the net residual improvement (NeRI).

Usage

ForwardSelection.Model.Res(size = 100, 
	                     fraction = 1, 
	                     pvalue = 0.05, 
	                     loops = 100, 
	                     covariates = "1", 
	                     Outcome, 
	                     variableList, 
	                     data, 
	                     maxTrainModelSize = 20, 
	                     type = c("LM", "LOGIT", "COX"), 
	                     testType=c("Binomial", "Wilcox", "tStudent", "Ftest"),
	                     timeOutcome = "Time",
	                     cores = 6,
	                     randsize = 0,
	                     featureSize=0)

Arguments

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

See Also

ForwardSelection.Model.Bin

Description

This function uses a wrapper procedure to select the best features of a non-penalized linear model that best predict the outcome, given the formula of an initial model template (linear, logistic, or Cox proportional hazards), an optimization procedure, and a data frame. A filter scheme may be enabled to reduce the search space of the wrapper procedure. The false selection rate may be empirically controlled by enabling bootstrapping, and model shrinkage can be evaluated by cross-validation.

Usage

FRESA.Model(formula,
	            data,
	            OptType = c("Binary", "Residual"),
	            pvalue = 0.05,
	            filter.p.value = 0.10,
	            loops = 32,
	            maxTrainModelSize = 20,
	            elimination.bootstrap.steps = 100,
	            bootstrap.steps = 100,
	            print = FALSE,
	            plots = FALSE,
	            CVfolds = 1,
	            repeats = 1,
	            nk = 0,
	            categorizationType = c("Raw",
	                                   "Categorical",
	                                   "ZCategorical",
	                                   "RawZCategorical",
	                                   "RawTail",
	                                   "RawZTail",
	                                   "Tail",
	                                   "RawRaw"),
	            cateGroups = c(0.1, 0.9),
	            raw.dataFrame = NULL,
	            var.description = NULL,
	            testType = c("zIDI",
	                         "zNRI",
	                         "Binomial",
	                         "Wilcox",
	                         "tStudent",
	                         "Ftest"),
	            lambda="lambda.1se",
	            equivalent=FALSE,
	            bswimsCycles=20,
	            usrFitFun=NULL
	            )

Arguments

Details

This important function of FRESA.CAD will model or cross validate the models. Given an outcome formula, and a data.frame this function will do an univariate analysis of the data (univariateRankVariables), then it will select the top ranked variables; after that it will select the model that best describes the outcome. At output it will return the bootstrapped performance of the model (bootstrapValidation_Bin or bootstrapValidation_Res). It can be set to report the cross-validation performance of the selection process which will return either a crossValidationFeatureSelection_Bin or a crossValidationFeatureSelection_Res object.

Value

Author(s)

Jose G. Tamez-Pena and Antonio Martinez-Torteya

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in medicine 27(2), 157-172.

Examples

## Not run: 

		# Start the graphics device driver to save all plots in a pdf format
		pdf(file = "FRESA.Model.Example.pdf",width = 8, height = 6)
		# Get the stage C prostate cancer data from the rpart package
		data(stagec,package = "rpart")
		options(na.action = 'na.pass')
		stagec_mat <- cbind(pgstat = stagec$pgstat,
		    pgtime = stagec$pgtime,
		    as.data.frame(model.matrix(Surv(pgtime,pgstat) ~ .,stagec))[-1])

		data(cancerVarNames)
		dataCancerImputed <- nearestNeighborImpute(stagec_mat)

		# Get a Cox proportional hazards model using:
		# - The default parameters
		md <- FRESA.Model(formula = Surv(pgtime, pgstat) ~ 1,
						  data = dataCancerImputed,
						  var.description = cancerVarNames[,2])
		pt <- plot(md$bootstrappedModel)
		sm <- summary(md$BSWiMS.model)
		print(sm$coefficients)


		# Get a 10 fold CV Cox proportional hazards model using:
		# - Repeat 10 times de CV
		md <- FRESA.Model(formula = Surv(pgtime, pgstat) ~ 1,
						  data = dataCancerImputed, CVfolds = 10, 
						  repeats = 10,
						  var.description = cancerVarNames[,2])
		pt <- plotModels.ROC(md$cvObject$Models.testPrediction,theCVfolds = 10)
		print(pt$predictionTable)

		pt <- plotModels.ROC(md$cvObject$LASSO.testPredictions,theCVfolds = 10)
		pt <- plotModels.ROC(md$cvObject$KNN.testPrediction,theCVfolds = 10)

		# Get a  regression of the survival time

		timeSubjects <- dataCancerImputed
		timeSubjects$pgtime <- log(timeSubjects$pgtime)

		md <- FRESA.Model(formula = pgtime ~ 1,
						  data = timeSubjects,
						  var.description = cancerVarNames[,2])
		pt <- plot(md$bootstrappedModel)
		sm <- summary(md$BSWiMS.model)
		print(sm$coefficients)

		# Get a logistic regression model using
		# - The default parameters and removing time as possible predictor

		dataCancerImputed$pgtime <- NULL

		md <- FRESA.Model(formula = pgstat ~ 1,
						  data = dataCancerImputed,
						  var.description = cancerVarNames[,2])
		pt <- plot(md$bootstrappedModel)
		sm <- summary(md$BSWiMS.model)
		print(sm$coefficients)

		# Get a logistic regression model using:
		# - residual-based optimization
		md <- FRESA.Model(formula = pgstat ~ 1,
						  data = dataCancerImputed,
						  OptType = "Residual",
						  var.description = cancerVarNames[,2])
		pt <- plot(md$bootstrappedModel)
		sm <- summary(md$BSWiMS.model)
		print(sm$coefficients)


		# Shut down the graphics device driver
		dev.off()

	
## End(Not run)

Description

All features from the data will be normalized based on the distribution of the reference data-frame

Usage

FRESAScale(data,refFrame=NULL,method=c("Norm","Order",
	           "OrderLogit","RankInv","LRankInv"),
    refMean=NULL,refDisp=NULL,strata=NA)

Arguments