Package 'innsight'

Description

innsight is an R package that interprets the behavior and explains individual predictions of modern neural networks. Many methods for explaining individual predictions already exist, but hardly any of them are implemented or available in R. Most of these so-called feature attribution methods are only implemented in Python and, thus, difficult to access or use for the R community. In this sense, the package innsight provides a common interface for various methods for the interpretability of neural networks and can therefore be considered as an R analogue to 'iNNvestigate' or 'Captum' for Python.

Details

This package implements several model-specific interpretability (feature attribution) methods based on neural networks in R, e.g.,

• Layer-wise relevance propagation (LRP)

  • Including propagation rules: $\epsilon$-rule and $\alpha$-$\beta$-rule

• Deep learning important features (DeepLift)

  • Including propagation rules for non-linearities: Rescale rule and RevealCancel rule

• DeepSHAP

• Gradient-based methods:

  • Vanilla Gradient, including Gradient$\times$Input

  • Smoothed gradients (SmoothGrad), including SmoothGrad$\times$Input

  • Integrated gradients (IntegratedGradient)

  • Expected gradients (ExpectedGradient)

• ConnectionWeights

• Model-agnostic methods:

  • Local interpretable model-agnostic explanation (LIME)

  • Shapley values (SHAP)

The package innsight aims to be as flexible as possible and independent of a specific deep learning package in which the passed network has been learned. Basically, a neural network of the libraries torch::nn_sequential, keras::keras_model_sequential, keras::keras_model and neuralnet::neuralnet can be passed to the main building block Converter, which converts and stores the passed model as a torch model (ConvertedModel) with special insights needed for interpretation. It is also possible to pass an arbitrary net in form of a named list (see details in Converter).

The scientific background and implementation details of innsight are described in the paper "Interpreting Deep Neural Networks with the Package innsight" by Koenen & Wright (2024), published in the Journal of Statistical Software. For a detailed explanation of the methods and use cases, please refer to the publication (doi: doi:10.18637/jss.v111.i08).

Author(s)

Maintainer: Niklas Koenen [email protected] (ORCID)

Other contributors:

• Raphael Baudeu [email protected] [contributor]

References

Koenen, N., & Wright, M. N. (2024). Interpreting Deep Neural Networks with the Package innsight. Journal of Statistical Software, 111(8), 1-52. doi: doi:10.18637/jss.v111.i08

See Also

Useful links:

• https://bips-hb.github.io/innsight/

• https://github.com/bips-hb/innsight/

• Report bugs at https://github.com/bips-hb/innsight/issues/

Description

The S4 class innsight_ggplot2 visualizes the results in the form of a matrix, with the output nodes (and also the input layers) in the columns and the selected data points in the rows. With these basic generic indexing functions, the plots of individual rows and columns can be accessed, modified and the overall plot can be adjusted accordingly.

Usage

## S4 method for signature 'innsight_ggplot2'
x[i, j, ..., restyle = TRUE, drop = TRUE]

## S4 method for signature 'innsight_ggplot2'
x[[i, j, ..., restyle = TRUE]]

## S4 replacement method for signature 'innsight_ggplot2'
x[i, j, ...] <- value

## S4 replacement method for signature 'innsight_ggplot2'
x[[i, j, ...]] <- value

Arguments

Value

• ⁠[.innsight_ggplot2⁠: Selects only the plots from the i-th rows and j-th columns and returns them as a new instance of innsight_ggplot2. If restyle = TRUE the facet stripes and axis labels of the original plot are transferred to the subplot, otherwise they are returned as they are.

• ⁠[[.innsight_ggplot2⁠: Selects only the subplot in row i and column j and returns it as a ggplot2::ggplot object. If restyle = TRUE the facet stripes and axis labels of the original plot are transferred to the subplot, otherwise they are returned as they are.

• ⁠[<-.innsight_ggplot2⁠: Replaces the plots in the rows i and columns j with those from value and returns the modified instance of innsight_ggplot2.

• ⁠[[<-.innsight_ggplot2⁠: Replaces the plot from the i-th row and j-th column with the plot from value and returns the modified instance of innsight_ggplot2.

See Also

innsight_ggplot2, print.innsight_ggplot2, +.innsight_ggplot2

Description

The S4 class innsight_plotly visualizes the results as a matrix of plots based on plotly::plot_ly. The output nodes (and also input layers) are displayed in the columns and the selected data points in the rows. With these basic generic indexing functions, the plots of individual rows and columns can be accessed.

Usage

## S4 method for signature 'innsight_plotly'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'innsight_plotly'
x[[i, j, ..., drop]]

Arguments

Value

• ⁠[.innsight_plotly⁠: Selects the plots from the i-th rows and j-th columns and returns them as a new instance of innsight_plotly.

• ⁠[[.innisght_plotly⁠: Selects only the single plot in the i-th row and j-th column and returns it as a plotly object.

See Also

innsight_plotly, print.innsight_plotly, plot.innsight_plotly, show.innsight_plotly

Description

This generic add function allows to treat an instance of innsight_ggplot2 as an ordinary plot object of ggplot2. For example geoms, themes and scales can be added as usual (see ggplot2::+.gg for more information).

Note: If e1 represents a multiplot (i.e., e1@mulitplot = TRUE), e2 is added to each individual plot. If only specific plots need to be changed, the generic assignment function should be used (see innsight_ggplot2 for details).

Usage

## S4 method for signature 'innsight_ggplot2,ANY'
e1 + e2

Arguments

See Also

innsight_ggplot2, print.innsight_ggplot2, [.innsight_ggplot2, [[.innsight_ggplot2, [<-.innsight_ggplot2, [[<-.innsight_ggplot2

Description

This is a super class for all implemented model-agnostic interpretability methods and inherits from the InterpretingMethod class. Instead of just an object of the Converter class, any model can now be passed. In contrast to the other model-specific methods in this package, only the prediction function of the model is required, and not the internal details of the model. The following model-agnostic methods are available (all are wrapped by other packages):

• Shapley values (SHAP) based on fastshap::explain

• Local interpretable model-agnostic explanations (LIME) based on lime::lime

Super class

innsight::InterpretingMethod -> AgnosticWrapper

Public fields

Methods

Public methods

• AgnosticWrapper$new()

• AgnosticWrapper$clone()

Method new()

Create a new instance of the AgnosticWrapper R6 class.

Usage

AgnosticWrapper$new(
  model,
  data,
  data_ref,
  output_type = NULL,
  pred_fun = NULL,
  output_idx = NULL,
  output_label = NULL,
  channels_first = TRUE,
  input_dim = NULL,
  input_names = NULL,
  output_names = NULL
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

AgnosticWrapper$clone(deep = FALSE)

Arguments

Description

This class implements the Connection weights method investigated by Olden et al. (2004), which results in a relevance score for each input variable. The basic idea is to multiply all path weights for each possible connection between an input feature and the output node and then calculate the sum over them. Besides, it is originally a global interpretation method and independent of the input data. For a neural network with $3$ hidden layers with weight matrices $W_1$, $W_2$ and $W_3$, this method results in a simple matrix multiplication independent of the activation functions in between:

$W_1 * W_2 * W_3.$

In this package, we extended this method to a local method inspired by the method Gradient$\times$Input (see Gradient). Hence, the local variant is simply the point-wise product of the global Connection weights method and the input data. You can use this variant by setting the times_input argument to TRUE and providing input data.

The R6 class can also be initialized using the run_cw function as a helper function so that no prior knowledge of R6 classes is required.

Super class

innsight::InterpretingMethod -> ConnectionWeights

Public fields

Methods

Public methods

• ConnectionWeights$new()

• ConnectionWeights$clone()

Method new()

Create a new instance of the class ConnectionWeights. When initialized, the method is applied and the results are stored in the field result.

Usage

ConnectionWeights$new(
  converter,
  data = NULL,
  output_idx = NULL,
  output_label = NULL,
  channels_first = TRUE,
  times_input = FALSE,
  verbose = interactive(),
  dtype = "float"
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

ConnectionWeights$clone(deep = FALSE)

Arguments

References

• J. D. Olden et al. (2004) An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological Modelling 178, p. 389–397

See Also

Other methods: DeepLift, DeepSHAP, ExpectedGradient, Gradient, IntegratedGradient, LIME, LRP, SHAP, SmoothGrad

Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

# Create nn_sequential model
model <- nn_sequential(
  nn_linear(5, 12),
  nn_relu(),
  nn_linear(12, 1),
  nn_sigmoid()
)

# Create Converter with input names
converter <- Converter$new(model,
  input_dim = c(5),
  input_names = list(c("Car", "Cat", "Dog", "Plane", "Horse"))
)

# You can also use the helper function for the initialization part
converter <- convert(model,
  input_dim = c(5),
  input_names = list(c("Car", "Cat", "Dog", "Plane", "Horse"))
)

# Apply method Connection Weights
cw <- ConnectionWeights$new(converter)

# Again, you can use a helper function `run_cw()` for initializing
cw <- run_cw(converter)

# Print the head of the result as a data.frame
head(get_result(cw, "data.frame"), 5)

# Plot the result
plot(cw)

#----------------------- Example 2: Neuralnet ------------------------------
if (require("neuralnet")) {
  library(neuralnet)
  data(iris)

  # Train a Neural Network
  nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
    iris,
    linear.output = FALSE,
    hidden = c(3, 2), act.fct = "tanh", rep = 1
  )

  # Convert the trained model
  converter <- convert(nn)

  # Apply the Connection Weights method
  cw <- run_cw(converter)

  # Get the result as a torch tensor
  get_result(cw, type = "torch.tensor")

  # Plot the result
  plot(cw)
}


# ------------------------- Example 3: Keras -------------------------------
if (require("keras") & keras::is_keras_available()) {
  library(keras)

  # Make sure keras is installed properly
  is_keras_available()

  data <- array(rnorm(10 * 32 * 32 * 3), dim = c(10, 32, 32, 3))

  model <- keras_model_sequential()
  model %>%
    layer_conv_2d(
      input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
      activation = "softplus", padding = "valid") %>%
    layer_conv_2d(
      kernel_size = 8, filters = 4, activation = "tanh",
      padding = "same") %>%
    layer_conv_2d(
      kernel_size = 4, filters = 2, activation = "relu",
      padding = "valid") %>%
    layer_flatten() %>%
    layer_dense(units = 64, activation = "relu") %>%
    layer_dense(units = 16, activation = "relu") %>%
    layer_dense(units = 2, activation = "softmax")

  # Convert the model
  converter <- convert(model)

  # Apply the Connection Weights method
  cw <- run_cw(converter)

  # Get the head of the result as a data.frame
  head(get_result(cw, type = "data.frame"), 5)

  # Plot the result for all classes
  plot(cw, output_idx = 1:2)
}


#------------------------- Plotly plots ------------------------------------
if (require("plotly")) {
  # You can also create an interactive plot with plotly.
  # This is a suggested package, so make sure that it is installed
  library(plotly)
  plot(cw, as_plotly = TRUE)
}

Description

This class stores all layers converted to torch in a module which can be used like the original model (but torch-based). In addition, it provides other functions that are useful for interpreting individual predictions or explaining the entire model. This model is part of the class Converter and is the core for all the necessary calculations in the methods provided in this package.

Usage

ConvertedModel(modules_list, graph, input_nodes, output_nodes, dtype = "float")

Arguments

Method forward()

The forward method of the whole model, i.e., it calculates the output $y=f(x)$ of a given input $x$. In doing so, all intermediate values are stored in the individual torch modules from modules_list.

Usage

self(x,
     channels_first = TRUE,
     save_input = FALSE,
     save_preactivation = FALSE,
     save_output = FAlSE,
     save_last_layer = FALSE)

Arguments

x

The input torch tensor for this model.

channels_first

If the input tensor x is given in the format 'channels first', use TRUE. Otherwise, if the channels are last, use FALSE and the input will be transformed into the format 'channels first'. Default: TRUE.

save_input

Logical value whether the inputs from each layer are to be saved or not. Default: FALSE.

save_preactivation

Logical value whether the preactivations from each layer are to be saved or not. Default: FALSE.

save_output

Logical value whether the outputs from each layer are to be saved or not. Default: FALSE.

save_last_layer

Logical value whether the inputs, preactivations and outputs from the last layer are to be saved or not. Default: FALSE.

Return

Returns a list of the output values of the model with respect to the given inputs.

Method update_ref()

This method updates the intermediate values in each module from the list modules_list for the reference input x_ref and returns the output from it in the same way as in the forward method.

Usage

self$update_ref(x_ref,
                channels_first = TRUE,
                save_input = FALSE,
                save_preactivation = FALSE,
                save_output = FAlSE,
                save_last_layer = FALSE)

Arguments

x_ref

Reference input of the model.

channels_first

If the tensor x_ref is given in the format 'channels first' use TRUE. Otherwise, if the channels are last, use FALSE and the input will be transformed into the format 'channels first'. Default: TRUE.

save_input

Logical value whether the inputs from each layer are to be saved or not. Default: FALSE.

save_preactivation

Logical value whether the preactivations from each layer are to be saved or not. Default: FALSE.

save_output

Logical value whether the outputs from each layer are to be saved or not. Default: FALSE.

save_last_layer

Logical value whether the inputs, preactivations and outputs from the last layer are to be saved or not. Default: FALSE.

Return

Returns a list of the output values of the model with respect to the given reference input.

Method set_dtype()

This method changes the data type for all the layers in modules_list. Use either 'float' for torch::torch_float or 'double' for torch::torch_double.

Usage

self$set_dtype(dtype)

Arguments

dtype

The data type for all the calculations and defined tensors.

Description

This class analyzes a passed neural network and stores its internal structure and the individual layers by converting the entire network into an nn_module. With the help of this converter, many methods for interpreting the behavior of neural networks are provided, which give a better understanding of the whole model or individual predictions. You can use models from the following libraries:

• torch (nn_sequential)

• keras (keras_model, keras_model_sequential),

• neuralnet

Furthermore, a model can be passed as a list (see vignette("detailed_overview", package = "innsight") or the website).

The R6 class can also be initialized using the convert function as a helper function so that no prior knowledge of R6 classes is required.

Details

In order to better understand and analyze the prediction of a neural network, the preactivation or other information of the individual layers, which are not stored in an ordinary forward pass, are often required. For this reason, a given neural network is converted into a torch-based neural network, which provides all the necessary information for an interpretation. The converted torch model is stored in the field model and is an instance of ConvertedModel. However, before the torch model is created, all relevant details of the passed model are extracted into a named list. This list can be saved in complete form in the model_as_list field with the argument save_model_as_list, but this may consume a lot of memory for large networks and is not done by default. Also, this named list can again be used as a passed model for the class Converter, which will be described in more detail in the section 'Implemented Libraries'.

Implemented methods

An object of the Converter class can be applied to the following methods:

• Layerwise Relevance Propagation (LRP), Bach et al. (2015)

• Deep Learning Important Features (DeepLift), Shrikumar et al. (2017)

• DeepSHAP, Lundberg et al. (2017)

• SmoothGrad including SmoothGrad$\times$Input, Smilkov et al. (2017)

• Vanilla Gradient including Gradient$\times$Input

• Integrated gradients (IntegratedGradient), Sundararajan et al. (2017)

• Expected gradients (ExpectedGradient), Erion et al. (2021)

• ConnectionWeights, Olden et al. (2004)

• Local interpretable model-agnostic explanation (LIME), Ribeiro et al. (2016)

• Shapley values (SHAP), Lundberg et al. (2017)

Implemented libraries

The converter is implemented for models from the libraries nn_sequential, neuralnet and keras. But you can also write a wrapper for other libraries because a model can be passed as a named list which is described in detail in the vignette "In-depth explanation"
(see vignette("detailed_overview", package = "innsight") or the website).

Public fields

Methods

Public methods

• Converter$new()

• Converter$print()

• Converter$clone()

Method new()

Create a new Converter object for a given neural network. When initialized, the model is inspected, converted as a list and then the a torch-converted model (ConvertedModel) is created and stored in the field model.

Usage

Converter$new(
  model,
  input_dim = NULL,
  input_names = NULL,
  output_names = NULL,
  dtype = "float",
  save_model_as_list = FALSE
)

Arguments

Returns

A new instance of the R6 class Converter.

Method print()

Print a summary of the Converter object. This summary contains the individual fields and in particular the torch-converted model (ConvertedModel) with the layers.

Usage

Converter$print()

Returns

Returns the Converter object invisibly via base::invisible.

Method clone()

The objects of this class are cloneable with this method.

Usage

Converter$clone(deep = FALSE)

Arguments

References

• J. D. Olden et al. (2004) An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological Modelling 178, p. 389–397

• S. Bach et al. (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, p. 1-46

• M. T. Ribeiro et al. (2016) "Why should I trust you?": Explaining the predictions of any classifier. KDD 2016, p. 1135-1144

• A. Shrikumar et al. (2017) Learning important features through propagating activation differences. ICML 2017, p. 4844-4866

• D. Smilkov et al. (2017) SmoothGrad: removing noise by adding noise. CoRR, abs/1706.03825 M. Sundararajan et al. (2017) Axiomatic attribution for deep networks. ICML 2017, p.3319-3328

• S. Lundberg et al. (2017) A unified approach to interpreting model predictions. NIPS 2017, p. 4768-4777

• G. Erion et al. (2021) Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nature Machine Intelligence 3, p. 620-631

Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

model <- nn_sequential(
  nn_linear(5, 10),
  nn_relu(),
  nn_linear(10, 2, bias = FALSE),
  nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)

# Convert the model (for torch models is 'input_dim' required!)
converter <- Converter$new(model, input_dim = c(5))

# You can also use the helper function `convert()` for initializing a
# Converter object
converter <- convert(model, input_dim = c(5))

# Get the converted model stored in the field 'model'
converted_model <- converter$model

# Test it with the original model
mean(abs(converted_model(data)[[1]] - model(data)))


#----------------------- Example 2: Neuralnet ------------------------------
if (require("neuralnet")) {
  library(neuralnet)
  data(iris)

  # Train a neural network
  nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
    iris,
    linear.output = FALSE,
    hidden = c(3, 2), act.fct = "tanh", rep = 1
  )

  # Convert the model
  converter <- convert(nn)

  # Print all the layers
  converter$model$modules_list
}


#----------------------- Example 3: Keras ----------------------------------
if (require("keras") & keras::is_keras_available()) {
  library(keras)

  # Make sure keras is installed properly
  is_keras_available()

  # Define a keras model
  model <- keras_model_sequential() %>%
    layer_conv_2d(
      input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
      activation = "relu", padding = "same") %>%
    layer_conv_2d(
      kernel_size = 8, filters = 4,
      activation = "tanh", padding = "same") %>%
    layer_conv_2d(
      kernel_size = 4, filters = 2,
      activation = "relu", padding = "same") %>%
    layer_flatten() %>%
    layer_dense(units = 64, activation = "relu") %>%
    layer_dense(units = 1, activation = "sigmoid")

  # Convert this model and save model as list
  converter <- convert(model, save_model_as_list = TRUE)

  # Print the converted model as a named list
  str(converter$model_as_list, max.level = 1)
}


#----------------------- Example 4: List  ----------------------------------

# Define a model

model <- list()
model$input_dim <- 5
model$input_names <- list(c("Feat1", "Feat2", "Feat3", "Feat4", "Feat5"))
model$input_nodes <- c(1)
model$output_dim <- 2
model$output_names <- list(c("Cat", "no-Cat"))
model$output_nodes <- c(2)
model$layers$Layer_1 <-
  list(
    type = "Dense",
    weight = matrix(rnorm(5 * 20), 20, 5),
    bias = rnorm(20),
    activation_name = "tanh",
    dim_in = 5,
    dim_out = 20,
    input_layers = 0, # '0' means model input layer
    output_layers = 2
  )
model$layers$Layer_2 <-
  list(
    type = "Dense",
    weight = matrix(rnorm(20 * 2), 2, 20),
    bias = rnorm(2),
    activation_name = "softmax",
    input_layers = 1,
    output_layers = -1 # '-1' means model output layer
    #dim_in = 20, # These values are optional, but
    #dim_out = 2  # useful for internal checks
  )

# Convert the model
converter <- convert(model)

# Get the model as a torch::nn_module
torch_model <- converter$model

# You can use it as a normal torch model
x <- torch::torch_randn(3, 5)
torch_model(x)

Description

This is an implementation of the deep learning important features (DeepLift) algorithm introduced by Shrikumar et al. (2017). It's a local method for interpreting a single element $x$ of the dataset concerning a reference value $x'$ and returns the contribution of each input feature from the difference of the output ($y=f(x)$) and reference output ($y'=f(x')$) prediction. The basic idea of this method is to decompose the difference-from-reference prediction with respect to the input features, i.e.,

$\Delta y = y - y' = \sum_i C(x_i).$

Compared to Layer-wise relevance propagation (see LRP), the DeepLift method is an exact decomposition and not an approximation, so we get real contributions of the input features to the difference-from-reference prediction. There are two ways to handle activation functions: the Rescale rule ('rescale') and RevealCancel rule ('reveal_cancel').

The R6 class can also be initialized using the run_deeplift function as a helper function so that no prior knowledge of R6 classes is required.

Super class

innsight::InterpretingMethod -> DeepLift

Public fields

Methods

Public methods

• DeepLift$new()

• DeepLift$clone()

Method new()

Create a new instance of the DeepLift R6 class. When initialized, the method DeepLift is applied to the given data and the results are stored in the field result.

Usage

DeepLift$new(
  converter,
  data,
  channels_first = TRUE,
  output_idx = NULL,
  output_label = NULL,
  ignore_last_act = TRUE,
  rule_name = "rescale",
  x_ref = NULL,
  winner_takes_all = TRUE,
  verbose = interactive(),
  dtype = "float"
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

DeepLift$clone(deep = FALSE)

Arguments

References

A. Shrikumar et al. (2017) Learning important features through propagating activation differences. ICML 2017, p. 4844-4866

See Also

Other methods: ConnectionWeights, DeepSHAP, ExpectedGradient, Gradient, IntegratedGradient, LIME, LRP, SHAP, SmoothGrad

Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

# Create nn_sequential model and data
model <- nn_sequential(
  nn_linear(5, 12),
  nn_relu(),
  nn_linear(12, 2),
  nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)
ref <- torch_randn(1, 5)

# Create Converter using the helper function `convert`
converter <- convert(model, input_dim = c(5))

# Apply method DeepLift
deeplift <- DeepLift$new(converter, data, x_ref = ref)

# You can also use the helper function `run_deeplift` for initializing
# an R6 DeepLift object
deeplift <- run_deeplift(converter, data, x_ref = ref)

# Print the result as a torch tensor for first two data points
get_result(deeplift, "torch.tensor")[1:2]

# Plot the result for both classes
plot(deeplift, output_idx = 1:2)

# Plot the boxplot of all datapoints and for both classes
boxplot(deeplift, output_idx = 1:2)

# ------------------------- Example 2: Neuralnet ---------------------------
if (require("neuralnet")) {
  library(neuralnet)
  data(iris)

  # Train a neural network
  nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
    iris,
    linear.output = FALSE,
    hidden = c(3, 2), act.fct = "tanh", rep = 1
  )

  # Convert the model
  converter <- convert(nn)

  # Apply DeepLift with rescale-rule and a reference input of the feature
  # means
  x_ref <- matrix(colMeans(iris[, c(3, 4)]), nrow = 1)
  deeplift_rescale <- run_deeplift(converter, iris[, c(3, 4)], x_ref = x_ref)

  # Get the result as a dataframe and show first 5 rows
  get_result(deeplift_rescale, type = "data.frame")[1:5, ]

  # Plot the result for the first datapoint in the data
  plot(deeplift_rescale, data_idx = 1)

  # Plot the result as boxplots
  boxplot(deeplift_rescale)
}


# ------------------------- Example 3: Keras -------------------------------
if (require("keras") & keras::is_keras_available()) {
  library(keras)

  # Make sure keras is installed properly
  is_keras_available()

  data <- array(rnorm(10 * 32 * 32 * 3), dim = c(10, 32, 32, 3))

  model <- keras_model_sequential()
  model %>%
    layer_conv_2d(
      input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
      activation = "softplus", padding = "valid") %>%
    layer_conv_2d(
      kernel_size = 8, filters = 4, activation = "tanh",
      padding = "same") %>%
    layer_conv_2d(
      kernel_size = 4, filters = 2, activation = "relu",
      padding = "valid") %>%
    layer_flatten() %>%
    layer_dense(units = 64, activation = "relu") %>%
    layer_dense(units = 16, activation = "relu") %>%
    layer_dense(units = 2, activation = "softmax")

  # Convert the model
  converter <- convert(model)

  # Apply the DeepLift method with reveal-cancel rule
  deeplift_revcancel <- run_deeplift(converter, data,
    channels_first = FALSE,
    rule_name = "reveal_cancel"
  )

  # Plot the result for the first image and both classes
  plot(deeplift_revcancel, output_idx = 1:2)

  # Plot the pixel-wise median reelvance image
  plot_global(deeplift_revcancel, output_idx = 1)
}


#------------------------- Plotly plots ------------------------------------
if (require("plotly")) {
  # You can also create an interactive plot with plotly.
  # This is a suggested package, so make sure that it is installed
  library(plotly)
  boxplot(deeplift, as_plotly = TRUE)
}

Description

The DeepSHAP method extends the DeepLift technique by not only considering a single reference value but by calculating the average from several, ideally representative reference values at each layer. The obtained feature-wise results are approximate Shapley values for the chosen output, where the conditional expectation is computed using these different reference values, i.e., the DeepSHAP method decompose the difference from the prediction and the mean prediction $f(x) - E[f(\tilde{x})]$ in feature-wise effects. The reference values can be passed by the argument data_ref.

The R6 class can also be initialized using the run_deepshap function as a helper function so that no prior knowledge of R6 classes is required.

Super class

innsight::InterpretingMethod -> DeepSHAP

Public fields

Methods

Public methods

• DeepSHAP$new()

• DeepSHAP$clone()

Method new()

Create a new instance of the DeepSHAP R6 class. When initialized, the method DeepSHAP is applied to the given data and the results are stored in the field result.

Usage

DeepSHAP$new(
  converter,
  data,
  channels_first = TRUE,
  output_idx = NULL,
  output_label = NULL,
  ignore_last_act = TRUE,
  rule_name = "rescale",
  data_ref = NULL,
  limit_ref = 100,
  winner_takes_all = TRUE,
  verbose = interactive(),
  dtype = "float"
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

DeepSHAP$clone(deep = FALSE)

Arguments

References

S. Lundberg & S. Lee (2017) A unified approach to interpreting model predictions. NIPS 2017, p. 4768–4777

See Also

Other methods: ConnectionWeights, DeepLift, ExpectedGradient, Gradient, IntegratedGradient, LIME, LRP, SHAP, SmoothGrad

Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

# Create nn_sequential model and data
model <- nn_sequential(
  nn_linear(5, 12),
  nn_relu(),
  nn_linear(12, 2),
  nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)

# Create a reference dataset for the estimation of the conditional
# expectation
ref <- torch_randn(5, 5)

# Create Converter
converter <- convert(model, input_dim = c(5))

# Apply method DeepSHAP
deepshap <- DeepSHAP$new(converter, data, data_ref = ref)

# You can also use the helper function `run_deepshap` for initializing
# an R6 DeepSHAP object
deepshap <- run_deepshap(converter, data, data_ref = ref)

# Print the result as a torch tensor for first two data points
get_result(deepshap, "torch.tensor")[1:2]

# Plot the result for both classes
plot(deepshap, output_idx = 1:2)

# Plot the boxplot of all datapoints and for both classes
boxplot(deepshap, output_idx = 1:2)

# ------------------------- Example 2: Neuralnet ---------------------------
if (require("neuralnet")) {
  library(neuralnet)
  data(iris)

  # Train a neural network
  nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
    iris,
    linear.output = FALSE,
    hidden = c(3, 2), act.fct = "tanh", rep = 1
  )

  # Convert the model
  converter <- convert(nn)

  # Apply DeepSHAP with rescale-rule and a 100 (default of `limit_ref`)
  # instances as the reference dataset
  deepshap <- run_deepshap(converter, iris[, c(3, 4)],
                           data_ref = iris[, c(3, 4)])

  # Get the result as a dataframe and show first 5 rows
  get_result(deepshap, type = "data.frame")[1:5, ]

  # Plot the result for the first datapoint in the data
  plot(deepshap, data_idx = 1)

  # Plot the result as boxplots
  boxplot(deepshap)
}


# ------------------------- Example 3: Keras -------------------------------
if (require("keras") & keras::is_keras_available()) {
  library(keras)

  # Make sure keras is installed properly
  is_keras_available()

  data <- array(rnorm(10 * 32 * 32 * 3), dim = c(10, 32, 32, 3))

  model <- keras_model_sequential()
  model %>%
    layer_conv_2d(
      input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
      activation = "softplus", padding = "valid") %>%
    layer_conv_2d(
      kernel_size = 8, filters = 4, activation = "tanh",
      padding = "same") %>%
    layer_conv_2d(
      kernel_size = 4, filters = 2, activation = "relu",
      padding = "valid") %>%
    layer_flatten() %>%
    layer_dense(units = 64, activation = "relu") %>%
    layer_dense(units = 16, activation = "relu") %>%
    layer_dense(units = 2, activation = "softmax")

  # Convert the model
  converter <- convert(model)

  # Apply the DeepSHAP method with zero baseline (wich is equivalent to
  # DeepLift with zero baseline)
  deepshap <- run_deepshap(converter, data, channels_first = FALSE)

  # Plot the result for the first image and both classes
  plot(deepshap, output_idx = 1:2)

  # Plot the pixel-wise median of the results
  plot_global(deepshap, output_idx = 1)
}


#------------------------- Plotly plots ------------------------------------
if (require("plotly")) {
  # You can also create an interactive plot with plotly.
  # This is a suggested package, so make sure that it is installed
  library(plotly)
  boxplot(deepshap, as_plotly = TRUE)
}

Description

The Expected Gradients method (Erion et al., 2021), also known as GradSHAP, is a local feature attribution technique which extends the IntegratedGradient method and provides approximate Shapley values. In contrast to IntegratedGradient, it considers not only a single reference value $x'$ but the whole distribution of reference values $X' \sim x'$ and averages the IntegratedGradient values over this distribution. Mathematically, the method can be described as follows:

$E_{x'\sim X', \alpha \sim U(0,1)}[(x - x') \times \frac{\partial f(x' + \alpha (x - x'))}{\partial x}]$

The distribution of the reference values is specified with the argument data_ref, of which n samples are taken at random for each instance during the estimation.

The R6 class can also be initialized using the run_expgrad function as a helper function so that no prior knowledge of R6 classes is required.

Super classes

innsight::InterpretingMethod -> innsight::GradientBased -> ExpectedGradient

Public fields

Methods

Public methods

• ExpectedGradient$new()

• ExpectedGradient$clone()

Method new()

Create a new instance of the ExpectedGradient R6 class. When initialized, the method Expected Gradient is applied to the given data and baseline values and the results are stored in the field result.

Usage

ExpectedGradient$new(
  converter,
  data,
  data_ref = NULL,
  n = 50,
  channels_first = TRUE,
  output_idx = NULL,
  output_label = NULL,
  ignore_last_act = TRUE,
  verbose = interactive(),
  dtype = "float"
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

ExpectedGradient$clone(deep = FALSE)

Arguments

References

G. Erion et al. (2021) *Improving performance of deep learning models with * axiomatic attribution priors and expected gradients. Nature Machine Intelligence 3, pp. 620-631.

See Also

Other methods: ConnectionWeights, DeepLift, DeepSHAP, Gradient, IntegratedGradient, LIME, LRP, SHAP, SmoothGrad

Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

# Create nn_sequential model and data
model <- nn_sequential(
  nn_linear(5, 12),
  nn_relu(),
  nn_linear(12, 2),
  nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)
ref <- torch_randn(1, 5)

# Create Converter
converter <- convert(model, input_dim = c(5))

# Apply method IntegratedGradient
int_grad <- IntegratedGradient$new(converter, data, x_ref = ref)

# You can also use the helper function `run_intgrad` for initializing
# an R6 IntegratedGradient object
int_grad <- run_intgrad(converter, data, x_ref = ref)

# Print the result as a torch tensor for first two data points
get_result(int_grad, "torch.tensor")[1:2]

# Plot the result for both classes
plot(int_grad, output_idx = 1:2)

# Plot the boxplot of all datapoints and for both classes
boxplot(int_grad, output_idx = 1:2)

# ------------------------- Example 2: Neuralnet ---------------------------
if (require("neuralnet")) {
  library(neuralnet)
  data(iris)

  # Train a neural network
  nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
    iris,
    linear.output = FALSE,
    hidden = c(3, 2), act.fct = "tanh", rep = 1
  )

  # Convert the model
  converter <- convert(nn)

  # Apply IntegratedGradient with a reference input of the feature means
  x_ref <- matrix(colMeans(iris[, c(3, 4)]), nrow = 1)
  int_grad <- run_intgrad(converter, iris[, c(3, 4)], x_ref = x_ref)

  # Get the result as a dataframe and show first 5 rows
  get_result(int_grad, type = "data.frame")[1:5, ]

  # Plot the result for the first datapoint in the data
  plot(int_grad, data_idx = 1)

  # Plot the result as boxplots
  boxplot(int_grad)
}


# ------------------------- Example 3: Keras -------------------------------
if (require("keras") & keras::is_keras_available()) {
  library(keras)

  # Make sure keras is installed properly
  is_keras_available()

  data <- array(rnorm(10 * 32 * 32 * 3), dim = c(10, 32, 32, 3))

  model <- keras_model_sequential()
  model %>%
    layer_conv_2d(
      input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
      activation = "softplus", padding = "valid") %>%
    layer_conv_2d(
      kernel_size = 8, filters = 4, activation = "tanh",
      padding = "same") %>%
    layer_conv_2d(
      kernel_size = 4, filters = 2, activation = "relu",
      padding = "valid") %>%
    layer_flatten() %>%
    layer_dense(units = 64, activation = "relu") %>%
    layer_dense(units = 2, activation = "softmax")

  # Convert the model
  converter <- convert(model)

  # Apply the IntegratedGradient method with a zero baseline and n = 20
  # iteration steps
  int_grad <- run_intgrad(converter, data,
    channels_first = FALSE,
    n = 20
  )

  # Plot the result for the first image and both classes
  plot(int_grad, output_idx = 1:2)

  # Plot the pixel-wise median of the results
  plot_global(int_grad, output_idx = 1)
}


#------------------------- Plotly plots ------------------------------------
if (require("plotly")) {
  # You can also create an interactive plot with plotly.
  # This is a suggested package, so make sure that it is installed
  library(plotly)
  boxplot(int_grad, as_plotly = TRUE)
}

Description

This is a generic S3 method for the R6 method InterpretingMethod$get_result(). See the respective method described in InterpretingMethod for details.

Usage

get_result(x, ...)

Arguments

Description

This method computes the gradients (also known as Vanilla Gradients) of the outputs with respect to the input variables, i.e., for all input variable $i$ and output class $j$

$d f(x)_j / d x_i.$

If the argument times_input is TRUE, the gradients are multiplied by the respective input value (Gradient$\times$Input), i.e.,

$x_i * d f(x)_j / d x_i.$

While the vanilla gradients emphasize prediction-sensitive features, Gradient$\times$Input is a decomposition of the output into feature-wise effects based on the first-order Taylor decomposition.

The R6 class can also be initialized using the run_grad function as a helper function so that no prior knowledge of R6 classes is required.

Super classes

innsight::InterpretingMethod -> innsight::GradientBased -> Gradient

Methods

Public methods

• Gradient$new()

• Gradient$clone()

Method new()

Create a new instance of the Gradient R6 class. When initialized, the method Gradient or Gradient$\times$Input is applied to the given data and the results are stored in the field result.

Usage

Gradient$new(
  converter,
  data,
  channels_first = TRUE,
  output_idx = NULL,
  output_label = NULL,
  ignore_last_act = TRUE,
  times_input = FALSE,
  verbose = interactive(),
  dtype = "float"
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

Gradient$clone(deep = FALSE)

Arguments

See Also

Other methods: ConnectionWeights, DeepLift, DeepSHAP, ExpectedGradient, IntegratedGradient, LIME, LRP, SHAP, SmoothGrad

Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

# Create nn_sequential model and data
model <- nn_sequential(
  nn_linear(5, 12),
  nn_relu(),
  nn_linear(12, 2),
  nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)

# Create Converter with input and output names
converter <- convert(model,
  input_dim = c(5),
  input_names = list(c("Car", "Cat", "Dog", "Plane", "Horse")),
  output_names = list(c("Buy it!", "Don't buy it!"))
)

# Calculate the Gradients
grad <- Gradient$new(converter, data)

# You can also use the helper function `run_grad` for initializing
# an R6 Gradient object
grad <- run_grad(converter, data)

# Print the result as a data.frame for first 5 rows
get_result(grad, "data.frame")[1:5,]

# Plot the result for both classes
plot(grad, output_idx = 1:2)

# Plot the boxplot of all datapoints
boxplot(grad, output_idx = 1:2)

# ------------------------- Example 2: Neuralnet ---------------------------
if (require("neuralnet")) {
  library(neuralnet)
  data(iris)

  # Train a neural network
  nn <- neuralnet(Species ~ ., iris,
    linear.output = FALSE,
    hidden = c(10, 5),
    act.fct = "logistic",
    rep = 1
  )

  # Convert the trained model
  converter <- convert(nn)

  # Calculate the gradients
  gradient <- run_grad(converter, iris[, -5])

  # Plot the result for the first and 60th data point and all classes
  plot(gradient, data_idx = c(1, 60), output_idx = 1:3)

  # Calculate Gradients x Input and do not ignore the last activation
  gradient <- run_grad(converter, iris[, -5],
                       ignore_last_act = FALSE,
                       times_input = TRUE)

  # Plot the result again
  plot(gradient, data_idx = c(1, 60), output_idx = 1:3)
}


# ------------------------- Example 3: Keras -------------------------------
if (require("keras") & keras::is_keras_available()) {
  library(keras)

  # Make sure keras is installed properly
  is_keras_available()

  data <- array(rnorm(64 * 60 * 3), dim = c(64, 60, 3))

  model <- keras_model_sequential()
  model %>%
    layer_conv_1d(
      input_shape = c(60, 3), kernel_size = 8, filters = 8,
      activation = "softplus", padding = "valid") %>%
    layer_conv_1d(
      kernel_size = 8, filters = 4, activation = "tanh",
      padding = "same") %>%
    layer_conv_1d(
      kernel_size = 4, filters = 2, activation = "relu",
      padding = "valid") %>%
    layer_flatten() %>%
    layer_dense(units = 64, activation = "relu") %>%
    layer_dense(units = 16, activation = "relu") %>%
    layer_dense(units = 3, activation = "softmax")

  # Convert the model
  converter <- convert(model)

  # Apply the Gradient method
  gradient <- run_grad(converter, data, channels_first = FALSE)

  # Plot the result for the first datapoint and all classes
  plot(gradient, output_idx = 1:3)

  # Plot the result as boxplots for first two classes
  boxplot(gradient, output_idx = 1:2)
}


#------------------------- Plotly plots ------------------------------------
if (require("plotly")) {
  # You can also create an interactive plot with plotly.
  # This is a suggested package, so make sure that it is installed
  library(plotly)

  # Result as boxplots
  boxplot(gradient, as_plotly = TRUE)

  # Result of the second data point
  plot(gradient, data_idx = 2, as_plotly = TRUE)
}

Description

Super class for gradient-based interpretation methods. This class inherits from InterpretingMethod. It summarizes all implemented gradient-based methods and provides a private function to calculate the gradients w.r.t. to the input for given data. Implemented are:

• Vanilla Gradients and Gradient$\times$Input (Gradient)

• Integrated Gradients (IntegratedGradient)

• SmoothGrad and SmoothGrad$\times$Input (SmoothGrad)

• ExpectedGradients (ExpectedGradient)

Super class

innsight::InterpretingMethod -> GradientBased

Public fields

Methods

Public methods

• GradientBased$new()

• GradientBased$clone()

Method new()

Create a new instance of this class. When initialized, the method is applied to the given data and the results are stored in the field result.

Usage

GradientBased$new(
  converter,
  data,
  channels_first = TRUE,
  output_idx = NULL,
  output_label = NULL,
  ignore_last_act = TRUE,
  times_input = TRUE,
  verbose = interactive(),
  dtype = "float"
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

GradientBased$clone(deep = FALSE)

Arguments

Description

The S4 class innsight_ggplot2 visualizes the results of the methods provided from the package innsight using ggplot2. In addition, it allows easier analysis of the results and modification of the visualization by basic generic functions. The individual slots are for internal use only and should not be modified.

Details

This S4 class is a simple extension of a ggplot2 object that enables a more detailed analysis of the results and a way to visualize the results of models with multiple input layers (e.g., images and tabular data). The distinction between one and multiple input layers decides the behavior of this class, and this information is stored in the slot multiplot.

One input layer (multiplot = FALSE)

If the model passed to a method from the innsight package has only one input layer, the S4 class innsight_ggplot2 is just a wrapper of a single ggplot2 object. This object is stored as a 1x1 matrix in the slot grobs and the slots output_strips and col_dims contain only empty lists because no second line of stripes describing the input layer is needed. Although it is an object of the class innsight_ggplot2, the generic function +.innsight_ggplot2 provides a ggplot2-typical usage to modify the representation. The graphical objects are simply forwarded to the ggplot2 object in grobs and added using ggplot2::+.gg. In addition, some generic functions are implemented to visualize or examine individual aspects of the overall plot in more detail. All available generic functions are listed below:

• +

• plot, print and show (all behave the same)

• [

• [[

Note: In this case, the generic function ⁠[<-⁠ is not implemented because there is only one ggplot2 object and not multiple ones.

Multiple input layers (multiplot = TRUE)

If the passed model has multiple input layers, a ggplot2 object is created for each data point, input layer and output node and then stored as a matrix in the slot grobs. During visualization, these are combined using the function gridExtra::arrangeGrob and corresponding strips for the output layer/node names are added at the top. The labels, column indices and theme for the extra row of strips are stored in the slots output_strips and col_dims. The strips for the input layer and the data points (if not boxplot) are created using ggplot2::facet_grid in the individual ggplot2 objects of the grob matrix. An example structure is shown below:

|      Output 1: Node 1      |      Output 1: Node 3      |
|   Input 1   |   Input 2    |   Input 1   |   Input 2    |
|---------------------------------------------------------|-------------
|             |              |             |              |
| grobs[1,1]  |  grobs[1,2]  | grobs[1,3]  | grobs[1,4]   | data point 1
|             |              |             |              |
|---------------------------------------------------------|-------------
|             |              |             |              |
| grobs[2,1]  |  grobs[2,2]  | grobs[2,3]  | grobs[2,4]   | data point 2
|             |              |             |              |

Similar to the other case, generic functions are implemented to add graphical objects from ggplot2, create the whole plot or select only specific rows/columns. The difference, however, is that each entry in each row and column is a separate ggplot2 object and can be modified individually. For example, adds + ggplot2::xlab("X") the x-axis label "X" to all objects and not only to those in the last row. The generic function [<- allows you to replace a selection of objects in grobs and thus, for example, to change the x-axis title only in the bottom row. All available generic functions are listed below:

• +

• plot, print and show (all behave the same)

• [

• [[

• [<-

• [[<-

Note: Since this is not a standard visualization, the suggested packages 'grid', 'gridExtra' and 'gtable' must be installed.

Slots

grobs

The individual ggplot2 objects arranged as a matrix (see details for more information)

multiplot

A logical value indicating whether there are multiple input layers and therefore correspondingly individual ggplot2 objects instead of one single object.

output_strips

A list containing the labels and themes of the strips for the output nodes. This slot is only relevant if multiplot is TRUE.

col_dims

A list of the length of output_strips assigning to each strip the column index of grobs of the associated strip.

boxplot

A logical value indicating whether the result of individual data points or a boxplot over multiple instances is displayed.

Description

The S4 class innsight_plotly visualizes the results of the methods provided from the package innsight using plotly. In addition, it allows easier analysis of the results and modification of the visualization by basic generic functions. The individual slots are for internal use only and should not be modified.

Details

This S4 class is a simple extension of a plotly object that enables a more detailed analysis of the results and a way to visualize the results of models with multiple input layers (e.g., images and tabular data).

The overall plot is created in the following order:

1. The corresponding shapes and annotations of the slots annotations and shapes are added to each plot in plots. This also adds the strips at the top for the output node (or input layer) and, if necessary, on the right side for the data point.

2. Subsequently, all individual plots are combined into one plot with the help of the function plotly::subplot.

3. Lastly, the global elements from the layout slot are added and if there are multiple input layers (multiplot = TRUE), another output strip is added for the columns.

An example structure of the plot with multiple input layers is shown below:

|      Output 1: Node 1      |      Output 1: Node 3      |
|   Input 1   |   Input 2    |   Input 1   |   Input 2    |
|---------------------------------------------------------|-------------
|             |              |             |              |
| plots[1,1]  |  plots[1,2]  | plots[1,3]  | plots[1,4]   | data point 1
|             |              |             |              |
|---------------------------------------------------------|-------------
|             |              |             |              |
| plots[2,1]  |  plots[2,2]  | plots[2,3]  | plots[2,4]   | data point 2
|             |              |             |              |

Additionally, some generic functions are implemented to visualize individual aspects of the overall plot or to examine them in more detail. All available generic functions are listed below:

• plot, print and show (all behave the same)

• [

• [[

Slots

plots

The individual plotly objects arranged as a matrix (see details for more information).

shapes

A list of two lists with the names shapes_strips and shapes_other. The list shapes_strips contains the shapes for the strips and may not be manipulated. The other list shapes_other contains a matrix of the same size as plots and each entry contains the shapes of the corresponding plot.

annotations

A list of two lists with the names annotations_strips and annotations_other. The list annotations_strips contains the annotations for the strips and may not be manipulated. The other list annotations_other contains a matrix of the same size as plots and each entry contains the annotations of the corresponding plot.

multiplot

A logical value indicating whether there are multiple input layers and therefore correspondingly individual ggplot2 objects instead of one single object.

layout

This list contains all global layout options, e.g. update buttons, sliders, margins etc. (see plotly::layout for more details).

col_dims

A list to assign a label to the columns for the output strips.

Description

Since all methods and the preceding conversion step in the innsight package were implemented using R6 classes and these always require a call to classname$new() for initialization, the following functions are defined to shorten the construction of the corresponding R6 objects:

• convert() for Converter

• run_grad() for Gradient

• run_smoothgrad() for SmoothGrad

• run_intgrad() for IntegratedGradient

• run_expgrad() for ExpectedGradient

• run_lrp() for LRP

• run_deeplift() for DeepLift

• run_deepshap for DeepSHAP

• run_cw for ConnectionWeights

• run_lime for LIME

• run_shap for SHAP

Usage

# Create a new `Converter` object of the given `model`
convert(model, ...)

# Apply the `Gradient` method to the passed `data` to be explained
run_grad(converter, data, ...)

# Apply the `SmoothGrad` method to the passed `data` to be explained
run_smoothgrad(converter, data, ...)

# Apply the `IntegratedGradient` method to the passed `data` to be explained
run_intgrad(converter, data, ...)

# Apply the `ExpectedGradient` method to the passed `data` to be explained
run_expgrad(converter, data, ...)

# Apply the `LRP` method to the passed `data` to be explained
run_lrp(converter, data, ...)

# Apply the `DeepLift` method to the passed `data` to be explained
run_deeplift(converter, data, ...)

# Apply the `DeepSHAP` method to the passed `data` to be explained
run_deepshap(converter, data, ...)

# Apply the `ConnectionWeights` method (argument `data` is not always required)
run_cw(converter, ...)

# Apply the `LIME` method to explain `data` by using the dataset `data_ref`
run_lime(model, data, data_ref, ...)

# Apply the `SHAP` method to explain `data` by using the dataset `data_ref`
run_shap(model, data, data_ref, ...)

Arguments

Value

R6::R6Class object of the respective type.

Description

The IntegratedGradient class implements the method Integrated Gradients (Sundararajan et al., 2017), which incorporates a reference value $x'$ (also known as baseline value) analogous to the DeepLift method. Integrated Gradients helps to uncover the relative importance of input features in the predictions $y = f(x)$ made by a model compared to the prediction of the reference value $y' = f(x')$. This is achieved through the following formula:

$(x - x') \times \int_{\alpha=0}^{1} \frac{\partial f(x' + \alpha (x - x'))}{\partial x} d\alpha$

In simpler terms, it calculates how much each feature contributes to a model's output by tracing a path from a baseline input $x'$ to the actual input $x$ and measuring the average gradients along that path.

Similar to the other gradient-based methods, by default the integrated gradient is multiplied by the input to get an approximate decomposition of $y - y'$. However, with the parameter times_input only the gradient describing the output sensitivity can be returned.

The R6 class can also be initialized using the run_intgrad function as a helper function so that no prior knowledge of R6 classes is required.

Super classes

innsight::InterpretingMethod -> innsight::GradientBased -> IntegratedGradient

Public fields

Methods

Public methods

• IntegratedGradient$new()

• IntegratedGradient$clone()

Method new()

Create a new instance of the IntegratedGradient R6 class. When initialized, the method Integrated Gradient is applied to the given data and baseline value and the results are stored in the field result.

Usage

IntegratedGradient$new(
  converter,
  data,
  x_ref = NULL,
  n = 50,
  times_input = TRUE,
  channels_first = TRUE,
  output_idx = NULL,
  output_label = NULL,
  ignore_last_act = TRUE,
  verbose = interactive(),
  dtype = "float"
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

IntegratedGradient$clone(deep = FALSE)

Arguments

References

M. Sundararajan et al. (2017) Axiomatic attribution for deep networks. ICML 2017, PMLR 70, pp. 3319-3328.

See Also

Other methods: ConnectionWeights, DeepLift, DeepSHAP, ExpectedGradient, Gradient, LIME, LRP, SHAP, SmoothGrad

Examples

#----------------------- Example 1: Torch ----------------------------------
library(torch)

# Create nn_sequential model and data
model <- nn_sequential(
  nn_linear(5, 12),
  nn_relu(),
  nn_linear(12, 2),
  nn_softmax(dim = 2)
)
data <- torch_randn(25, 5)
ref <- torch_randn(1, 5)

# Create Converter
converter <- convert(model, input_dim = c(5))

# Apply method IntegratedGradient
int_grad <- IntegratedGradient$new(converter, data, x_ref = ref)

# You can also use the helper function `run_intgrad` for initializing
# an R6 IntegratedGradient object
int_grad <- run_intgrad(converter, data, x_ref = ref)

# Print the result as a torch tensor for first two data points
get_result(int_grad, "torch.tensor")[1:2]

# Plot the result for both classes
plot(int_grad, output_idx = 1:2)

# Plot the boxplot of all datapoints and for both classes
boxplot(int_grad, output_idx = 1:2)

# ------------------------- Example 2: Neuralnet ---------------------------
if (require("neuralnet")) {
  library(neuralnet)
  data(iris)

  # Train a neural network
  nn <- neuralnet((Species == "setosa") ~ Petal.Length + Petal.Width,
    iris,
    linear.output = FALSE,
    hidden = c(3, 2), act.fct = "tanh", rep = 1
  )

  # Convert the model
  converter <- convert(nn)

  # Apply IntegratedGradient with a reference input of the feature means
  x_ref <- matrix(colMeans(iris[, c(3, 4)]), nrow = 1)
  int_grad <- run_intgrad(converter, iris[, c(3, 4)], x_ref = x_ref)

  # Get the result as a dataframe and show first 5 rows
  get_result(int_grad, type = "data.frame")[1:5, ]

  # Plot the result for the first datapoint in the data
  plot(int_grad, data_idx = 1)

  # Plot the result as boxplots
  boxplot(int_grad)
}


# ------------------------- Example 3: Keras -------------------------------
if (require("keras") & keras::is_keras_available()) {
  library(keras)

  # Make sure keras is installed properly
  is_keras_available()

  data <- array(rnorm(10 * 32 * 32 * 3), dim = c(10, 32, 32, 3))

  model <- keras_model_sequential()
  model %>%
    layer_conv_2d(
      input_shape = c(32, 32, 3), kernel_size = 8, filters = 8,
      activation = "softplus", padding = "valid") %>%
    layer_conv_2d(
      kernel_size = 8, filters = 4, activation = "tanh",
      padding = "same") %>%
    layer_conv_2d(
      kernel_size = 4, filters = 2, activation = "relu",
      padding = "valid") %>%
    layer_flatten() %>%
    layer_dense(units = 64, activation = "relu") %>%
    layer_dense(units = 2, activation = "softmax")

  # Convert the model
  converter <- convert(model)

  # Apply the IntegratedGradient method with a zero baseline and n = 20
  # iteration steps
  int_grad <- run_intgrad(converter, data,
    channels_first = FALSE,
    n = 20
  )

  # Plot the result for the first image and both classes
  plot(int_grad, output_idx = 1:2)

  # Plot the pixel-wise median of the results
  plot_global(int_grad, output_idx = 1)
}


#------------------------- Plotly plots ------------------------------------
if (require("plotly")) {
  # You can also create an interactive plot with plotly.
  # This is a suggested package, so make sure that it is installed
  library(plotly)
  boxplot(int_grad, as_plotly = TRUE)
}

Description

This is a super class for all interpreting methods in the innsight package. Implemented are the following methods:

• Deep Learning Important Features (DeepLift)

• Deep Shapley additive explanations (DeepSHAP)

• Layer-wise Relevance Propagation (LRP)

• Gradient-based methods:

  • Vanilla gradients including Gradient$\times$Input (Gradient)

  • Smoothed gradients including SmoothGrad$\times$Input (SmoothGrad)

  • Integrated gradients (IntegratedGradient)

  • Expected gradients (ExpectedGradient)

• Connection Weights (global and local) (ConnectionWeights)

• Also some model-agnostic approaches:

  • Local interpretable model-agnostic explanations (LIME)

  • Shapley values (SHAP)

Public fields

Methods

Public methods

• InterpretingMethod$new()

• InterpretingMethod$get_result()

• InterpretingMethod$plot()

• InterpretingMethod$plot_global()

• InterpretingMethod$print()

• InterpretingMethod$clone()

Method new()

Create a new instance of this super class.

Usage

InterpretingMethod$new(
  converter,
  data,
  channels_first = TRUE,
  output_idx = NULL,
  output_label = NULL,
  ignore_last_act = TRUE,
  winner_takes_all = TRUE,
  verbose = interactive(),
  dtype = "float"
)

Arguments

Method get_result()

This function returns the result of this method for the given data either as an array ('array'), a torch tensor ('torch.tensor', or 'torch_tensor') of size (batch_size, dim_in, dim_out) or as a data.frame ('data.frame'). This method is also implemented as a generic S3 function get_result. For a detailed description, we refer to our in-depth vignette (vignette("detailed_overview", package = "innsight")) or our website.

Usage

InterpretingMethod$get_result(type = "array")

Arguments

Returns

The result of this method for the given data in the chosen type.

Method plot()

This method visualizes the result of the selected method and enables a visual in-depth investigation with the help of the S4 classes innsight_ggplot2 and innsight_plotly.
You can use the argument data_idx to select the data points in the given data for the plot. In addition, the individual output nodes for the plot can be selected with the argument output_idx. The different results for the selected data points and outputs are visualized using the ggplot2-based S4 class innsight_ggplot2. You can also use the as_plotly argument to generate an interactive plot with innsight_plotly based on the plot function plotly::plot_ly. For more information and the whole bunch of possibilities, see innsight_ggplot2 and innsight_plotly.

Notes:

1. For the interactive plotly-based plots, the suggested package plotly is required.

2. The ggplot2-based plots for models with multiple input layers are a bit more complex, therefore the suggested packages 'grid', 'gridExtra' and 'gtable' must be installed in your R session.

3. If the global Connection Weights method was applied, the unnecessary argument data_idx will be ignored.

4. The predictions, the sum of relevances, and, if available, the decomposition target are displayed by default in a box within the plot. Currently, these are not generated for plotly plots.

Usage

InterpretingMethod$plot(
  data_idx = 1,
  output_idx = NULL,
  output_label = NULL,
  aggr_channels = "sum",
  as_plotly = FALSE,
  same_scale = FALSE,
  show_preds = TRUE
)

Arguments

Returns

Returns either an innsight_ggplot2 (as_plotly = FALSE) or an innsight_plotly (as_plotly = TRUE) object with the plotted individual results.

Method plot_global()

This method visualizes the results of the selected method summarized as boxplots/median image and enables a visual in-depth investigation of the global behavior with the help of the S4 classes innsight_ggplot2 and innsight_plotly.
You can use the argument output_idx to select the individual output nodes for the plot. For tabular and 1D data, boxplots are created in which a reference value can be selected from the data using the ref_data_idx argument. For images, only the pixel-wise median is visualized due to the complexity. The plot is generated using the ggplot2-based S4 class innsight_ggplot2. You can also use the as_plotly argument to generate an interactive plot with innsight_plotly based on the plot function plotly::plot_ly. For more information and the whole bunch of possibilities, see innsight_ggplot2 and innsight_plotly.

Notes:

1. This method can only be used for the local Connection Weights method, i.e., if times_input is TRUE and data is provided.

2. For the interactive plotly-based plots, the suggested package plotly is required.

3. The ggplot2-based plots for models with multiple input layers are a bit more complex, therefore the suggested packages 'grid', 'gridExtra' and 'gtable' must be installed in your R session.

Usage

InterpretingMethod$plot_global(
  output_idx = NULL,
  output_label = NULL,
  data_idx = "all",
  ref_data_idx = NULL,
  aggr_channels = "sum",
  preprocess_FUN = abs,
  as_plotly = FALSE,
  individual_data_idx = NULL,
  individual_max = 20
)

Arguments

Returns

Returns either an innsight_ggplot2 (as_plotly = FALSE) or an innsight_plotly (as_plotly = TRUE) object with the plotted summarized results.

Method print()

Print a summary of the method object. This summary contains the individual fields and in particular the results of the applied method.

Usage

InterpretingMethod$print()

Returns

Returns the method object invisibly via base::invisible.

Method clone()

The objects of this class are cloneable with this method.

Usage

InterpretingMethod$clone(deep = FALSE)

Arguments

Description

The R6 class LIME calculates the feature weights of a linear surrogate of the prediction model for a instance to be explained, namely the local interpretable model-agnostic explanations (LIME). It is a model-agnostic method that can be applied to any predictive model. This means, in particular, that LIME can be applied not only to objects of the Converter class but also to any other model. The only requirement is the argument pred_fun, which generates predictions with the model for given data. However, this function is pre-implemented for models created with nn_sequential, keras_model, neuralnet or Converter. Internally, the suggested package lime is utilized and applied to data.frame.

The R6 class can also be initialized using the run_lime function as a helper function so that no prior knowledge of R6 classes is required.

Note: Even signal and image data are initially transformed into a data.frame using as.data.frame() and then lime::lime and lime::explain are applied. In other words, a custom pred_fun may need to convert the data.frame back into an array as necessary.

Super classes

innsight::InterpretingMethod -> innsight::AgnosticWrapper -> LIME

Methods

Public methods

• LIME$new()

• LIME$clone()

Method new()

Create a new instance of the LIME R6 class. When initialized, the method LIME is applied to the given data and the results are stored in the field result.

Usage

LIME$new(
  model,
  data,
  data_ref,
  output_type = NULL,
  pred_fun = NULL,
  output_idx = NULL,
  output_label = NULL,
  channels_first = TRUE,
  input_dim = NULL,
  input_names = NULL,
  output_names = NULL,
  ...
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LIME$clone(deep = FALSE)

Arguments

See Also

Other methods: ConnectionWeights, DeepLift, DeepSHAP, ExpectedGradient, Gradient, IntegratedGradient, LRP, SHAP, SmoothGrad

Examples

#----------------------- Example 1: Torch -----------------------------------
library(torch)

# Create nn_sequential model and data
model <- nn_sequential(
  nn_linear(5, 12),
  nn_relu(),
  nn_linear(12, 2),
  nn_softmax(dim = 2)
  )
data <- torch_randn(25, 5)

# Calculate LIME for the first 10 instances and set the
# feature and outcome names
lime <- LIME$new(model, data[1:10, ], data_ref = data,
                 input_names = c("Car", "Cat", "Dog", "Plane", "Horse"),
                 output_names = c("Buy it!", "Don't buy it!"))

# You can also use the helper function `run_lime` for initializing
# an R6 LIME object
lime <- run_lime(model, data[1:10, ], data_ref = data,
                 input_names = c("Car", "Cat", "Dog", "Plane", "Horse"),
                 output_names = c("Buy it!", "Don't buy it!"))

# Get the result as an array for the first two instances
get_result(lime)[1:2,, ]

# Plot the result for both classes
plot(lime, output_idx = c(1, 2))

# Show the boxplot over all 10 instances
boxplot(lime, output_idx = c(1, 2))

# We can also forward some arguments to lime::explain, e.g. n_permutatuins
# to get more accurate values
lime <- run_lime(model, data[1:10, ], data_ref = data,
                 input_names = c("Car", "Cat", "Dog", "Plane", "Horse"),
                 output_names = c("Buy it!", "Don't buy it!"),
                 n_perturbations = 200)

# Plot the boxplots again
boxplot(lime, output_idx = c(1, 2))

#----------------------- Example 2: Converter object --------------------------
# We can do the same with an Converter object (all feature and outcome names
# will be extracted by the LIME method!)
conv <- convert(model,
                input_dim = c(5),
                input_names = c("Car", "Cat", "Dog", "Plane", "Horse"),
                output_names = c("Buy it!", "Don't buy it!"))

# Calculate LIME for the first 10 instances
lime <- run_lime(conv, data[1:10], data_ref = data, n_perturbations = 300)

# Plot the result for both classes
plot(lime, output_idx = c(1, 2))

#----------------------- Example 3: Other model -------------------------------
if (require("neuralnet") & require("ranger")) {
  library(neuralnet)
  library(ranger)
  data(iris)

  # Fit a random forest unsing the ranger package
  model <- ranger(Species ~ ., data = iris, probability = TRUE)

  # There is no pre-implemented predict function for ranger models, i.e.,
  # we have to define it ourselves.
  pred_fun <- function(newdata, ...) {
    predict(model, newdata, ...)$predictions
  }

  # Calculate LIME for the instances of index 1 and 111 and add
  # the outcome labels (for LIME, the output_type is required!)
  lime <- run_lime(model, iris[c(1, 111), -5],
                   data_ref = iris[, -5],
                   pred_fun = pred_fun,
                   output_type = "classification",
                   output_names = levels(iris$Species),
                   n_perturbations = 300)

  # Plot the result for the first two classes and all selected instances
  plot(lime, data_idx = 1:2, output_idx = 1:2)

  # Get the result as a torch_tensor
  get_result(lime, "torch_tensor")
}