CytobankAPI advanced analysis guide


The CytobankAPI package is designed to make interacting with Cytobank API endpoints easy via R. This document is an accompanying overview of the package to learn concepts and see basic examples. View the Cytobank API Endpoint Documentation for a comprehensive list of API endpoints for Cytobank.

Within the CytobankAPI package, there are endpoints to interact with advanced analyses via R. This documentation is an overview of the different ways to utilize advanced analyses. To find more general documentation on using the CytobankAPI package, view the Cytobank quickstart guide.

All advanced analyses are encapsulated within an object. This guide will be an overview of advanced analyses object structures:

  1. Advanced Analyses Objects: What are advanced analyses objects
  2. Interactions: How to interact with the advanced analyses objects

Advanced Analyses Objects


Representation


Every advanced analysis is represented as an object. Creating a new advanced analysis will return an object that is passed to all of their other respective advanced analysis endpoints.

Important information to note:

  1. Each advanced analysis object returned can be edited directly
  2. Each advanced analysis object is a representation, but does not necessarily mean it is the same as what is seen in the GUI. In order to get the most current settings, utilize the show endpoint, or update the advanced analysis with the current settings using the update endpoint
  3. When the update method is called on the advanced analysis object, the existing record for the run on the Cytobank is overwritten according to the settings in the object
viSNE_analysis <- visne.show(cyto_session, experiment_id=22, visne_id=214)
viSNE_analysis@name
#>  [1] "My viSNE analysis example"

# Update the viSNE analysis object name directly
viSNE_analysis@name <- "My updated viSNE analysis name"
# Update the viSNE analysis using the 'visne.update' endpoint
updated_viSNE <- visne.update(cyto_session, viSNE_analysis)
updated_viSNE@name
#>  [1] "My updated viSNE analysis name"

Common features


There are common features for all advanced analyses:

  1. Name: The name of the advanced analysis

  2. Compensation ID: The compensation ID used for the advanced analysis

    • uncompensated compensation_id = -1
    • file-internal compensation_id = -2
    • external compensation matrix compensation_id listed in compensations.list
  3. Channels: The channels being analyzed within the algorithm (clustering or for general analysis)

  4. Source experiment: The experiment the advanced analysis belongs to (all advanced analyses belong to an experiment)

  5. Status: The state of the advanced analysis (new, running, done, canceled, etc.)

  6. Available FCS files: FCS files available for the advanced analysis

  7. Available channels: Channels available for the advanced analysis

  8. Available populations: Populations available for the advanced analysis

Unique features for each advanced analysis method


There are special settings that pertain to each advanced analysis algorithm. These settings affect how the advanced analysis algorithm is ran. For each advanced analysis, you can view their respective settings and slots as shown below.

CITRUS_object <- citrus.new(cyto_session, experiment_id, citrus_name="My new Cytobank CITRUS analysis")
slotNames(CITRUS_object)
#>  [1] "citrus_id"                "population_id"           
#>  [3] "file_grouping"            "association_models"      
#>  [5] "cluster_characterization" "statistics_channels"     
#>  [7] "event_sampling_method"    "events_per_file"         
#>  [9] "minimum_cluster_size"     "cross_validation_folds"  
#> [11] "false_discovery_rate"     "normalize_scales"        
#> [13] "plot_theme"               "attachment_id"           
#> [15] "channels"                 "compensation_id"         
#> [17] "name"                     "source_experiment"       
#> [19] "status"                   ".available_channels"     
#> [21] ".available_files"         ".available_populations"

Learn more about CITRUS settings.

FlowSOM_object <- flowsom.new(cyto_session, experiment_id, flowsom_name="My new Cytobank FlowSOM analysis")
slotNames(FlowSOM_object)
#>  [1] "author"                                 
#>  [2] "type"                                   
#>  [3] "flowsom_id"                             
#>  [4] "selected_population_name"               
#>  [5] "population_id"                          
#>  [6] "num_fcs_files"                          
#>  [7] "fcs_files"                              
#>  [8] "event_sampling_method"                  
#>  [9] "desired_events_per_file"                
#> [10] "desired_total_events"                   
#> [11] "sampled_event_total"                    
#> [12] "num_events_to_actually_sample"          
#> [13] "random_seed"                            
#> [14] "som_creation_method"                    
#> [15] "clustering_method"                      
#> [16] "expected_metaclusters"                  
#> [17] "expected_clusters"                      
#> [18] "iterations"                             
#> [19] "normalize_scales"                       
#> [20] "created_experiment"                     
#> [21] "attachment_id"                          
#> [22] "auto_seed"                              
#> [23] "external_som_analysis_info"             
#> [24] "external_som_analysis_id"               
#> [25] "channels_to_plot"                       
#> [26] "cluster_size_type"                      
#> [27] "fixed_cluster_size"                     
#> [28] "gate_set_names_to_label"                
#> [29] "max_relative_cluster_size"              
#> [30] "output_file_type"                       
#> [31] "show_background_on_legend"              
#> [32] "show_background_on_channel_colored_msts"
#> [33] "show_background_on_population_pies"     
#> [34] "final_result"                           
#> [35] "completed"                              
#> [36] "canceled"                               
#> [37] "channels"                               
#> [38] "compensation_id"                        
#> [39] "name"                                   
#> [40] "source_experiment"                      
#> [41] "status"                                 
#> [42] ".available_channels"                    
#> [43] ".available_files"                       
#> [44] ".available_populations"

Learn more about FlowSOM settings.

SPADE_object <- spade.new(cyto_session, experiment_id=22, spade_name="My new Cytobank SPADE analysis")
slotNames(SPADE_object)
#>  [1] "created_experiment"         "down_sampled_events_target"
#>  [3] "down_sampled_events_type"   "fold_change_groups"        
#>  [5] "population_id"              "spade_id"                  
#>  [7] "target_number_nodes"        "channels"                  
#>  [9] "compensation_id"            "name"                      
#> [11] "source_experiment"          "status"                    
#> [13] ".available_channels"        ".available_files"          
#> [15] ".available_populations"

Learn more about SPADE settings.

viSNE_object <- visne.new(cyto_session, experiment_id, visne_name="My new Cytobank viSNE analysis")
slotNames(viSNE_object)
#>  [1] "iterations"             "perplexity"             "channels"              
#>  [4] "compensation_id"        "population_selections"  "sampling_total_count"  
#>  [7] "sampling_target_type"   "seed"                   "theta"                 
#> [10] "visne_id"               "analysis_id"            "type"                  
#> [13] "name"                   "status"                 "source_experiment"     
#> [16] "created_experiment"     ".available_channels"    ".available_files"      
#> [19] ".available_populations"

Learn more about viSNE settings.

slotNames(dimensionality_reduction_object)
#> [1] "analysis_id"            "type"                   "name"                  
#> [4] "status"                 "source_experiment"      "created_experiment"    
#> [7] ".available_channels"    ".available_files"       ".available_populations"

Learn more about dimensionality reduction settings.

Interacting with advanced analyses objects


See each section below for instructions on how to interact with the object for each advanced analysis.

CITRUS


Updating general CITRUS settings

Directly update CITRUS settings via their slot names.

The following slots can be updated directly:

  • Required settings:

  • Optional settings (defaults in parentheses):

    • compensation_id: (experiment-wide compensation)
    • association_models: (“pamr”), sam”, and/or “glmnet”
    • cluster_characterization: (“abundances”) or “medians”
    • event_sampling_method: (“equal”) or “max-per-file”
    • events_per_file (5000 or the number in the smallest file, if less than 5000)
    • minimum_cluster_size (5)
    • cross_validation_folds (5)
    • false_discovery_rate (1)
    • normalize_scales: TRUE or (FALSE)
    • plot_theme: (“white”) or “black”
# Set association models, plot theme and compensation
CITRUS_object@association_models <- c("sam", "pamr", "glmnet")
CITRUS_object@plot_theme <- "black"
CITRUS_object@compensation_id <- 22

# Bulk update the changes made to the CITRUS object
CITRUS_object <- citrus.update(cyto_session, CITRUS_object)

Updating CITRUS file grouping


The core functionality of CITRUS is establishing biological explanations for why samples between two or more groups differ from each other. CITRUS file grouping is used to categorize different files into these groups. There is 1 important setting to pay attention to:

  • group_name: the group that each file is associated with

    • The minimum number of samples per group is three. However, for more robust statistical analysis and to avoid spurious results, at least eight samples are recommended per group (see documentation for more information here)

    • There must be at least 2 groups in order to run a CITRUS analysis

Directly update CITRUS file grouping data.

# Set 'file1.fcs' through 'file4.fcs' to 'Group 1' and 'file5.fcs' through 'file8.fcs' to 'Group 2', exclude 'file9.fcs' from analysis
CITRUS_object@file_grouping[CITRUS_object@file_grouping$id <= 44856,]$group_name <- "Group 1"
CITRUS_object@file_grouping[is.element(c(44857, 44858, 44859, 44860), CITRUS_object@file_grouping$id),]$group_name <- "Group 2"

View(CITRUS_object@file_grouping)
id name group_name
44853 file1.fcs Group 1
44854 file2.fcs Group 1
44855 file3.fcs Group 1
44856 file4.fcs Group 1
44857 file5.fcs Group 2
44858 file6.fcs Group 2
44859 file7.fcs Group 2
44860 file8.fcs Group 2
44861 file9.fcs Unassigned

Learn more about CITRUS file grouping.

FlowSOM


Updating general FlowSOM settings

Directly update FlowSOM settings via their slot names.

The following slots can be updated directly:

  • Basic settings:
    • population_id: gateSetId from [email protected]_populations
    • channels: can be set as a list of channel IDs or a list of channel long names (long names must correlate to a unique short channel name)
    • fcs_files
    • random_seed
    • compensation_id: must match experiment-wide compensation
  • Event sampling settings:
    • event_sampling_method: “equal”, “proportional”, or “all”
    • desired_events_per_file: for equal sampling
    • desired_total_events: for proportional sampling
  • Algorithm settings (default in parentheses):
    • som_creation_method: (“create_new”) or “import_existing”
      • external_som_analysis_id: if importing existing SOM, character of source FlowSOM ID
    • clustering_method: (“consensus”), “hclust”, or “kmeans”
    • expected_metaclusters (10)
    • expected_clusters (100)
    • iterations (10)
  • Transformations settings (default in parentheses):
    • normalize_scales: TRUE or (FALSE)
  • PDF output settings (default in parentheses):
    • channels_to_plot: channels can be set as a list of channel IDs or a list of channel long names (long names must correlate to a unique short channel name)
    • cluster_size_type: (“relative”), “fixed” or “both”
    • fixed_cluster_size (8)
    • max_relative_cluster_size (15)
    • gate_set_names_to_label: populations for pie charts
    • output_file_type: (“pdf”), “png” or “both”
    • Toggle metacluster background on plots: TRUE or FALSE
      • show_background_on_legend: (FALSE)
      • show_background_on_channel_colored_msts (FALSE)
      • show_background_on_population_pies (TRUE)

If the required channels, fcs_files and random_seed slots are not present, updates will not occur to the FlowSOM analysis.

# Set a clustering method and target number of nodes
FlowSOM_object@clustering_method <- "kmeans"
FlowSOM_object@num_expected_clusters <- 144

# Update FCS file selection to the first 5 files
FlowSOM_object@fcs_files <- FlowSOM_object@.available_files$id[1:5]

# Update channel selection
FlowSOM_object@channels <- list("CD3", "CD4")

# Bulk update the changes made to the FlowSOM object
FlowSOM_object <- flowsom.update(cyto_session, FlowSOM_object)

SPADE


Updating general SPADE settings

Directly update SPADE settings via their slot names.

The following slots can be updated directly:

  • Required settings:
    • population_id
    • channels: channels can be set as a list of channel IDs or a list of channel long names (long names must correlate to a unique short channel name)
  • Additional settings if changing from default (in parentheses):
# Set a new population, target number of nodes, and compensation
SPADE_object@population_id <- 2
SPADE_object@target_number_nodes <- 150
SPADE_object@compensation_id <- 22

# Update channels
channel_ids_list <- list(2, 3, 5, 8)
SPADE_object@channels <- channel_ids_list

# Update channels by long channel names
channel_names_list <- list("channel1", "channel2", "channel3")
SPADE_object@channels <- channel_names_list

# Bulk update the changes made to the SPADE object
SPADE_object <- spade.update(cyto_session, SPADE_object)

SPADE_object@population_id
#> [1] 2
SPADE_object@target_number_nodes
#> [1] 150
SPADE_object@compensation_id
#> [1] 22
SPADE_object@channels
#> [[1]]
#> [1] "channel1"
#>
#> [[2]]
#> ...

Updating SPADE fold change groups


SPADE fold change groups are used to categorize different files into separate collections that will be compared amongst each other. There are 2 important settings to pay attention to:

  1. group_name: The group a specific file belongs to
  2. baseline: The file(s) used as the baseline in order to calculate fold change

Directly update SPADE fold change groups data.

# Set 'file6.fcs' and 'file7.fcs' as the baseline for 'Group 1'
SPADE_object@fold_change_groups[grep("my_file6|my_file7", 
    SPADE_object@fold_change_groups$name),]$baseline <- TRUE

# Set 'file2.fcs', 'file4.fcs', and 'file8.fcs' as part of 'Group 2', and set 'file2.fcs' as the baseline
SPADE_object@fold_change_groups[grep("file2|file4|file8", 
    SPADE_object@fold_change_groups$name),]$group_name <- "Group 2"
SPADE_object@fold_change_groups[SPADE_object@fold_change_groups$name=="file2.fcs",]$baseline <- TRUE

View(SPADE_object@fold_change_groups)
id name baseline group_name
44853 file1.fcs FALSE Group 1
44854 file2.fcs TRUE Group 2
44855 file3.fcs FALSE Group 1
44856 file4.fcs FALSE Group 2
44857 file5.fcs FALSE Group 1
44858 file6.fcs TRUE Group 1
44859 file7.fcs TRUE Group 1
44860 file8.fcs FALSE Group 2

Learn more about SPADE fold change groups.

viSNE


Updating general viSNE settings

Directly update viSNE settings via their slot names.

The following slots can be updated directly:

  • sampling_target_type

  • sampling_total_count

  • channels

    • channels can be set as a list of channel IDs or a list of channel long names (long names must correlate to a unique short channel name)
  • compensation_id

  • iterations

  • perplexity

  • theta

  • seed

The following slots must be updated via helper functions:

# Set a new sampling target type, sampling total count, and compensation
viSNE_object@sampling_target_type <- "equal"
viSNE_object@sampling_total_count <- 150000
viSNE_object@compensation_id <- 22

# Bulk update the changes made to the viSNE object
viSNE_object <- visne.update(cyto_session, viSNE_object)

Updating viSNE population selections


Adding viSNE population selections is slightly more difficult because the same file can be used in the analysis in combination with multiple populations. Because of this complexity, the visne.helper.set_populations helper function is used to set files for a selected population.

Parameters for visne.helper.set_populations:

  1. visne: The viSNE object to set populations for
  2. population_id: The gate set ID for the specified population (different than the actual population ID, and can be obtained by looking at the .available_populations slot)
  3. fcs_files: A vector/list of FCS files to set for the population

Set files for a specific population through the visne.helper.set_populations helper function.

Setting files for a specific population will overwrite the files previously set for the population in question.

# Set files for different populations
viSNE_object <- visne.helper.set_populations(viSNE_object, population_id=1, fcs_files=c(44853))
viSNE_object <- visne.helper.set_populations(viSNE_object, population_id=2, fcs_files=c(44867,44868))
viSNE_object <- visne.helper.set_populations(viSNE_object, population_id=4, fcs_files=unlist(visne@.available_files[grep("file4|file5|file6", visne@.available_files$filename),]$id))
# Overwrite 'population_id=2' FCS file selection, note that 'file1.fcs' and 'file2.fcs' are in both 'Population 1', as well as 'Population 2'
viSNE_object <- visne.helper.set_populations(viSNE_object, population_id=2, fcs_files=c(44854,44855, 44853, 44867))

# Update the changes made to viSNE population selections
viSNE_object <- visne.update(cyto_session, viSNE_object)

View(viSNE_object@population_selections)
id name samplingCount eventCount populationId populationName
44853 file1.fcs NA NA 1 Population 1
44856 file4.fcs NA NA 4 Population 4
44857 file5.fcs NA NA 4 Population 4
44858 file6.fcs NA NA 4 Population 4
44854 file2.fcs NA NA 2 Population 2
44855 file3.fcs NA NA 2 Population 2
44853 file1.fcs NA NA 2 Population 2
44856 file4.fcs NA NA 2 Population 2

Learn more about selecting viSNE populations.

Dimensionality Reduction Suite


Setting up a new analysis run using one of the algorithms that are available in Dimensionality Reduction Suite

The following example shows how to set up and launch a tSNE-CUDA analysis run.


# create a new dimensionality reduction analysis object and set the analysis type to be tSNE-CUDA
dimensionality_reduction_object<-dimensionality_reduction.new(cyto_session, experiment_id=22, analysis_name = 'use API to launch a tSNEcuda run', analysis_type = 'tSNE-CUDA')

# select clustering channels
dimensionality_reduction_object@clustering_channels<-list('CD4','CD8','CD3')

# Bulk update the change
dimensionality_reduction_object<-dimensionality_reduction.update(cyto_session,dimensionality_reduction_object)

# Launch the configured analysis run
dimensionality_reduction.run(cyto_session,dimensionality_reduction_object)

# Check the status of the launched analysis run
dimensionality_reduction.status(cyto_session,cyto_tsnecuda)

Updating general settings for the algorithms in Dimensionality Reduction Suite

Directly update settings via their slot names.

The following slots can be updated directly:

  • tSNE-CUDA, opt-SNE and UMAP:
    • event_sampling_method
    • desired_total_events
    • clustering_channels
      • channels can be set as a list of channel IDs or a list of channel long names (long names must correlate to a unique short channel name)
      • channel names can be obtained by looking at the .available_channels slot
    • normalize_scales
  • Additional slots for tSNE-CUDA:
    • perplexity
    • iterations
    • auto_iterations
    • learning_rate
    • auto_learning_rate
    • early_exaggeration
  • Additional slots for opt-SNE:
    • perplexity
    • auto_learning_rate
    • learning_rate
    • early_exaggeration
    • max_iterations
    • random_seed
  • Additional slots for UMAP:
    • collapse_outliers
    • min_distance
    • num_neighbors
# Set a new sampling target type, sampling total count, and compensation
dimensionality_reduction_object@event_sampling_method <- "equal"
dimensionality_reduction_object@desired_total_events <- 150000

# Bulk update the changes made to the dimensionality reduction object
dimensionality_reduction_object <- dimensionality_reduction.update(cyto_session, dimensionality_reduction_object)