---
title: "Functionality Guide"
description: >
  A detailed guide to isoreader2 functionality.
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Functionality Guide}
  %\VignetteEncoding{UTF-8}
  %\VignetteEngine{knitr::rmarkdown}
editor_options:
  chunk_output_type: console
---

> This step-by-step functionality guide explores the functions in the [package structure flowchart](https://isoreader2.isoverse.org/index.html#package-structure). All functions below labelled with an `*` are required steps of the standard data reading flow. Everything else is optional. Rarely used additional features that are mentioned here but not part of the standard flowchart are labeled as `bonus`.

```{r, include=FALSE}
# default chunk options
knitr::opts_chunk$set(collapse = FALSE, message = TRUE, comment = "")
```

```{r, message=FALSE}
# libraries
library(isoreader2) # load isoreader2 R package
library(dplyr) # for select syntax and mutating data frames
```

```{r, include=FALSE}
# copy the bundled example files (and any extraction cache) to a local
# folder so reading does not write into the installed package directory
if (!dir.exists("tmp_examples")) {
  dir.create("tmp_examples")
}
ir_examples_folder() |>
  list.files(full.names = TRUE) |>
  file.copy(to = "tmp_examples", overwrite = TRUE)
```

# Reading isotope files

The first step is finding and reading your isotope data files. isoreader2 supports a range of continuous flow, dual inlet, and scan file formats:

```{r}
# supported file types
ir_get_supported_file_types()
```

## `ir_find_isofiles()` *

Point isoreader2 at a folder to discover the data files inside it. The generic `ir_find_isofiles()` finds all supported types, while `ir_find_continuous_flow()`, `ir_find_dual_inlet()`, and `ir_find_scans()` narrow the search to specific measurement types. The bundled example files are available via `ir_examples_folder()`.

```{r}
# path to your data folder (here a local copy of the example files)
data_folder <- file.path("tmp_examples")

# find continuous flow files (.dxf/.cf) in the folder
file_paths <- data_folder |> ir_find_continuous_flow()

# show what was found
file_paths
```

## `ir_read_isofiles()` *

Read the discovered files. This returns an `ir_isofiles` object - a tibble with one row per file and the extracted datasets stored in nested columns.

```{r}
# read the files
isofiles <- file_paths |> ir_read_isofiles()

# show what was read
isofiles
```

### bonus combine collections with `c()`

Multiple `ir_isofiles` collections can be combined into one with a simple `c()`, which row-binds them while preserving the object type.

```{r}
# combine two collections (here just the same files twice for illustration)
c(isofiles, isofiles)
```

### bonus `ir_save_isofiles()` / `ir_load_isofiles()`

You can store an entire `ir_isofiles` collection to disk (as an RDS file) and read it back exactly as it was, without re-reading the original data files.

```{r}
# save and reload the read isofiles
isofiles |> ir_save_isofiles(file.path("tmp_examples", "my_isofiles"))
reloaded <- ir_load_isofiles(file.path("tmp_examples", "my_isofiles"))
```

# Aggregating data

The nested `ir_isofiles` structure is convenient for reading but not for analysis. Aggregation pulls the data together into a tidy set of data frames (metadata, traces, cycles, scans, resistors) that are easy to work with.

## `ir_aggregate_isofiles()` *

```{r}
# aggregate the data
dataset <- isofiles |> ir_aggregate_isofiles()

# show all that was recovered
# (as well as what was ignored / not aggregated)
dataset
```

### bonus intensity units

By default the raw signals are reported in millivolts (`mV`). Pass a different
`intensity_units` to report voltage (`mV`, `V`), current (`fA`, `pA`, `nA`,
`µA`, `mA`, `A`), or counts per second (`cps`) instead. Conversions that cross
the voltage/current/cps boundary use the recorded resistor values automatically
(the same machinery is exposed as `ir_convert_intensity()`).

```{r}
# report intensities in nanoamperes instead of the default millivolts
isofiles |>
  ir_aggregate_isofiles(intensity_units = "nA") |>
  ir_get_data(traces = c("species", "mass", "time.s", "intensity.nA"))
```

### bonus `ir_get_aggregator()`

You can optionally use a different aggregator. Besides the default `standard` aggregator, the package ships (from smallest to largest) the `metadata` aggregator (metadata only), the `minimal` aggregator (metadata plus the core data series), and the `extended` aggregator, which is more elaborate and provides access to additional columns (such as the resistor/cup configuration and the vendor data table) from the data files.

```{r}
# minimal vs. extended aggregator
ir_get_aggregator("minimal")
ir_get_aggregator("extended")

# using the extended aggregator instead of the default (standard)
isofiles |> ir_aggregate_isofiles(aggregator = "extended")
```

### bonus `ir_register_aggregator()`

Or build your own aggregator with `ir_start_aggregator()` and/or expand an existing one with `ir_add_to_aggregator()`, then register it via `ir_register_aggregator()`. This functionality is rarely needed and thus not part of the package structure flowchart.

```{r}
my_agg <-
  ir_get_aggregator("minimal") |>
  # pull the "Identifier 1" metadata field out under a friendlier name
  ir_add_to_aggregator("metadata", "sample_id", source = "Identifier 1") |>
  ir_register_aggregator(name = "my_aggregator")

# show my aggregator summary
my_agg

# use it
isofiles |> ir_aggregate_isofiles(aggregator = "my_aggregator")
```

### bonus `ir_get_problems()` / `ir_show_problems()`

Reading and aggregation are designed to be fail-safe: instead of stopping at the first error, problems are collected and reported. There were no problems with these example files so the result is empty, but this can be very helpful for figuring out what went wrong.

`ir_get_problems()` returns the problems as a data frame for further inspection:

```{r}
isofiles |> ir_get_problems()
dataset |> ir_get_problems()
```

`ir_show_problems()` instead just prints out all the problems directly:

```{r}
isofiles |> ir_show_problems()
dataset |> ir_show_problems()
```

# Accessing the data

## `ir_get_data()`

At any point you can pull the data of interest out of the aggregated dataset with `ir_get_data()`, selecting columns from each dataset using `dplyr::select()` syntax. Columns selected from more than one dataset are automatically combined with a join.

```{r}
# direct access to the individual datasets
dataset$metadata
dataset$traces

# retrieve + combine data with dplyr select syntax
dataset |>
  ir_get_data(
    metadata = c("file_name", "analysis", sample = "Identifier 1"),
    traces = c("species", "mass", "time.s", "intensity.mV")
  )
```

### shortcuts `ir_get_metadata()` / `ir_get_traces()` / ...

For the common case of grabbing a whole dataset, the shortcut functions `ir_get_metadata()`, `ir_get_traces()`, `ir_get_cycles()`, `ir_get_scans()`, and `ir_get_resistors()` retrieve all columns of the respective dataset.

```{r}
# all metadata columns
dataset |> ir_get_metadata()

# all traces (joined with the file metadata)
dataset |> ir_get_traces()
```

### bonus `ir_get_vendor_data_table()`

Some file formats also store the vendor's own computed results table (the table Isodat shows in its results view: per-peak or per-cycle intensities, retention times, ratios, deltas, etc.). This `vendor_data_table` is **only available from a few formats** — the isodat `.dxf`, `.cf`, `.did`, and `.caf` files — and it is **only aggregated by the `"extended"` aggregator** (it is dropped by the default `"standard"` aggregator as well as the `"minimal"` and `"metadata"` aggregators). Its columns are kept under their original names (with units, e.g. `"Rt [s]"`), numeric by default with the rare string/integer columns (e.g. `"Ref. Name"`, `"Nr."`) kept as-is.

```{r}
# aggregate with the extended aggregator so the vendor data table is included
dataset_ext <- isofiles |> ir_aggregate_isofiles(aggregator = "extended")

# retrieve the vendor data table (joined with the file metadata)
dataset_ext |> ir_get_vendor_data_table()
```

## `ir_filter_metadata()` / `ir_mutate_metadata()` / `ir_join_metadata()`

You can filter, add to, or join into the metadata while keeping the rest of the datasets consistent. `ir_filter_metadata()` cascades the filter to all other datasets so they always stay in sync.

```{r}
# keep only continuous flow files and add a derived label column
dataset |>
  ir_filter_metadata(type == "cf") |>
  ir_mutate_metadata(label = paste(file_name, analysis, sep = " / ")) |>
  ir_get_metadata(metadata = c("file_name", "analysis", "label"))
```

These functions also work directly on an unaggregated `ir_isofiles` object (applied individually to each file), though that is significantly slower than working on an aggregated dataset.

### bonus `ir_filter_for_continuous_flow()` / `ir_filter_for_dual_inlet()` / `ir_filter_for_scans()`

If a dataset mixes measurement types, these convenience filters keep only the files of one type (filtering on the metadata `type` field) and cascade to all datasets, just like `ir_filter_metadata()`.

```{r}
# keep only the continuous flow files
dataset |>
  ir_filter_for_continuous_flow() |>
  ir_get_metadata(metadata = c("file_name", "type"))
```

## bonus `ir_save_aggregated_data()` / `ir_load_aggregated_data()`

Aggregated data can be stored to (and loaded from) a parquet file for fast, language-independent access. This requires the suggested **arrow** package.

```{r, eval=FALSE}
# save and reload the aggregated data
dataset |> ir_save_aggregated_data(file.path("tmp_examples", "my_dataset"))
reloaded <- ir_load_aggregated_data(file.path("tmp_examples", "my_dataset"))
```

# Calculating ratios

## `ir_calculate_ratios()`

Isotope work is all about *ratios*. `ir_calculate_ratios()` adds two columns - `ratio_name` (e.g. `"45/44"`) and `ratio` - to the `traces`, `cycles`, and/or `scans` of an aggregated dataset. By default each mass is divided by the numerically lowest ("base") mass of the same species at each time/cycle/scan position.

```{r}
dataset_ratios <- dataset |> ir_calculate_ratios()

dataset_ratios |>
  ir_get_data(
    metadata = "file_name",
    traces = c("species", "mass", "time.s", "ratio_name", "ratio")
  )
```

### bonus base mass, additive offsets, and normalization

The base mass can be overridden per species by name (e.g. `N2 = 28`, `CO2 = 44`). For continuous flow (`traces`) and `scans` data a small additive offset is added to numerator and denominator before dividing, which stabilizes the ratio where the signal is near the baseline; it is configurable per unit family (`num_add.V`/`denom_add.V` for voltage, `num_add.nA`/`denom_add.nA` for current, `num_add.cps`/`denom_add.cps` for counts) and is always `0` for dual inlet `cycles`. Finally, `normalize_ratios` can divide each ratio by a per-file summary function such as `mean`, `median`, `min`, or `max`.

```{r}
dataset |>
  ir_calculate_ratios(N2 = 28, normalize_ratios = median) |>
  ir_get_data(
    metadata = "file_name",
    traces = c("species", "mass", "ratio_name", "ratio")
  )
```

# Visualizing data

isoreader2 provides quick-look plotting functions for each measurement type. They operate on aggregated data and return regular `ggplot` objects that you can further customize. To add your own ggplot2 layers (e.g. `+ labs(...)` or `+ theme(...)`), attach ggplot2 with `library(ggplot2)` first.

## `ir_plot_continuous_flow()`

```{r}
#| fig.width: 8
#| fig.height: 5
dataset |> ir_plot_continuous_flow(facet = file_name)
```

Narrow the plot down with `species`/`mass` and zoom in with a time window given
either in seconds (`time_window.s`) or minutes (`time_window.min`):

```{r}
#| fig.width: 8
#| fig.height: 5
dataset |>
  ir_plot_continuous_flow(
    species = "CO2",
    facet = file_name,
    time_window.min = c(4.5, 8.5)
  )
```

Add calculated `ratio`s to the plot. When more than one data type is present
(intensities and ratios), they are automatically split into facet rows
(`data_type_as_facet = auto()`):

```{r}
#| fig.width: 8
#| fig.height: 6
dataset |>
  ir_calculate_ratios(normalize_ratios = median) |>
  ir_plot_continuous_flow(
    species = "CO2",
    ratio = "45/44",
    facet = file_name,
    time_window.min = c(4.5, 8.5)
  )
```

The plots are ordinary `ggplot` objects (the package theme `ir_default_theme()`
is always applied), so you can keep customizing with ggplot2 layers - and tune
the aesthetics with arguments like `color`, `scientific`, `linetype`, or
`color_values`:

```{r}
#| fig.width: 8
#| fig.height: 5
library(ggplot2)
dataset |>
  ir_plot_continuous_flow(
    species = "N2",
    facet = file_name,
    scientific = TRUE
  ) +
  labs(title = "N2 traces") +
  theme(legend.position = "bottom")
```

### bonus `ir_generate_traces_tibble()` / `ir_generate_cycles_tibble()` / `ir_generate_scans_tibble()`

If you want the exact tibble that a plot would draw (e.g. to build your own plot or do further analysis), the `ir_generate_*_tibble()` helpers return it without plotting. They take the same `species`/`mass`/`ratio` arguments and add the `trace`, `data_type`, and `value` columns the plotting functions use.

```{r}
dataset |> ir_generate_traces_tibble(species = "CO2")
```

## `ir_plot_dual_inlet()`

```{r}
#| fig.width: 8
#| fig.height: 7
di_dataset <-
  data_folder |>
  ir_find_dual_inlet() |>
  ir_read_isofiles() |>
  ir_aggregate_isofiles("V") |>
  ir_calculate_ratios()

di_dataset |> ir_plot_dual_inlet(facet = file_name)
```

The same `species`/`mass`/`ratio` filters apply, and `cycle_window` zooms to a
range of cycles:

```{r}
#| fig.width: 8
#| fig.height: 5
di_dataset |> ir_plot_dual_inlet(mass = c(44, 45, 46), facet = file_name)
```

## `ir_plot_scans()`

For scan files with more than one scan type, specify which one to plot
(`x_window` zooms the x axis the same way `time_window.s` does for continuous
flow):

```{r}
#| fig.width: 8
#| fig.height: 6
data_folder |>
  ir_find_scans() |>
  ir_read_isofiles() |>
  ir_aggregate_isofiles("V") |>
  ir_plot_scans(scan_type = "high voltage")
```

# Exporting data

## `ir_export_to_excel()`

Finally, you can export individual data frames (typically retrieved with the `ir_get_*()` functions) to an Excel file, one sheet per data frame. Named arguments set the sheet names; unnamed ones use `"Sheet{position}"`. This requires the suggested **openxlsx** package.

```{r}
ir_export_to_excel(
  metadata = dataset |> ir_get_metadata(),
  traces = dataset |> ir_get_traces(),
  file = "tmp_examples/my_dataset.xlsx"
)
```

# Package options

## bonus `ir_options()` / `ir_get_option()`

A handful of package-wide options control isoreader2's behavior. `ir_get_options()` lists them, `ir_get_option()` reads one, and `ir_options()` sets them. For example, `debug = TRUE` disables the fail-safe error catching so problems surface with a full traceback, which is useful when troubleshooting a reader.

```{r}
# list available options and read one
names(ir_get_options())
ir_get_option("debug")

# set an option (here enable debug mode), then restore the default
invisible(ir_options(debug = TRUE))
ir_get_option("debug")
```

```{r, include=FALSE}
# reset options
invisible(ir_options(debug = FALSE))
# clean up the local data folder created for this vignette
unlink("tmp_examples", recursive = TRUE)
```