---
title: "aidR: Tandem Source export format"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{aidR: Tandem Source export format}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---
```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```
```{r setup, message = FALSE}
library(aidR)
```
```{r, include=FALSE}
path <- system.file("extdata", "tandem_source_example.zip", package = "aidR")
tmpdir <- file.path(tempdir(), "tandem_source_example")
unzip(path, exdir = tmpdir)
files <- list.files(tmpdir, recursive = TRUE)
```
## Overview

A Tandem Source export is a folder (possibly zipped) containing CSV files for several data types. Filenames combine the participant, serial number, and export date with a data-type suffix, for example:

```{r, comment = "", echo = FALSE}
files
```

Each file records the device type (e.g. `Tandem Mobi™ System`) and serial number on every row:
```{r, comment = "", echo = FALSE}
writeLines(readLines(file.path(tmpdir, files[4]), n = 10))
```

## How aidR detects this format

Because the headers in a Tandem Source export are language-specific, `aidR` cannot identify a file from its header names. It relies instead on the *structure* of the file, and distinguishes between data types by the expected number of columns, which columns are numeric, and, where available, a language-independent keyword such as `"Tandem"` or `"t:slim X2"` for the files containing CGM, SMBG and basal rates, as well as `"Bolus"` for the file containing the bolus data.

## Files and the data types they produce

To inspect the raw files, we parse the export with `clean = FALSE`, which returns the full information provided by Tandem Source:
```{r}
data <- parse_data(id = "1", paths = path, clean = FALSE)
```
Each entry below describes one raw file and shows an example.

- **`*-CGM.csv`**: CGM data.
```{r, comment=""}
head(data$cgm)
```
- **`*-Basal-doses.csv`**: the basal insulin dose commanded at each event. Tandem reports doses rather than rates and gives no explicit duration, so `aidR` derives the duration of each entry from the gap to the following one and converts the dose into a rate in U/h.
```{r, comment=""}
head(data$basal)
```
- **`*-Bolus.csv`**: insulin boluses, including the bolus type and delivery method, the entered carbohydrate amount, the food and correction components, and the completion status.
```{r, comment=""}
head(data$bolus)
```
- **`*-BG.csv`**: self-monitored blood glucose (SMBG).
```{r, comment=""}
head(data$SMBG)
```
- **`*-Hourly-basal-delivery.csv`**: basal insulin aggregated per hour, with one column per hour of the day (the hours are named either `12 AM` ... `11 PM` or `00` ... `23`, depending on the export). This aggregated file is not used to build the cleaned `basal`, which is derived from `*-Basal-doses.csv` instead; it is used for `total_basal`.
```{r, comment=""}
head(data$hourly_basal)
```
## Cleaning files

When `parse_data` is run with `clean = TRUE`, the cleaned data types are built from the following raw files and attributes:

Cleaned data type | Used files                      | Used attributes from `clean = FALSE` |
|-----------------|---------------------------------|-----------------|
| cgm             | *-CGM.csv                       | `data$cgm`        |
| basal           | *-Basal-doses.csv               | `data$basal`      |
| bolus           | *-Bolus.csv                     | `data$bolus`      |
| carbs           | *-Bolus.csv                     | `data$bolus`      |
| SMBG            | *-BG.csv                        | `data$bg`         |
| total_basal     | *-Hourly-basal-delivery.csv     | `data$hourly_basal` |
| total_bolus     | *-Bolus.csv                     | `data$bolus`      |

## Notes

- CGM and SMBG values are reported in mg/dL or mmol/L depending on the account settings; `aidR` converts them to mg/dL automatically.
- `timezone_offset` is always `NA`, since Tandem Source exports report only local (wall-clock) time.
- The device type is stored as `pump_name` for insulin data and `sensor_name` for glucose data.
- The hourly basal file reports the basal insulin delivered in each hour of the day, one row per day and one column per hour. `aidR` sums the 24 hourly columns into `total_basal`, which therefore carries `source = "reported"`. `total_bolus`, in contrast, is summed from the cleaned `bolus` data and carries `source = "summed"`.
- Because basal durations are inferred from consecutive timestamps, the final basal entry of an export has no successor and its duration is therefore unknown.
- Each row of the bolus file holds a bolus together with the carbohydrates entered for it, so `bolus` and `carbs` are linked by `link_id`. Boluses without carbohydrates have `link_id = NA`.