---
title: "Manifests and safe downloads with nemoR"
author: "Maciej Pietrzak"
date: "`r Sys.Date()`"
output:
  rmarkdown::html_vignette:
    toc: true
    toc_depth: 2
vignette: >
  %\VignetteIndexEntry{2. Manifests and safe downloads with nemoR}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

## Introduction

The central reproducibility object in `nemoR` is a manifest. A manifest is a
table that records which NeMO files were selected, where they can be
downloaded, where they should live locally, and what happened during download.
This vignette demonstrates the manifest and safety layers without downloading
large external data.

`nemoR` complements Bioconductor workflows by making the data-access step
explicit before data are loaded into objects such as `SingleCellExperiment`.
The loading helpers are deliberately optional because different NeMO file
formats require different Bioconductor or CRAN readers.

## Installation

```{r install-cran, eval=FALSE}
install.packages("nemoR")
```

```{r load}
library(nemoR)
```

## Create a small manifest

The manifest below uses example URLs, so it is suitable for documentation and
tests. A real NeMO manifest can be created with `nemo_search_manifest()` or
`nemo_fetch(dry_run = TRUE)`.

```{r make-manifest}
manifest <- nemo_manifest_from_urls(
  urls = c(
    "https://example.com?file=processed_counts.h5ad",
    "https://example.com?file=parameters.json"
  ),
  collection_id = "example-collection",
  file_id = c("example-counts", "example-parameters")
)

manifest$data_type <- c("counts", "parameters")
manifest$size <- c(5 * 1024^2, 2048)
manifest
```

## Validate and summarize

Validation checks that the minimum columns required by the download and loading
workflow are present.

```{r validate}
nemo_validate_manifest(manifest)
nemo_download_plan(manifest, max_size_gb = 1)
```

## Understand download status columns

`nemo_download()` updates `local_path`, `download_status`, and
`checksum_verified`. These fields make the manifest useful as a record of what
happened, not only what was requested.

```{r status-columns}
manifest[, c("file_id", "file_name", "download_status", "checksum_verified")]
```

## Check a local checksum

When NeMO provides an MD5 checksum, `nemo_download(verify_checksum = TRUE)` can
compare it with the downloaded file. The small local example below shows the
MD5 value that would be stored in a manifest without downloading external data.

```{r checksum}
tmp <- tempfile()
writeLines("nemo", tmp)

checksum <- unname(tools::md5sum(tmp))
checksum
identical(checksum, unname(tools::md5sum(tmp)))
```

## Download real files

Actual downloads are not run in this vignette because NeMO files can be large
and depend on external archive availability. The code below is the intended
interactive pattern after the manifest and size plan have been inspected.

```{r download-real-files, eval=FALSE}
downloaded <- nemo_download(
  manifest,
  destdir = "nemo_downloads",
  max_size_gb = 5,
  verify_checksum = TRUE
)

nemo_write_manifest(downloaded, file.path("nemo_downloads", "nemo_manifest.tsv"))
```

## Load downloaded data

Loading requires local files and optional reader packages. The first-pass
helpers support H5AD and 10x HDF5 files for `SingleCellExperiment` or Seurat
workflows.

```{r load-downloaded-files, eval=FALSE}
downloaded <- nemo_read_manifest("nemo_downloads/nemo_manifest.tsv")
sce <- nemo_load(downloaded, format = "SingleCellExperiment")
seurat_obj <- nemo_load(downloaded, format = "Seurat")
```

## Session information

```{r session-info}
sessionInfo()
```