NEWS
projectData 0.1.0
- Initial CRAN release.
- Register installed R data packages as named collections.
- Provide deferred access and metadata-only resource inventories.
- Support explicit name-conflict resolution and source-qualified access.
- Preserve package provenance and installation identity.
- Introduce a public loader interface for specialized data formats.
- Fixed removal of temporary staging links on Windows (junctions and directory
symbolic links), and made link-ownership checks compare locations rather than
path text.
- Tests and the
adapters vignette no longer assume directory-link creation
succeeds: environments without link support (e.g. some CRAN Windows builds)
now exercise the documented projectData_staging_error path instead of
failing, and the vignette's executable source-tree-authoring example uses a
portable installed-layout directory that needs no links.
Changes in this version
- Changed installations are now detected reliably. The registered installation
is compared with the metadata on disk at its recorded root (never a loaded
namespace), and the library entry must still lead to that root. Previously a
library link repointed to another installation (pak/renv-style stores) left
the physically resolved root untouched, so
data_attach() was a silent no-op
and bindings stayed stale. The identity also covers every file in Meta/
and the manifest, so same-version reinstalls are caught without relying on one
timestamp.
- The stale-installation error names the collection, path, registered and
current version and the exact
data_attach(..., replace = TRUE) command, and
states that objects already held elsewhere keep the old value. It is raised
before registration, bindings, preferences or attachment change.
data_attach(replace = TRUE) for an existing id keeps omitted lib.loc
(the library of the current installation, not .libPaths()), suffix and
include; explicit arguments win. A failed replacement keeps the previous
state (class projectData_replace_failed). Old resource records and caches are
discarded whenever the installation changed. A lib.loc that selects a
different installation for an existing id now fails with
projectData_installation_conflict (and with the stale-installation error if
the registered one changed) unless replace = TRUE.
- Resource working directories (for loaders that need a writable copy) are now
built from safe path components. Both the collection id and the resource name
are encoded injectively (only
a-z, 0-9 and - pass through; every other
byte, including uppercase letters, becomes %XX), so path separators or ..
can no longer move a directory out of projectData-work, and distinct
identities such as a/b and a_b never share a directory. Directory names for
ids or resources containing uppercase letters, _ or . differ from earlier
versions. Caching, directory reuse and lifetime are unchanged; existing
directories are never deleted.
projectData 0.0.0.9006
Changes in this version
- Manifest validation and authoring against a verified package source tree now
accept
Source paths that pass through a symbolic link under inst/ (e.g.
inst/extdata -> /shared/data), provided the final path exists. Source
stays package-relative. data/, installed packages and standalone
directories keep strict physical containment, and a directory with
installation artifacts (Meta/, R/<pkg>.rdb) is never treated as a source
tree. Authors must still verify that built archives contain the data.
projectData 0.0.0.9005
Changes in this version
- One rule now selects the file behind a dataset. For
data/ files, the
built-in reader and the inventory both use the selection utils::data()
makes: a name in the lazy-load map wins; otherwise the first supported
extension in data()'s precedence order (R, r, RData, rdata, rda,
then tables, then csv) for which <dataset>.<extension> exists. Previously
inventory recorded whichever matching file was listed last, so with both
dual.R and dual.rda the catalog could name the .rda while the script was
opened, and removing the unused .rda could block retrieval. Now catalog
source and the first-access message name the file actually read, lazy-load
datasets still share data/Rdata.rdb, and removing an unused alternative has
no effect. If the selected file is removed, opening fails with an error naming
it and never falls back to another file; if a file with higher precedence has
appeared since registration, opening is refused with a request to re-register
(replace = TRUE) instead of silently reading something other than the
recorded source (a dataset registered with no located file records no source,
so a file restored later is simply read). Selection uses the directory listing
and the small lazy-load map only. Files whose extension data() does not understand (for example
x.rda.gz) are now reported as unsupported instead of being inventoried.
- Cleanup of the temporary view used to author against a package source tree is
portable and checked. Links are removed as links and the verified-empty
staging directory with
unlink() instead of file.remove(); only entries
created by the current operation are removed, and a link that was replaced by
something else, or a directory that gained foreign files, is left in place.
Link targets are never followed or deleted. Every removal is checked, and a
failure is reported with a projectData_cleanup_warning naming what remains,
without replacing the authoring error that triggered the cleanup. Cleanup is
safe after partial staging, after success, and when repeated. Nothing is ever
copied. This was developed and tested on Linux; it has not been run on
Windows.
- The search-path environment that
data_attach() creates is now removed by
identity (its current position) in data_detach() and when the package is
unloaded, and is captured by identity when it is attached. An unrelated
search-path entry with the same name is left alone.
- Adapter work directories skip names that already exist, so reloading the
package inside one R session no longer collides with directories made before.
- Package metadata: authors, maintainer and copyright holder are filled in from
information in the repository (see
DESCRIPTION); README, DESCRIPTION and
LICENSE agree on the MIT license. The optional monocle3/BPCells check moved
to dev/integration/, outside the package tests and the source tarball.
Minimum versions of the test packages are declared (testthat 3.2.0 for
local_mocked_bindings(), withr, tools), and tests remove the libraries and
scratch files they create.
Behavior of this version
Attaching and retrieving
data_attach() registers installed data packages (metadata only) and
attaches them in one step as deferred active bindings in an environment named
projectData:data. Nothing is attached when projectData loads. Objects are
read on first use, after a message (data_config(messages = )).
data_get(name, package = ) registers the package automatically if needed
and returns the object without attaching anything; explicit and attached
access share one resource cache. data_query() looks at packages without
remembering them, data_detach() removes attached collections and forgets
every registration, attached or not. It never deletes files.
- Name conflicts are never resolved by attachment order. Use a primary
collection (
data_primary()), per-object preferences (data_prefer()),
suffix aliases or qualified data_get(name, collection = ). Resolution uses
names only.
- Installed packages are resolved once, with their installation identity; if the
installation changes before an object is opened, opening fails and asks for
replace = TRUE.
Opening, caching, failure
- A resource is the loadable unit and can produce several objects; aliases and
siblings share one canonical opening. Opening is guarded against reentrancy;
a failed resource stays failed until
data_retry(). on_open runs inside
that guard.
- Object names recorded at registration are unchecked until a resource is
opened.
data_get() is strict about a missing object; data_inspect()
replaces the recorded mapping with the observed names (rejecting duplicates
within a collection, atomically).
- Script datasets (
data/*.R) run only on first access, in a fresh output
environment whose enclosing lookup chain is the fixed set of standard startup
packages (stats, graphics, grDevices, utils, datasets, methods, base) and
excludes the global workspace and attached packages. The chain is built only
for script datasets. utils is detached again after utils::data() when it
was not attached beforehand. See ?data_attach.
Catalog and provenance
data_catalog() is metadata only: it never opens a resource or inspects
values. Columns include origin (index, adapter, manifest), mapping
(recorded, unknown, verified, mismatch), loader_status, storage
and state. storage of a ready resource is the snapshot taken when it was
opened, so later loader changes do not relabel it.
Loaders and adapters
data_loader() defines contract version 1 (discover, validate, open);
data_loader_add() and data_loader_adapter() register loaders atomically,
and nothing is installed automatically. validate receives the same context
as open and runs before every uncached opening. projectData::rds and
projectData::dataset are built in.
- Adapter claims on standard datasets are ignored with a warning
(
data_config(ignored_claims = )); a manifest entry assigns such a resource to
a specialized reader. Ambiguous adapter claims are errors.
- Disk-backed objects: detaching never deletes backing files; an object that
references installed files can still break if the installation is replaced or
removed. Adapters state whether they return installed-file references or a
working copy (
needs_work_dir).
Manifests
- An optional versioned DCF file,
inst/projectData/manifest.dcf, declares
loaders and mappings as data (never code). Reading, data_manifest_validate()
and data_manifest_write() apply the same rules: one declaration per resource,
unique object names across resources, strictly parsed Options.
data_manifest_draft() and data_manifest_write() author manifests from the
registered adapters' discovery. root can be a package source tree (needs
symbolic links, see above), an installed package, or a standalone directory;
Source paths are always installed-relative. Existing manifests are never
overwritten without overwrite = TRUE, and nothing is written inside an
installed package.
Not included
- A monocle3/BPCells adapter is deferred; the tiny disk-backed backend in
inst/examples/tinydisk.R is the tested adapter example.
- ExperimentHub or other remote retrieval is a possible extension, not
implemented.