| Title: | Structurally Faithful Development Surrogates for Tabular Data |
|---|---|
| Description: | Turns a confidential tabular dataset - a single table, a folder of files, or a multi-sheet workbook - into a structurally faithful synthetic clone suitable for pipeline development. Experimental-design allocation and the NA pattern are preserved exactly; categorical design, treatment, and covariate vocabularies can be reversibly aliased; outcome and numeric-covariate values are re-simulated via a Gaussian copula that preserves the global covariance structure. Columns shared across tables are aliased consistently so the synthetic tables still join. A private recipe object round-trips a pipeline written against the synthetic clone onto the original data. Not a differential-privacy or anonymisation tool: outputs are development surrogates, not public-release-safe artefacts. |
| Authors: | Max Moldovan [aut, cre] (ORCID: <https://orcid.org/0000-0001-9680-8474>, affiliation: Adelaide University) |
| Maintainer: | Max Moldovan <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.16.0 |
| Built: | 2026-11-19 03:52:24 UTC |
| Source: | https://github.com/cran/masque |
Forward translation: takes an original-namespace data frame and
returns it renamed and re-labelled to match the synthetic namespace
produced by mask(). Use this to run a pipeline (trained on the
synthetic) against the original data without modifying pipeline code.
apply_recipe(original, rec, check_integrity = TRUE)apply_recipe(original, rec, check_integrity = TRUE)
original |
A data frame in the original namespace. |
rec |
A |
check_integrity |
Logical. When |
Operations applied (in order):
Re-apply cleaning and conform decisions: the legalised names and
trimmed labels from clean_table(), then the label merges and type
changes from conform_table() when the clone was built on a
conformance object.
Verify integrity by comparing the NA mask of original to the
SHA-256 fingerprint stored on the recipe (controlled by
check_integrity).
Drop every column whose resolved action was "drop" on the
roles table mask() masked with, in either mode. A linked join key
of a set is aliased in the clone and is kept.
Subset and reorder to the columns the recipe knows about.
Re-label factors / characters for any column with a level map
held by the recipe (i.e., treatment and categorical covariates in
collaborate mode; or treatment in local mode with opt-in
permutation). Unknown non-NA values fail closed.
Rename columns per recipe@column_name_map when mask() was
called with alias_names (otherwise NULL, and names are
unchanged).
Numeric columns are passed through unchanged: the synthetic-namespace for numeric columns is the same as the original. NA cells in the input remain NA in the output (no synthesis is performed here).
A data.table in the synthetic namespace, ready for the pipeline.
r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" r$role[r$col == "Species"] <- "covariate" m <- mask(iris, r, mode = "collaborate", seed = 1) rec <- recipe(m) iris_in_synth_space <- apply_recipe(iris, rec) head(iris_in_synth_space)r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" r$role[r$col == "Species"] <- "covariate" m <- mask(iris, r, mode = "collaborate", seed = 1) rec <- recipe(m) iris_in_synth_space <- apply_recipe(iris, rec) head(iris_in_synth_space)
Returns a per-column audit data.table, with table-level checks attached, and
prints a severity-grouped report.
Auto-runs in mode = "collaborate" at mask() time and stores the
result on the masque object (m@audit); for local-mode audits or
explicit re-audits, pass the original data frame via original.
audit_mask(m, original = NULL, print = TRUE)audit_mask(m, original = NULL, print = TRUE)
m |
A |
original |
Optional. Required when |
print |
Logical; if TRUE (default), print a styled report. |
Each row of the returned data.table holds:
col: column name in the original.
role: assigned role.
kind: storage kind.
leakage_class: low, medium, or high.
n_unique_levels: distinct non-NA values (categorical only).
freq_min: minimum per-level frequency (categorical only).
exact_match_pct: percentage of synthetic cells equal to the
original cell (numeric only; cell-by-cell).
na_pct: percentage of NA cells in the original column.
na_pattern_uniqueness: fraction of rows in the original with
a globally unique NA pattern (one number per data frame, repeated
on every row).
alias_status: aliased, permuted, passthrough, or
dropped.
notes: short human summary.
same_row_pct: percentage of rows where the clone value equals
the original value in the same row, read back through the level map
(categorical, logical, date and two-valued numeric columns).
baseline_pct: the same share expected by chance, under a random
permutation of the original column: over its
level frequencies (within the conditioning strata for a scrambled
numeric column of a conditional clone).
excess_pct: same_row_pct - baseline_pct.
The table-level checks sit in attr(audit, "joint"), a data.table with
one row per check (check, cols, observed, baseline, excess,
sd, leakage_class, notes). Each baseline comes from independent
permutations of the original content columns (outcomes, covariates,
dates, kept text), drawn under a fixed internal seed that leaves the
caller's random number stream untouched; design, treatment, id, aliased
text and linked columns are held fixed:
same_row_joint: rows whose every content column equals the
original row.
whole_row: clone rows equal to some original row on every
retained column.
rare_cross: in the cross of visible (not aliased) categorical
covariates and outcomes, clone cells holding one row whose
combination is also a single row in the original.
unit_sizes: a note, never graded, when design units differ in
size, since the multiset of sizes can link a clone to a public trial
list.
Classification heuristics:
Retained PII-pattern column -> high.
Outcome with action keep in collaborate -> high, or medium
when mask() was called with allow_kept_outcome = TRUE, which the
recipe records.
Treatment unaliased in collaborate -> high. A numeric treatment
(a dose) is aliased like a categorical one.
Categorical covariate with a frequency-1 level in collaborate ->
high.
Outcome with exact-match-pct > 1\
Numeric covariate with exact-match-pct > 5\
medium.
Two-valued numeric columns skip the two exact-match rules: a 0/1
column matches its own cell about half the time by chance. So do
columns synthesised once per design unit (see grain in mask())
and whole-valued numeric covariates, which repeat real values by
construction; all three are graded on excess_pct instead.
Content column whose excess_pct is at least 10 points and four
binomial standard deviations in collaborate -> medium.
Joint match excess of at least 20 points and four permutation
standard deviations -> high (5 points -> medium); rare-cross
excess of at least 3 cells and four standard deviations -> high
(1 cell and three -> medium). In collaborate mode these verdicts
raise masque_high_leakage and block package-managed writers; in
local mode they are reported in the notes and graded low.
Ignore column retained in local -> low (informational).
A column constant within a design unit that was synthesised row by row
(grain = FALSE, or a collaborate grain with fewer than five units) carries
a note saying the clone does not keep that grain.
As a reference point, a collaborate-mode clone of agridat::besag.met
(yield as the outcome, design columns kept) scores low on every
column, and none of its yields equals the original value.
The audit data.table, returned invisibly. For a masque_set, a
named list of per-table audit data.tables, printed table by table.
r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" r$role[r$col == "Species"] <- "covariate" m <- mask(iris, r, mode = "collaborate", seed = 1) audit_mask(m)r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" r$role[r$col == "Species"] <- "covariate" m <- mask(iris, r, mode = "collaborate", seed = 1) audit_mask(m)
Real custodian tables arrive with column names that are not valid R
names ("Yield (t/ha)", "Site Name"), leading or trailing
whitespace in names and factor / character values, and the
occasional near-duplicate label ("north" vs "North" vs
" north"). clean_table() legalises names and trims whitespace, and
reports each fix. Near-duplicate labels are only reported, since merging
labels that look alike is a judgement call; conform_table() makes it.
clean_table(df, clean = c("auto", "report", "off"), quiet = FALSE)clean_table(df, clean = c("auto", "report", "off"), quiet = FALSE)
df |
A data frame. |
clean |
One of |
quiet |
Logical. When |
The corrections are returned alongside the cleaned data, so mask()
records them in the recipe and apply_recipe() re-applies the same
cleaning to the original.
Two input columns with the same name are refused with a
masque_duplicate_names_refusal error: the recipe records renamed
columns by their original name, so the two could not be told apart on
the round trip. Rename them first.
An object of class masque_cleaning: a list with
data - the cleaned (or, under report / off,
unchanged) data.table;
name_map - named character original -> clean for every
column whose name changed (empty if none);
level_fixes - named list, one entry per column whose
values were trimmed, each a named character original -> clean;
near_duplicates - a data.table of report-only label pairs
(col, a, b, kind) that look like typos but were left
untouched. Labels that differ only inside a run of digits
("G01" and "G02", "Plot12" and "Plot13") are different
codes and are not listed. The printed report shows the first 20
pairs and the count, and a recipe keeps the same;
mode - the clean mode applied.
df <- data.frame( `Site Name` = c("north ", "north", "South"), `Yield (t/ha)` = c(3.1, 2.9, 5.0), check.names = FALSE ) cl <- clean_table(df, quiet = TRUE) names(cl$data) cl$near_duplicatesdf <- data.frame( `Site Name` = c("north ", "north", "South"), `Yield (t/ha)` = c(3.1, 2.9, 5.0), check.names = FALSE ) cl <- clean_table(df, quiet = TRUE) names(cl$data) cl$near_duplicates
clean_table() makes the fixes that are unambiguously safe: it legalises
column names and trims whitespace, and it reports near-duplicate labels
without merging them. conform_table() is the next step and makes the two
decisions that need judgement: deciding that
"NSW" and "Nsw" are one category, and deciding that a character column
holding six labels is a factor.
conform_table( df, merge_labels = c("report", "auto", "off"), types = c("report", "auto", "off"), max_levels = 20L, clean = c("auto", "report", "off"), quiet = FALSE )conform_table( df, merge_labels = c("report", "auto", "off"), types = c("report", "auto", "off"), max_levels = 20L, clean = c("auto", "report", "off"), quiet = FALSE )
df |
A data frame. |
merge_labels |
One of |
types |
One of |
max_levels |
Cardinality at or below which a character column is
treated as categorical. Default |
clean |
Passed to |
quiet |
Logical. When |
Both are off by default: the default "report" mode lists every change
it would make, in plain words, and applies none. Set a mode to "auto" to
apply them; the assumption behind each one is recorded in the returned
object.
Storage is decided before categories, so a column of numbers or dates
held as text is typed first and "4.2" is never merged into "4.4". Only
columns that remain categorical are considered for merging, and a
one-character edit between labels shorter than four characters is ignored
("a" and "b" are one edit apart).
Two labels that differ by an edit are merged only when one spelling is
clearly more common; a column is converted only when every value parses;
numeric columns keep their storage. Anything not changed for these reasons
is reported as a not applied row with the reason.
Merges follow chains: if "Nsw" merges onto "nsw" and "nsw" onto
"NSW", both end on "NSW". A label whose merges end on two different
labels ("colur" beside "color" and "colour" that occur equally
often) is not merged; it is reported, and a masque_conform_ambiguous_merge
warning is raised under merge_labels = "auto". Codes that differ only
inside a run of digits ("G01" and "G02") are never treated as
near-duplicates. Which columns hold categories is decided the same way
whatever types is, so types = "off" still merges labels and leaves
storage as it is.
Pass the result to mask() (or as a table of mask_set()): the recipe
records the merges and type changes, and apply_recipe() replays them on
the original so a pipeline built on the conformed clone sees the same
labels and storage.
An object of class masque_conformance: a list with
data - the conformed data.table;
cleaning - the masque_cleaning record from clean_table();
merges - one row per near-duplicate pair considered
(col, from, to, n_from, n_to, applied, reason);
merge_map - the merges as applied to the data (col, from,
to), one row per merged-away label, each to a label left in
the column. This is what mask() records and apply_recipe()
replays;
types - one row per column whose storage was considered
(col, from, to, applied, reason);
assumptions - every decision as a plain sentence
(col, assumption, applied), which is the thing to read;
modes - the modes applied;
max_levels - the cardinality threshold used.
clean_table() for the safe fixes, propose_roles() for what a
column is, mask() for the round-trip.
df <- data.frame( state = c("NSW", "Nsw", "NSW", "VIC", "VIC"), yield = c("3.1", "2.9", "5.0", "4.2", "3.8"), sown = c("2024-05-01", "2024-05-03", "2024-05-01", "2024-05-08", "2024-05-08"), stringsAsFactors = FALSE ) # report only: nothing is changed cf <- conform_table(df, quiet = TRUE) cf$assumptions # apply the decisions cf2 <- conform_table(df, merge_labels = "auto", types = "auto", quiet = TRUE) str(cf2$data)df <- data.frame( state = c("NSW", "Nsw", "NSW", "VIC", "VIC"), yield = c("3.1", "2.9", "5.0", "4.2", "3.8"), sown = c("2024-05-01", "2024-05-03", "2024-05-01", "2024-05-08", "2024-05-08"), stringsAsFactors = FALSE ) # report only: nothing is changed cf <- conform_table(df, quiet = TRUE) cf$assumptions # apply the decisions cf2 <- conform_table(df, merge_labels = "auto", types = "auto", quiet = TRUE) str(cf2$data)
Describes a clone in a data card (card version 0.1): its size, its design columns and, for each column, a role, a kind and counts. An analysis planner can read the card in place of the data. No value of any column is written: a factor's levels are counted, never listed.
data_card( m, file = NULL, name = NULL, env = NULL, spatial = NULL, unit = NULL, primary_event = 1L )data_card( m, file = NULL, name = NULL, env = NULL, spatial = NULL, unit = NULL, primary_event = 1L )
m |
A |
file |
Where to write the card in YAML, or |
name |
The table name on the card. Defaults to |
env, spatial, unit
|
Optional names of clone columns: the environment
columns (trials, sites, years), the columns giving a spatial or temporal
position, and the one column identifying the unit of repeated
measurement. Environment columns must have the role |
primary_event |
Which event pair of the first |
The card is computed from the clone alone: synthetic(), the role table
from roles(), the mode, the package version and the column names of any
events spec given to mask(). No other part of the private recipe is
read, so the card can travel with the clone. On a clone whose column
names were aliased, the card uses the aliases throughout.
The card as a list, invisibly when file is given. For a set, a
named list of cards, one per table.
The design block is copied from the role table, never re-detected. The
treatment columns are those with the role treatment, and the block
columns those with the role design, less any named in env,
spatial or unit. Environment, spatial and unit columns are named by
the caller, since the role table does not tell them apart from blocks.
No design class is written. For a linked-event spec, the event pair
chosen by primary_event gives the event time and status columns, the
other status columns of that spec are listed as competing, and the card
records the number of rows with the primary status equal to 1 and the
share of rows with neither the primary nor a competing event (censored).
Every statistic carries a provenance class.
structure: exact in the clone, so equal to the same statistic on the
original table. Row and column counts, missing counts and shares, the
level counts and zero shares of kept and aliased columns and of event
status columns, the treatment, block and event columns, the
environment count and the event count.
clone_estimate: re-simulated, so approximate. The level counts and
zero shares of scrambled columns, including event times, and the
censored share.
analyst: named in this call: the environment, spatial and unit
columns.
The card is written by a small deterministic writer: keys in the order of
the contract, every string double-quoted and escaped, counts as integers,
shares with six decimals, UTF-8 with \n line ends. The same clone gives
the same bytes.
roles(), synthetic(), mask(), mask_set().
r <- propose_roles(iris) r <- set_role(r, "Species", role = "treatment", action = "alias") r <- set_role(r, "Sepal.Length", role = "outcome") m <- mask(iris, r, seed = 1) card <- data_card(m) card$table card$design path <- tempfile(fileext = ".yaml") data_card(m, file = path) head(readLines(path), 12)r <- propose_roles(iris) r <- set_role(r, "Species", role = "treatment", action = "alias") r <- set_role(r, "Sepal.Length", role = "outcome") m <- mask(iris, r, seed = 1) card <- data_card(m) card$table card$design path <- tempfile(fileext = ".yaml") data_card(m, file = path) head(readLines(path), 12)
Inspects df and returns an S7 design_summary. Environment scope and
experimental-design class are separate conclusions: a table can be a
multi-environment trial (MET) even when its within-environment randomisation
cannot be recovered from the recorded columns.
detect_design( df, roles = NULL, interactive = FALSE, threshold = 0.5, tie_delta = 0.02, env = NULL )detect_design( df, roles = NULL, interactive = FALSE, threshold = 0.5, tie_delta = 0.02, env = NULL )
df |
A data frame. |
roles |
Optional roles table (as returned by |
interactive |
If |
threshold |
Minimum top-rule score for a class to be reported.
Below this, |
tie_delta |
Score difference within which two rules are treated
as tied. Default |
env |
Environment specification. |
With env = NULL, exact environment names and a bounded set of site-year
patterns are assessed conservatively. A site-only candidate auto-resolves
only when treatments are replicated across sites. Weak or competing
evidence gives an explicit uncertain result, never a guessed single
trial. Supply env to define the environment basis, or use env = FALSE
to run the pre-0.9 whole-table path exactly.
Before the scope step, the pooled legacy detector runs six independent
design rules. Each returns a score in . The highest-scoring
class above threshold is one of "CRD", "RCBD",
"IBD/alpha-lattice", "row-column", "split-plot", "factorial",
or "none". Ties within tie_delta favour the simpler design. For a MET,
per-environment classes, treatment connectivity, and near-disjoint
experiment groups are diagnostics only. Dense connectivity calculations
that exceed the package safety bound are reported as not_computed.
None of these diagnostics proves the original randomisation protocol.
When scope resolves, block_cols is the block basis the environments
agree on, or empty when they disagree; an environment column is never a
block. evidence$block_revision records the change.
For a column without a treatment-pattern name, a treatment hint from the rules is auto-applied only when the column is crossed with a design column. A column with one value per level of a design column, or with an outcome-pattern name, is a covariate and never a candidate.
The detector never edits df. Its job is to recommend a role
assignment, surface the evidence, and (optionally) draw a sanity
check via plot().
An S7 design_summary object. Legacy design fields include
class_label,
treatment_col, block_cols, whole_plot_col, sub_plot_col,
spatial_cols, scores, evidence, recommended_roles,
candidates, and warnings. Scope fields include scope_label,
scope_status, scope_confidence, is_met, env_cols, env_method,
n_env, group_cols, connectivity, per_env, and
within_design_label.
propose_roles() for the role table that feeds detection and
plot_design_summary() for the sanity-check visualisation.
# Classic alpha-lattice (24 genotypes, 3 reps, 6 blocks per rep). if (requireNamespace("agridat", quietly = TRUE)) { d <- agridat::john.alpha ds <- detect_design(d) print(ds) } # Observational data frame -> class_label "none". detect_design(mtcars) # Explicit two-environment trial. met <- expand.grid( env = factor(c("E1", "E2")), rep = factor(seq_len(2L)), gen = factor(c("G1", "G2", "G3")) ) met$yield <- seq_len(nrow(met)) met_design <- detect_design(met, env = "env") met_design@scope_label met_design@connectivity$status# Classic alpha-lattice (24 genotypes, 3 reps, 6 blocks per rep). if (requireNamespace("agridat", quietly = TRUE)) { d <- agridat::john.alpha ds <- detect_design(d) print(ds) } # Observational data frame -> class_label "none". detect_design(mtcars) # Explicit two-environment trial. met <- expand.grid( env = factor(c("E1", "E2")), rep = factor(seq_len(2L)), gen = factor(c("G1", "G2", "G3")) ) met$yield <- seq_len(nrow(met)) met_design <- detect_design(met, env = "env") met_design@scope_label met_design@connectivity$status
Displaces latitude / longitude pairs by a small random distance, in place,
so a synthetic table can carry realistic coordinates without revealing the
true location of a field, farm or site. Two geographic-masking schemes are
offered. The default "donut" displaces every point by a distance drawn
uniformly (by area) from an annulus between min_km and max_km, in a
random direction: this guarantees a minimum displacement (a privacy floor –
no point is left almost where it started) while bounding the maximum drift.
The "gaussian" scheme adds independent normal noise of standard deviation
sd_km to each axis; it is simpler but has an unbounded tail, so a handful
of points can move much further than intended.
jitter_coordinates( df, lat_col, lon_col, by = NULL, method = c("donut", "gaussian"), min_km = 5, max_km = 20, sd_km = 10, on_land = TRUE, max_tries = 100L, seed = NULL, .group = NULL )jitter_coordinates( df, lat_col, lon_col, by = NULL, method = c("donut", "gaussian"), min_km = 5, max_km = 20, sd_km = 10, on_land = TRUE, max_tries = 100L, seed = NULL, .group = NULL )
df |
A data frame containing the coordinate columns. |
lat_col, lon_col
|
Column names of the latitude and longitude (numeric, in decimal degrees). |
by |
How rows are grouped into sites. |
method |
|
min_km, max_km
|
Inner and outer radii of the donut, in kilometres
(used when |
sd_km |
Per-axis standard deviation in kilometres (used when
|
on_land |
Controls the land constraint. |
max_tries |
Maximum re-draws per site before giving up. Default |
seed |
Optional integer seed for reproducibility. The caller's RNG state
is preserved. Several published jitters of the same point average toward
it: keep the seed (a |
.group |
Internal. A pre-computed grouping vector of length
|
The displacement is computed in kilometres and converted to degrees with a
cos(latitude) correction on longitude, so the true ground distance matches
the requested magnitude at any latitude. Each candidate is checked against a
land mask and re-drawn until it falls on land, so a coastal point is never
pushed offshore. The original NA pattern is preserved cell-by-cell and the
two axes stay paired (if either coordinate is missing, both are set to NA).
df, with lat_col and lon_col overwritten by the jittered
coordinates (rounded to five decimal places, about one metre).
One displacement is drawn per site, not per row, and broadcast to every
row of that site. A table that holds many rows per physical place – plots
within a trial, samples within a paddock, observations within a farm –
therefore comes back with one masked coordinate per place, exactly as the
source table has one true coordinate per place. by chooses how a site is
identified:
by = NULL (default) treats rows that share an identical input coordinate
pair as one site. This needs no user action and cannot alter genuinely
point-level data, where every input pair is already distinct.
by = c("trial") (one or more column names) treats rows sharing those
values as one site, for the case where a site's recorded coordinates differ
slightly between rows. Where such a group holds more than one distinct
input coordinate, the group's row-weighted centroid is used and the
consolidation is reported.
by = FALSE restores the pre-0.10.0 behaviour of displacing every row
independently, and warns if the input carries repeated coordinate pairs.
It exists only so a table masked by an earlier version can be reproduced.
Displacing each row of a repeated coordinate separately leaks the site. Donut displacement is isotropic, so the mean of many independent draws around one true site converges on that site: averaging the 360 rows of a single trial recovers the true position to a median 0.58 km, against the 5 km floor the donut was asked for. One draw per site removes the estimator and leaves the full displacement in place.
One displacement per site holds within a masked table. It does not hold across several. Each rebuild with a fresh seed draws independently, so a site masked repeatedly and kept in several versions can be averaged back out, exactly as its rows could be before. Measured at the 5-20 km default:
| Rebuilds kept | Median recovery error |
| 1 | 14.2 km |
| 2 | 10.1 km |
| 4 | 6.4 km |
| 8 | 4.5 km |
| 16 | 3.1 km |
Eight kept rebuilds fall below the 5 km floor the donut was asked for.
The remedy is to reuse the seed. The same seed reproduces the
same displacement for the same sites, so any number of rebuilds yields one
point and nothing to average. Draw a fresh seed only when you intend to
supersede every earlier version, and retire the ones you replace.
Choose the unit carefully. It is the finest grouping that denotes one physical place at one time. For a multi-year trial series that is location by year, not location alone: trials at one named location in different seasons usually sit in different paddocks, and one displacement for all of them would misplace every season but one.
The right displacement depends on the density of the entities you are
protecting: the masked point should be spatially k-anonymous (roughly, at
least k comparable entities lie closer to the masked point than the true
one). Individual-level urban health data is
typically masked with a standard deviation of about 1 km, because cities are
dense. Agricultural fields and farms are orders of magnitude sparser, so a
comparable level of protection needs a much larger displacement – a donut of
roughly 5 to 20 km (the default) moves a point across several properties
while keeping it in the same agroclimatic region. For a formal guarantee,
calibrate min_km / max_km to the local field density to reach a target
k-anonymity.
Hampton, K. H., Fitch, M. K., Allshouse, W. B., Doherty, I. A., Gesink, D. C., Leone, P. A., Serre, M. L., & Miller, W. C. (2010). Mapping health data: improved privacy protection with donut method geomasking. American Journal of Epidemiology, 172(9), 1062-1069. doi:10.1093/aje/kwq248
Zandbergen, P. A. (2014). Ensuring confidentiality of geocoded health data: assessing geographic masking strategies for individual-level data. Advances in Medicine, 2014, 567049. doi:10.1155/2014/567049
synthesise_geospatial() to instead re-anchor coordinates at
user-supplied fake centroids.
# Four plots at each of three sites: one masked coordinate per site. df <- data.frame( site = rep(c("A", "B", "C"), each = 4), lat = rep(c(-34.9, -35.2, -33.6), each = 4), lon = rep(c(138.6, 142.0, 148.2), each = 4) ) if (requireNamespace("maps", quietly = TRUE)) { out <- jitter_coordinates(df, "lat", "lon", min_km = 5, max_km = 20, seed = 1) nrow(unique(out[c("lat", "lon")])) # 3, one pair per site }# Four plots at each of three sites: one masked coordinate per site. df <- data.frame( site = rep(c("A", "B", "C"), each = 4), lat = rep(c(-34.9, -35.2, -33.6), each = 4), lon = rep(c(138.6, 142.0, 148.2), each = 4) ) if (requireNamespace("maps", quietly = TRUE)) { out <- jitter_coordinates(df, "lat", "lon", min_km = 5, max_km = 20, seed = 1) nrow(unique(out[c("lat", "lon")])) # 3, one pair per site }
Takes one data frame and a user-edited roles table (from
propose_roles(), possibly adjusted with set_role()) and produces
a synthetic clone according to each column's action. Returns a
masque S7 object holding the synthetic data and a private
masque_recipe.
mask( df, roles, mode = c("local", "collaborate"), seed = NULL, clean = c("auto", "report", "off"), alias_names = FALSE, conditional = FALSE, ladder = c("hierarchy", "levels"), coords = NULL, events = NULL, grain = TRUE, allow_unmasked_coords = FALSE, allow_kept_outcome = FALSE, quiet = FALSE, .shared_maps = list(), .cleaning = NULL, ... )mask( df, roles, mode = c("local", "collaborate"), seed = NULL, clean = c("auto", "report", "off"), alias_names = FALSE, conditional = FALSE, ladder = c("hierarchy", "levels"), coords = NULL, events = NULL, grain = TRUE, allow_unmasked_coords = FALSE, allow_kept_outcome = FALSE, quiet = FALSE, .shared_maps = list(), .cleaning = NULL, ... )
df |
A data frame, or the |
roles |
A roles table from |
mode |
Either |
seed |
Optional integer for reproducibility. |
clean |
Label and column-name hygiene before masking, passed to
|
alias_names |
Hide the column names themselves. |
conditional |
Logical scalar (default The stratum is chosen by a coarsening ladder. Treatment crossed
with every retained design column is the finest rung, but on a
replicated factorial that rung holds one row per cell, which is too
thin to synthesise. The ladder then drops design columns, in the order
set by |
ladder |
How the coarsening ladder orders the design columns it may
drop, and what it keeps of a dropped one. |
coords |
Optional geographic-coordinate declaration. Supply one or more
latitude/longitude pairs and each is coarsened in place by an on-land
jitter (see |
events |
Optional linked time-to-event outcomes. One spec per group of
linked events, |
grain |
Single logical (default |
allow_unmasked_coords |
Single logical. A masked table must not carry
a real coordinate, so by default |
allow_kept_outcome |
Single logical. In collaborate mode an outcome
with the action |
quiet |
Single logical. |
.shared_maps |
Internal. A named list of pre-computed
|
.cleaning |
Internal. The cleaning record of a table |
... |
Must be empty. An unused argument, for example a misspelled name, is an error. |
mode = "local" marks the synthetic for owner development only; the
reminder is recorded on the recipe and shown when the object prints.
mode = "collaborate" additionally jitters re-simulated numerics
within their measurement resolution (stochastically rounding
integers) and runs audit_mask() automatically; a HIGH finding is
raised as a classed warning (masque_high_leakage) and blocks the
package-managed writers (masque()'s out, write_set()) until it
is resolved or explicitly overridden. Which columns are aliased,
kept, or dropped is decided by the action column of roles -
propose_roles() resolves mode-appropriate defaults, so the table
you reviewed is the plan that runs.
Collaborate mode changes the transformations and runs the audit. Whether a synthetic table is appropriate for a given collaborator, environment or jurisdiction is a release decision for the data custodian; masque informs that decision, it does not make it.
A masque S7 object. Use synthetic() and recipe() to
extract the components.
keepByte-identical pass-through, both modes.
scrambleNumeric outcome / covariate columns are re-simulated jointly via a Gaussian copula on the global Pearson covariance, with empirical-quantile marginals. Categorical, date, and text columns are row-permuted within non-NA positions, class preserved. Treatment columns get a seeded label permutation - the assignment structure never moves.
aliasAs scramble where applicable, plus opaque label
substitution: treatments become trt_NNN (<col>_trt_NNN when
two or more treatments are aliased. A numeric treatment becomes
text, and unmask() restores its exact values), categorical
covariates <col>_LNNN, design labels <col>_DNNN (in place -
structure intact), ids <col>_INNN (in place - row linkage
intact), text values <col>_TNNN. A key mask_set() shares across
tables becomes <col>_KNNN. NNN has at least three digits.
dropColumn excluded from the synthetic, both modes.
The NA mask of every retained column is preserved cell-by-cell. RNG state is preserved across the call.
propose_roles(), set_role(), roles_validate(),
synthetic(), recipe(), roles(), reveal_maps(), audit_mask(),
data_card().
r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") m <- mask(iris, r, seed = 1) head(synthetic(m))r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") m <- mask(iris, r, seed = 1) head(synthetic(m))
The set-level counterpart to mask(). Takes a folder of tabular
files, an Excel workbook, or a named list of data frames and produces
one synthetic table per input table plus a single private recipe
bundle. Columns that are shared across tables - a site code, a
genotype name, a plot id appearing in several tables - are aliased
identically everywhere they occur, so a join written against the
synthetic set still resolves on the masked data.
mask_set( input, roles = NULL, links = NULL, mode = c("local", "collaborate"), seed = NULL, clean = c("auto", "report", "off"), alias_names = FALSE, conditional = FALSE, ladder = c("hierarchy", "levels"), grain = TRUE, allow_kept_outcome = FALSE, quiet = FALSE, sheets = NULL, info = NULL, ... )mask_set( input, roles = NULL, links = NULL, mode = c("local", "collaborate"), seed = NULL, clean = c("auto", "report", "off"), alias_names = FALSE, conditional = FALSE, ladder = c("hierarchy", "levels"), grain = TRUE, allow_kept_outcome = FALSE, quiet = FALSE, sheets = NULL, info = NULL, ... )
input |
A folder path, an |
roles |
Optional named list of roles tables, one per input table
(names must match). When |
links |
Optional override for link detection: a character vector
of column names to treat as cross-table links, or |
mode |
Either |
seed |
Optional integer for reproducibility. |
clean |
Hygiene mode passed to |
alias_names |
Hide column names. |
conditional |
Logical scalar (default |
ladder |
Drop order of the conditioning ladder, |
grain |
Single logical (default |
allow_kept_outcome |
Single logical, passed to each |
quiet |
Logical; suppress the link / hygiene report, including the hint that a table looks like documentation. |
sheets |
For an Excel workbook, the sheets to read (default: all);
passed to |
info |
Optional character vector of table names (sheet names, or
file names without extension) to set aside as documentation, for
example a data dictionary. They are set aside before masking: not
masked, not in the synthetic set, not written by |
... |
Must be empty: any argument here is refused with
|
A masque_set S7 object. Use synthetic() for the named list
of synthetic tables and recipe() for the private recipe bundle.
A link is a column name that appears in two or more tables with a
compatible kind and overlapping values - the join keys of the set.
mask_set() proposes links automatically and prints them; pass
links to override. Linked columns are aliased in place with a shared
map (built over the union of values across all tables), so identical
original values map to identical aliases in every table. The aliases
read <col>_K001, <col>_K002, ..., assigned at random and, when seed
is set, the same for the same seed. Set a linked column's action to
keep in its roles table to pass it through unmasked instead.
mask(), read_set(), write_set(), audit_mask(),
data_card(), reveal_maps().
tables <- list( plots = data.frame( site = rep(c("A", "B", "C", "D", "E"), each = 2), gen = rep(c("x", "y"), 5), yield = c(3.1, 2.9, 4.0, 3.7, 3.3, 3.5, 2.8, 3.0, 3.9, 3.6) ), sites = data.frame( site = c("A", "B", "C", "D", "E"), rainfall = c(420, 560, 390, 610, 480) ) ) roles <- lapply(tables, propose_roles, mode = "collaborate") roles <- lapply(roles, set_role, "site", action = "alias") m <- mask_set(tables, roles = roles, seed = 1, quiet = TRUE) synthetic(m)$plots$site synthetic(m)$sites$site # same aliases -> the join still workstables <- list( plots = data.frame( site = rep(c("A", "B", "C", "D", "E"), each = 2), gen = rep(c("x", "y"), 5), yield = c(3.1, 2.9, 4.0, 3.7, 3.3, 3.5, 2.8, 3.0, 3.9, 3.6) ), sites = data.frame( site = c("A", "B", "C", "D", "E"), rainfall = c(420, 560, 390, 610, 480) ) ) roles <- lapply(tables, propose_roles, mode = "collaborate") roles <- lapply(roles, set_role, "site", action = "alias") m <- mask_set(tables, roles = roles, seed = 1, quiet = TRUE) synthetic(m)$plots$site synthetic(m)$sites$site # same aliases -> the join still works
The front door. masque() walks the whole procedure - read the data,
propose column roles, (in an interactive session) pause for you to
review the plan, mask, audit, and optionally write the result - from a
single call. It dispatches on the input: a single file or data frame
goes through mask(); a folder, an Excel workbook, or a named list of
tables goes through mask_set().
masque( input, roles = NULL, out = NULL, mode = c("local", "collaborate"), seed = NULL, clean = c("auto", "report", "off"), alias_names = FALSE, conditional = FALSE, ladder = c("hierarchy", "levels"), ask = interactive(), overwrite = FALSE, quiet = FALSE, allow_high = FALSE, detect = TRUE, sheets = NULL, info = NULL, ... )masque( input, roles = NULL, out = NULL, mode = c("local", "collaborate"), seed = NULL, clean = c("auto", "report", "off"), alias_names = FALSE, conditional = FALSE, ladder = c("hierarchy", "levels"), ask = interactive(), overwrite = FALSE, quiet = FALSE, allow_high = FALSE, detect = TRUE, sheets = NULL, info = NULL, ... )
input |
A data frame, a single tabular file ( |
roles |
Optional. A roles table (single-table input) or named list of roles tables (set input). When supplied, the interactive review is skipped. |
out |
Optional output path. For a single table, a |
mode |
Either |
seed |
Optional integer for reproducibility. |
clean |
Hygiene mode passed to |
alias_names |
Hide column names; see |
conditional |
Logical scalar (default |
ladder |
Drop order of the conditioning ladder, |
ask |
Whether to pause for interactive review when |
overwrite |
Passed to the writer when |
quiet |
Suppress progress messages. Warnings - including the HIGH-leakage finding - are never suppressed. |
allow_high |
Logical (default |
detect |
Logical (default |
sheets |
For an Excel workbook, the sheets to read (default: all).
See |
info |
For a set, names of documentation tables to set aside: not
masked and not written. See |
... |
Further arguments for the masking verb: |
It is also fully scriptable. Pass an edited roles table (or named
list of them) to skip the interactive review, and an out path to
write the masked result in one go. The returned object is the same
masque / masque_set you would get from the lower-level verbs, so
anything you can do with those you can do with the result here.
A masque object (single-table input) or a masque_set
object (set input), invisibly. Use synthetic() and recipe().
Read the input into one or more clean rectangular tables.
Propose roles for every column (skipped if you pass roles).
Review - in an interactive session with no roles supplied,
the proposed plan is printed and you are asked to proceed, edit, or
stop. Editing opens the roles table in the spreadsheet editor
(utils::edit()) where the platform provides one; when it cannot
start (for example macOS without XQuartz, or a headless session), a
console editor takes over - pick a column, then a role and an
action from numbered menus, with every change applied through
set_role() so default actions re-resolve exactly as on the
scriptable path. With ask = FALSE (the default in
non-interactive use) the proposed plan is used as-is, with a note.
Mask the data in the chosen mode.
Audit - in collaborate mode the leakage audit runs and its
headline is printed. A HIGH finding surfaces as a classed warning
(masque_high_leakage) - it is never silenced by the guided flow.
Write - if out is set, the masked data is written there
(mirroring the input format), unless the audit left unresolved
HIGH findings: then the write is refused (nothing is written) and
the flagged columns are listed. Resolve them and mask again, or
pass allow_high = TRUE after your own review - the override is
warned (masque_high_override) and recorded on the recipe. The
private recipe is never written automatically; persist it yourself
with save_recipe().
mask(), mask_set(), propose_roles(), set_role(),
apply_recipe() and unmask() for the round trip, save_recipe() to
keep the recipe, audit_mask(), data_card() and roles() to describe
the clone, write_set().
# Scripted single-table use (no prompt because roles are supplied): r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") m <- masque(iris, roles = r, seed = 1, ask = FALSE, quiet = TRUE) head(synthetic(m))# Scripted single-table use (no prompt because roles are supplied): r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") m <- masque(iris, roles = r, seed = 1, ask = FALSE, quiet = TRUE) head(synthetic(m))
Plots the structure that drove the detect_design() verdict. A detected MET
uses a compact environment-coverage overview by default. Set environment
to draw one selected environment. A single-trial panel depends on the
detected design class:
plot_design_summary( x, df, engine = c("base", "ggplot2"), environment = NULL, ... )plot_design_summary( x, df, engine = c("base", "ggplot2"), environment = NULL, ... )
x |
A |
df |
The data frame that was passed to |
engine |
|
environment |
Optional single environment label. For a detected MET, the default is a compact all-environment overview. Supplying a label draws the selected environment's field layout or legacy design diagnostic. |
... |
Ignored. |
CRD, factorial, none -> frequency-of-treatment + NA-pattern.
RCBD, IBD/alpha-lattice -> treatment x block replication tile.
row-column -> spatial layout tile (row x col, fill =
treatment).
split-plot -> factor-nesting tree + within-block replication.
Output is purely diagnostic. Do not use it as a publication figure
(use desplot::desplot() or ggplot2-based packages for that).
With engine = "ggplot2", the ggplot object, so it can be
modified or saved with ggplot2::ggsave(); nothing is drawn until it is
printed. With engine = "base", the input x invisibly, after the plot
has been drawn.
if (requireNamespace("agridat", quietly = TRUE)) { d <- agridat::john.alpha ds <- detect_design(d) plot(ds, df = d) } met <- expand.grid( env = factor(c("E1", "E2")), rep = factor(seq_len(2L)), gen = factor(c("G1", "G2", "G3")) ) met$yield <- seq_len(nrow(met)) met_design <- detect_design(met, env = "env") plot(met_design, df = met)if (requireNamespace("agridat", quietly = TRUE)) { d <- agridat::john.alpha ds <- detect_design(d) plot(ds, df = d) } met <- expand.grid( env = factor(c("E1", "E2")), rep = factor(seq_len(2L)), gen = factor(c("G1", "G2", "G3")) ) met$yield <- seq_len(nrow(met)) met_design <- detect_design(met, env = "env") plot(met_design, df = met)
Generates a heuristic two-axis classification for every column of
df: a role (what the column is) and an action (what mask()
will do to it). The user is expected to inspect this table and edit
it - directly or via set_role() - before passing it to mask().
propose_roles(df, mode = c("local", "collaborate"), detect = TRUE)propose_roles(df, mode = c("local", "collaborate"), detect = TRUE)
df |
A data frame. Must have at least one column. |
mode |
The masking mode the table is being prepared for:
|
detect |
Logical scalar (default |
A data.table with one row per column: col, role, action,
kind (storage kind: numeric, integer, factor, character,
logical, date, datetime, other), freq_or_range,
pii_suspected, and notes. The target mode is stored as
attr(roles, "mode").
role describes the column and determines the mechanics of any
synthesis:
designExperimental / structural columns (site, block,
rep, plot, year). Mechanics of alias: labels are substituted in
place, structure intact.
treatmentAssignment columns (variety, genotype, dose).
Labels are remapped in place - the assignment structure never
moves. scramble = seeded label permutation; alias = opaque
labels (trt_001).
outcomeNumeric response columns. scramble re-simulates
via the Gaussian copula, jointly with scrambled numeric
covariates. Multiple outcomes are supported.
covariateEverything measured alongside. Numeric:
copula re-simulation. Categorical: row permutation, plus opaque
label aliasing under alias.
dateDate / POSIX / difftime columns. scramble row-
permutes within the observed values; class and NA pattern are
preserved.
idIdentifier columns. Never scrambled (that would break
row linkage); alias substitutes opaque per-value labels in
place, preserving linkage.
textFree-text columns. scramble row-permutes; alias
tokenises each distinct string.
otherClasses masque cannot synthesise (list columns, exotic S4, ...). Keep or drop only.
action sets the masking depth per column:
keepByte-identical pass-through, both modes.
scrambleRe-simulate (numeric) or row-permute (categorical / date / text); original label vocabulary remains visible.
aliasScramble where applicable, plus opaque label substitution - the vocabulary itself is hidden.
dropColumn excluded from the synthetic, both modes.
The proposed action column is resolved for mode, so the table you
edit shows the actual masking plan. Re-assigning a column's role
with set_role() re-resolves its default action; a direct
roles$role[...] <- ... edit leaves action untouched (set it to
NA to have mask() re-resolve the default).
PII-pattern column names (contact, email, phone, gps,
latitude / longitude, postcode, ssn, password, owner,
farmer, operator, etc., case-insensitive substring) ->
pii_suspected = TRUE and action drop in both modes.
Re-role deliberately if the column must survive.
Date / POSIXct / POSIXlt / difftime columns -> role date,
action scramble (row permutation).
ID-pattern names (\\bid\\b, _id$, ^id_) -> role id; kept
in local mode, dropped in collaborate mode.
Design-pattern names (rep, block, row, col(umn)?,
range, plot(no)?, site, site_name, trial_site,
env(ironment)?, trial, year, season, colrep, tos; the
two-word names also spaced or dotted, as in Site Name and
Site.Name) -> role design, action keep.
Treatment-pattern names (treatment, variety, cultivar,
genotype, ^trt, ^dose) -> role treatment; kept in local
mode, aliased in collaborate mode.
Numeric columns with an outcome-pattern name (yield, yield_t_ha,
Yield..t.ha., grain_yield, response, outcome) -> role
outcome, action scramble.
Character columns with > 50% unique values on non-NA -> role
text; kept in local mode, dropped in collaborate mode.
Unsupported classes -> role other, action keep, with a note.
Everything else -> role covariate, action scramble. Re-role
other response variables as outcome.
No outcome is required: with no column roled outcome, the copula
simply re-simulates all scrambled numeric columns jointly.
Since masque 0.3.0, propose_roles() also calls detect_design() by
default (detect = TRUE) and applies the detected design's
recommended_roles on top of the name-based heuristic, re-resolving
the default action for any promoted column. The design summary is
stashed as attr(roles, "design"). Pass detect = FALSE for the
name-only heuristic.
For a column without a treatment-pattern name, a treatment hint from the
design rules is applied only when the column is crossed with a design
column: a design-pattern name, a column roled design, or a detected
environment. A column with one value per level of a design
column is a unit-level covariate and is never promoted. Hints that fail
are listed in attr(roles, "design")@recommended_roles with
auto_apply = FALSE and the reason.
Since masque 0.9.1, high-confidence environment columns detected by
detect_design() are promoted to role design. In local mode their values
remain byte-identical. In collaborate mode categorical environment labels
default to alias, which preserves row assignment and factor codes while
hiding the vocabulary. Numeric environment columns default to keep and
raise a masque_environment_disclosure warning for explicit review.
Weak or competing environment candidates never change roles automatically.
Inspect attr(roles, "design") and use set_role() when domain knowledge
should override an uncertain result. Preserving environment allocation does
not imply that synthesised outcomes preserve treatment-by-environment
effects.
An explicitly chosen action is pinned and is not overwritten by a later
design-role promotion.
set_role() to edit; roles_validate() for the fail-closed
validation applied at mask() time.
propose_roles(iris) propose_roles(iris, mode = "collaborate") met <- expand.grid( env = factor(c("E1", "E2")), rep = factor(seq_len(2L)), gen = factor(c("G1", "G2", "G3")) ) propose_roles(met, mode = "collaborate")[, c("col", "role", "action")]propose_roles(iris) propose_roles(iris, mode = "collaborate") met <- expand.grid( env = factor(c("E1", "E2")), rep = factor(seq_len(2L)), gen = factor(c("G1", "G2", "G3")) ) propose_roles(met, mode = "collaborate")[, c("col", "role", "action")]
Loads a recipe written by save_recipe(). Validates that the file
contains a masque_recipe or a masque_recipe_set bundle and informs
(does not error) if it was written by a different package version than
the one currently installed. Bundles are read from masque 0.16.0 on.
read_recipe(path)read_recipe(path)
path |
File path. |
A masque_recipe, or a masque_recipe_set bundle.
r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" m <- mask(iris, r, mode = "collaborate", seed = 1) tmp <- tempfile(fileext = ".rds") save_recipe(recipe(m), tmp) rec2 <- read_recipe(tmp) rec2r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" m <- mask(iris, r, mode = "collaborate", seed = 1) tmp <- tempfile(fileext = ".rds") save_recipe(recipe(m), tmp) rec2 <- read_recipe(tmp) rec2
Ingests a multi-table dataset into a named list of data frames, ready
for mask_set(). Three input shapes are accepted:
read_set(input, sheets = NULL, pattern = NULL)read_set(input, sheets = NULL, pattern = NULL)
input |
A folder path, an |
sheets |
For an Excel workbook, an optional character vector of
sheet names to read (default: all sheets). Any other input with
|
pattern |
For a folder, an optional regular expression to select
files (default: all |
Every .csv, .tsv, and .fst file in the
folder becomes one table, named by its file name (without
extension). CSV / TSV are read with data.table::fread(); .fst
needs the Suggested fst package.
.xlsx / .xls)Every sheet becomes one
table, named by its sheet name. Needs the Suggested readxl
package.
Returned as-is after validation.
Only clean rectangular tables are supported. A sheet or file that does
not read as a rectangle - missing header row (auto-named ...1
columns), zero rows, or zero columns - fails with an explanatory error
naming the offending table. masque does not attempt to recover
merged cells, multi-row headers, or junk rows; tidy the source first.
A named list of data.tables.
tables <- list( plots = data.frame(site = c("A", "B"), yield = c(3.1, 4.2)), sites = data.frame(site = c("A", "B"), rainfall = c(400, 550)) ) read_set(tables)tables <- list( plots = data.frame(site = c("A", "B"), yield = c(3.1, 4.2)), sites = data.frame(site = c("A", "B"), rainfall = c(400, 550)) ) read_set(tables)
The recipe is private: at least as sensitive as the original data.
Never share alongside the synthetic. By default print(recipe(m))
redacts all level maps; use reveal_maps() for an explicit reveal.
recipe(m)recipe(m)
m |
A |
A masque_recipe, or for a set a masque_recipe_set bundle.
synthetic(), reveal_maps(), mask(), mask_set().
r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" m <- mask(iris, r, seed = 1) recipe(m)r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" m <- mask(iris, r, seed = 1) recipe(m)
Explicit, audited reveal: prints each original -> synthetic level map
held by the recipe. Use sparingly. Recipe maps are at least as
sensitive as the original data; printing them defeats the redaction
built into the print(), format(), str() and knitr::knit_print()
methods of recipes, recipe bundles, masque and masque_set objects.
saveRDS() and dput() write the maps by design.
reveal_maps(rec, table = NULL)reveal_maps(rec, table = NULL)
rec |
A |
table |
For a bundle, the names of the tables whose maps to reveal.
|
For each recipe it prints the column-name map (when names were
aliased), every level map as original -> alias, and the seed. The
alias labels follow the scheme described under alias in mask():
trt_NNN or <col>_trt_NNN for treatments, <col>_LNNN, <col>_DNNN,
<col>_INNN and <col>_TNNN for covariates, design columns, ids and
text, and <col>_KNNN for a key shared across the tables of a set.
rec, invisibly.
r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" r$role[r$col == "Species"] <- "covariate" m <- mask(iris, r, mode = "collaborate", seed = 1) rec <- recipe(m) reveal_maps(rec)r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" r$role[r$col == "Species"] <- "covariate" m <- mask(iris, r, mode = "collaborate", seed = 1) rec <- recipe(m) reveal_maps(rec)
Renders the two-axis vocabulary as a table: every role paired with
every action, the storage kinds the pair works for, and the reason
the pair is constrained when it is. The table is generated from the
same compatibility rules roles_validate() applies at mask() time,
so what it shows is exactly what a roles table is allowed to say.
role_options(kind = NULL)role_options(kind = NULL)
kind |
Optional storage kind to filter for: one of |
A column's kind is never chosen: propose_roles() derives it from
the column's class. To move a column to a different kind, convert the
column in the data and re-propose.
A data.table with one row per role-action pair and four columns:
role, action, kinds (the storage kinds the pair is valid for:
"all", "none", "all except other", or a comma-separated
list), and notes (why the pair is constrained, empty when it is
not).
propose_roles(), set_role(), roles_validate().
role_options() role_options(kind = "factor") subset(role_options(), role == "design")role_options() role_options(kind = "factor") subset(role_options(), role == "design")
Returns the role each column of the clone was masked under, in the
clone's own vocabulary: one row per column of synthetic(), in the same
order and under the same names. When the column names were aliased
(alias_names in mask()), the rows carry the aliases, so the table can
travel with the clone without disclosing the original names. A dropped
column is not in the clone and has no row.
roles(m)roles(m)
m |
A |
Only the four public columns are returned. The proposal's notes and its
value summaries describe the original table and stay on the private
recipe, as does the full role table used on the original
(recipe(m)@roles).
For a masque, a data.table with columns col, role,
action and kind. For a masque_set, a named list of such tables,
one per table of the set.
synthetic(), data_card(), mask().
r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") m <- mask(iris, r, seed = 1) roles(m)r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") m <- mask(iris, r, seed = 1) roles(m)
Fail-closed validation of a two-axis roles table before mask()
consumes it. Returns the validated table - with any NA actions
resolved to their (role, kind, mode) defaults - so callers can use
the return value directly.
roles_validate(roles, df = NULL, mode = NULL)roles_validate(roles, df = NULL, mode = NULL)
roles |
A roles table from |
df |
Optional data frame. If supplied, |
mode |
Optional mode ( |
Tables produced by masque 0.5.0 and earlier (no action column;
v1 vocabulary with keep / ignore roles and the optional
mask_levels column) are upgraded in place with a deprecation
warning. The upgrade preserves the v1 semantics exactly: v1 keep
becomes action keep; v1 ignore becomes role id / text /
other with action keep in local mode and drop in collaborate
mode; v1 treatment mask_levels = "permute" becomes action
scramble.
Hard errors:
missing required columns (col, role, action, kind);
unknown role (not in design, treatment, outcome,
covariate, date, id, text, other) or unknown action
(not in keep, scramble, alias, drop);
any NA role (an NA action is allowed - it resolves to
the default for the row's role and kind);
an incompatible (role, action, kind) combination, e.g.
design + scramble, numeric + alias, id + scramble,
other + anything but keep / drop;
duplicate col entries;
if df supplied: any df column missing from roles, or
any roles column missing from df.
Loud advisories (warnings, not errors):
every action is keep - the "synthetic" would equal the
original byte-for-byte;
the table was proposed for one mode but is being validated for another (actions are taken as-is; defaults are not re-resolved).
The validated (and possibly upgraded / resolved) roles table, invisibly.
r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") roles_validate(r, iris)r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") roles_validate(r, iris)
Writes the recipe to a single .rds file. The default is
runtime-minimal:
no simulator state (copula covariance, raw margins) is written, only the
translation maps, factor metadata, storage classes, integrity fingerprint,
and warnings. This keeps the saved artefact small and reduces the
information that would leak if the recipe file alone were shared.
save_recipe(rec, path, include_simulator = FALSE)save_recipe(rec, path, include_simulator = FALSE)
rec |
A |
path |
File path ending in |
include_simulator |
Logical. Has no effect; the recipe is always written without simulator state. |
include_simulator = TRUE is accepted and has no effect: the recipe
does not carry simulator state.
Recipes are at least as sensitive as the original data. Protect the saved
file at the same security class as the original. Note that
save_recipe() writes plain R serialisation - it does not encrypt.
The saved recipe is a re-identification key: store it under your
organisation's access controls and key-management practice, not
alongside the synthetic output.
path, invisibly.
r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" m <- mask(iris, r, mode = "collaborate", seed = 1) tmp <- tempfile(fileext = ".rds") save_recipe(recipe(m), tmp) rec2 <- read_recipe(tmp)r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" m <- mask(iris, r, mode = "collaborate", seed = 1) tmp <- tempfile(fileext = ".rds") save_recipe(recipe(m), tmp) rec2 <- read_recipe(tmp)
Ergonomic editor for the two-axis roles table returned by
propose_roles(). Setting a role without an explicit action
re-resolves the action to the default for the new role, the column's
kind, and the mode the table was proposed for – so a re-roled column
never silently carries a stale action from its previous role.
set_role(roles, cols, role = NULL, action = NULL)set_role(roles, cols, role = NULL, action = NULL)
roles |
A roles table from |
cols |
Character vector of column names to edit. Every entry
must be present in |
role |
Optional single role to assign to all of |
action |
Optional single action to assign to all of |
Direct edits (roles$role[roles$col == "x"] <- "outcome") also work,
but leave roles$action unchanged.
The edited roles table.
propose_roles(), roles_validate().
r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") r <- set_role(r, "Species", action = "keep") r[, c("col", "role", "action")]r <- propose_roles(iris) r <- set_role(r, "Sepal.Length", role = "outcome") r <- set_role(r, "Species", action = "keep") r[, c("col", "role", "action")]
Replaces the latitude / longitude values in a masqued data frame with coordinates anchored at user-supplied centroids and clustered to preserve the original's site-count-per-anchor structure. The function never reads the real coordinates beyond counting how many distinct sites the original holds per anchor level – so it leaks the replication-per-site distribution and the count of distinct sites, nothing more.
synthesise_geospatial( synth, original, anchor_col, lat_col, lon_col, anchor_centroids, site_spread_deg = 0.6, jitter_deg = 0.05, seed = NULL )synthesise_geospatial( synth, original, anchor_col, lat_col, lon_col, anchor_centroids, site_spread_deg = 0.6, jitter_deg = 0.05, seed = NULL )
synth |
A synthetic data frame (typically |
original |
The original data frame from which |
anchor_col |
Name of the column whose levels anchor each cluster
(e.g., |
lat_col, lon_col
|
Column names of the latitude and longitude
fields to overwrite in |
anchor_centroids |
Named list keyed by anchor levels; each
element is a length-2 numeric named |
site_spread_deg |
Half-width of the box (in decimal degrees)
around each anchor centroid within which fake site centroids are
uniformly placed. Default |
jitter_deg |
Within-site uniform jitter (in decimal degrees)
added to each row's assigned site centroid. Default |
seed |
Optional integer seed for reproducibility. The function
uses |
Typical use: after mask() produces a synthetic with copula-drawn or
missing coordinates, call synthesise_geospatial() to substitute
plausible points. The synthetic ends up with:
(e.g., if the original has five distinct trial sites in NSW, the synthetic will have five fake sites in NSW);
(each fake site receives a share of the synthetic rows proportional to its real counterpart's count);
(small jitter within site; larger spread between sites within each anchor centroid's neighbourhood).
What the function does not preserve:
the real positions of the sites (they are random within a user-defined neighbourhood of each anchor centroid);
the relative spacing or bearings between real sites;
any spatial autocorrelation in the outcome.
Coordinates that are NA in the original remain NA in the
synthetic – the NA pattern is preserved cell-by-cell.
synth, with lat_col and lon_col overwritten by the
re-anchored coordinates.
# Toy example: 50 rows split across two states. set.seed(1) n <- 50 df <- data.frame( state = sample(c("NSW", "VIC"), n, replace = TRUE), lat = stats::rnorm(n, -33, 0.3), lon = stats::rnorm(n, 145, 0.3), y = stats::rnorm(n) ) roles <- propose_roles(df, detect = FALSE) roles$role[roles$col == "y"] <- "outcome" roles$role[roles$col %in% c("lat", "lon")] <- "covariate" roles$role[roles$col == "state"] <- "design" m <- mask(df, roles, mode = "collaborate", seed = 1L) centroids <- list( NSW = c(lat = -32.5, lon = 147), VIC = c(lat = -36.5, lon = 144) ) synth_geo <- synthesise_geospatial( synthetic(m), df, anchor_col = "state", lat_col = "lat", lon_col = "lon", anchor_centroids = centroids, seed = 2L ) head(synth_geo[, c("state", "lat", "lon")])# Toy example: 50 rows split across two states. set.seed(1) n <- 50 df <- data.frame( state = sample(c("NSW", "VIC"), n, replace = TRUE), lat = stats::rnorm(n, -33, 0.3), lon = stats::rnorm(n, 145, 0.3), y = stats::rnorm(n) ) roles <- propose_roles(df, detect = FALSE) roles$role[roles$col == "y"] <- "outcome" roles$role[roles$col %in% c("lat", "lon")] <- "covariate" roles$role[roles$col == "state"] <- "design" m <- mask(df, roles, mode = "collaborate", seed = 1L) centroids <- list( NSW = c(lat = -32.5, lon = 147), VIC = c(lat = -36.5, lon = 144) ) synth_geo <- synthesise_geospatial( synthetic(m), df, anchor_col = "state", lat_col = "lat", lon_col = "lon", anchor_centroids = centroids, seed = 2L ) head(synth_geo[, c("state", "lat", "lon")])
Extract the synthetic data from a masque object
synthetic(m)synthetic(m)
m |
A |
For a masque, a data.table (the synthetic table); for a
masque_set, a named list of synthetic tables.
r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" m <- mask(iris, r, seed = 1) head(synthetic(m))r <- propose_roles(iris) r$role[r$col == "Sepal.Length"] <- "outcome" m <- mask(iris, r, seed = 1) head(synthetic(m))
Inverse of apply_recipe(). Accepts a data frame or an atomic vector.
For an atomic factor / character vector with a recipe that holds
multiple level maps, column must name which map to invert. Atomic
numeric, integer, logical, and Date / POSIXct vectors are
returned unchanged (no inverse map applies – these are pass-through
under apply_recipe() too).
unmask(x, rec, column = NULL)unmask(x, rec, column = NULL)
x |
A data frame or an atomic vector to translate from synthetic-namespace to original-namespace. |
rec |
A |
column |
Optional column name. Only consulted for atomic
factor / character |
The most common pattern is round-tripping pipeline predictions:
fit <- my_model(synthetic(m)) # train on synthetic orig_in_synth_space <- apply_recipe(original, recipe(m)) # forward preds_synth <- predict(fit, orig_in_synth_space) preds_orig <- unmask(preds_synth, recipe(m)) # inverse
An unknown level (a synthetic alias not in the recipe's map) is an
error; it is never coerced to NA.
An object of the same type as x, in the original namespace.
r <- propose_roles(iris) r$role[r$col == "Species"] <- "treatment" m <- mask(iris, r, mode = "collaborate", seed = 1) synth <- synthetic(m) rec <- recipe(m) # round-trip a synthetic-namespace value back to its original label head(unmask(synth$Species, rec, column = "Species"))r <- propose_roles(iris) r$role[r$col == "Species"] <- "treatment" m <- mask(iris, r, mode = "collaborate", seed = 1) synth <- synthetic(m) rec <- recipe(m) # round-trip a synthetic-namespace value back to its original label head(unmask(synth$Species, rec, column = "Species"))
Writes the synthetic tables of a mask_set() result to disk. The
output format mirrors the request: a .xlsx path produces one
workbook with one sheet per table; a folder path produces one .csv
per table. The private recipe bundle is never written by this
function - persist it separately with save_recipe() and protect it
at the same security class as the original data.
write_set(m, path, overwrite = FALSE, allow_high = FALSE)write_set(m, path, overwrite = FALSE, allow_high = FALSE)
m |
A |
path |
Output location. A path ending in |
overwrite |
Logical. When |
allow_high |
Logical (default |
In collaborate mode the mask-time audit gates the write: unresolved
HIGH leakage findings refuse the write (nothing is written) until the
flagged columns are re-roled, aliased, or dropped - or the refusal is
explicitly overridden with allow_high = TRUE, which raises a
masque_high_override warning so the exception stays visible.
path, invisibly.
mask_set(), read_set(), save_recipe().
tables <- list( plots = data.frame(site = c("A", "B"), yield = c(3.1, 4.2)), sites = data.frame(site = c("A", "B"), rain = c(420, 560)) ) m <- mask_set(tables, seed = 1, quiet = TRUE) dir <- file.path(tempdir(), "masked_set") write_set(m, dir) list.files(dir)tables <- list( plots = data.frame(site = c("A", "B"), yield = c(3.1, 4.2)), sites = data.frame(site = c("A", "B"), rain = c(420, 560)) ) m <- mask_set(tables, seed = 1, quiet = TRUE) dir <- file.path(tempdir(), "masked_set") write_set(m, dir) list.files(dir)