Package 'ballast'

Default inference settings for stress_inference()

Description

Default inference settings for stress_inference()

Usage

default_inference()

Value

A named list of list(method = , vcov = ) pairs: Satterthwaite, Kenward-Roger, linear Kenward-Roger (the SAS DDFM=KR default), residual, sandwich with between-within degrees of freedom (SAS EMPIRICAL) and the jackknife sandwich.

Examples

names(default_inference())

Description

Replaces subject identifiers in the labels of the leave-one-subject-out and deletion-search rows with sequence numbers (S001, S002, ...) in the order of the spec's subject levels, so that a census, report or page can leave the organisation without participant identifiers. The mapping is returned as attr(, "id_map").

Usage

mask_ids(census, spec)

Arguments

Value

The same object with masked labels.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:40), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
cen <- stress_census(spec, stress_loo(spec))
head(mask_ids(cen, spec)$label)

Description

The empirical cumulative distribution of the p-values of every counted refit, against the uniform diagonal that a result with no true effect would produce (the p-value of each refit of a null result is uniform, so the curve of a null census hugs the diagonal; a census far above it is far from random). One curve per census when several are given, with the pooled curve in black; the share below alpha is annotated.

Usage

plot_pcdf(x, alpha = NULL, pooled = TRUE)

Arguments

Details

The refits are not independent, so the curve is a description, not a test.

Value

A ggplot object.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
cen <- stress_census(spec, stress_covariance(spec, c("cs", "ar1h")), stress_loo(spec, by = "RACE"))
if (requireNamespace("ggplot2", quietly = TRUE)) plot_pcdf(cen)

Description

Draws the p-value against the parameter for every family of rows that carries a param value (from stress_tipping() or stress_curve()): one line per family, the alpha reference line, the anchor point (the row of kind = "alternative", usually the unshifted analysis) emphasised, and a dotted vertical line at each interpolated crossing with its value.

Usage

plot_pcurve(
  rows,
  alpha = NULL,
  xlab = "Perturbation",
  ylab = "p-value",
  log_p = FALSE
)

Arguments

Value

A ggplot object.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:80), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL)
tp <- stress_tipping(spec, deltas = c(0, -4, -8), m = 3)
if (requireNamespace("ggplot2", quietly = TRUE)) plot_pcurve(tp)

Description

Each dot is one refit with one unit removed, on a p-value axis (linear by default, log on request), one strip per leave-one-out family; the diamond is the pre-specified analysis with nothing removed and the dashed line is alpha. Counts of deletions below alpha label each strip.

Usage

plot_pstrip(x, alpha = NULL, log_p = FALSE, xmax = NULL, label_family = TRUE)

Arguments

Value

A ggplot object.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
cen <- stress_census(spec, stress_loo(spec, by = "RACE"), stress_loo(spec, by = "SEX"))
if (requireNamespace("ggplot2", quietly = TRUE)) plot_pstrip(cen)

Description

One row per census. Each bubble is one distinct estimate among the counted alternative analyses (rounded to digits), with area proportional to the number of analyses giving that value and shade by the share of those analyses that are significant; bubbles holding five or more analyses are numbered. A ring marks the pre-specified estimate, the shaded band holds the central 95% of the distinct values, the heavy vertical line is no difference, and the least favourable alternative is labelled. Deletions are a strip of small light dots beneath the swarm.

Usage

plot_swarm(
  x,
  scale = c("percent", "estimate"),
  digits = 2,
  r0 = 0.004,
  favours = NULL,
  xlab = NULL,
  xlim = NULL
)

Arguments

Value

A ggplot object (uses a fixed aspect ratio so bubbles are round).

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
cen <- stress_census(spec, stress_covariance(spec, c("us", "cs", "ar1h", "toep")),
                     stress_covariates(spec), stress_loo(spec, by = "RACE"))
if (requireNamespace("ggplot2", quietly = TRUE)) plot_swarm(cen)

Description

Each counted analysis is a point at its estimate as a percent of the base estimate, filled by significance; the shaded band is the base confidence interval on the same axis and the vertical line is the base estimate. Deletions are drawn in a separate panel from alternative analyses.

Usage

## S3 method for class 'stress_census'
plot(x, ...)

Arguments

Value

A ggplot object.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:80), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
cen <- stress_census(spec, stress_covariance(spec, c("cs", "ar1h")),
                     stress_loo(spec, by = "RACE"))
if (requireNamespace("ggplot2", quietly = TRUE)) plot(cen)

Description

Plot power and robustness power against the sample size

Usage

## S3 method for class 'stress_design'
plot(x, target = 0.8, ...)

Arguments

Value

A ggplot: the probability of a significant result and of a significant and robust result against subjects per arm, one panel per effect size.

Description

Runs every family in a stress_plan(), assembles the census and the verdict, and keeps the rows by family.

Usage

run_plan(plan, cores = 1L, seed = 1)

Arguments

Value

An object of class stress_result: plan, rows (a list of stress_rows by family), census, verdict, base and elapsed.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
plan <- stress_plan(spec, covariance = list(structures = c("cs", "ar1h")),
                    inference = NULL, covariates = NULL, weights = FALSE,
                    loo = list(by = "RACE"), permute = list(n = 9), reference = NULL,
                    estimator = list(estimators = "ancova"), fragility = list(max_k = 1))
res <- run_plan(plan)
res

Description

Run several plans (several endpoints or comparisons) and compare

Usage

run_plans(plans, cores = 1L, seed = 1)

Arguments

Value

A named list of stress_result objects of class stress_results; summary() gives one line per plan and plot_swarm() accepts the censuses (lapply(results, function(r) r$census)).

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
mk <- function(s) stress_plan(s, covariance = list(structures = c("cs", "ar1h")), inference = NULL,
                              covariates = NULL, weights = FALSE, loo = NULL, permute = NULL,
                              reference = NULL, estimator = NULL, fragility = NULL)
res <- run_plans(list(VIS4 = mk(spec), VIS3 = mk(spec_update(spec, contrast_visit = "VIS3"))))
summary(res)

Description

Multivariate-normal responses at n_visits post-baseline visits with an AR(1) correlation, a baseline covariate, a treatment effect growing linearly to delta at the last visit, and dropout after each visit whose probability rises as the current value falls (missing at random).

Usage

simulate_trial(
  n_per_arm = 50,
  delta = 0,
  n_visits = 4,
  sigma = 10,
  rho = 0.6,
  dropout = 0.08,
  baseline_beta = 0.5,
  errors = c("normal", "t"),
  t_df = 3,
  hetero = 1,
  mnar = 0,
  interaction = 0,
  seed = NULL
)

Arguments

Value

A long data frame with id, arm ("REF", "ACT"), base, visit ("V1", ...) and y, missing rows removed.

Examples

d <- simulate_trial(40, delta = 3, seed = 1); head(d)

Description

Returns a copy of spec with the named settings replaced. Settings that change the data conventions (reference, visit, treatment, subject, response) cannot be changed here; build a new spec instead.

Usage

spec_update(spec, ...)

Arguments

Value

The modified stress_spec.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
spec_update(spec, covariance = "ar1h", method = "Satterthwaite", vcov = "Asymptotic")

Description

A recommended, citable set of families with fixed settings, so that a statistical analysis plan can say "the standard stress test of ballast" and mean one thing. It contains, in order:

Usage

standard_plan(
  spec,
  site = NULL,
  strata = site,
  baseline = NULL,
  deltas = NULL,
  shift_visits = NULL,
  ice = NULL,
  m = 25,
  n_perm = 2000,
  max_k = 5
)

Arguments

Details

1. covariance structure: us, ar1h, ar1, cs, csh, toep, toeph;

2. inference: Satterthwaite, Kenward-Roger, linear Kenward-Roger, residual, sandwich with between-within df, jackknife sandwich;

3. covariate set: each covariate dropped, all dropped;

4. LS-mean weighting: equal and observed margins;

5. leave-one-out: every subject, and every level of site if given;

6. re-randomisation test: 2,000 permutations, within strata if given;

7. reference-based multiple imputation: MAR, J2R, CR, CIR, 25 imputations;

8. delta-adjusted multiple imputation over deltas if given;

9. estimator class: complete cases, GEE independence, ANCOVA, robust ANCOVA, plus a random intercept for site and cLDA for baseline if given;

10. greedy deletion search for the loss of significance, up to 5 subjects;

11. intercurrent-event strategies if ice is given (treatment policy, hypothetical, while on treatment).

Value

A stress_plan.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
standard_plan(spec, site = "RACE", baseline = "FEV1_BL")

Description

Simulates trials from a planned design and runs a light stress test on each (covariance, inference, covariate, weighting and leave-one-out families, plus the greedy deletion search if fragility = TRUE), so that a sponsor can see, before unblinding, how often a trial of this design would be classed robust, sensitive or fragile at each power, and how often the marginal flag would be set.

Usage

stress_calibrate(
  n_per_arm = 50,
  power = c(0, 0.5, 0.8, 0.95),
  n_trials = 100,
  n_visits = 4,
  sigma = 10,
  rho = 0.6,
  dropout = 0.08,
  baseline_beta = 0.5,
  errors = "normal",
  t_df = 3,
  hetero = 1,
  mnar = 0,
  interaction = 0,
  fragility = FALSE,
  cores = 1L,
  seed = 1
)

Arguments

Value

A data frame of class stress_calibration with one row per power: the effect size, observed power, and the proportion of trials with each verdict, the marginal flag, and (if run) a one-subject fragility; attr(, "trials") holds the per-trial rows.

Examples

stress_calibrate(n_per_arm = 20, power = c(0, 0.8), n_trials = 2)

Description

Binds the rows returned by the family functions and scores each against the pre-specified (base) fit: percent of the base estimate, shift in base standard errors, whether the estimate lies inside the base confidence interval, keeps the base direction, is significant at alpha, and whether significance changed. Rows that reproduce the base (is_base), did not converge, are tipping-point searches, or duplicate an earlier row are not counted.

Usage

stress_census(spec, ..., base = NULL)

Arguments

Value

A data frame of class stress_census with the family rows plus pct_of_base, shift_se, inside_ci, same_direction, significant, crossed, duplicate (a later row numerically identical to an earlier counted row, so the same specification reached by two families), counted; the base row is attr(, "base").

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
cen <- stress_census(spec, stress_covariance(spec, c("us", "cs", "ar1h")),
                     stress_covariates(spec))
summary(cen)
stress_verdict(cen)

Description

Refits the pre-specified model under each covariance structure. The row that reproduces the spec's own structure is flagged is_base.

Usage

stress_covariance(
  spec,
  structures = c("us", "ar1h", "ar1", "cs", "csh", "toep", "toeph"),
  cores = 1L
)

Arguments

Value

A stress_rows data frame, one row per structure.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_covariance(spec, c("us", "cs", "ar1h"))

Description

Refits the model with each covariate dropped in turn, with no covariates, and with any user-supplied covariate sets.

Usage

stress_covariates(
  spec,
  drop_each = TRUE,
  drop_all = TRUE,
  sets = list(),
  cores = 1L
)

Arguments

Value

A stress_rows data frame.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_covariates(spec, sets = list("with weight" = ~ RACE + SEX + FEV1_BL + WEIGHT))

Description

Applies a user-supplied modification to the spec's data at each value of a continuous parameter, refits the pre-specified model and returns one row per value with the value in param. This generalises tipping-point searches that are not multiple-imputation based: assigning a contested participant a post-deviation trajectory, shifting one arm's observed values, moving a baseline definition, and so on. Rows are kind = "search" except the row at anchor, which is a counted alternative.

Usage

stress_curve(
  spec,
  values,
  modify,
  family = "perturbation curve",
  anchor = NULL,
  role = "sensitivity",
  cores = 1L
)

Arguments

Value

A stress_rows data frame, one row per value, with param set.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL)
# shift every observed value of one subject on the active arm
shift_pt3 <- function(d, v) { k <- d$USUBJID == "PT3"; d$FEV1[k] <- d$FEV1[k] + v; d }
cv <- stress_curve(spec, values = c(-20, -10, 0, 10, 20), modify = shift_pt3, anchor = 0)
tipping_point(cv)

Description

The generic data-decision family. Each alternative is a data frame in the spec's layout (an intercurrent-event handling, a different baseline definition, a contested record reversed, a different visit window). The pre-specified model is refitted on each.

Usage

stress_data(
  spec,
  alternatives,
  family = "data handling",
  role = "supplementary",
  cores = 1L
)

Arguments

Value

A stress_rows data frame, one row per alternative.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
alt <- mmrm::fev_data
alt$FEV1[alt$USUBJID == "PT1"] <- NA   # one subject's records set missing
stress_data(spec, list("PT1 removed" = alt))

Description

Simulates trials from a planned design over a grid of sample sizes and effect sizes, runs the light stress test on each (covariance, inference, covariate, weighting and leave-one-subject-out families) and reports, for every cell, the conventional power next to the robustness power: the probability that the pre-specified analysis is significant and classed robust. The deletion search and a delta-adjusted tipping grid are optional and give the expected fragility index and the shift the result can absorb. This is the design-stage use of the stress test: choose the sample size so that a true effect is expected to come out robust rather than only significant, and write the expected verdict rates into the statistical analysis plan before unblinding so that the observed verdict has a pre-registered comparison.

Usage

stress_design(
  n_per_arm,
  delta,
  n_trials = 100,
  n_visits = 4,
  sigma = 10,
  rho = 0.6,
  dropout = 0.08,
  baseline_beta = 0.5,
  covariance = "us",
  method = "Satterthwaite",
  vcov = "Asymptotic",
  alpha = 0.05,
  alternatives = c("us", "ar1h", "cs"),
  loo = TRUE,
  fragility = FALSE,
  max_k = 5,
  tipping = FALSE,
  shifts = seq(0, 2, by = 0.25),
  m = 5,
  errors = "normal",
  t_df = 3,
  hetero = 1,
  mnar = 0,
  interaction = 0,
  cores = 1L,
  seed = 1
)

## S3 method for class 'stress_design'
print(x, target = 0.8, ...)

Arguments

Details

The light family set offers the alternative covariance structures, three inference methods, the model without the baseline covariate, observed-margin weighting and, if loo = TRUE, every single-subject deletion. Deletions dominate the run time at large sample sizes; loo = FALSE gives a fast first pass whose verdicts omit the deletion family.

Value

A data frame of class stress_design with one row per cell: n_per_arm, delta, trials, power (proportion significant), robustness_power (significant and robust), robust_given_sig, sensitive_given_sig, fragile_given_sig, marginal_given_sig, k1_given_sig (one deletion loses significance), k_reached_given_sig and k_median (deletion search, if run), tipping_margin (median shift that crosses alpha among significant trials, as a multiple of delta or sigma) and tipping_absorbed (share of significant trials with no crossing within the grid), and max_shift_se (median largest shift in pre-specified standard errors). attr(, "trials") holds the per-trial rows and attr(, "design") the settings. plot.stress_design() draws power and robustness power against the sample size.

See Also

stress_calibrate() for the verdict rates of one design at a set of planned powers; stress_sap() for the analysis-plan text.

Examples

des <- stress_design(n_per_arm = c(40, 80), delta = c(0, 5), n_trials = 4, loo = FALSE)
des

Description

Declares the intercurrent events that occurred and derives the analysis data set implied by each handling strategy, then refits the pre-specified model on each. Strategies per event type:

Usage

stress_estimand(
  spec,
  ice,
  strategies = c("treatment_policy", "hypothetical", "while_on_treatment", "composite"),
  composite_value = NULL,
  cores = 1L
)

Arguments

Details

• "treatment_policy": use the data as observed, whatever happened.

• "hypothetical": set the responses at and after the event to missing; the model's missing-at-random assumption then supplies the value the subject would have had without the event.

• "while_on_treatment": drop the records at and after the event.

• "composite": replace the responses at and after the event with an unfavourable value: a number, or a function ⁠function(data, rows)⁠ returning one value per affected row (the worst observed change in the subject's arm at that visit, say).

With several event types the strategies are crossed, one row per combination, and the row that applies "treatment_policy" to every type reproduces the pre-specified data and is flagged is_base. Every other row targets a different estimand, so they are role = "supplementary"; a hypothetical strategy can be followed by stress_reference() on the derived data for its reference-based sensitivity analyses.

Value

A stress_rows data frame, one row per strategy combination; attr(, "data") holds the derived data sets by label.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:80), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
ice <- data.frame(USUBJID = c("PT2", "PT5", "PT9"), AVISIT = c("VIS3", "VIS2", "VIS4"),
                  type = c("discontinuation", "discontinuation", "rescue"))
strat <- list(discontinuation = c("treatment_policy", "hypothetical"),
              rescue = c("treatment_policy", "composite"))
stress_estimand(spec, ice, strategies = strat, composite_value = 20)

Description

Refits the question with estimators other than the pre-specified MMRM:

Usage

stress_estimator(
  spec,
  estimators = c("complete_cases", "gee_independence", "ancova", "robust_ancova",
    "random_intercept", "clda"),
  random = NULL,
  baseline = NULL,
  seed = 1,
  cores = 1L
)

Arguments

Details

• "complete_cases": the pre-specified MMRM on subjects observed at every visit (no reliance on missing-at-random within the likelihood).

• "gee_independence": working-independence weighted least squares with a visit-specific residual variance and a cluster-robust (sandwich) variance, between-within degrees of freedom; no within-subject covariance is modelled. Needs nlme.

• "ancova": analysis of covariance at the contrast visit, observed cases, residual degrees of freedom. Covariate terms involving the visit variable are dropped.

• "robust_ancova": MM-type robust regression at the contrast visit (MASS::rlm), normal-theory inference. Needs MASS.

• "clda": constrained longitudinal data analysis: the raw outcome (baseline plus the response, which must be a change from baseline) is modelled at baseline and every post-baseline visit with one common baseline mean across arms, an unstructured covariance over all visits and the other covariates as fixed effects; the contrast is the difference in the arms' means at the contrast visit. Needs baseline.

• "random_intercept": the pre-specified fixed effects with random as a random intercept instead of a fixed covariate, the spec's covariance structure within subject where nlme has an equivalent (us, ar1h, ar1, cs, csh); normal-theory inference. Needs nlme and random.

Every estimator reports the same least-squares-mean contrast at the contrast visit (class covariates at equal weights, continuous covariates at their means).

Value

A stress_rows data frame, one row per estimator.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_estimator(spec, c("complete_cases", "ancova"))

Description

For one row of a census (an alternative that crossed alpha, say), attributes the difference between its estimate and the pre-specified estimate to subjects. Each subject's influence is the change in the contrast when that subject is removed, computed under the pre-specified settings and again under the alternative's settings; the difference of the two influences says how much more (or less) that subject drives the result under the alternative. Subjects are ranked by the absolute difference.

Usage

stress_explain(spec, row, data = NULL, top = 10, cores = 1L)

Arguments

Details

The alternative's model settings are read from the row (covariance, method, vcov, covariates). For an alternative that used different data, pass that data as data.

Value

A data frame of class stress_explanation with one row per subject: subject, arm, n_visits, infl_base, infl_alt, difference, sorted by absolute difference; the row explained is attr(, "row") and the two full-data estimates are attr(, "estimates").

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:30), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
cen <- stress_census(spec, stress_covariates(spec))
stress_explain(spec, cen[1, ], top = 5)

Description

Fits the model described by a stress_spec() (optionally on alternative data or with alternative settings) and returns the treatment contrast at the contrast visit as a single row in the package's common schema. This is the engine every family function calls.

Usage

stress_fit(
  spec,
  data = NULL,
  label = spec$label,
  family = "base",
  kind = "alternative",
  is_base = is.null(data),
  note = "",
  covariates = spec$covariates,
  covariance = spec$covariance,
  method = spec$method,
  vcov = spec$vcov,
  at = list(),
  role = "sensitivity",
  weights = spec$weights
)

Arguments

Value

A one-row data frame of class stress_rows with columns family, label, kind, role, is_base, converged, note, estimate, stderr, df, lcl, ucl, p, n_subj, n_obs, aicc, param (the numeric value of a continuous perturbation, NA otherwise), covariance, method, vcov, covariates, alpha.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_fit(spec)

Description

A greedy forward search: at each step every remaining subject is removed in turn from the current data, the pre-specified model is refitted, and the subject whose removal moves the result furthest towards the target is dropped for good. The search stops when the target is reached or after max_k steps. The number of steps needed is a continuous-outcome analogue of the fragility index; because the search is greedy it is an upper bound on the true minimum.

Usage

stress_fragility(
  spec,
  target = c("significance", "ci", "direction"),
  max_k = 5,
  candidates = NULL,
  cores = 1L
)

Arguments

Value

A stress_rows data frame with one row per step (kind = "search", so the rows are not counted in a census, since the search crosses the target by construction; param = number removed, label = the subjects removed so far). attr(, "fragility") is a list with target, k (the step at which the target was reached, NA if never), subjects and reached.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
fr <- stress_fragility(spec, max_k = 2)
attr(fr, "fragility")

Description

Fits every combination of the supplied covariance structures, inference settings, covariate sets and data sets (a specification curve). The cell that reproduces the spec is flagged is_base.

Usage

stress_grid(
  spec,
  covariance = spec$covariance,
  inference = list(base = list(method = spec$method, vcov = spec$vcov)),
  covariates = list(base = spec$covariates),
  data = list(base = NULL),
  cores = 1L
)

Arguments

Value

A stress_rows data frame, one row per cell, labelled "covariance | inference | covariates | data".

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_grid(spec, covariance = c("us", "cs"),
            covariates = list(full = ~ RACE + SEX + FEV1_BL, baseline = ~ FEV1_BL))

Description

Writes an HTML page (its JavaScript dependencies in a lib folder beside it) with the verdict, the summary table and two interactive figures built with plotly: the bubble swarm, where hovering a bubble lists the specifications that give that estimate, and the deletion strip, where hovering a dot names the subject removed and shows the refit's estimate and p-value. Needs plotly and htmlwidgets.

Usage

stress_html(result, file, digits = 2, mask_ids = TRUE)

Arguments

Value

The file path, invisibly.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
plan <- stress_plan(spec, covariance = list(structures = c("cs", "ar1h")), inference = NULL,
                    covariates = NULL, weights = FALSE, loo = list(by = "RACE"), permute = NULL,
                    reference = NULL, estimator = NULL, fragility = NULL)
if (requireNamespace("plotly", quietly = TRUE) && requireNamespace("htmlwidgets", quietly = TRUE))
  stress_html(run_plan(plan), file.path(tempdir(), "stress.html"))

Description

Perturbation family: degrees-of-freedom method and variance estimator

Usage

stress_inference(spec, settings = default_inference(), cores = 1L)

Arguments

Value

A stress_rows data frame, one row per setting.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_inference(spec, default_inference()[c("Satterthwaite", "Kenward-Roger (linear)")])

Description

Records the plan text and its MD5 hash, the seed, the package and R versions, the platform, the elapsed time and the verdict, as JSON if jsonlite is installed and as plain text otherwise. The hash of the plan text lets a reviewer confirm that the stress test run is the one that was pre-specified.

Usage

stress_log(result, file = NULL, seed = 1)

Arguments

Value

The log as a list, invisibly.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
plan <- stress_plan(spec, covariance = list(structures = c("cs", "ar1h")), inference = NULL,
                    covariates = NULL, weights = FALSE, loo = NULL, permute = NULL,
                    reference = NULL, estimator = NULL, fragility = NULL)
str(stress_log(run_plan(plan)))

Description

Refits the model with each subject removed, or with each level of a grouping variable (site, cohort, country) removed. Rows are kind = "deletion" and are summarised separately from alternative analyses.

Usage

stress_loo(spec, by = NULL, cores = 1L)

Arguments

Value

A stress_rows data frame, one row per deleted unit.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:40), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL)
stress_loo(spec, by = "RACE")

Description

Permutes treatment labels across subjects, optionally within strata, refits the pre-specified model each time and returns the permutation p-value for the observed contrast. The returned row carries the observed estimate and the permutation p; the null distribution is attached as attr(, "null").

Usage

stress_permute(
  spec,
  n = 1000,
  strata = NULL,
  statistic = c("estimate", "t"),
  seed = 1,
  cores = 1L
)

Arguments

Value

A one-row stress_rows data frame with p equal to ⁠(1 + number of |null| >= |observed|) / (n + 1)⁠.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL)
stress_permute(spec, n = 19)

Description

A plan names the families to run and their settings before unblinding, so the stress test can be written into a statistical analysis plan and run with one call (run_plan()). Each family argument is NULL to skip the family, an empty list() to run it with its defaults, or a named list of arguments for the family function. Printing the plan gives the text a statistical analysis plan can carry.

Usage

stress_plan(
  spec,
  covariance = list(),
  inference = list(),
  covariates = list(),
  weights = TRUE,
  population = NULL,
  data = NULL,
  loo = list(),
  grid = NULL,
  permute = list(n = 2000),
  tipping = NULL,
  reference = list(),
  estimator = list(),
  fragility = list(),
  curve = NULL,
  estimand = NULL,
  visits = NULL,
  title = "Pre-specified stress test"
)

Arguments

Value

An object of class stress_plan.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
plan <- stress_plan(spec, covariance = list(structures = c("us", "cs", "ar1h")),
                    inference = list(), covariates = list(), loo = list(by = "RACE"),
                    permute = NULL, reference = NULL, estimator = list(estimators = "ancova"),
                    fragility = list(max_k = 2))
plan

Description

Each rule is a function of the spec's data returning a logical vector of rows to keep (a per-protocol flag, a completer definition, a cohort).

Usage

stress_population(spec, rules, role = "supplementary", cores = 1L)

Arguments

Value

A stress_rows data frame, one row per rule.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_population(spec, list("female" = function(d) d$SEX == "Female"))

Description

Model-free tests of the treatment difference in the response at the contrast visit: the Wilcoxon rank-sum test (normal approximation with the tie correction and continuity correction, as SAS ⁠PROC NPAR1WAY⁠ reports it) and, if strata is given, the van Elteren test (the Cochran-Mantel- Haenszel row-mean-score statistic on modified ridit scores within strata). The rows carry a p-value and the Hodges-Lehmann shift as the estimate, with no standard error or interval; they count towards significance only.

Usage

stress_ranktest(spec, strata = NULL, cores = 1L)

Arguments

Value

A stress_rows data frame with one or two rows.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_ranktest(spec, strata = "RACE")

Description

Imputes the missing responses from a multivariate-normal imputation model (an MMRM on the observed data with treatment by visit, the subject-level covariates and an unstructured covariance) under the standard reference-based assumptions (Carpenter, Roger and Kenward, 2013), refits the pre-specified model on each completed data set and pools with Rubin's rules:

Usage

stress_reference(
  spec,
  methods = c("MAR", "J2R", "CR", "CIR"),
  m = 25,
  seed = 1,
  delta = 0,
  arm = NULL,
  impute_covariates = NULL,
  same_cov = TRUE,
  max_tries = 3L,
  cores = 1L
)

Arguments

Details

• MAR: every subject's missing values follow their own arm.

• J2R (jump to reference): an active-arm subject's visits after the last observed one take the reference arm's means.

• CR (copy reference): an active-arm subject with any missing value follows the reference arm's means at every visit.

• CIR (copy increments in reference): after the last observed visit the subject keeps their own-arm mean at that visit and then changes as the reference arm does.

Visits missing before the last observed one are treated as missing at random under every assumption. Parameter uncertainty is carried by approximate Bayesian draws: for each imputation the imputation model is refitted on a bootstrap sample of subjects (stratified by arm) and its fixed effects and covariance are used to draw the missing values from the conditional normal distribution given the subject's observed values. An optional delta is added to every imputed value of the active arm.

Imputation i uses seed seed + i on whichever worker draws it, so the result is the same for any number of cores.

Value

A stress_rows data frame, one counted row per method (role = "sensitivity"), with the number of imputations and the Monte Carlo standard error in note.

References

Carpenter JR, Roger JH, Kenward MG (2013). Analysis of longitudinal trials with protocol deviation: a framework for relevant, accessible assumptions, and inference via multiple imputation. Journal of Biopharmaceutical Statistics 23, 1352-1371.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:100), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
stress_reference(spec, methods = c("MAR", "J2R"), m = 3)

Description

Writes a Markdown report with the plan, the base analysis, the per-family summary table, the verdict and the standard figures (census dot plot, bubble swarm, deletion p-value strip, p-value curves where the result has rows with a parameter) as PNG files beside it.

Usage

stress_report(
  result,
  dir,
  name = "stress_report",
  width = 9,
  height = 5,
  dpi = 150,
  mask_ids = FALSE
)

Arguments

Value

The path of the Markdown file, invisibly.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
plan <- stress_plan(spec, covariance = list(structures = c("cs", "ar1h")), inference = NULL,
                    covariates = NULL, weights = FALSE, loo = list(by = "RACE"), permute = NULL,
                    reference = NULL, estimator = NULL, fragility = NULL)
stress_report(run_plan(plan), dir = tempdir())

Description

Returns the paragraphs a statistical analysis plan can carry to pre-specify a stress test: the purpose, the pre-specified analysis, the families and their settings (from the plan), the counting rule, the classification, what will be reported, and the plan's MD5 hash so the run after unblinding can be shown to match what was pre-specified. The same text is installed as a template at system.file("templates", "sap_stress_test.md", package = "ballast").

Usage

stress_sap(plan, file = NULL, trial = "the study")

Arguments

Value

A character vector of paragraphs, invisibly if file is given.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
cat(stress_sap(standard_plan(spec, site = "RACE"), trial = "Study 001"), sep = "\n\n")

Description

A stress_spec holds the analysis data and every decision of the pre-specified mixed model for repeated measures: the response, subject, visit and treatment variables, the covariates, the visit at which the treatment contrast is taken, the within-subject covariance structure, the degrees-of-freedom method, the variance estimator and the significance level. Every family function in the package takes a spec and returns rows in the schema of stress_fit().

Usage

stress_spec(
  data,
  response,
  subject,
  visit,
  treatment,
  reference = NULL,
  active = NULL,
  covariates = NULL,
  contrast_visit = NULL,
  covariance = "us",
  method = "Kenward-Roger",
  vcov = "Kenward-Roger-Linear",
  alpha = 0.05,
  weights = c("equal", "observed"),
  label = "pre-specified",
  sas_conventions = TRUE,
  time = NULL
)

Arguments

Details

The fixed effects are covariates + treatment + visit + treatment:visit; the treatment contrast at contrast_visit is the difference of least-squares means (active minus reference) with class covariates weighted equally over their levels and continuous covariates at their means, the convention of SAS LSMEANS.

Value

An object of class stress_spec.

Examples

spec <- stress_spec(mmrm::fev_data, response = "FEV1", subject = "USUBJID",
                    visit = "AVISIT", treatment = "ARMCD", reference = "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL, contrast_visit = "VIS4")
spec

Description

Subsets a CDISC ADaM BDS data set (ADEFF, ADQS and the like) to one parameter, the analysis-flagged post-baseline records, orders the visits by their numeric code and builds the spec with the baseline value as a covariate by default.

Usage

stress_spec_adam(
  bds,
  paramcd,
  response = "CHG",
  subject = "USUBJID",
  visit = "AVISIT",
  visitn = "AVISITN",
  treatment = "TRTP",
  reference = NULL,
  active = NULL,
  covariates = ~BASE,
  contrast_visit = NULL,
  analysis_flag = "ANL01FL",
  drop_baseline = TRUE,
  filter = NULL,
  covariance = "us",
  method = "Kenward-Roger",
  vcov = "Kenward-Roger-Linear",
  alpha = 0.05,
  weights = "equal",
  label = paste(paramcd, response)
)

Arguments

Value

A stress_spec.

Examples

bds <- transform(mmrm::fev_data, PARAMCD = "FEV1", CHG = FEV1 - FEV1_BL, BASE = FEV1_BL,
                 AVISITN = VISITN, TRTP = ARMCD, ANL01FL = "Y")
stress_spec_adam(bds, "FEV1", response = "CHG", visit = "AVISIT", reference = "PBO")

Description

Takes the data, response, subject, visit, covariance structure, degrees-of-freedom method and variance estimator from an mmrm fit. The treatment variable is the factor that interacts with the visit variable unless given.

Usage

stress_spec_mmrm(
  fit,
  treatment = NULL,
  reference = NULL,
  active = NULL,
  contrast_visit = NULL,
  alpha = 0.05,
  weights = "equal",
  label = "pre-specified",
  sas_conventions = TRUE
)

Arguments

Value

A stress_spec.

Examples

fit <- mmrm::mmrm(FEV1 ~ RACE + SEX + FEV1_BL + ARMCD * AVISIT + us(AVISIT | USUBJID),
                  data = mmrm::fev_data)
stress_spec_mmrm(fit, reference = "PBO")

Description

Writes the per-family summary, the rows that crossed alpha or left the interval, the deletion-search path and the full census. CSV files are always written; with flextable and officer installed, a Word document with the tables in a standard clinical-report layout is written too, and an RTF version of each table.

Usage

stress_tables(
  result,
  dir,
  name = "stress",
  formats = c("csv", "docx", "rtf"),
  digits = 3,
  mask_ids = FALSE
)

Arguments

Value

The paths written, invisibly.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL,
                    covariance = "cs", method = "Satterthwaite", vcov = "Asymptotic")
plan <- stress_plan(spec, covariance = list(structures = c("cs", "ar1h")), inference = NULL,
                    covariates = NULL, weights = FALSE, loo = list(by = "RACE"), permute = NULL,
                    reference = NULL, estimator = NULL, fragility = NULL)
stress_tables(run_plan(plan), dir = tempdir(), formats = "csv")

Description

Imputes the missing responses by fully conditional specification (regression method, one column per visit, subject-level covariates and treatment as predictors), adds delta to every imputed value of the active arm, refits the pre-specified model on each completed data set and pools with Rubin's rules. delta = 0 is the missing-at-random multiple-imputation analysis and is a counted alternative; other deltas are tipping-point searches (kind = "search"). Rows carry the delta in param.

Usage

stress_tipping(
  spec,
  deltas = seq(0, -10, by = -2),
  m = 25,
  seed = 1,
  arm = NULL,
  shift_visits = NULL,
  impute_covariates = NULL,
  nbiter = 10,
  shift_every = TRUE,
  cores = 1L
)

Arguments

Details

Covariates that vary within subject are excluded from the imputation model and are kept in the analysis model; subject-visit rows absent from the data are created for the imputed response with those covariates missing, so they drop out of the refit.

Value

A stress_rows data frame, one row per delta; note records the number of imputations and the Monte Carlo standard error of the pooled estimate.

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:80), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL)
stress_tipping(spec, deltas = c(0, -5), m = 3)

Description

Applies the definitions: fragile if any counted row reverses the base direction or lies outside the base confidence interval; otherwise sensitive if any counted row changes significance at alpha; otherwise robust. Marginal is a separate flag on the base p-value, true when it lies within alpha * marginal_band.

Usage

stress_verdict(census, marginal_band = c(0.5, 2))

Arguments

Value

A list of class stress_verdict: stability over every counted row, stability_sensitivity over the rows that keep the estimand (role = "sensitivity"), marginal, the counts behind them, and the rows that drove the classification.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
cen <- stress_census(spec, stress_covariance(spec, c("us", "cs")), stress_covariates(spec))
v <- stress_verdict(cen)
v$stability
v

Description

Fits the pre-specified model once per visit and reports the treatment contrast at each; the contrast at the pre-specified visit is flagged is_base. A different visit is a different estimand, so the rows are role = "supplementary".

Usage

stress_visits(spec, visits = NULL, cores = 1L)

Arguments

Value

A stress_rows data frame, one row per visit.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_visits(spec)

Description

Refits the pre-specified model and forms the treatment contrast with the other weighting of the class covariates: equal weights over levels (SAS LSMEANS default) versus the observed covariate distribution of the analysis rows (SAS OBSMARGINS, g-computation). The point estimate differs only when a class covariate interacts with treatment or visit; the two rows otherwise coincide.

Usage

stress_weights(spec, cores = 1L)

Arguments

Value

A stress_rows data frame with two rows, one flagged is_base.

Examples

spec <- stress_spec(mmrm::fev_data, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO",
                    covariates = ~ RACE + SEX + FEV1_BL)
stress_weights(spec)

Description

Summarise a census by family

Usage

## S3 method for class 'stress_census'
summary(object, ...)

Arguments

Value

A data frame with one row per family, kind and role: counted analyses, number significant, number that crossed alpha, number outside the base confidence interval, number reversing direction, the range of percent of base, and the largest shift in base standard errors. Closing rows total the sensitivity analyses (same estimand) and all counted rows.

Description

For rows that carry a param value (from stress_tipping() or stress_curve()), interpolates linearly between adjacent parameter values to find where the p-value crosses alpha, separately for each family.

Usage

tipping_point(rows, alpha = NULL)

Arguments

Value

A data frame with one row per family: family, n points, param_min, param_max, p_min, p_max, crossing (the interpolated parameter value, NA when the curve does not cross) and direction ("up" when p rises through alpha with increasing parameter, "down" otherwise).

Examples

small <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:60), ]
spec <- stress_spec(small, "FEV1", "USUBJID", "AVISIT", "ARMCD", "PBO", covariates = ~ FEV1_BL)
shift_pt3 <- function(d, v) { k <- d$USUBJID == "PT3"; d$FEV1[k] <- d$FEV1[k] + v; d }
tipping_point(stress_curve(spec, values = c(-20, 0, 20), modify = shift_pt3))