After a case in which the reference group (150 respondents) was smaller than the batch under examination (210): the functions refused to run, and with subsamples of 100 the level-matched control samples came out almost identical (the same gap in every sample), so the p values overstated the evidence.
compare_batches() and neighbor_test() gain min_ratio (default 3).
When the reference has fewer than min_ratio times the focal
respondents, including a reference smaller than the batch, each sample
takes a random subsample of the batch of floor(n_reference / min_ratio)
respondents and pairs the reference with it. The focal value is the
median over the focal subsamples (focal_gap_sample in
neighbor_test()). min_ratio = 1 gives the former behaviour whenever
the reference is at least as large as the batch.settings$matched_overlap) and warn when it is
above 80%: the control then barely varies. In the case above the share
was 100% with the whole reference, 87% with min_ratio = 2 and 63% with
min_ratio = 3; with 5 the focal samples had 30 respondents and the
near-duplicate test lost its power, hence the default of 3.compare_batches()$tests gains n_sample, the size of the samples
compared.Tools to judge a whole batch of data, after a case in which 40 late answers of a survey looked suspicious (many 4s, weak item correlations) and turned out to be explained by their level and by the flat responders of the first batch.
compare_batches(): compares the item correlation structure of a batch
with a reference group by permutation (mean inter-item correlation and the
within-minus-between-scales structure), after leaving out flat responders
(drop_flat) and against reference samples paired by response level
(match_level); adds the Jennrich (1970) test. plot() types "null",
"heatmap" and "structure".order_test(): are consecutive rows (items and, optionally,
demographic profile columns) more alike than in a random order? Detects
rows copied from the previous one without a submission time. plot()
types "null" and "sequence".neighbor_test(): distance to the most similar respondent, against
data shuffled by item and against level-matched control samples; returns
the closest pairs. plot() types "null" and "distance".scan_responses() gains the even-odd consistency index and flag
(evenodd, evaluated with eo_min_scales = 6 scales or more), and the
reference and ref_quantile arguments: the cut-offs of low_irv,
mahal, psychsyn and evenodd are then taken from a reference group
without flat responders.flat is judged only in scales of flat_min_items (6) items or more, or in
all items together when no scale is that long. In three- or four-item
scales many genuine respondents give the same answer everywhere.batch_sim: two phases, with a hand-filled block in the
second one.Adjustments after screening two short one-factor scales (7 and 10 items) in a large real sample (N = 4911), where three criteria flagged far too many people.
psychsyn uses disjoint pairs: each item enters one pair only, strongest
first. In a short one-factor scale almost every pair passes syn_r and the
pairs reused the same items, so the within-person correlation was noise
(45 % flagged). New argument syn_min_pairs (default 5): with fewer
disjoint pairs the criterion is reported as not evaluable.duplicate is evaluated only with at least dup_min_items items (default
15). With few items, genuine respondents share answers by chance (28-45 %
flagged in the real case). Vectors that are flat in every scale are
no longer counted as duplicates; they are already flagged by flat.low_irv is computed among respondents with some variability (IRV > 0).
Before, when more than 5 % of the sample responded flat the cut fell to 0
and low_irv repeated flat exactly.exclude_responses() warns, and compare_fit() adds a note, when a rule
removes more than half of the sample.compare_fit() adds a note when an exclusion removes respondents for
their Zh: Zh is estimated from the same items, so part of the gain in fit
is circular.plot() returns the
ggplot object invisibly, so it can be modified or saved with ggsave();
new argument base_size.plot.response_scan() and of plot.fit_comparison()
explains how to read each view.exclude_responses(): returns the data without the respondents that
meet the chosen flags, patterns, number of flags or rows, ready for any
model.compare_fit(): fits the same CFA (lavaan syntax) or EFA with each
exclusion and compares fit indices, mean loading and factor correlations;
improper solutions are reported. print() and plot() methods.plot() gains five views: "days", "overlap", "profiles", "styles"
and "scatter".as.data.frame() prefixes the flags with flag_ so that they do not clash
with the index of the same name.scan_responses() with print(), summary(),
as.data.frame() and plot() methods (overview, indices, patterns,
heatmap and time), and the simulated data set survey_sim.