estimate_effort_aerial_glmm() now reports a total across the sampled days,
the same basis estimate_effort() uses, instead of a single average day
(#363).
The two functions could be called on one design and returned quantities that differed by a factor of the number of sampled days, with nothing in either signature or either result to say so. On the package's own aerial fixture that was 378.6 against 5092.5 — a gap a reader would naturally attribute to the diurnal correction the GLMM exists to apply. It is now 4728.6 against 5092.5, and the 7% between them is that correction.
target = "mean_day" restores the single-day figure for anyone who wants it,
and the basis is recorded on the result as effort_target, so which question
was answered can be read off the object rather than inferred.
Both targets are expectations, and both now carry a retransformation factor
the previous code did not apply. The fitted curve is a fixed-effects
prediction — the day whose random intercept is zero — which on a log link is
the median day, not the mean. Reporting it as a mean understates by
exp(sigma^2 / 2): 4% on the package fixture, more where days vary more. A
mean_day estimate is therefore about 4% higher than the old return value,
not identical to it.
The standard error scales with the expansion, so the CV is unchanged. It does not carry the uncertainty in the variance component itself, which makes it mildly optimistic; that is documented rather than hidden, and closing it needs a variance method neither the delta nor the bootstrap path offers.
This does not make the GLMM usable by the total estimators. They take no effort argument, and every design carries strata, so they always multiply per-stratum effort by per-stratum CPUE while this function returns one ungrouped row. Connecting the two needs stratified GLMM estimation, which is a modelling change and is tracked separately.
estimate_effort_aerial_glmm(boot = TRUE) now returns NA confidence
interval bounds when a multiplier's uncertainty was declared unknown, instead
of an interval that quietly left it out.
The total carries uncertainty from three places: the GLMM fit, the visibility
correction, and the angler-to-people ratio. Declaring one unknown —
visibility_correction = "none", or an NA angler_ratio_se — makes the
bootstrap skip that component, because there is nothing to draw from. The
standard error already went NA for precisely that reason. The interval did
not: it was built independently from the bootstrap quantiles and kept a value.
What that value described was an interval conditional on the unknown
multiplier being exact. Measured, with the same data and the same seed:
declaring the visibility correction unknown and declaring it known with zero
uncertainty produced bit-for-bit identical bounds of [322.920, 449.732],
distinguished only by se being NA in one and 33.98 in the other. "Never
studied" was reported exactly as "studied, and found no uncertainty" — the one
equivalence this package must never assert.
The interval now goes NA alongside the standard error. The delta path
already behaved this way, because its interval is derived from the SE and
inherited the NA for free; the bootstrap path built quantiles independently
and had to say so explicitly. The two paths now agree.
The point estimate is untouched — this is a reporting change, not an
estimation one. To get an interval back, state the correction's uncertainty
(visibility_se, angler_ratio_se). Declaring it known and zero is a claim
you can make deliberately; it is just no longer what silence means.
Examples no longer contain commented-out code. The estimate_total_catch(),
estimate_total_harvest() and estimate_harvest_rate() examples now run the
lines they previously only showed, and get_site_contributions(), which had
no runnable example, now has one. CRAN's review of 7.0.0 asked for this.
The validate_incomplete_trips() example and the incomplete-trips vignette
restore the equivalence-threshold option they change.
Four vignettes described behaviour the package does not have. Each was found by running the vignette and reading its own output, not by reading the source.
incomplete-trips.Rmd told a reader whose validation had just failed to
"stick with use_trips = "complete" (default)". The default is NULL, and on
a roving design — which is what that vignette is about — an unspecified
use_trips selects the all-trip mean-of-ratios estimator of Hoenig et al.
(1997), not complete trips. Following the parenthetical produced the estimator
the line above it warned against. Measured on the example data, catch rate
differs by 30% between the two (mean-of-ratios-cpue at 1.257 against
ratio-of-means-cpue at 0.967). The same claim appeared twice more, once in a
code comment. All three are corrected, and a new section demonstrates the
routing with a runnable example rather than asserting it.
aerial-glmm.Rmd compared a one-day GLMM total against a twelve-day total in a
table headed "side-by-side comparison", and attributed the resulting 13.5x gap
to the diurnal correction. The table now states each row's basis, and the
like-for-like rows differ by 10.8%, which is the diurnal correction the prose
actually describes. Its "Downstream Estimation" section also claimed that GLMM
effort "feeds directly into the standard downstream estimators"; it does not —
estimate_total_catch() takes no effort argument and derives effort itself, so
the section now says what the code does.
aerial-surveys.Rmd explained that ci_lower / ci_upper "give the 95%
confidence interval" directly beneath output where both were NA, because the
design declares visibility_correction = "none". Both aerial vignettes now
explain why an undeclared correction yields NA rather than a number, and
where to supply visibility_se to get an interval.
camera-surveys.Rmd said negative-duration rows are "set to NA and excluded
from the daily sum". They are excluded, but the day reports a smaller number
rather than NA — on the package fixture one flipped ingress/egress pair takes
a day from 11 hours to 7.75 — so the warning is the only place the exclusion is
visible.
The PDF reference manual builds again. Six characters across four help pages
had no LaTeX definition under pdflatex — tau, kappa, pi, a combining macron,
U+2212 minus and U+1D62 subscript i — and stopped R CMD check from producing
the manual at all. They are now \eqn{} math.
The README names vignettes rather than linking to the published article pages.
Those vignettes ship inside the tarball, so vignette("bus-route-surveys")
works from an installed package where a link to the website only works with a
network. It also means the README no longer depends on a checking machine
being able to reach the documentation host.
Every exported function now has a runnable example, and nothing is wrapped in
\dontrun{}.
Seventeen exported functions had no example at all, and fourteen help pages
wrapped theirs in \dontrun{}. Most of those wrappers were not protecting
anything: six ran as written once unwrapped, and the rest referenced objects
that were never defined — design_a, my_effort, counts_df — so the block
was a sketch rather than an example. One demonstrated writing a CSV into the
user's working directory.
Examples that genuinely need a suggested package are guarded with
@examplesIf rlang::is_installed(...) rather than hidden:
estimate_effort_aerial_glmm() needs lme4 and summarize_by_county() needs
zipcodeR. All 125 example topics run, each in under five seconds.
DESCRIPTION cites the methods the estimators implement — Hoenig, Jones,
Pollock, Robson and Wade (1997) doi:10.2307/2533116 for roving catch rates
and Kinloch, McGlennon, Nicoll and Pike (1997)
doi:10.1016/s0165-7836(97)00068-4 for bus-route designs — and records the
maintainer as copyright holder. The README shows install.packages().
Printing a validate_incomplete_trips() result no longer leaves the plot
margins changed for the rest of the session. print() widened par(mar) to
draw its scatter plot and never put it back; it now restores the caller's
setting on exit.
simulate_creel_data() and simulate_creel_catch() no longer leave the
caller's random number stream reset.
Both called set.seed(seed) and never put back what was there. seed is a
convenience for reproducing one simulation, not a licence to take over the
session — but a script that seeded its own analysis and then called either
function part-way through silently continued from our seed. Every draw
after that point was determined by an argument passed for one function's
benefit, with nothing to say so.
Measured: after set.seed(999), the next runif(1) was 0.389071 on its
own and 0.685170 with a simulate_creel_catch(seed = 42) call in between.
Both now give 0.389071.
The stream is captured before set.seed() and restored on exit. A session
that had never drawn at all is left without a .Random.seed again, rather
than inheriting ours. Reproducibility is unchanged: the same seed still
produces the same data, and omitting seed still consumes randomness
normally.
Identifier columns are normalised to character at every join
(add_catch(), add_lengths(), add_ages()).
A uid is a label, not a quantity: nothing is summed or ordered by magnitude on it, and what it must do is join. The CSV reader infers a bare integer id column as numeric while a JSON API serves the same ids as strings, so the same survey reached a design with a numeric uid from one backend and a character uid from the other.
Base R hid this — merge() and %in% coerce — but dplyr::left_join()
refuses outright with "Can't join x$interview_uid with y$interview_uid
due to incompatible types", and identical() is silently FALSE. Anyone
joining a fetched interviews frame to a fetched catch frame across backends
hit it.
design$interviews, design$catch, design$lengths and design$ages now
carry character ids whatever the source gave.
The coercion is deliberately not as.character(): R renders a numeric
100000 as "1e+05", so a naive conversion silently rewrites every id at or
above 1e5 — in the join key, the one column where a corrupted value cannot be
noticed by looking at a total. Whole numbers go through sprintf("%.0f", ...),
which is exact for every value R can hold; a non-whole value is left alone
rather than truncated into a collision.
summarize_boat_composition() counts the events it excludes (#337).
A count event yields an angler-boat share only when the boats were counted and some were present. Both exclusions are real; both happened with no trace that the event had occurred.
keep <- (ab + nb) > 0 is NA when either count is NA, and an NA
subscript does not drop a row — it selects a phantom all-NA one. The
event survived the subset with an NA month and day type, and
aggregate(by = ) then dropped it for having an NA grouping value. Two
mechanisms chained, neither visible: on a 12-day fixture, three unrecorded
boat counts took the event total to 9 and moved a reported share from
73.4% to 76.7%.
Two new integer columns, n_unknown_boats and n_nonpositive_boats, close
the accounting:
n_events + n_unknown_boats + n_nonpositive_boats
== count events in that month and day type
A month and day type whose every event was excluded now keeps its row,
reporting NA for pct_angler_boats rather than disappearing.
An unrecorded grouping value is now genuinely pooled with a category the data
already records as "Unknown", rather than forming a second group that merely
prints under the same label (#337).
#333 tracks missingness on an internal sentinel so a real "Unknown" keeps
its own counts. Where a column holds both, that kept them separate through
aggregation and then rendered both as "Unknown" — so one recorded and one
unrecorded fish in the same length bin came out as two rows reading N = 1
and 50% each, instead of one reading N = 2 and 100%. The warning already
emitted for this case says the counts are pooled; now they are.
summarize_cws_rates() and summarize_hws_rates() exclude interviews whose
sought species was not recorded, and report how many (#336).
The numerator counts fish of the species the party was targeting. With no target recorded nothing in the catch table can match, so such an interview fell through the join exactly as a party that caught none of its target does — and was scored the same way, as a zero. That asserted these parties caught none of something nobody recorded, and it dragged down every group they belonged to.
Measured on the shipped example data, blanking the sought species on 7 of 22
interviews: the boat group's mean_rate went from 0.393 to 0.254, a 35%
drop, with N unchanged at 9. No row was dropped, no group was missing
and nothing warned — which made it harder to see than #333, where at least
something disappeared.
An interview whose effort was not recorded is treated the same way, for
the same reason: a rate needs an effort to divide by. One unrecorded effort
used to turn the whole group's mean into NA — mean() of anything
containing an NA is NA — while N went on counting it. Those interviews
are now excluded and counted in n_unknown_effort, and the group keeps a real
rate from the rest.
Interviews whose effort is not positive are counted too, in
n_nonpositive_effort (#339). A zero is a real record — a party interviewed
before it started fishing — and a negative one is a data error that
add_interviews() already warns about; neither yields a rate, and both used
to be dropped with no trace at all. A table could report 20 of 22 interviews
with nothing in it to say the other two existed.
Four changes to the returned table:
a new integer column n_unknown_target;
a new integer column n_unknown_effort;
a new integer column n_nonpositive_effort;
N now counts the interviews that produced a rate, not every interview
in the group. The accounting closes exactly:
N + n_unknown_target + n_unknown_effort + n_nonpositive_effort
== interviews in the group
The three exclusion counts are mutually exclusive, in that precedence, so an interview missing more than one thing is counted once.
The estimand is now the rate among parties with a known target. That equals the rate among all parties only if the target went unrecorded independently of what was caught — an assumption about the data, not the code, which is why the count is reported beside every rate instead of the exclusion being silent.
A party that genuinely caught none of a recorded target is a real zero and
still counts, per add_catch(). A group with no interview left to rate keeps
its row, reporting NA for mean_rate, se and the interval.
The Calamus 2016 validation fixture ships again, so the tests that depend on it can run (#337 follow-up).
It lived under inst/extdata/, which .Rbuildignore excludes — a correct
exclusion when that directory held 17 MB of real waterbody data and a
pdfs/ directory of published journal articles. Those are long gone;
inst/extdata/ was down to the 32 KB fixture alone, so the exclusion had
stopped protecting anything and was only withholding the one file it should
have shipped.
The consequence was invisible. tidycreel.connect's
test-composition-calamus.R resolves the fixture through system.file(),
so all eight of its tests skipped on every run, in CI included — and they are
that package's only end-to-end assertions against real reference numbers.
That took connect from 437 passing with 9 skips to 461 with 1.
(Stated as the delta rather than a running total: a current count written
into a release note is wrong by the next release.)
The fixture moved to inst/calamus-2016/ rather than un-ignoring
inst/extdata/, so the directory-level guard stays in place: anything
dropped in there later still cannot reach a build.
summarize_length_freq(by = ) keeps every fish (#337).
stats::aggregate(by = ) drops every row whose grouping value is NA, so a
length record with no recorded value for a by column left the distribution
entirely — taking its weight with it. Measured on the shipped example data,
blanking species on 6 of 20 length rows took the total from 37 fish to
26: more than six, because a binned release row carries a count rather
than one fish.
The ungrouped total was never affected, which is what kept this invisible — and is why the existing tests could not fail on it.
An unrecorded grouping value is now reported under "Unknown", sorted last,
as it is in the six functions #333 swept. This is the last of that class:
#333 could not reach this function because its by selects columns of the
lengths frame, and no fixture built from the shipped data got that far.
summarize_cws_rates() and summarize_hws_rates() no longer fail with
non-numeric argument to binary operator when every sought species is
unrecorded (#336).
Comparing anything with NA yields NA, and an NA subscript selects a
phantom all-NA row rather than nothing (#324's shape). Those phantoms made
the filtered frame look non-empty, so the aggregate returned zero rows with a
logical key and the join produced a non-numeric target count that failed
later, naming nothing the caller had set. The comparison now tests the sought
species for NA explicitly.
The two rate functions share one implementation of their grouping, rate and interval steps (#336). Those steps were byte-identical copies in both, which is how a seam fixed in one twin comes to survive in the other.
Six summary functions dropped an interview whose grouping value was not recorded, out of its own row and out of the total (#333).
table() and stats::aggregate(by = ) both discard every record whose
grouping value is NA. summarize_by_angler_type(), summarize_by_method(),
summarize_by_species_sought(), summarize_successful_parties(),
summarize_cws_rates(by = ) and summarize_hws_rates(by = ) all grouped that
way, so an interview with no recorded angler type, method or sought species
left the table entirely. sum(N) silently stopped equalling the number of
interviews attached to the design, and nothing errored or warned.
Measured on the shipped example data, blanking 7 of 22 interviews: every one
of the six went from accounting for 22 interviews to 15, and
summarize_successful_parties() lost two whole rows. The groups that survived
lost their own members, so this was never only a missing row — in
summarize_cws_rates(by = "angler_type") the boat group's mean rate moved
from 0.393 to 0.762, because the interviews that vanished were the ones
holding it down.
An unrecorded grouping value is now reported under "Unknown", sorted last,
matching summarize_by_zip() and summarize_by_county(), which have always
done this. "Unknown" labels the absence; it is never a category anyone
selected, and nothing is imputed. (The survey-weighted estimators use
<unknown> via group_value_labels(); the two conventions still differ.)
This is the svyby() sweep of #321 reaching the functions that sweep could
not see: these group with base R, not survey::svyby().
summarize_successful_parties() reports NA, not 0, where success cannot
be determined (#333).
A party is successful when it caught some of the species it sought. Where
the sought species was not recorded there is nothing to compare the catch
against, so N_successful and percent are now NA for those rows rather
than 0 and 0.0%, which asserted that the parties had failed. N_total
still counts them: the interviews happened.
An unrecorded angler type is a different case and is not blanked — the sought species is known, so whether it was caught is knowable, and only the reporting group is unknown. Those rows carry real counts.
summarize_cws_rates(by = ) and summarize_hws_rates(by = ) report NA,
not 0, for a group whose sought species was not recorded (#333).
The numerator counts fish of the species the party was targeting. With no
target recorded there is nothing to count, and the upstream fill made that
count 0 — so the group reported a mean rate of exactly 0.000, asserting
that these parties caught none of their target. Grouping by anything else
leaves the rate determinable, because the catch and effort are the
interviews' own, and those groups are unaffected.
Missingness is tracked internally rather than inferred from the "Unknown"
label, so a category genuinely recorded as "Unknown" keeps its own counts
(#333). A sought species the interviewer recorded as unknown is a real answer,
not a missing one. A column holding both unrecorded values and the literal
value "Unknown" now warns, because the two are pooled into one row and
cannot be told apart in the output.
summarize_successful_parties() no longer dies inside base R when a grouping
column is entirely unrecorded (#333). aggregate() returned a zero-row frame
and the failure surfaced as replacement has 1 row, data has 0, naming
nothing the caller had set.
summarize_successful_parties() now reads add_catch()'s catch-type model,
so a party that recorded only harvested/released rows counts as successful
(#329).
add_catch() documents the caught row as optional: when a pair has
none, its total catch is harvested + released. This function held a private
copy of the rule with no such fallback — it asked whether a caught row
existed, full stop — so a party that recorded only its dispositions was
reported as unsuccessful while its own harvest was positive.
This was live on the package's own example data. Four species-interview
pairs in example_catch record dispositions and no caught row — interviews
11, 12 and 14 for walleye, and 13 for bass. All four demonstrably caught the
species they sought, and all four were counted as failures. The successful
party total on the shipped data was 6; it is 10.
Dropping every caught row — which the documentation says is legal, and which
leaves each pair's harvested and released rows untouched — previously took the
total from 6 to zero, with every reported rate reading 0.0%. It now
changes nothing, which is the property that makes the rule real.
A recorded catch of zero still counts as unsuccessful: that is data, not an
absence. The fallback now comes from species_counts_per_interview(), the
package's one implementation of the catch-type rule, applied per
species-interview pair (#318, #320).
This was #317's ninth instance and the last one outstanding; that defect class is now closed.
A large release, and a breaking one. The theme running through most of it is a single defect class: a quantity that was unknown or absent was allowed to behave like a zero or like nothing at all. Every instance produced a plausible number, no error and no warning.
Numbers that move, and why:
Grouped estimates gain rows. survey::svyby() drops rows whose by= value
is NA, so any grouping column with a missing value silently lost those
records. estimate_total_catch(by = zone) reported 565.2 fish against an
ungrouped 784.6 — 28% of the catch missing from a table that read as
complete. Every grouped estimator now reports the unknown group, so the parts
sum back to the whole (#317, #321).
Length-based standard errors get larger. The bins' covariance matrix was
discarded, and est_compliance(), est_mean_length() and est_biomass() each
rebuilt a variance assuming the bins were independent. est_compliance()
reported a standard error 32% below an independently computed
survey::svyratio() reference; it now reproduces that reference exactly.
compliance_se, mean_length_se and biomass_se all change. Point
estimates do not move (#311).
add_catch() refuses an unknown count. count = NA was accepted and read
as a genuine zero everywhere downstream — the unknown and the zero produced
byte-identical output. Now an error (#324).
Length and age totals are on a catch basis. They previously expanded the measured subsample as if it were a census. Ratios were right; totals were not (#310).
If you pin any standard error from the length or age path, or read any grouped table, re-run before comparing to earlier output.
Grouped effort now reports an unknown group instead of dropping it, and
refuses a supplied within-day variance that is NA (#317).
Two separate ways an unknown quantity was behaving like nothing at all.
The unknown group. survey::svyby() drops rows whose by= value is NA,
with no warning and no row in the result. The effort did not move anywhere
else — it simply stopped being reported, so the grouped parts no longer summed
to the ungrouped whole. On a design whose gear was unrecorded on three days,
estimate_effort(by = gear) returned groups totalling 662.5 angler-hours
against an ungrouped total of 773: 110.5 hours, 14% of the fishery, gone
silently. With within-day variance attached the same request instead died
with a bare missing value where TRUE/FALSE needed, because the two sides of
the variance match built their group keys with different base idioms.
NA is now a group like any other here, exactly as it already is on the
interview side. Its row appears with its own estimate, standard error and n,
and the parts sum back to the whole. A group genuinely labelled "NA" remains
a different group — the two collided under paste(), which renders a
missing value as the string "NA", and that collision also made
add_counts() report such a row as a repeated sampling unit.
The supplied variance. add_counts() accepts a precomputed
within_day_var column and validated nothing about it. The consumer reads a
missing value as "this unit had a single count, so its within-day term is
zero" — true for a unit absent from the table, false for one present with an
unknown sum of squares. Three unknown PSUs took se_within from 26.46 to
23.45 and the reported SE from 29.56 to 26.90, with no warning. Supplying NA
in within_day_var or n_counts now raises
creel_error_na_within_day_var: an unknown variance component is not a zero
one, and this table cannot carry the difference.
The party-size expansion component travels with it. That producer keyed its
groups with the old idiom, so an unknown group matched nothing and the
consumer read "no match" as "this group contributed no expanded boats" — a
confident zero in place of a real component. Caught by the ensemble review:
the unknown group's se_expansion was 0.0 where it is 0.8.
Grouped effort results gain a row wherever a grouping column contains
NA, and a counts table supplying NA in either within-day column is now
rejected at attach time rather than silently understating the SE.
The sweep over the remaining grouped estimators followed in #321, below.
The length bins' covariance now reaches the ratio consumers, which were reporting standard errors up to a third too small (#311).
svytotal() over the bin columns estimates a full covariance matrix, and
two_phase_rescale() propagates it. It then died at the
est_length_distribution() seam, and est_compliance(), est_mean_length()
and est_biomass() each rebuilt a variance from the diagonal alone — the
quadratic form \eqn{w' \Sigma w} with every off-diagonal set to zero.
The bins are a partition of the same fish, clustered within interviews and
rescaled onto a single reported total, so they are strongly dependent. On the
package's own example data the maximum off-diagonal correlation is 1.0.
Against a survey::svyratio() reference built independently in base R:
| | proportion | SE |
|---|---|---|
| est_compliance(), before | 0.5714286 | 0.1422436 |
| svyratio() reference | 0.5714286 | 0.2093703 |
| est_compliance(), now | 0.5714286 | 0.2093703 |
The point estimates agreed all along — only the uncertainty was wrong, which is why nothing looked wrong. The package standard error was 32% too small, on a proportion that gets compared against a legal size limit.
All three consumers now take the quadratic form against the full matrix, and
est_compliance() reproduces the independent reference exactly. The matrix
travels as an attribute keyed by reported group, with bin labels as dimnames,
so a consumer aligns to it by bin rather than by position — a caller who
keeps a subset of rows gets that subset's block rather than a misaligned one.
When the matrix is unavailable — an object from an older version, or one
subsetted in a way that dropped the attribute — the independence form is used
and a creel_warning_bin_vcov_unavailable warning says so. An absent
covariance is unknown, not zero; falling back in silence would restore the
defect by the back door.
est_biomass() was the one consumer that documented the omission. It is no
longer an omission, and its @details now says so, as do the other two.
Note for anyone pinning these numbers: compliance_se, mean_length_se and
biomass_se all change, and in the direction of being larger. The point
estimates do not move.
add_catch() now refuses an unknown count, which was indistinguishable
from a genuine zero everywhere downstream (#324).
add_catch() documents that an angler who caught none of a species need not
appear in the catch table at all. Every consumer therefore reads a missing
row as a catch of none. That reading is right for an absent row and wrong for
a row that is present but carries NA, and nothing could tell the two apart:
the summaries left-join the catch table onto the interviews and fill the join
miss with zero, which swallowed the NA along with it.
Changing a single harvested count on the package's own example data:
| that pair's count | summarize_hws_rates() | se |
|---|---|---|
| 5 (known) | 0.2332251 | 0.07883204 |
| NA (unknown) | 0.1682900 | 0.05512425 |
| 0 (genuine zero) | 0.1682900 | 0.05512425 |
The unknown and the zero produced byte-identical output — the rate understated and the confidence interval narrowed, with no error and no warning.
count may no longer contain NA (CATCH-07, class
creel_error_na_catch_count). Refused at entry rather than repaired
downstream, for the same reason #322 refuses an NA within_day_var: the
consumers' reading of a join miss is correct exactly when the table carries no
unknowns, so this is the one place that can make it true.
It also removes two accidents in the caught >= harvested + released check
(CATCH-04). An unknown harvested count was zeroed by that check's own
sub_total fill and so passed in silence. An unknown caught count made
the comparison evaluate to NA, and indexing a data frame with NA yields a
phantom all-NA row — so it did abort, but for the wrong reason, naming the
offending pair as "NA/NA" and telling the user nothing about which row to
fix. Both are now unreachable, and the refusal names the real pair.
The distinction this protects is pinned in both directions: an interview absent from the catch table still reads as a zero, exactly as documented.
Note that dropping a row states something rather than nothing. A dropped
"harvested" or "released" row means none of that disposition; a dropped
"caught" row makes total catch derive from harvested + released instead.
The error message says which, so the remediation it suggests is not itself a
silent change of claim.
Every remaining grouped estimator now reports the unknown group too, and the ratio-estimation sample-size floor is applied to it (#321).
#317 fixed grouped effort. survey::svyby() drops rows whose by= value is
NA with no warning and no row, and the other grouped estimators reached it
through their own paths, so each one went on dropping the group silently.
The parts stopped summing to the whole in three different places:
estimate_catch_rate(by = gear) and estimate_harvest_rate(by = gear)
reported those 32 and said nothing at all about the other 16 — a third of
the sample.estimate_total_catch(by = zone) on counts whose zone
was unrecorded on a third of the days returned 565.2 fish against an
ungrouped 784.6 — 28% of the catch missing, in a table that reads as
complete.proportion
column, whose denominator always included the unknown group, quietly summed
to 0.64 — the one visible trace, in a column nobody reads as a completeness
check.Two seams behind those numbers were worse than the missing rows:
The sample-size floor never saw the unknown group. n >= 10 per group
was counted with aggregate(.count ~ ., ...), whose formula method drops
NA rows — so a grouped ratio request was admitted or refused on a
denominator that had silently lost interviews. The floor now counts on the
group key, and an unknown group must clear the same n >= 10 as any other.
A second private copy of the key rule. expansion_stratum_key() built
its own key with paste(), which renders a missing value as the literal
string "NA" — so an unknown stratum and a stratum genuinely labelled "NA"
received the same key and had their party-size expansion components pooled,
with the total still looking right. It now delegates to group_key(), the
package's one answer to that question.
Grouped results gain a row wherever a grouping column contains NA, in
estimate_catch_rate(), estimate_harvest_rate(), estimate_total_catch(),
estimate_total_harvest() and the bus-route effort, catch-total and
catch-rate paths. A grouped ratio request whose unknown group falls below
n >= 10 now errors where it previously returned an estimate for the other
groups.
A reported grouping column keeps its own type. Making the unknown group
survive svyby() means promoting the column to a factor — but only when it
contains a missing value, so the same estimator on the same column returned a
factor when something was unknown and a numeric when nothing was, and
depth > 15 or depth + 1 on the result worked only in the second case. The
factor is an implementation detail of the svyby() call and is now undone on
the way out: the unknown group comes back as an NA of the column's own type,
which is exactly what that column can say about a group whose value was never
recorded. This also repairs the grouped effort path shipped under #317.
A column supplied as a factor comes back as that factor too, with its own
levels and its ordered class, and with a true NA rather than the
promoted addNA() level — so is.na(result$group) finds the unknown row for
a factor exactly as it does for every other type.
Undoing it exposed a defect the factor had been hiding. The stratified product
sum aggregated with aggregate(cbind(...) ~ ., ...), whose formula method
drops NA rows — so an unknown group was dropped from the product sum
outright. It had survived only because an addNA() level is not is.na() at
the integer level, so the formula method could not see it. That aggregation
now splits on the group key, which does not depend on the column's type at
all.
The sample-size floor's message names the unknown group. Its bullets were
built with paste0(by_vars, "=", vals), which renders a missing value as the
literal string "NA" — the very label a group may genuinely carry. A design
with both produced two bullets that both read Group gear=NA: n=6, with no
way to tell which group had failed. An absent value now reads
gear=<unknown>.
Two estimators named in the issue turned out not to be reachable, and are
recorded here so the question is not re-opened: the camera path groups by
strata rather than by a user variable, and add_counts() rejects an NA
stratum at Tier 1; the aerial path calls svytotal() and supports no by=
at all. The sectioned paths group by a registered section, and
add_sections() rejects an unregistered NA.
Species-level catch now reads add_catch()'s catch-type model per
species-interview pair, as that model is documented, instead of once per
species across the whole catch table (#318).
add_catch() states the rule per pair: a "caught" row is the pair's total
and is optional, and when it is absent, total catch is inferred as
harvested + released. CATCH-04 enforces caught >= harvested + released
for each pair. The code applied that rule table-wide instead — it asked
whether a "caught" row existed anywhere for the species, and if one did,
every other pair holding only harvested/released rows read as a catch of
zero, while its own harvest stayed positive.
On the package's own example data that made reported harvest exceed reported catch for all three species — walleye 33 caught against 55 harvested plus released, bass 10 against 25, panfish 7 against 13. No error, no warning, and the contradiction CATCH-04 rejects row by row was reintroduced at the total.
Species catch totals rise wherever a catch table mixes the two shapes.
This reaches estimate_total_catch(by = species), the species CPUE path,
exploitation rate (which divides harvest by catch), and the bus-route
equivalents, all of which read the same helper. Tables recording "caught"
rows for every pair, or none at all, are unaffected — the old rule and the
new one agree there, which is why no existing test moved.
An interview absent from the catch table entirely still counts as zero for
that species. add_catch() documents that too, and the two absences are
different: a missing row means the angler caught none, while a missing
"caught" row alongside recorded dispositions is an instruction to derive.
This is the eighth instance of the pattern tracked in #317 — a quantity that is unknown or absent behaving like a zero.
est_length_distribution() and est_age_distribution() now read the same
per-pair rule when they build the reported total they scale onto, and they
refuse per group rather than per species (#317).
Both distributions are rescaled onto a design-estimated reported total
(see #310 below). That total was built by a second, private copy of the
add_catch() catch-type model — the table-wide copy #318 removed everywhere
else. A species with a "caught" row on any one interview therefore had every
other interview's harvested/released rows read as a catch of zero, and the
distribution was scaled onto the understated total. On the package's own
example data, grouping a catch distribution by species and an interview
attribute returned totals built from the "caught" rows alone: 7 / 18 / 5 / 15
where the reported catch is 10 / 28 / 18 / 27.
The refusal moved with it. When no reported rows exist, these functions abort rather than scale onto zero — but the test asked whether the species had rows anywhere in the catch table. A group whose own interviews reported nothing still passed, because some other group's interviews carried the species, and its measured fish were scaled onto a total of zero with no warning. The test is now whether any of the group's own interviews records the species — on the key, not the value, so a recorded count of zero is still data.
Grouped length and age distribution totals rise wherever a catch table
mixes the two shapes, and a group that reports nothing of its own now raises
creel_error_no_rescale_total instead of returning zeros. Shares (percent)
are unaffected. The refusal message now names the group, not only the species.
This is #317's own finding 7 — the same “absent behaves like a zero” pattern, in the rescaling path.
est_length_distribution() and est_age_distribution() now scale their
totals onto the design-estimated reported catch instead of expanding the
measured subsample (#310).
Lengths and ages are read from a subsample of the catch. Expanding that
subsample through the interview design estimated the total number of fish
that happened to be measured — on the package's example data, 14 against a
reported harvest of 77, with no error and no warning. est_biomass()
multiplies those counts by weight-at-length and calls the result total
biomass, so a headline number was low by whatever fraction of the catch got
measured. Measuring every fish twice doubled the reported biomass.
The estimator is now two-phase (double sampling, Cochran 1977 §12.9 — the
structure already used for the camera calibration ratio): the bin proportion
among measured fish, scaled by the reported total. Both parts come from one
svytotal() call, so the bin-to-total covariance is estimated rather than
assumed away, and the standard error is the delta method over it.
estimate, se and the confidence bounds change value, as does every
column of est_biomass(). percent and cumulative_percent do not: a share
is invariant to how many fish were measured, which is why the shape of the
distribution was always right and only its level was wrong.
est_mean_length() and est_compliance() keep their point estimates for
the same reason — both are ratios. Their standard errors do move, because the
per-bin standard errors they read now carry the reported total's variance
through the delta method rather than being a pure rescaling of the old ones:
on the example data mean_length_se goes 40.04 to 37.88 and compliance_se
0.1554 to 0.1422. The cross-bin covariance those two still assume away is a
separate defect, tracked as #311.
Two consequences for existing code. Grouping by species now requires
add_catch(), because only that table is species-resolved and scaling one
species by the all-species total would be worse than refusing. And a design
with no matching total — a harvest distribution with no harvest = supplied
to add_interviews() — is now an error rather than a subsample total.
Every call that rescales warns, naming the measured total, the reported total and the factor between them.
targeted = FALSE on a by = species request now tests that species' own
catch, not the total catch, under the mean-of-ratios estimator (#304).
targeted = FALSE is documented as excluding zero-catch trips so that the
result is the rate among trips that caught the species. The exclusion ran
inside the mean-of-ratios branch, above the species split, and tested the
design's total catch column. A trip that caught one fish of any species
therefore counted as non-zero however many of the requested species it held,
so on a survey where every interview caught something the argument excluded
nothing at all and returned the untargeted rate with no warning:
# before: identical to the digit
estimate_catch_rate(design, by = species, estimator = "mor", targeted = TRUE)
estimate_catch_rate(design, by = species, estimator = "mor", targeted = FALSE)
Both the exclusion and the accompanying ">70% of trips have zero catch" mis-specification warning now test the species' own zero-filled count, which is the only reading that makes sense for a per-species rate. The regression estimator already did this (#290); the two now agree.
Three consequences for existing calls:
targeted = FALSE with by = species under "mor"/"mortr" returns
different numbers. It previously returned the fishery-wide rate, so no
caller loses a correct estimate — but any recorded output changes.n >= 10 ratio
floor, so a call that silently returned a number now aborts. That is the
intended trade: a targeted rate from a handful of trips is not estimable,
and an error naming the sample size is preferable to an untargeted number
wearing a targeted label.targeted = TRUE (the default) emits the mis-specification warning per
species where it previously could not fire. No estimate produced with
default arguments changes.estimator = "ratio-of-means" continues to ignore targeted, as documented.
Sectioned results now report their sections under the column the design
registered, instead of a hardcoded section (#282).
add_sections(section_col = ) lets a design name its section column whatever
the survey calls it, but every sectioned estimator returned a column literally
named section regardless. On a design registered with section_col = reach:
# design registered with section_col = reach
names(estimate_catch_rate(design))
# before: "section" "estimate" "se" ...
# after: "reach" "estimate" "se" ...
The result could not be joined back to the caller's own section table by name,
and it disagreed with the package's own error message: by = reach is refused
with "reach is already how the result is split" while no output column was
named reach. It was also inconsistent with grouped results, which have
always echoed the caller's column — by = day_type returns a day_type
column.
This affects all seven sectioned paths: estimate_effort(),
estimate_catch_rate(), estimate_harvest_rate(), estimate_release_rate(),
and the three estimate_total_*(). The lake-wide aggregate row is unchanged
in substance — .lake_total is a value in the section column, so it now
appears under the caller's name too. prop_of_lake_total and
se_prop_of_lake_total are not caller columns and keep their fixed names.
A design whose section column is already called section is completely
unaffected, which is every fixture in this repo and the common case. That is
also why no existing test could catch this: with section_col == "section"
the hardcoded name and the correct name are the same string. The new tests use
a fixture that names the column something else, so they can fail.
Code that reads est$section on a design registered under a different name
should read est[[design$section_col]].
as_hybrid_svydesign() now takes one long-form counts table plus a
frame_col naming the column that partitions it, instead of two pre-split
tables named access_data and roving_data (#248). The two *_fraction
arguments are replaced by a single fraction, a named list with one entry
per frame.
# before
as_hybrid_svydesign(
access_data = boat, roving_data = bank, calendar = cal,
access_fraction = c(weekday = 0.5), roving_fraction = c(weekday = 0.4),
trips_disjoint = TRUE
)
# after
as_hybrid_svydesign(
counts, frame_col = "angler_type", calendar = cal,
fraction = list(boat = c(weekday = 0.5), bank = c(weekday = 0.4)),
trips_disjoint = TRUE
)
The arguments borrowed the interview vocabulary for something that is not
an interview mode. Access and roving describe how anglers are interviewed and
select the catch-rate estimator; tidycreel carries that axis on
add_interviews(interview_type =). What this function combines are disjoint
count frames, usually angler-type domains. Pope et al. (Chapter 17) carry
exactly this as an anglerType column beside the stratum, which is what
frame_col now names, and taking a partitioning column rather than pre-split
tables is how the rest of the package already works.
The frame labels now come from the data, so the design speaks the caller's
vocabulary. attr(design, "component_col") names the frame column, and the
design data carries that column unchanged rather than a renamed copy.
The two-frame ceiling is gone: three or more frames stratify, expand and weight exactly as two did. Nothing in the arithmetic was ever limited to two.
A frame label that is missing, and a frame_col holding fewer than two
distinct frames, are both refused rather than silently forming a stratum. The
repeated-day refusal is now keyed on the frame as well as the date and
stratum, so frames sampling a shared date are not mistaken for repeat counts.
Two further refusals come with the caller-supplied labels. The internal
stratum key is paste(stratum, frame, sep = "."), so a . inside either
value can make two different combinations land on one key -- stratum "a"
with frame "b.c" and stratum "a.b" with frame "c" both give "a.b.c" --
which survey would pool into one stratum, counting n_h over the union of
their sampled dates and getting the day expansion and the fpc wrong for both.
Ambiguous keys are now named and refused; a . that cannot collide is still
allowed. A fraction entry naming a frame absent from the data, naming one
frame twice, or naming one stratum twice inside a frame's vector, is also
refused rather than silently ignored, since each leaves the caller believing a
fraction was applied that never was.
The method on a sectioned catch rate now names the estimator that produced
it (#284). estimate_catch_rate_sections() passed the caller's estimator down
and computed with it, then labelled every result "ratio-of-means-cpue-sections"
whatever had run, so a mean-of-ratios rate reported a ratio-of-means name --
and that is the roving default, because the auto-route resolves to "mor" with
no argument from the caller. A sectioned MOR catch rate is now
"mean-of-ratios-cpue-sections", or "mean-of-ratios-truncated-cpue-sections"
when estimator = "mortr". Code matching on method for sectioned catch rates
should expect the new values. No estimate changes; the numbers were already
mean-of-ratios.
estimate_harvest_rate() and estimate_release_rate() gained an estimator
argument and now resolve it from the design's interview_type, matching
estimate_catch_rate(). estimate_total_harvest() and
estimate_total_release() gained estimator and truncate_at and follow
their own rate functions through the resolver added in #268 (#271).
Harvest and release rates and totals on a roving design change by default.
Where neither use_trips nor estimator is specified and the design was built
with add_interviews(interview_type = "roving"), all four now use all trips
with the truncated mean-of-ratios estimator. Previously mean-of-ratios HPUE and
RPUE existed nowhere outside the bus-route path: the standard internals were
ratio-of-means only, so a roving survey got a mean-of-ratios catch rate and a
ratio-of-means harvest rate off one design, with no message.
Hoenig et al. (1997) recommend the truncated mean of ratios for a roving survey because the clerk intercepts trips mid-stream. That argument is about the interview, not about which fish are counted — harvest and release are recorded in the same interception and are length-biased the same way.
To keep the previous numbers, pass use_trips = "complete" or
estimator = "ratio-of-means"; naming either one suppresses the routing.
Access-point designs are unaffected. Bus-route and ice designs never
auto-route, and their totals now refuse a mean-of-ratios estimator rather than
accepting and ignoring it.
This completes #268. The rule established there is unchanged — each total resolves by the same rule its own rate function uses — and only what that rule resolves to has moved, because the harvest and release rate functions now have an estimator to follow.
The method on a harvest or release estimate now names the estimator that
produced it: "mean-of-ratios-hpue" / "mean-of-ratios-rpue" (and the
-truncated- and -sections variants) rather than always reporting
"ratio-of-means-". Code matching on method for these two metrics should
expect the new values on MOR paths.
estimate_total_catch() now resolves its rate estimator from the design's
interview_type instead of always requesting ratio-of-means, and gained
estimator and truncate_at arguments (#268).
Total catch on a roving design changes by default. On a design built with
add_interviews(interview_type = "roving"), and where the caller specified
neither use_trips nor estimator, the total is now built from all trips
using the truncated mean-of-ratios estimator -- the same specification
estimate_catch_rate() already chose for that design. Previously the rate
function used all-trip MOR while the total used complete-trip ratio-of-means
on the same object, so a survey's reported catch rate and its reported total
catch came from different estimators and different trip sets, with no message.
Mean-of-ratios and ratio-of-means are not two spellings of one quantity: MOR averages per-interview ratios and is the estimator justified for mid-trip interception, and Hoenig et al. (1997) recommend it, truncated at 30 minutes, "to estimate catch rate and hence total catch under the roving creel survey design". The truncation is part of the estimator rather than a tuning knob, because the untruncated mean-of-ratios estimator has infinite variance.
To keep the previous numbers, pass use_trips = "complete" or
estimator = "ratio-of-means" explicitly; naming either one suppresses the
automatic routing. Access-point designs are unaffected, as are bus-route and
ice designs, which estimate a completed-trip Horvitz-Thompson total, never
auto-route, and now refuse a mean-of-ratios estimator rather than accepting
and ignoring it.
estimate_total_harvest() and estimate_total_release() are unchanged:
estimate_harvest_rate() and estimate_release_rate() offer no estimator
selection to follow, so routing their totals would have re-created the same
rate-versus-total disagreement in the other direction. All three totals now
resolve through one function, so when those two rate functions grow estimator
selection (#271) their totals follow without further change.
use_trips on all three estimate_total_*() functions now defaults to NULL
("not specified") rather than "complete", so a design that should be routed
can be told apart from a caller who asked for complete trips. "complete"
remains what an unspecified value resolves to everywhere except the roving
catch case above. An unrecognised value is still refused, with the wording now
"Must be one of" rather than match.arg()'s "should be one of".
estimate_total_harvest() and estimate_total_release() gain a use_trips
argument and now estimate from complete trips by default, and
estimate_total_catch() honours use_trips on the paths that were silently
discarding it (#266). This changes the numbers all three return on any
design that records trip status and holds a mix of complete and incomplete
interviews.
A rate estimated from incomplete trips is length-biased: an interview taken mid-trip reports the catch so far against the effort so far, and the two do not scale together over the trip. Excluding incomplete trips is the documented default of the rate estimators for that reason, and a total is effort times a rate, so it inherits the bias while still returning a plausible number.
Three separate failures, one shape:
estimate_total_harvest() and estimate_total_release() had no
use_trips argument and no trip filter on any path. They were built
from every interview, while estimate_harvest_rate() and
estimate_release_rate() default to the complete ones -- so the total and
the rate on one design disagreed about which interviews they came from.estimate_total_catch() threaded use_trips into each dispatch branch
separately. It reached the ungrouped and grouped paths and was dropped at
the call sites for by = <species> and for sectioned designs, where the
argument was accepted, documented and inert: "complete" and "all"
returned the same number, which was the "all" number.use_trips = "all",
as estimate_total_catch() already did. Those estimate a completed-trip
Horvitz-Thompson total, where an uncompleted trip would contribute
catch-so-far under the inclusion probability of a completed one.All three now apply one filter, once, before any dispatch, so the ungrouped,
grouped, species and sectioned paths are built from the same interviews. The
filter runs ahead of the creel_warning_pooled_domain_mix check, so that
warning now describes the interviews the estimate is built from: under
use_trips = "complete" it previously reported a rate spread, and per-level
rates, drawn from the incomplete trips it had just excluded.
Passing the value down each branch is what allowed two of them to lose it.
The filter is deliberately quieter than the rate functions' use_trips
block: no messages, no minimum-sample abort, and none of "incomplete",
"diagnostic" or the roving auto-routing, which belong to the rate a caller
can estimate directly.
Designs that record no trip status are unaffected -- there is nothing to
filter on -- and callers wanting the previous behaviour can ask for it with
use_trips = "all". The new argument sits after target, matching
estimate_total_catch(), so positional calls that reached
aggregate_sections or later without naming them will land one argument
short.
The totals still never consult interview_type and are hard-wired to
ratio-of-means, so a roving design's total does not follow its rate function
to mean-of-ratios. That is a separate defect, filed as #268.
estimate_total_harvest() and estimate_total_release() no longer ignore a
design's sections when grouping by species (#255). This changes the numbers
those two calls return on a sectioned design.
Both resolved species before the section dispatch, so
estimate_total_harvest(sectioned_design, by = species) returned a lake-wide
species total: effort estimated by pooling the sections rather than per
section, reported with no section column and nothing to say the sectioning
had been ignored. It was a believable number for a different estimand.
estimate_total_catch() was not affected in the same way -- it dispatched to
sections first and then failed, with tidyselect's "Column species doesn't
exist", because species lives in the catch table and so is in neither the
counts nor the interviews.
All three now dispatch to the section path first and resolve species inside it, returning one row per section per species. Catch is apportioned against each section's own whole effort, which is what makes a species total formable per section at all: it rides on whole effort rather than splitting it.
As with any other grouping on a sectioned design, a species result carries no
.lake_total row and no prop_of_lake_total. Grouping decides that, not the
absence of by -- a species-only call leaves no grouping columns behind, and
a gate keyed on those alone would have summed a lake total across species.
A grouping variable alongside species must still be present in the counts,
since effort is genuinely split by it. That refusal is the same
creel_error_count_unobservable_by raised on unsectioned designs (#241),
rather than a second wording.
as_hybrid_svydesign() now requires a calendar and estimates a period
total rather than a sampled-day total (#246).
access_fraction and roving_fraction are within-day quantities -- the
proportion of a component's frame the count enumerated on a sampled day.
Since #229 the PSU is the date, and those fractions were passed straight to svydesign(fpc = ),
which is a stage-1 correction over the date PSUs. survey therefore computed
each stratum's population as sampled dates / fraction and read the result
as a count of days: three sampled dates at 0.5 made a "six day" weekday
stratum, where a June weekday stratum holds about twenty. A fraction that
should not shrink the stage-1 variance at all was shrinking it as though half
the calendar had been enumerated. Because each component divided by its own
fraction, access and roving also implied different calendars for the same
stratum -- 6 days and 7.5 days at once, the same arithmetic signature #229
fixed, displaced from within a stratum to across the pair of them. The point
estimate carried no signal either way; only the standard error moved.
The population now comes from a required calendar argument giving the days
each stratum holds, counted as distinct dates the way creel_design() counts
them, and shared by both components -- one stratum is one span of the season,
whichever method observed it. Both expansions live in the weight: the
within-day fraction to the whole of a sampled day, and N_h / n_h to the
season. Only the second drives the finite-population correction.
Totals from this design are now season totals and are larger than before by
the day expansion. calendar has no default, and every sampled date must
appear in it under the same stratum.
as_hybrid_svydesign() now refuses a calendar that assigns one date to more
than one stratum (#246).
A day counted in two strata lengthens the season by a day in each, so the period total expands to a calendar larger than the one that exists. On the probe fixture the weekend stratum grew from 5 days to 6 and the total moved from 1078.75 to 1195.5, with no error and no warning. A date repeated within one stratum is still accepted: \eqn{N_h} counts distinct dates, so it changes nothing.
as_hybrid_svydesign() now refuses an fpc that is not a non-missing logical
scalar (#246). It is branched on with a bare if, where NA surfaced as base
R's missing value where TRUE/FALSE needed and a length-2 vector silently took
its first element -- building the design with a correction the caller never
chose.
as_hybrid_svydesign() now requires Date date columns and refuses a missing
date or stratum in access_data, roving_data or calendar (#246).
Dates and strata are the keys the two components, the calendar and the day
expansion are all joined on, and they are compared through as.character(),
which renders NA as the string "NA" and then matches it to every other
NA. A missing calendar date was counted as one more day in \eqn{N_h}: on a
four-day weekday calendar with two sampled days, one NA row moved the total
from 156 to 195, with no error and no warning. A missing calendar stratum
silently withheld that day from the stratum it belonged to, and a missing
sampled date reached survey::svydesign(), which aborted with
missing values in `id' -- an error naming an internal column the caller
never supplied. Date is now required for the same reason the rest of the
package requires it: the keys must mean the same day on both sides of the
join.
as_hybrid_svydesign() now refuses repeated counts on one date (#246).
A day-level expansion is only defined when a sampled day is one row per
component. Two counts on a date are two looks at that date, not two sampled
days; they were summed, which multiplied the total by the number of counts
per day, and the new day expansion would have multiplied that again. This is
the defect class refuse_duplicate_psus() (#193, #197) guards on the
creel_design path. Clustering on the date (#229) corrected the variance but
left the point estimate inflated, so the construction now aborts with a
creel_error_repeated_psus condition. Average repeats to one row per date
before constructing the design.
as_hybrid_svydesign() now requires trips_disjoint, stratifies on the
stratum-by-component interaction, and clusters on the date (#229). Three
seams, one construction.
The two components were pooled into a single stratum while each kept its own
sampling fraction, so survey derived a population size from a row count
that mixed access and roving rows -- one stratum carried two population sizes
at once (weekday was simultaneously 10 and 12.5 in the function's own
example), and the only signal was an unexplained fpc' varies within strata
warning from survey next to a number that looked fine. Access and roving
sample different frames at different rates, so each is now its own stratum
and carries its own population size.
ids = ~1 made every row a PSU, so two counts taken on one date were two
independent sampling units -- the defect class refuse_duplicate_psus()
(#193) exists to prevent on the creel_design path, routed around by this
bridge. Observations are now clustered on date_col. Point estimates are
unchanged; standard errors were understated and are now larger. A component
that sampled only one date within a stratum now leaves that stratum with a
single PSU, so survey will refuse to compute its variance where it
previously returned one from rows mistaken for days.
Adding the two component totals is valid only if the components sample
disjoint sets of angler trips, and nothing in date, strata or count can
establish that. trips_disjoint is now required with no default: pass TRUE
to affirm the precondition holds. component still names a survey method,
not an angler population -- either method can cover either angler type -- so
disjointness is a fact about the protocol and never about the labels.
The documentation also said the returned object was "suitable for effort
estimation via estimate_effort()", which refuses it: it is a
survey.design2, not a creel_design. The help page now sends users to
survey::svytotal(), and its example is no longer wrapped in \dontrun{},
so R CMD check executes the documented path. The text described a hybrid
design as suiting both components "within the same sampling frame" -- the
condition under which the sum is wrong -- and gave differing coverage as an
example of bias when differing coverage is what makes the total valid.
The within-day variance component is now keyed by the sampling unit rather
than by the PSU alone (#227). add_counts() keys design$within_day_var by
the full unit key -- the section, the site, or whatever unit_cols named --
and two consumers rebuilt a narrower c(psu_col, strata_cols) key from the
design instead of reading the one the table was built with.
Nothing errored, because a join on too few columns does not fail: it returns
more rows than it was given. On a three-section design each section's 12 count
rows matched three within-day rows apiece and became 36, so every section
summed the lake-wide sum of squares, and the inflated row count also became
n_sampled in the variance divisor. On a fixture where two of three sections
are counted identically at both count times -- no within-day variation
whatsoever -- all three reported the same se_within of 849.9, roughly 85% of
each section's total standard error, against between-day components of 110 to
228.
The same wrong key scaled the component to effort units. ss_d is multiplied
by T_d^2, and match() on the date returned the first row carrying it, so
every section of a date was scaled by whichever section sorted first. A
section open 6 hours sitting alongside sections open 12 was scaled by 12^2
instead of 6^2: fourfold too large, silently. Sections with different open
hours is an ordinary field situation.
This moves standard errors, confidence intervals and every downstream
product for any design whose unit key is wider than (psu, strata):
sectioned designs, site-structured designs, and any use of unit_cols. Point
estimates are unchanged. estimate_total_catch(), estimate_total_harvest()
and estimate_total_release() all build products from estimate_effort_total()
and inherit the correction.
The same defect crashed rather than lying when the extra unit-key column came
from unit_cols and the caller grouped by it: the narrow join renamed the
duplicated column and the estimator died inside base R with replacement has 0 rows. That path now returns per-group within-day variance.
Both consumers now read the key off the table itself, via the new internal
within_day_key_cols(), so a key written one way and read another cannot
recur.
Found by the sectioned/hybrid seam audit; the .lake_total row's standard
error omitted this component too, corrected separately below (#228).
The .lake_total row of a sectioned effort estimate now reports the same
variance components as the section rows above it (#228). Section rows come
from estimate_effort_total(), whose se is
sqrt(var_between + var_within). The lake row came from a pure
survey::svyby() + svycontrast() aggregation, which carries the between-day
component and its across-section covariance and nothing else -- two
definitions of variance in one column.
The result was a lake-wide standard error smaller than that of every section it contained. On a fixture whose three sections have genuine within-day variation the sections reported 405.6, 698.8 and 186.8 while the total reported 472.3, and that figure did not move at all when a section's within-day spread was widened from 0.1 to 0.9.
se_between and se_within were reported as NA on that row. The package's
convention is that NA means unknown, so a missing component read as a
decomposition that could not be performed rather than one that was never
added; both are now reported.
The within-day component is second-stage sampling error inside one unit, so on
a shared day it is independent across sections and the per-section components
add; the between-day covariance the sections do share is already inside the
svycontrast() figure. A present section carrying an unknown component
propagates to an NA lake se rather than to the between-day figure alone.
This widens the lake-wide standard error and confidence interval for any
sectioned design with more than one count per day. Point estimates are
unchanged, and a design with a single count per day is unaffected: its
within-day component is a true zero. Applies to both method = "correlated"
and method = "independent".
Found by the sectioned/hybrid seam audit. Depended on #227: the correct per-section components are its input.
A sectioned effort estimate now carries the party-size expansion component
into its .lake_total row and reports it on the returned object (#230).
estimate_effort_sections() called new_creel_estimates() without
se_expansion, and built the lake row's variance from the between-day and
within-day components alone.
Both halves were wrong. se_expansion came back NULL, which is this
package's signal that a component does not apply and was indistinguishable
from a design where no party-size standard error was ever supplied. More
seriously, the lake row's se did not depend on party_size_se at all: on a
two-section fixture the section rows moved from 15.811 to 15.937 and from
7.906 to 8.108 when a party-size SE was introduced, while the lake row stayed
bit-identical at 14.577. It now reports 15.065.
The combination now routes through combine_section_variances(), the helper
the three estimate_total_*_sections() twins already share, so the structure
classification cannot drift apart from theirs. A multiplier estimated once and
applied across sections is one random quantity common to them, so its
contributions add before squaring; per-section multipliers are independent and
combine in quadrature; groups that straddle the sections unevenly are resolved
exactly from the per-group decomposition, and only fall back to NA when no
decomposition is available to resolve them.
That helper gained an expansion_in_section_var argument to make this
possible. The twins' base is sum(section_var), which already holds each
section's contribution as an independent term; the effort base is between-day
plus within-day, which does not, so the independent terms are added before the
covariance correction replaces them. The argument defaults to the twins'
behaviour, leaving their arithmetic bit-for-bit unchanged.
This widens the lake-wide standard error and confidence interval for any
sectioned effort design carrying a party-size expansion with a supplied
standard error. Point estimates are unchanged, and a design with no expansion
is unaffected -- it still reports se_expansion as NULL and an unchanged
se, rather than a zero that would be indistinguishable from a component that
never propagated.
expansion_decomposition is reported alongside the component. The
creel_estimates contract is one entry per row of estimates, NULL exactly
when se_expansion is; returning a component with no decomposition behind it
broke that, leaving se_expansion recoverable from nothing and giving a
combination over a wider partition no group index to work from. The
.lake_total entry is keyed by party-size group rather than by section -- the
groups summed across the sections -- so squaring and summing any row's
decomposition reproduces that row's component.
The group-wise sum is now combine_section_decompositions(), factored out of
exact_expansion_var() so the aggregated variance and the decomposition
reported beside it cannot describe different geometries.
Found by the sectioned/hybrid seam audit. The issue as filed described this as metadata-only; re-deriving it before implementing showed the lake row's arithmetic was affected too.
Harvest and release rates on sectioned designs now honour use_trips, and no
longer include incomplete trips by default (#263).
estimate_harvest_rate() and estimate_release_rate() dispatched to their
section path before the block that validates use_trips, filters the
interviews and reports what it did. On a sectioned design the argument was
therefore inert: use_trips = "all" and use_trips = "complete" returned the
same number, an unrecognised value was accepted rather than refused, and no
filtering message was emitted. estimate_catch_rate() has always dispatched
after that block, so on one design the three estimators disagreed about which
interviews they were built from -- catch rate from the complete trips, harvest
and release from every interview.
This changes numbers. HPUE and RPUE estimated from incomplete trips are length-biased: an interview taken mid-trip reports the catch so far against the effort so far, and the two do not scale together across the trip. Excluding incomplete trips is the documented default for exactly that reason, and a sectioned design was silently opting out of it while returning a plausible-looking result.
Callers who want the previous behaviour can ask for it explicitly with
use_trips = "all", which now reaches the sectioned path.
The sectioned path of estimate_total_catch() discards use_trips for a
different reason -- the argument is never passed to the section helper -- and
is tracked separately as #266.
Grouped totals on sectioned designs now report the party-size expansion component too (#260).
A sectioned design splits by section; naming another variable in by= splits
within each section, and that branch delegates to the grouped estimator --
which reports a component per row of its own result. Only the estimates were
kept at the section boundary, so every grouped sectioned total reported
se_expansion = NULL while its standard error carried the party-size term.
This is the same reporting gap as #259 one branch over, and it outlived that
fix in a way that made it visible: afterwards by = c(<species>, day_type)
reported the components that by = day_type still returned NULL for, on the
same design and over the same partition.
Point estimates and standard errors are unchanged. An absent section's
placeholder row reports NA, as it does for species.
Species totals now report the party-size expansion component their standard error carries (#259).
estimate_total_catch(by = <species>) and its two near-twins reported
se_expansion = NULL on designs whose counts were expanded from boats by an
estimated party size, while the standard error demonstrably moved with
party_size_se. A reader decomposing the standard error was told the
party-size contribution was absent when it was inside the number -- the one
thing the component contract forbids, since NULL is how the package says a
term was never propagated.
The component was computed correctly and then dropped. The species estimators
move the species column to the front of the result, and on the ungrouped
product branch that result is a base data.frame, whose [ keeps only names,
row.names and class. The grouped branch returns a tibble, which preserves
attributes, so by = c(<species>, x) was unaffected and the gap was invisible
there. On sectioned designs a second gap sat downstream: the wrapper filled
its per-section component only on the ungrouped branch, so a sectioned species
total reported nothing for its own reason.
Point estimates and standard errors are unchanged -- the term was always in
se. What changes is that se_expansion and the party_size entry of
se_components are now populated, one value per reported row, with NA for
the placeholder row a missing section contributes.
Totals pooled over a domain the counts never classified now say so (#242).
Counts bound what a total can be broken down by. When a domain is present in
the interviews but not in the counts, the only available total is
E_total * rate_pooled, and the pooled rate is a ratio of means weighted by
the interview sample's composition over that domain. Had the domain been
classified in the counts it would be a stratum, and the total would be
sum(E_h * rate_h) -- unbiased whatever the interview composition.
The two agree only when the interview sample's effort composition matches the true effort composition, and interview selection is not proportional to effort by construction: access interviews intercept completed trips, over-representing anglers who must return to a fixed point, and roving interviews are length-biased toward longer trips. Malvestuto (1996) states it directly -- "it is usually impossible to sample all angler types proportional to their level of effort", a particular problem for bank anglers "widely dispersed along the shoreline". So the mix differs by design rather than by accident, and where levels differ in rate the pooled total inherits that difference. Nothing in the reported output distinguished the two situations.
estimate_total_catch(), estimate_total_harvest() and
estimate_total_release() now raise a warning of class
creel_warning_pooled_domain_mix when the interviews carry an unclassified
categorical domain and a crude rate differs by more than 20% across its
levels. Both conditions are required: a domain the counts miss is only a
problem when the levels actually differ in rate, and a warning that fires
where nothing is wrong is one users learn to ignore. It fires once per
estimator and domain, and on the sectioned paths as well.
The screen is a ratio of sums taken straight off the interview columns, not a
survey-weighted estimate. estimate_catch_rate(by = ) refuses sparse
interview data, which is precisely the case most at risk of a mismatched mix,
so a check built on it would fail where it is needed most.
The warning names a risk, not a defect, and is worded that way: this is unverifiable from within the data, because the counts hold no composition to compare against. The three estimators' help pages gain a "What the pooled total assumes" section stating the same thing, and pointing at classifying the domain in the counts as what removes the assumption.
creel_schema() gains harvest_lengths_table and release_lengths_table,
for a source that keeps harvest and release lengths in separate tables
(#185). Both fall back to lengths_table, so a single-table source needs
neither and nothing changes for an existing schema.
tidycreel.connect's YAML loader has offered both keys since #176 and passed them straight to this constructor, where they arrived as unused arguments: a profile setting either aborted with a base error naming no cause. The connect DBI backend reads the two lengths tables through them.
estimate_harvest_rate() and estimate_release_rate() gain a targeted
argument, matching estimate_catch_rate() (#307).
targeted = FALSE restricts the domain to the interviews that recorded some
of the species being estimated, so the result is the harvest or release rate
among trips that took that species rather than the fishery-wide rate. It is
read by the mean-of-ratios forms and ignored by ratio-of-means, exactly as
for the catch rate, and the per-species exclusion is now shared by all three
rate functions through one internal helper rather than copied.
targeted = FALSE requires by = species on these two functions and
raises creel_error_targeted_needs_species otherwise. Without a species
there is no per-species count to test, and the only available test would be
"recorded nothing at all" — a different estimand, and one the package has
never estimated for harvest or release. Refusing is deliberate: an argument
that silently does nothing is the defect #304 fixed, and this avoids
repeating it. The check runs above the section dispatch, so it reaches
sectioned designs too.
No estimate produced with default arguments changes. The per-species ">70% recorded none of this species" diagnostic warning is new on these two functions and fires only under the mean-of-ratios estimator.
estimator = "regression" now runs the regression on a by = species
request, instead of silently returning ratio-of-means (#290).
The species dispatch sits above the regression route, so a species request
reached estimate_cpue_species(), whose only branches were ratio-of-means and
mean-of-ratios. "regression" fell through to ratio-of-means. No error, no
warning, and a believable number under an estimator the caller did not ask
for. It affected flat designs as well as sectioned ones, which is what
separated it from #285.
reg <- estimate_catch_rate(design, by = species, estimator = "regression")
rom <- estimate_catch_rate(design, by = species, estimator = "ratio-of-means")
# before: identical estimates, method "ratio-of-means-cpue-species"
# after: method "regression-cpue-species", and the estimates differ
The species form is CPUE₃ restricted to one species: the slope of that
species' catch on the same angler effort, forced through the origin, with the
same leave-one-out jackknife SE. That is Petrere et al. (2010) eq. 3,
sum(C_i f_i) / sum(f_i^2), computed on the trip set the call already uses.
Both the flat and the sectioned species paths route through it, so a sectioned
request reports "regression-cpue-sections" with per-section rows.
Zero-catch interviews are kept by default. An interview that caught none
of the target species contributes a 0 at positive effort, and it is a real
observation: Petrere et al. evaluated all three estimators with zeros present,
using a delta distribution with a 10% probability of zero precisely because
"zero catches are fairly common". Dropping them would change the estimand from
the catch rate of that species per angler-hour to the rate among anglers who
caught it.
targeted = FALSE still makes that second choice available, and on the
regression species path only the exclusion is applied per species: a trip
that caught none of the species being estimated is dropped, whichever other
species it caught, and the warning names the species and the percentage
excluded.
It is confined to the regression form on purpose. targeted has always been
read inside the mean-of-ratios branch, which tests the design's total catch
column and is therefore usually inert on a species request — on the release
fixture no interview has zero total catch while 20 of 22 have zero bass.
Widening the per-species test to the other estimators would move numbers
existing ratio-of-means callers already get, so it is filed as #304 rather
than changed here.
This replaces the refusal added alongside #285, which was a deliberate placeholder while the modelling question was open.
Corrected the Petrere et al. (2010) reference in two more places (#290).
#233 fixed this citation in one block of simulate_creel_data() and missed
two others — a second block in the same file, and the compare_cpue_estimators()
documentation. All three gave Fish. Res. 106: 325-333; the paper is
Braz. J. Biol. 70: 483-491, 10.1590/S1519-69842010005000010.
A grep for the journal name, rather than for the sentence, would have caught
all three at once.
@param targeted said zero-effort trips were excluded; the code excludes
zero-catch trips (#290).
A species-level regression request was held to the ratio-estimation n >= 10
floor (#290). That floor is a ratio rule; the regression slope has its own
"fewer than 3 interviews" rule, and the ungrouped regression path was never
held to the ratio one. The species path therefore refused a defined estimator
with a message about a different one — reachable via use_trips = "all",
where the ungrouped regression ran at n = 5 while the species form aborted.
The floor still applies to the estimators it belongs to.
A species-level regression result reported variance_method as the caller's
variance argument rather than "jackknife" (#290). The slope's SE is a
leave-one-out jackknife computed inside the regression internals, which never
consult variance; the ungrouped and sectioned regression paths already
reported it correctly, so only the species path named a variance that had not
run — the same class of mislabel as #284.
by = no longer accepts the interview id or an internal .-prefixed column
as a grouping variable (#293).
On a design carrying catch data, estimate_total_*(by = everything())
selected the species column, routed to the species branch, and grouped by
every interview column — including interview_id, which is unique per
row. Nothing refused it. On a 22-interview fixture the call did not return
within 900 seconds.
The statistical stake is not the runtime. A key column as a grouping variable
puts one interview in each group, so every group's rate has n = 1 and no
within-group variance is estimable. Had the call returned, it would have
produced a table of per-interview "totals" each carrying an uncertainty that
could not have been computed — the failure mode this package treats as the
dangerous one, since the number looks fine.
Two refusals now sit in the shared by = resolver, so every estimator that
groups by interview columns gets them:
estimate_total_release(design, by = everything())
#> Error: `by` names the interview key `interview_id`.
#> x It holds one value per interview, so every group would be a single
#> interview and no within-group variance could be estimated.
estimate_total_release(design, by = .angler_effort)
#> Error: `by` names a column the package derived: `.angler_effort`.
#> x It is computed by `add_interviews()`, not data you supplied.
Both tests are structural — they ask what the design registered, never what a name looks like or what values happen to hold.
The key comes from whichever of add_catch(), add_lengths() or
add_ages() registered the interview id. Testing for distinct values instead
would be wrong: on a short survey a real grouping column such as date can
be unique per row without being a key.
The derived set is read from the design one field at a time, because a
leading . is not the test either. A user column literally named
.se_expansion is a supported grouping variable (#259). And a design now
records whether it computed the trip duration, rather than the check
inferring it from the column being called .trip_duration_hrs: a caller may
supply a column of their own by that name, and it stays groupable. A column
of your own is never treated as derived, whatever it is called.
Derived columns are treated by how they were selected: a wildcard such as
everything() means every column the user brought, so they are dropped
silently, while asking for one specifically is an error rather than a silent
substitution. The two are told apart by re-resolving the selector with the
derived columns removed — if that leaves nothing to select, the selector was
asking for them. One consequence is deliberate and worth knowing:
starts_with(".") is a refusal on a design whose only dot-named column is
derived, and a silent drop on one that also has a user column such as
.se_expansion.
estimate_effort() is unaffected: it resolves by = against the counts,
which carry neither an interview key nor these derived columns.
Two limits worth stating. A design with no catch, lengths or ages attached
registers no id column, so an id there is still accepted — nothing in the
design says it is a key. And the sparse-group warning still only warns at
n < 3; whether n = 1 should be refused generally is a wider question this
did not settle.
creel_n_camera() no longer warns that a stratum is below a Feltz and
Middaugh (2025) camera-day minimum (#234). The 12 weekday and 7 weekend days
that check used are the study's per-month well-performing schedule, while
n_h allocates over the whole period named in N_h, so the comparison ran
across scales and under-fired by roughly the number of months surveyed. On the
function's own documented example -- about three months -- the cited schedule
is near 36 weekday and 21 weekend camera-days, but the check fired only below
12 and 7 and passed a plan of 27 and 12 in silence.
Scaling the benchmark would have fixed only that one gap. Three others remain
and the function cannot close any of them: the 12/7 scenario is specifically
at 1 count/day and nothing here knows the counts per day; its error band is
fixed by the study rather than taken from cv_target, so a caller asking for
a tight CV was judged against the loose row; and the simulations measured
boat-trailer counts on six Arkansas reservoirs, whereas ybar_h and s2_h
are whatever the caller piloted. The benchmark is now stated in ?creel_n_camera
in the study's own units, with those conditions, as design context rather than
a threshold.
Which benchmark applied was also chosen by matching "weekday" or "weekend"
as a substring of a caller-supplied stratum name, so weekday_holiday took 12,
weekend_evening took 7 by luck, and Sat/Sun took neither and drew a
permanent "unclassified stratum" advisory instead. No stratum name now changes
the result.
No sample size changes. The check only ever emitted a warning; n_h,
total and allocated were always returned as computed.
The error raised when a by variable is missing from the interview data now
carries the condition class creel_error_by_missing_in_interviews and is
attributed to the estimate_total_*() call that produced it, rather than to an
internal helper (#254). Grouped totals require each by variable in both the
count and the interview data; the count-side half already raised a classed,
catchable condition and this half raised an unclassed one, so only one of the
two could be handled programmatically. The message is unchanged.
estimator = "regression" now runs on a sectioned design instead of being
silently discarded (#285). estimate_catch_rate() dispatches on sections
before it dispatches on the estimator, so a sectioned regression request fell
through to estimate_cpue_total(), whose estimator test names only the
mean-of-ratios variants -- and "regression" landed in the ratio-of-means
branch. No error, no warning, and a believable number.
The visible symptom was in compare_cpue_estimators(), whose purpose is making
estimator divergence visible: on every sectioned design it reported the
regression row as numerically identical to the ratio-of-means row, carrying a
jackknife standard error because that function requests one for regression. A
ratio-of-means point estimate with a jackknife SE under the regression label
corresponds to no estimator in the literature.
A sectioned regression now fits one regression per section, on that section's
interviews, and reports method = "regression-cpue-sections" with
variance_method = "jackknife" -- the variance that actually ran, rather than
the caller's Taylor default. force_origin reaches the sectioned path, which
previously had no such argument at all. The section-level jackknife SE rests on
that section's interviews rather than the whole sample and is correspondingly
less stable; this is documented on estimate_catch_rate().
species in by combined with estimator = "regression" is now refused
rather than answered with a different estimator (#290). The species dispatch
also sits above the regression route, so this affected flat designs too,
not only sectioned ones: the call returned ratio-of-means numbers labelled
"ratio-of-means-cpue-species" while the caller had asked for regression.
Refused rather than implemented, because a per-species regression is a modelling decision and not a correction: a species that was not caught on a trip contributes a zero at positive effort, and whether those rows belong in the slope changes the estimate. That question is open in #290.
The mean-of-ratios diagnostic banner now describes the trips the estimate was actually built from, and every metric that takes the estimator prints one (#276). Two problems, both exposed by mean-of-ratios spreading beyond the catch rate.
HPUE got no banner at all. estimate_cpue_total() and
estimate_cpue_grouped() returned a mean-of-ratios object; the harvest
internals computed the same truncation metadata and then discarded it,
returning a plain result. The same estimator = "mor" request therefore
produced a caveat and a truncation report for CPUE and RPUE and silence for
HPUE. Harvest now returns the same object its twins do, ungrouped and grouped.
The banner also said "This estimate uses incomplete trip interviews (n of N
total)" whatever trips had been used. That wording dates from when
mean-of-ratios was the incomplete-trip estimator; since the roving
auto-route (#268 for catch, #271 for harvest and release) the default
mean-of-ratios path uses all trips, so a roving default rate announced an
incomplete-trip caveat while using every trip it had, and
use_trips = "complete" announced one while using none. The banner now names
the trip set -- "All Trips", "Complete Trips" or "DIAGNOSTIC: ... (Incomplete
Trips)" -- and the length-of-stay caveat and the validate_incomplete_trips()
pointer appear only for the incomplete set, which is what
mor_estimation_warning() already did at run time. The truncation report
appears on every path, because truncation is part of the estimator rather than
a diagnostic detail. The "n of N" denominator is gone: trip filtering happens
upstream, so N had become the filtered count and the incomplete path printed
a literal "24 of 24 total".
The counts the banner reports are taken from the trips that survived truncation, not the set that entered it. Reported from before truncation they contradicted the truncation line printed directly beneath them -- "over all 48 interviews" above "Truncation: 12 trips excluded", when 36 ratios had been averaged. This affected all three metrics, in both the shared truncation helper and the catch rate's own filtering block.
Interviews the rate internals discard are no longer counted as used. Those
internals drop missing effort, zero effort and missing catch or harvest after
the design-level counts are stamped, so a design with six unusable interviews
printed "over 48 interviews" beside an estimate whose own n was 42, on all
three metrics and on the grouped paths.
Two things found while making the above change and fixed with it. A
mean-of-ratios rate reported no unit -- the constructor had no unit argument,
so every MOR rate read NA while the ratio-of-means rate beside it read
"fish/angler-hour"; routing harvest through that constructor would have taken
harvest's unit away. And the incomplete-trip count on a design with no trip
status column is now NA rather than 0, because that count was never
measured; the banner omits the clause instead of reporting an absence as a
zero. No estimate values change.
Estimates now record which estimator produced them, in a new estimator
component on the returned object (#275). A total's method names the product
form -- "product-total-catch" whichever rate estimator built it -- so
ratio-of-means, mean-of-ratios and truncated mean-of-ratios were the same
string, and the design slot carries the normalised estimator, where a
"mortr" request is indistinguishable from "mor" at the default threshold.
The field records the estimator as you asked for it: "mortr" stays
"mortr". It is NULL on paths that take no estimator argument, such as
effort totals, which is deliberately distinct from recording a default that
was never chosen.
The sectioned rates gained the matching method labels
("mean-of-ratios-truncated-{cpue,hpue,rpue}-sections"), and
"mean-of-ratios-truncated-cpue" gained the display label its HPUE and RPUE
counterparts already had in print(), format() and autoplot(). No
estimate changes.
The roving auto-route still resolves to "mor" rather than "mortr" and
still reports itself untruncated: truncation runs at the 0.5 default there,
but the caller did not ask for it to be mandatory.
Naming the section column in by= on a sectioned design is now refused on
every rate estimator, instead of failing inside tibble (#265). A sectioned
result is already one row per section, so estimate_catch_rate(d, by = section)
asks for a split that has happened; all three rate estimators answered with
"Column name section must not be duplicated", raised by tibble::add_column()
and naming neither the design nor what to do instead. The refusal and its
wording already existed -- refuse_section_in_by(), error class
creel_error_section_in_by -- and were wired into the three totals only
(#255). All three rate paths now use it, at the same point in the call.
Grouping within sections by anything else is unaffected, and no estimate that
returned a number before returns a different one.
Two degenerate MOR truncation inputs now abort with tidycreel wording rather
than falling through to base R or to the survey package (#279). truncate_at = NA passed the validator's numeric and length checks and reduced it to
NA <= 0, so if (NA) aborted with "missing value where TRUE/FALSE needed";
all three validators -- the catch-rate one, the totals resolver's, and the
bus-route one -- carried the same gap. A threshold that truncates away every
interview left an empty sample to reach rowSums(), which aborts with "all
arguments must have the same length", a message about matrix conformability
for a condition entirely about the chosen threshold; the refusal now names
truncate_at and the duration column, as the bus-route incomplete-trip path
already did. Both are pre-existing, and no estimate that returned a number
before returns a different one.
The MOR truncation message now reports a percentage of the interviews it
actually truncated. It divided by the incomplete-trip count regardless of
which trip set was being estimated, so use_trips = "complete" with
estimator = "mor" divided by zero and reported Inf% -- always taking the
"high truncation rate may indicate data quality issues" branch -- and
use_trips = "all" reported the share of the incomplete trips rather than of
all of them, roughly doubling it on a half-incomplete sample. Both the rate
and the total paths were affected. The denominator also excludes interviews
with no recorded trip duration, which are reported separately: a trip with no
duration was never eligible to be judged short, and counting it diluted the
short-trip rate enough to hide it below the 10% threshold that triggers the
data-quality warning. No estimate changes; the message is what a caller reads
to judge whether the threshold is discarding too much data.
estimate_catch_rate() no longer aborts when an interview has no recorded
trip duration and MOR truncation is in effect (#272). The truncation filter
compared duration against the threshold without guarding for NA, and
NA >= truncate_at is NA: a logical index carrying NA subsets a data
frame to an all-NA row rather than dropping it. That phantom row reached
svydesign() as a missing stratum and aborted inside the survey package with
missing values in 'strata', a message naming neither trip duration nor
tidycreel.
Trips with no recorded duration are now dropped, because a trip whose duration
is unknown cannot be shown to clear the threshold, and they are counted and
warned about separately from the short trips excluded by truncation — a
missing-duration loss is a data-quality fact, not an estimator decision, and a
caller needs to tell a threshold that excluded six short trips from a duration
column that is half empty. mor_n_truncated now counts only short trips.
Affects every truncating path in estimate_catch_rate(): use_trips = "incomplete", use_trips = "all" including the roving auto-route, and
estimator = "mortr". This is the guard truncate_interviews_for_mor()
(#268) already applied on the totals and br_incomplete_harvest_rate()
already applied on the bus-route path; the standard rate path was the one site
left without it. Designs with a complete duration column are unaffected.
estimate_catch_rate() no longer aborts on a bus-route or ice design built
with add_interviews(interview_type = "roving") (#270). The roving auto-route
fired before the bus-route/ice dispatch and was then undone inside it, but the
undo read a flag the route itself had to clear, so use_trips reached the
bus-route validator as "all" — which it refuses. Such a design could not
produce a catch rate at all unless the caller passed use_trips explicitly.
The route is now excluded at the point of resolution for these designs, as
resolve_total_rate_spec() already excluded it and as the harvest and release
rates get structurally by returning first. Bus-route and ice results are
unchanged, and a roving one now matches the access-point one, as it should:
these designs estimate a completed-trip Horvitz-Thompson total, for which
use_trips = "all" names no estimator that exists. Standard designs are
unaffected.
estimate_harvest_rate() and estimate_release_rate() can now group by
species on a sectioned design (#257).
Both failed with tidyselect's Column \species` doesn't existwhileestimate_catch_rate()succeeded on the same design. On a sectioned design the public estimators return into the section path before their own species dispatch runs, and the harvest and release section helpers resolvedby=`
against the interviews alone -- species lives in the catch table, so it is in
neither the interviews nor the counts.
Both now resolve by= the way the catch-rate path does and estimate per
species inside each section, from that section's own interviews. Grouping a
further interview variable alongside species works too, and a missing section
still produces its placeholder row.
No count-observability constraint applies here, unlike the totals: a rate is estimated from interviews alone, so it may be grouped by attributes a total may not (#241). Nothing is apportioned and there is no lake row or share.
Grouping effort or a total by an attribute the counts do not carry now
explains the constraint instead of reporting the column as non-existent
(#241). estimate_effort(), the three estimate_total_*() functions and
their sectioned paths resolve by= against the count data, which is correct
-- effort comes from counts, so it can only be split by what the counter
could see. But the refusal arrived as tidyselect's "Column target doesn't
exist", which is false to the user's situation: the column does exist, in the
interviews, and estimate_catch_rate(by = target) accepts it one call
earlier.
The message now says the column is in the interview data but not the count
data, lists the columns that can group effort, points at
estimate_catch_rate(by=) for a rate over that attribute and, for the
totals, at the by = <species> route that apportions catch against whole
effort. It also names the wrong workaround the old message invited: copying
the column into the counts fabricates a classification the counter never
made. Errors carry class creel_error_count_unobservable_by; a column
present in neither table still raises tidyselect's own error unchanged.
The three estimate_total_*_sections() product totals now report
se_prop_of_lake_total alongside prop_of_lake_total (#243). The share of
the lake-wide total was the only quantity in the table without an error, so a
reader comparing sections had nothing to judge the comparison by.
This is not a port of the #231 fix for estimate_effort_sections(). There the
proportion is a domain total over an overall total from one survey design, so
svyratio() returns the ratio and its error together. Here
prop_h = (E_h * rate_h) / sum_k (E_k * rate_k) has a numerator and
denominator that are each products of two estimates from different designs
-- effort from the counts, rate from the interviews -- and the numerator is one
of the denominator's own terms. The error is derived by delta method, carrying
both the cross-section covariance a shared party-size multiplier induces and
the correlation from the numerator appearing in the denominator.
The denominator variance and the cross terms come from the same
combine_section_variances() call that builds the .lake_total row's own
standard error, so the reported error belongs to the number beside it rather
than to a parallel derivation free to drift from it (#134). With two sections
the shares sum to 1, so one is a linear function of the other and their errors
come out equal -- an identity a derivation that dropped the
numerator-in-denominator correlation would fail.
.lake_total reports 0, not NA: its share of itself is exactly 1 by
construction and was never estimated. A section with no data reports NA for
both columns, since a zero would claim it held none of the total rather than
that nothing was observed there. Neither column is produced on the grouped
path, unchanged. No existing number moved.
The three sectioned product totals now report expansion_decomposition
alongside se_expansion (#238). estimate_total_catch(),
estimate_total_harvest() and estimate_total_release() each gathered the
per-section decomposition, used it in combine_section_variances(), and then
called new_creel_estimates() without it.
The constructor states the invariant where it stores the field: one entry per
row of estimates, NULL exactly when se_expansion is. A component was
reported with nothing behind it, so se_expansion was recoverable from
nothing and a combination over a wider partition -- a season, or several water
bodies -- had no group index to work from. That index is what lets a
"partial" geometry be resolved exactly instead of refused, which is the
mechanism behind #150.
Each section's entry is scaled by that section's rate, as se_expansion
already was, so the per-row identity
sqrt(sum(decomposition^2)) == se_expansion holds on the reported product
scale. The .lake_total entry is keyed by party-size group rather than by
section -- the groups summed across the sections -- because the section index
is recoverable from the rows while the group index is not.
No estimate or standard error moves. The lake row already consumed the
decomposition through combine_section_variances(); only the reporting was
missing. A design carrying no party-size expansion still returns NULL for
both fields.
Found while completing #230, which fixed the same defect in
estimate_effort_sections().
estimate_effort_sections() now forwards target to the per-section
estimator, and reports a standard error for prop_of_lake_total (#231).
The target argument was dropped by a positional call, so every section was
computed on "sampled_days" while the returned object was labelled with the
caller's target. The lake row read design$survey directly rather than going
through get_effort_target_design(), so it was stuck on sampled days too.
On a calendar holding three times the sampled days, target = "stratum_total"
returned 104, 50, 154 -- the sampled-day figures, unchanged in every row --
while reporting effort_target = "stratum_total". Nothing in the table
disagreed with anything else; only the label was wrong. It now returns
312, 150, 462.
This was unreachable from estimate_effort(), which aborts for a sectioned
design whenever target != "sampled_days". That abort describes itself as
temporary, and this is the estimand-mislabelling class rather than an
arithmetic error, so it is cheaper to forward the argument now than to
remember it when the guard is lifted. Behaviour for "sampled_days" is
unchanged: get_effort_target_design() returns design$survey for that
target.
prop_of_lake_total was a bare division of two survey estimates, reported
without uncertainty in a table where every other quantity carries a standard
error. It is a ratio of a domain total to the overall total, both estimated
from the same design and therefore correlated -- the denominator contains the
numerator. The new se_prop_of_lake_total column and the proportion now come
from one survey::svyratio() call, which handles that correlation and keeps
the reported error attached to the number beside it rather than to a parallel
derivation free to drift from it. The point estimate is unchanged.
The .lake_total row reports se_prop_of_lake_total = 0. That is a
structural zero rather than an unpropagated component: the lake total's share
of itself is exactly 1 by construction and was never estimated, so NA would
assert an uncertainty that does not exist. An absent section reports NA for
both, since it has no share to report.
Found by the sectioned/hybrid seam audit.
estimate_total_catch(), estimate_total_harvest() and
estimate_total_release() now document why they have no targeted
argument (#307).
A targeted rate is conditional on having recorded the species; total effort is not. Multiplying them applies a conditional rate to an unconditional base: on the package's example data one species' rate is 0.48 fish/hr over all 50 trips and 2.00 fish/hr over the 12 that caught it, so expanding the targeted rate by total effort returns roughly 223 fish where 30 were actually caught. The domain-consistent product needs the season-wide effort of species-catching trips, which no creel design observes. A targeted rate is therefore available and a targeted total is not, as a property of the estimand rather than a gap in the implementation.
impute_camera_counts() no longer attributes either of its imputation
models to a paper that does not contain it (#297).
Both citations named the wrong work. The negative binomial GLMM was credited to a real Afrifa-Yamoah et al. (2020) paper — but the group's climate time-series paper, which imputes weather data with expectation maximisation and LSTM neural networks, and which the relevant paper itself cites for that purpose. The relevant one is
Afrifa-Yamoah, E., Taylor, S.M., Fisher, A. & Mueller, U. (2020). Imputation of missing data from time-lapse cameras used in recreational fishing surveys. ICES Journal of Marine Science 77(7-8): 2984-2994.
and swapping it in unqualified would have repeated the defect at a finer
grain: that paper evaluates nine models in a fully conditional specification
multiple-imputation framework and concludes that zero-inflated Poisson
models "were generally ranked best", reporting the negative binomial fits as
slow and cumbersome to converge. Its fixed effects are climatic covariates
and its random intercepts are temporal classes; neither appears here. It is
now cited for what it does support — the multiple-imputation framing behind
m > 1 and est_effort_camera_mi().
The Poisson GLM default was attributed in-text to "Hartill 2016", with no matching reference entry to follow. Hartill et al. (2016) do impute camera outages with a GLM, but a cross-site one: the outage ramp's daily count is predicted from the counts at two other ramps the same day, square-root transformed as third-order polynomials, given fishing year, season and day-type. The word "Poisson" does not appear in the paper, and their stated reason for a cross-site model is that same-ramp neighbouring days were "not considered to be sufficiently representative" — an argument away from, not towards, a local mean. Both papers now carry full reference entries saying what each does and does not support, and a new "Where these imputation models come from" section states plainly that the two models offered are the package's own choices.
Also corrected in passing: the high-missingness warning told the user that
results "may be unreliable (Afrifa-Yamoah 2020)", where that paper reports
"no clear systematic trend in the performance of the models with respect to
... the proportion of missing data" and successfully imputed months of
complete outage. The warning is kept — over half a stratum being model
predictions is worth saying — but it no longer claims a source that says the
opposite. A code comment describing the GLMM's (1 | site_col) term as a
random slope now calls it a random intercept, matching the formula and the
site_col documentation. The description also no longer calls strata_col
the model's "sole predictor": it partitions the data, and a separate
intercept-only model is fitted within each level.
Documentation only; no estimate, imputed value or model changes.
Corrected two bad references in simulate_creel_data() (#233) — one
fabricated, one mis-cited.
The reference read "Greene, B.T. (1995). The ANGLER simulation model. N. Am. J. Fish. Manage. 15: 743-750." No paper by that title exists, and every field of the citation was wrong. The work the simulator actually draws on is
Greene, C.J., Hoenig, J.M., Barrowman, N.J. & Pollock, K.H. (1995). Programs to simulate catch rate estimation in a roving creel survey of anglers. DFO Atlantic Fisheries Research Document 95/99.
— a Department of Fisheries and Oceans technical report describing two S-PLUS functions that build an angler population and simulate a roving clerk, not a journal article, which is why no Crossref search for it returns anything. The author is Colin J. Greene with three coauthors, not a solo "B.T. Greene", and it never appeared in North American Journal of Fisheries Management.
Found while verifying that one, and a different kind of error: the Petrere reference is a real paper, cited with the right title, authors and year, but given the wrong journal, volume and pages — "Fish. Res. 106: 325-333" for a paper published in Brazilian Journal of Biology 70: 483-491. Corrected, with its DOI added.
Su & Clapp (2013) was checked at the same time and is correct as cited.
The @details sentence claimed the generative model "follows Su & Clapp
(2013) and Greene (1995)" without saying which part came from which. Greene
et al.'s simulated anglers are deterministic — evenly spaced around the
shoreline, all starting one hour into an eight-hour day, trip lengths
alternating between 3 and 6 hours — so the three distributional levels are
not from that paper. Only the roving-clerk step is: length-biased
interception, with catch recorded up to the interview time. The docs now say
so.
No computation changes. Documentation only.
Corrected the attribution of the camera calibration ratio, which cited
Hartill et al. (2020) for an estimator that paper does not contain (#236).
est_effort_camera() estimates rho, the hours of effort per camera count,
as a ratio of sums over the days carrying both a count and interviews, and
applies it to the stratum's full count total.
Hartill et al. (2020) is a review of digital camera monitoring. It presents no estimator and no variance, and where it discusses combining cameras with creel data it cites others. The earlier sweep in #235 replaced a fabricated Hartill reference with the real one and verified that the DOI resolved; it did not ask whether the resolved work supports the formula attached to it, which is a separate question that metadata cannot answer.
The citation is now split by what each source actually carries. The estimator and its variance are Cochran (1977): the counts are the first phase of a double sample and the interview days the second, which is the structure of Chapter 12 (Section 12.9, p. 343), and the ratio's variance is eq. 2.46 with the finite-population correction omitted. The practice of calibrating camera counts against paired creel observations is credited to Hartill et al. (2016), van Poorten et al. (2015) and Eckelbecker et al. (2022) — each of which uses a different estimator: a per-day classification proportion, a hierarchical Bayesian model, and a fitted linear correction respectively.
?est_effort_camera now states plainly that the ratio-of-totals form is this
package's own application of standard double-sampling ratio estimation, not a
reproduction of a published fisheries estimator. In particular it is not
Hartill et al.'s (2016) rho, which is a dimensionless proportion of
observed boats that were fishing, estimated per day from interviews that are
a subsample of the camera's own frame, with a bootstrap variance.
impute_camera_counts() also carried the Hartill et al. (2020) reference,
for a function that imputes camera outages by a per-stratum Poisson GLM or a
negative binomial GLMM. The review supports neither, and the entry is
removed.
No computation changes. This release alters documentation, comments and the camera vignette only.
Corrected the framing of as_hybrid_svydesign(), which described access
and roving as though they were count methods (#246).
They are not. In the creel literature access and roving describe how anglers
are interviewed -- access interviews intercept completed trips as anglers
leave, roving interviews intercept incomplete trips while anglers are still
fishing, and the two require different catch-rate estimators (Pollock et al.
1994). A survey mixing them is a hybrid interview design. Counts are
described by their own methods: instantaneous, progressive, bus-route, camera
or aerial -- the values creel_schema() accepts for survey_type, none of
which is "access" or "roving". tidycreel already carries the interview axis
on add_interviews() via interview_type.
The help page previously stated that component "names a survey method",
and the glossary defined a hybrid design as "fixed access-point counts plus
roving-route counts". Both are now corrected: the two components are two
disjoint count frames, in practice angler-type domains such as boat and
bank anglers, and the access/roving argument names are inherited from the
interview vocabulary and flagged as under review. No behaviour changed.
Corrected five references that named papers which do not exist, or whose DOI resolved to an unrelated paper. Found by checking every DOI in the package against Crossref after the camera citation turned out to be wrong.
est_effort_camera(),
estimate_effort_camera() and impute_camera_counts(), gave a title, an
author list and a journal that belong to no paper, and a DOI
(10.1016/j.fishres.2020.105706) that resolves to a study of age
determination in sawsharks. The real reference is Hartill, Taylor, Keller
and Weltersbach 2020, Digital camera monitoring of recreational fishing
effort: applications and challenges, Fish and Fisheries 21:204-215,
\doi{10.1111/faf.12413}.estimate_angler_n() and the mark-recapture
vignette, used 10.1139/f58-002, which is The Abundance and Distribution
of the Northern Sea Lion. The correct DOI is 10.1139/f58-003; it is one
article later in the same issue.estimate_effort_aerial_glmm(),
example_aerial_glmm_counts and the aerial GLMM vignette, had the right
DOI but an invented title and the wrong pages, and the vignette named four
authors none of whom wrote it. It is Angler effort estimates from
instantaneous aerial counts, NAFM 38:194-209.simulate_creel_data(), is in Transactions of
the American Fisheries Society 142:234-246 under the title Evaluation of
sample design and estimation methods for Great Lakes angler surveys, not
in NAFM 33:895-909 under the title given.creel_n_camera(), was recorded as
in press under a title the paper does not carry. It is published as
Improving efficiency of estimating angler effort using low-frequency
time-lapse camera data, NAFM 45:322-332.No estimator changed. What changed is that following a reference now reaches
the work it claims to. Two related questions are tracked separately: the
provenance of the creel_n_camera() camera-day minimums, which were
attributed to the Feltz and Middaugh title that does not exist (#234), and
the unverified Greene 1995 citation in simulate_creel_data() (#233).
creel_n_effort() and creel_n_camera() now share one internal
implementation instead of holding two copies of the same 37 lines (#295).
The two are the same stratified allocation reached through two vocabularies —
sampling days for angler contact, camera-days for a camera deployment — and
after #234 removed the camera-only warning their bodies were byte-identical.
Two copies of one computation is how a fix lands in one twin and not the
other, which this package has hit repeatedly with the three near-twin
creel-estimates-total-*.R files.
No user-visible change. Both functions keep their exports, their separate help pages and their own vocabulary; validation moved into the shared internal but the checkmate assertions name the same arguments, so error messages are unchanged. Verified over 2,000 random inputs against the previous implementation: zero differences. A test now pins the two entry points as identical, so a future re-copy that edits one of them fails.
summarize_by_day_type() and summarize_boat_composition() now resolve the
day type column instead of assuming it is the first stratum (#221). Both read
design$strata_cols[1] and labelled whatever they found there day_type. But
creel_design() preserves the order the caller declared their strata in, so
that index is a declaration order, not a definition: a design declaring
strata = c(site, day_type) produced a table of site names under a day_type
header, with no warning, and the real weekday / weekend breakdown absent
entirely.
Resolution order is now an explicit day_type_col argument, then a stratum
actually named day_type, then the first stratum -- which warns and names the
column it chose when the design declares more than one. A single-stratum
design resolves silently and is unaffected, so the documented
strata = day_type workflow does not change.
This moves numbers for multi-stratum designs. On a six-day two-site
fixture whose boat composition is driven by site, summarize_boat_composition()
reported 90% / 10% -- the per-site means under a day_type header -- where the
per-day-type figures are 63.3% / 36.7%. summarize_by_day_type() moves labels
rather than counts in the balanced case, which is what made it invisible: the
6 / 6 site split and the 6 / 6 weekday / weekend split are the same numbers.
A stratum has no canonical name in this package -- the caller names their own
calendar columns -- so this is a resolution with a documented fallback, not a
lookup. Pass day_type_col when neither inference applies.
This is the same defect class as #216, which was the identical
strata_cols[1] shortcut on the camera calibration path. Found while fixing
that issue and recorded rather than fixed inline.
estimate_effort() and the three total estimators now refuse camera designs
(#214). A camera count is a daily ingress total -- a count of arrivals -- not
an instantaneous count of anglers present. The dispatch chain in
estimate_effort() branches on bus_route, ice and aerial, and camera had
no branch, so it fell through to the instantaneous path and its counts were
summed as though they were snapshots of how many anglers were present. The
result was a plausible number with a plausible standard error: on the package's
own example data, 613 "angler visits" where the calibrated estimator returns
111 angler-hours.
The camera vignette documented that route. It called estimate_effort() for
both sub-modes, stated that camera designs "feed into the same
estimate_effort() ... pipeline -- no changes", never mentioned
est_effort_camera(), and wrapped every call in suppressWarnings(). So the
guards added by #136, #137, #142 and #158 all sit in a function the
documentation never reached: a design carrying imputed counts, a stratum with
one paired interview day, a repeated count date, and an uncalibrated raw
expansion each went unreported on the documented path.
The refusal is raised at all four entry points, not only in
estimate_effort(), because estimate_total_catch(),
estimate_total_harvest() and estimate_total_release() call
estimate_effort_total() directly and never pass through it. Guarding only
estimate_effort() would have left the totals building a product from the same
arrival count -- multiplying a rate per angler-hour by a count of arrivals and
reporting it as fish.
Refusing rather than dispatching is deliberate. est_effort_camera() already
implements the calibrated estimator and carries the guards; giving those guards
a second caller to be right about is how the split arose. It also takes
arguments the generic signature has nowhere to put -- interviews,
n_anglers, h_open, calibration -- so a silent dispatch would have to guess
them.
To fix an affected analysis, call est_effort_camera(design, interviews = , n_anglers = ) for the calibrated estimate, or
est_effort_camera(design, calibration = "none", h_open = ) to expand the raw
counts under a declared assumption of one angler-hour per count per hour open.
Catch rates are unaffected -- they come from the interviews and never touch the
camera -- but there is no camera catch total, and the vignette now says so
rather than demonstrating one.
estimate_angler_n(method = "schumacher", ci_method = "bootstrap") is now
refused rather than silently ignored (#209). The Schumacher-Eschmeyer branch
appended no ci_lo_boot/ci_hi_boot columns and attached no boot_samples,
and raised nothing at all -- so an explicitly requested inference method
vanished, and estimate_mr_harvest(ci_method = "bootstrap") then aborted
telling the caller to do what they had already done.
The bootstrap is not implemented for this estimator on statistical grounds
rather than for want of effort. The other three methods resample recaptures,
m_k ~ Binomial(n_k, m_k/n_k), which is coherent where the recaptures are the
random component. Schumacher-Eschmeyer's published variance is the residual
mean square of a weighted regression through the origin (Seber 1982
eq. 4.17) -- the scatter of the observed points about the fitted line, which
is a different quantity from binomial noise in m. Resampling m_k alone
would report a narrower, differently-defined uncertainty under the same column
names.
This breaks any call that combined the two. Such a call previously returned a
correct point estimate and a correct regression interval, so the fix is to
drop ci_method = "bootstrap", which changes nothing about the numbers
returned. "logit" and "delta" both give that interval. Use
method = "schnabel" where a bootstrap interval is genuinely required.
Repeated sampling units with no count time are now refused at estimation
rather than warned about (#193). Two counts on one day are two looks at that
day, not two sampled days: the day's effort is the mean of its counts, and
the spread between them is the within-day variance component. That averaging
has always been what count_time_col triggers -- but rows repeating a unit
without one bypassed it entirely and reached svytotal(), which sums them.
The reported effort came back multiplied by the number of counts per unit
(measured at exactly k-fold for k = 1..4) and propagated undiminished into
catch, harvest and release totals, while se_within was reported as 0,
indistinguishable from a within-day component that had been evaluated and
found to be nil.
add_counts() warned about this, and its sibling check already aborted on
rows identical in every column -- so the harmless case (a double entry) was
refused while the dangerous one (a genuine second count) was merely
announced. The package cannot tell the two apart from the table: rows sharing
a unit key are either repeat counts, which average, or undeclared distinct
units, which sum, and only the surveyor knows which. It now asks rather than
guesses.
To fix an affected analysis, say what separates the rows -- count_time_col
for repeat counts, or unit_cols for distinct units -- after which they are
aggregated correctly and the within-day spread is retained.
The refusal is raised by estimate_effort(), not add_counts(), so
estimators that never sum these rows are unaffected:
estimate_effort_aerial_glmm() models counts against their flight time and
keeps its several rows per day. add_counts() still warns, so the problem is
reported next to the call that introduced it.
est_effort_camera() now estimates the calibration ratio within every
stratum the design declares, not within the first stratum column only
(#216). A design created with strata = c(day_type, site) has strata
day_type x site, but the camera ratio path read design$strata_cols[1]
and keyed both the calibration and the count total on it. One pooled
hours-per-count ratio was formed over the coarser partition and applied to
counts belonging to a stratum that never contributed to it.
Multi-column-stratified camera estimates change. On an eight-day two-site
fixture where north fishes 40 h on 10 counts and south 10 h on 100 counts,
with interviews on three north days and one south day, the estimate was
440 against a per-stratum truth of 200 -- a 2.2x overestimate with no
warning. The factor is set by how unevenly interview effort is allocated
across the dropped columns, so it is unbounded in principle.
Where every day is an interview day the ratio of sums telescopes and the
point estimate is unchanged, but the standard error still moves: on the
balanced version of that fixture the calibration component was 107.2
against a within-stratum truth of 0, because pooling two dissimilar site
regimes inflates the ratio residuals.
Estimating within the declared strata puts fewer paired days in each
stratum, so est_effort_camera() may now report an NA standard error
where it previously reported a number: a stratum with one paired
interview/count day has no measurable ratio variance (#136), and a sum
missing an unknown term is a lower bound rather than a standard error. The
warning names the stratum. Adding a second matched interview day in that
stratum recovers the SE.
interviews must now contain every column in design$strata_cols. A
missing one is an error naming the column, where before the calibration
proceeded on whichever columns the table happened to carry.
Single-column-stratified camera designs -- every fixture in the package's own examples and tests -- are bit-identical.
est_effort_camera() no longer treats a missing camera count as a
zero-effort day on the ratio-calibration path (#215). na.rm = TRUE was
passed to svytotal through svyby, so an outage day's count was dropped
from the numerator while its population day stayed in the frame -- making it
contribute exactly zero hours to the total, with no error, no warning and no
message.
On the package's own five-day fixture, setting one non-interview day's count
to NA moved the estimate from 18.97 to 15.00, a 21% undercount. That
15.00 was bit-identical to the estimate obtained by deleting the row
outright, which is the demonstration: the missing day contributed nothing
while n still reported 5.
The raw-count branch of the same function passed no na.rm and already
returned NA for the same input, so one function answered one input two
opposite ways depending on which branch it took. Both now return NA, and
both now warn -- naming the affected dates, the camera_status values that
explain them, and impute_camera_counts() as the remedy. The count is not
imputed or reweighted here: which day is missing is informative, so the
treatment is the caller's to choose.
Because missing rows are no longer dropped, survey::SE() can now report
NaN for a stratum whose total is NA. That is normalised to NA_real_,
since the calibration component already uses NA for the same condition
(#136) and one function should not report one unknown two ways.
The suppressWarnings() around the stratified count total is removed, so
survey's own diagnostics reach the caller. It previously swallowed every
warning svyby raised, which is half of why an outage produced a confident
wrong number. It was also unnecessary: the benign "No weights or
probabilities supplied" note it was presumably there for comes from
svydesign() when the design is built, not from svyby(), and removing the
wrapper surfaces no new warnings across the test suite.
Camera surveys with complete counts are unaffected.
The three total estimators now derive the reported unit from their two
factors instead of writing the literal "fish" (#213). A total is "fish"
only when a per-angler-hour rate multiplies an effort in angler-hours;
anything else reports NA_character_.
There are two ways to fail to cancel, and both were labelled "fish":
design$effort_unit is NA whenever
add_counts() received no period_length_col, because a bare count column
may be an instantaneous head count or effort the caller already expanded,
and nothing can tell the two apart. Unknown times known is unknown.warn_party_hours_product() already
reported that seam, but the result still carried a confident label through
it.This changes the reported unit for the common workflow. The package's own
example_counts has no period-length column, so a design built from it now
reports unit = NA on its totals where it previously reported "fish".
Point estimates, standard errors and confidence intervals are unchanged --
only the label moves. Supply period_length_col to add_counts(), and
n_anglers to add_interviews(), to make the unit derivable.
The same literal appeared three more times on the bus-route and ice total
paths (R/creel-estimates-bus-route.R), which reach a different constructor.
Those are keyed on interview_effort_unit() rather than
design$effort_unit, since that is the effort those totals are built from.
Bus-route designs whose interviews carry a party size are unaffected: their
units already cancelled, and now they are shown to.
This follows the rule estimate_effort_per_acre() already used -- compose
the unit from its inputs, and an unknown input yields an unknown result.
est_effort_camera() now reports the within-day variance component instead
of a literal 0 (#217). add_counts(count_time_col = ) averages several
counts on one day into a daily mean and stores the within-day components
(ss_d, k_d) on the design; the camera estimators never read them. The
standard and aerial estimators have always called
compute_within_day_var_contribution() for exactly this, so the machinery
existed and only the call was missing.
On a five-day fixture with two counts per day, widening the within-day spread
from zero to +/-30 counts -- holding every daily mean, and therefore the
point estimate, fixed -- moved design$within_day_var$ss_d from 0 to
1800 per day and left the reported SE bit-identical at 3.266133. It is
now 5.991888, with se_within of 5.023454 where it was 0.
The component is scaled by the stratum's calibration ratio on the ratio path
and by h_open on the raw path, because the stored quantity is a variance of
the stratum count total and each path multiplies that total by a different
factor. It is combined with the between-day component at the variance level
rather than by adding two standard errors in quadrature, so a within-day
variance of exactly zero leaves existing estimates bit-identical.
se_within remains 0 for a design with one count per day. That is the one
case where a zero is right: there is no within-day variation to measure, so
the component is nil by construction rather than unknown. Designs built
without count_time_col are therefore unaffected.
estimate_exploitation_rate() requires se_C when C is a bare number
(#208). It previously reached if (se_C < 0) holding a NULL and failed as
argument is of length zero -- loud, so no wrong number ever escaped, but
uninformative on an entirely plausible call. The error now names the argument
and points at the object route added in #206, which supplies the standard
error itself. Defaulting the absent case to 0 was rejected: a zero standard
error cannot be told apart from a variance that never propagated.
The reporting_rate documentation said the correction adjusts the
exploitation rate downward, contradicting the formula printed beside it
(#207). It adjusts upward: under-reporting means the recoveries actually
observed understate how many tagged fish were removed, so dividing by
lambda < 1 restores them and at lambda = 0.5 the estimate doubles. The
code was correct throughout and is unchanged; only the wording was wrong. The
direction is now also asserted in the test suite, so prose and arithmetic
cannot drift apart again silently.
estimate_exploitation_rate() now accepts the estimate_total_harvest()
result itself for C, and checks it (#206). u = (C/T)(m/n)/lambda is the
fraction of the tagged cohort removed over the whole season, so C must be
a period total while T is the full cohort -- but estimate_total_harvest()
defaults to target = "sampled_days", and C arrived as a bare number with
its estimand stripped off. The shortest correct-looking pipeline was therefore
the wrong one, and it failed silently: on a survey sampling 6 of 30 days the
exploitation rate came back understated five-fold, inside [0, 1] so the
range guard never fired, with a standard error that scaled down with it. The
factor is the sampling fraction, so sparser surveys were wrong by more.
Passing the object lets the target be read and a sampled-day total refused.
It also makes the catch-for-harvest substitution detectable -- released fish
were never removed from the tagged cohort, and while both totals are counts of
fish, the method is recorded on the object. A stratum total warns rather than
aborting, since it is correct when T is that stratum's cohort. Supplying
se_C alongside an object is an error; the standard error is read from it.
Bare numeric C keeps working unchanged and now reports that its target could
not be verified. No estimate changes on any existing call -- the object and
numeric paths return identical results for the same total.
read_schedule() restored only four column types, so window_id came back
as character (#194). The column is added by attach_count_times() rather than
by generate_schedule(), and coerce_schedule_columns() matches an
allow-list by name, so a write_schedule() -> read_schedule() round trip
was not type-stable for it: a join against an integer window_id, an
arithmetic comparison, or an identical() check silently saw a character
vector. window_id is now restored to integer using the same guard
period_id already used, so numeric ids become integers while character
window labels are preserved. No estimate changes -- schedules carry no
quantities that reach an estimator.
The aerial-glmm vignette compared the GLMM against the simple aerial
estimator on one design holding four overflights per day, with no count time
declared. The simple estimator sums, so the figure it published was four
times the correct one (roughly 20,370 angler-hours against 5,092.5). The
comparison now builds a second design that declares the flights via
count_time_col, aggregating them to daily means; the GLMM continues to read
the individual flights, which is what it fits the diurnal curve against.
add_counts() now records unit_cols on the design. It was previously
validated and discarded, leaving the design unable to distinguish a declared
multi-column sampling unit from an undeclared repeat.
estimate_effort() on an ice design renamed its estimate column to record
the effort type, so tidy() returned total_effort_hr_on_ice (or
total_effort_hr_active) and no estimate at all (#199). Ice was the only
design to do this -- including the degenerate bus route, which is what an ice
design is. Generic code reading the documented accessor, tidy(x)$estimate,
a rollup across strata or species, or a report template, received NULL, and
sum(NULL) is 0: a season total came back as zero rather than as an error.
The effort-type column is now an alias rather than a replacement, so both
names are present and agree.
Bus-route and ice standard errors were computed over interview rows rather
than over the sampling unit (#198). build_interview_survey() passed
ids = ~1, declaring every interview its own PSU. Malvestuto (1996,
section 20.2.3) defines this design as stratified two-stage probability
sampling -- fishing days are the primary sampling units, and secondary units
are chosen within them -- so several anglers contacted on one day are not
independent draws from the frame.
Reported precision was therefore a function of interview-recording
convention. Splitting one interview into two half-effort rows at the same
site left the Horvitz-Thompson estimate exactly unchanged and shrank the
standard error by 1/sqrt(2): an agency recording one row per angler looked
more precise than one recording one row per party for the same survey.
The bus-route and ice estimators now use the ultimate-cluster estimator,
taking the variance between PSU totals, so partitioning interview rows within
a day leaves both the estimate and the standard error unchanged. Access-point
and roving designs are untouched -- there the interview genuinely is the
sampling unit and ids = ~1 is correct.
No point estimate changes. Standard errors and confidence intervals on
bus-route and ice designs do change, and not all in one direction: on the
Calamus 2016 fixture catch_total falls by more than half while
effort_total roughly doubles. Direction is a property of the data.
The stratified sample-size functions reported a total that was not the sum
of the per-stratum values, and power_creel() rendered it as though it were
(#195). total is Cochran's n, solved from the variance equation before
allocation; each stratum is then rounded up from it independently, so the
parts sum to as much as k - 1 more for k strata. Printed beneath rows
named after strata, in a column named n_required, that row read as their
sum and under-booked the survey by the difference.
total is unchanged — it is a real quantity and was documented as such.
creel_n_effort(), optimal_n() and creel_n_camera() now additionally
return allocated, the sum of the per-stratum values, which is what the
returned allocation commits to and the number to budget against.
power_creel() reports both rows. Code reading these results by name is
unaffected; code depending on the length or exact names of the returned
vector will see one more element.
No estimate changes. This is a planning-stage reporting fix — the
per-stratum values were correct throughout, and rounding each up is
deliberate, keeping every stratum at or better than its share of
cv_target.
Expanded effort targets (target = "stratum_total" and "period_total")
understated the total whenever a sampled day carried more than one count row
(#183). The two sides of the expansion factor N_h / n_h counted different
things: the numerator counted calendar rows, the denominator counted rows of
the attached counts table. A day holding k rows — two shift periods, three
spatial sections — therefore divided every weight by k, and the season total
came back low by exactly that factor, with no error and no warning.
Both sides now count distinct sampling units, so the ratio is days over days however many rows a day carries. Rows sharing a day are summed into it before the expansion, which is what the Horvitz-Thompson estimator intends.
Two configurations change value, and both were wrong before: a design whose
counts carry a within-day dimension (the shipped example_sections_counts
expanded to 282 where the hand calculation gives 846), and one whose calendar
lists the frame at a finer resolution than the day, which expanded as though
the season held more days than it does. A design with one count row per
sampled day — the shape of every existing test — is unaffected.
Estimates now report when a day carries several rows, so the reader can see that the quantity being expanded is a day rather than a row.
Registered sectioned designs never reached this: add_sections() refuses
expanded targets outright.
creel_schema() gains count_time_col, naming the time a count was taken
(#129). A count row is one observation at one moment, not a day's total, and
sources routinely record several on a sampled day; the time is the only thing
that tells those rows apart. Map it whenever the source records one and pass
the fetched count_time to add_counts()'s count_time_col. Optional, and
carried through as character rather than parsed — it is a label that
distinguishes observations, not a quantity, and a source may write a clock
time in any format.
creel_schema() gains length_bin_col and length_count_col, the pair a
source needs when it reports released fish as length groups rather than
measurements (#127). A binned row is frequency-weighted — "350-400, 5 fish" is
five fish — so the count has to travel with the label. Both are optional and
absent from the required-column set: a source that measures every fish maps
neither and is unaffected. Map the label to length_bin_col rather than
length_mm_col, whose name asserts a unit the label does not carry.
creel_schema() gains value_maps, declaring what a source's codes mean for
the three columns whose meaning is a fixed vocabulary rather than a number —
trip_status, catch_type, length_type (#128). Each entry maps the
source's own codes to canonical values, c("1" = "complete", "2" = "incomplete"). tidycreel.connect applies the map at the fetch, so a coded
source reaches add_interviews() speaking the vocabulary every downstream
filter matches. Map targets are checked against the canonical vocabulary at
construction, so a typo'd target is caught where the map is written rather
than several stages later against the data.
New creel_vocabulary() returns those canonical vocabularies. Exported
because tidycreel.connect translates source codes and must check its targets
against the same list this package filters on — a second copy would be free to
drift from this one.
creel_schema() gains strata_cols, naming the stratum columns to carry
through from the source (#171). It is the one mapping here with no canonical
tidycreel name on the other side: add_counts() matches design$strata_cols
— the caller's own calendar column names — against the names of the counts
frame, so the mapping is two-sided. Names are the column the design refers to,
values the source column holding it: strata_cols = c(day_type = "DayType").
An unnamed entry, c("day_type"), means the source already uses the design's
name.
The advanced-use warning issued by as_creel_svydesign() (formerly
as_survey_design()) no longer prints unevaluated cli markup. It was raised
with rlang::warn(), which does not interpolate cli fields, so the line
reached users as Most users should use {.fn estimate_effort} instead. It now
uses cli::cli_warn(), as every other warning in the file already did. The
test covering the message could not have caught this: its assertions sat
behind an if (!is.null(result)) that was never entered once the
once-per-session warning had been consumed by an earlier test, and it closed
with expect_true(TRUE). It now resets that state and asserts
unconditionally.
The Calamus 2016 validation script now runs (#130). inst/validation/calamus-2016-validation.R
aborted at add_counts() — the fixture carries three numeric count columns
and the call named none of them — so the package's only end-to-end validation
of its own reference outputs had not executed at all. It also called
estimate_harvest_rate(), which returns HPUE (0.4226 here), where
reference-outputs.csv records the Horvitz–Thompson total that
estimate_total_harvest() produces; a comment argued explicitly for the wrong
one. Both fixed, and the script now reports 3/3 estimands within tolerance.
tests/testthat/test-validation-guard.R can now fail when that script is
broken (#130). It previously accepted any error that was not the
working-directory guard — its comment said "any other error (e.g. from
load_all or estimators) is acceptable" — so it stayed green for the entire
period the script was aborting. It now asserts the script runs to completion
and that no estimand reports FAIL.
add_lengths() now accepts a length column whose name is not literally
length (#127). length is one of this function's own arguments, so an
unqualified length() call in its body made R force that argument while
searching for a function of that name, and any other column aborted with
object 'length_mm' not found before a row was read. Every example and test
passed length = length, which resolves to base::length and hid it — while
the two names tidycreel.connect's fetch layer actually produces, length_mm
and length_bin, both failed. The documented connect-to-design handoff for
length data could not be run as written.
Bus-route and ice totals now refuse a design with no complete trips by name
(#128). estimate_total_catch(), estimate_total_harvest(),
estimate_total_release() and the bus-route rate estimators filter to
completed trips, and a Horvitz–Thompson assembly handed a zero-row frame does
not notice: it failed several calls later inside rowSums() with all arguments must have the same length, which names nothing the caller can act
on and reads like a package bug. The standard designs already aborted by name
here; these now say the same thing, name the quantity that could not be
produced, and point at use_trips = "incomplete" or "diagnostic" — never
"all", which a bus-route design does not accept, because an uncompleted trip
supports a rate but never a total. No estimate changes: every affected call
already failed, just unreadably.
print() on a creel_schema now groups n_counted_col under interviews
rather than counts (#170). Both enumeration columns live on the interviews
table — add_interviews() resolves them against the interviews frame and
get_enumeration_counts() reads them back off it — so a bus-route user
reading the printed schema was told the enumeration count belonged to a table
it is not in, while its own denominator was listed under another. Display
only; no estimate was affected.
summarize_by_zip() and summarize_by_county() gain a zip_col argument,
defaulting to "zip_code". Both previously required a hardcoded raw field
name from one agency's database, which no general-purpose package should
assume. Rename the column, or pass zip_col, to keep existing code working.
add_interviews() now warns rather than informs when n_anglers is
omitted (#126). The assumption it states is a claim about the data, not a
note about a default: with any party larger than one, .angler_effort is
party-hours while count-derived effort is angler-hours, so every rate
denominator is wrong by the mean party size with no error raised. Pass
n_anglers = 1 to declare that the interviews really are one angler each;
that silences the warning and, unlike omission, marks the effort as genuine
angler-hours.
add_counts() refuses a counts table containing rows identical in every
column (#152). svytotal() sums the rows of design$counts, so a repeated
row was counted twice: a six-day table rose from 65 to 77 angler-days and its
standard error from 5.26 to 15.61, with only a warning. Previously CNT-06
warned; it now aborts, naming the affected rows.
The check is on the whole row, not the sampling-unit key, and is deliberately
independent of unit_cols. Two rows sharing a key are ordinary structure —
two sections, two effort types, two counts within a day — and differ
somewhere. Two rows differing in no column carry nothing that could
distinguish one unit from another, so the table is malformed under every key,
including a key that is wrong (as it has twice been: #155, #162).
Tables where the repeat is a genuine second observation are unaffected, since the counts themselves differ; CNT-06 still warns about those.
derive_angler_count() now removes the columns it consumed (bank,
boat_anglers, boat_count) from its result. They are superseded by the
derived count and by expansion_basis, and leaving them in produced a table
that varied between sub-counts of one sampling unit — indistinguishable, to
add_counts(), from a structural dimension it had not been told about
(#162). The destination column is never dropped, even when it is also an
input. Code reading a raw component off the result must read it from the
input table instead.
add_counts() aborts, rather than silently taking a first value, when
within-day aggregation would collapse rows that differ in a column the
sampling-unit key does not contain (#162). The error names the column and
supplies a ready-made unit_cols call.
as_survey_design() is renamed to as_creel_svydesign() (#167). The old name
is srvyr's principal entry point, and srvyr is the natural companion for tidy
survey work, so attaching both packages masked one with the other depending on
load order. A user who loaded srvyr second and called as_survey_design(design)
got srvyr's generic failing to dispatch on creel_design, with an error that
said nothing about masking. The new name also matches the sibling
as_hybrid_svydesign() and states what the function does: it extracts the
internal survey object rather than constructing a design. as_survey_design()
keeps working and now warns; it delegates to as_creel_svydesign(), so the two
cannot diverge.The calamus-2016 reference outputs record a re-baselined catch_total
standard error, 55.7239 becoming 52.9963 (#178). The point estimate is
unchanged and always was. The file was written once at v1.7.0 and never
regenerated, so it had gone on recording a number the package stopped
producing at v3.0.0, when the dimensional seam audit routed all three
bus-route totals through br_complete_trips_only() — a filter the harvest
total had always applied and the catch total never had.
Nothing about the estimator changed here; only the record of what it produces.
The fixture's two incomplete-trip rows both carry catch_count = 0, so
dropping them cannot move a Horvitz-Thompson sum — it moves only the interview
count behind the variance, 24 to 22. That is worth stating plainly, because
the divergence was first misread as evidence against the trip filter on the
grounds that the point estimate was invariant to it: with zero-catch rows,
invariance is guaranteed by construction and says nothing about the SE.
The row was regenerated only after the responsible release was identified from
the source history and the pre-v3.0.0 value reproduced exactly by disabling
the filter on current code. inst/extdata/calamus-2016/README.md is new and
records that reasoning, together with the standing rule that these outputs are
not re-baselined to match current behaviour without it. effort_total and
harvest_total are untouched and have reproduced bit-for-bit since v1.7.0.
inst/validation/calamus-2016-validation.R now compares standard errors as
well as point estimates — six comparisons where there were three (#178).
Comparing estimates alone is the reason the stale SE above survived three
major versions: the script is the only thing that exercises the reference
outputs, and the one quantity that had moved was the one it never looked at.
Its guard test gains two related fixes. It now asserts which comparisons
ran, not merely that none failed — a script that quietly stopped checking
standard errors would otherwise still report no failure, which is the original
blind spot one level up. And it no longer wraps the script in
suppressMessages(): the script reports through message(), so suppressing
them left the captured output empty and the existing "no FAIL in output" check
passing on a zero-length vector, testing nothing.
The sampling unit is now declarable: add_counts() gains unit_cols (#162).
Until now the unit was inferred from the design alone — the PSU column plus
strata, section, and site — so a counts table carrying a dimension the design
does not model was read as repeated units. That is exactly what
prep_counts_daily_effort() produces: it emits one row per
(date, strata, effort_type), and bank and boat counts on the same day are
two units, not one day counted twice.
With count_time_col supplied, the consequence was a wrong number and no
warning. All rows for a day collapsed into one, so the effort types were
averaged rather than summed and the surviving row kept the first row's
label: a four-day example whose true total is 121 angler-days reported 60.5,
and the rows that vanished were labelled bank. With k effort types the
estimate was off by a factor of k.
Inference is kept as the default, so existing correct code is untouched, but
it can no longer fail quietly: where the unit is ambiguous the call now
aborts. This is the third appearance of one root cause — the key omitted
section (#155), then effort_type (#162) — which is why the fix stops
enumerating dimensions and lets the caller state the unit instead.
creel_schema() gains site_col and circuit_col (#126). A bus-route
interview has to name the site and circuit it was taken at, or
add_interviews() cannot join the site inclusion probability — but the schema
had no way to say which source columns hold them, so the connect layer dropped
them and the join aborted with an error that pointed nowhere near the cause.
Both default to NULL; nothing else changes.
The second major bump. It closes one defect class opened by the 2026-08-14 seam audits: a parameter estimated from data, then consumed as though it were known. Six issues (#135, #137, #138, #139, #157, #158) turned out to be one bug wearing six hats — a visibility correction, an angler-to-people ratio, a camera calibration, a harvest rate, a reporting rate, and an imputed count were each divided or multiplied into an estimate while contributing nothing to its standard error. Every one of them produced a plausible number and no warning.
Read the Breaking changes section before upgrading. Aerial and camera designs must now state their corrections or they abort, and several standard errors move upward — including some that were previously smaller than the single term they had omitted.
Aerial designs must supply visibility_correction and angler_ratio. Both
arguments previously defaulted silently to 1.0. Not supplying a correction is
not the same claim as declaring that none applies, and only one of those should
be silent. To declare that none applies, pass the string "none": the point
estimate uses 1 and the corresponding standard-error component is reported as
NA, never 0, because a zero is indistinguishable from a term that never
propagated. To assert instead that a multiplier is known exactly, supply its
standard error as 0 deliberately (visibility_correction = 1, visibility_se = 0).
est_effort_camera() without interviews must pass calibration = "none",
and then reports NA standard error. Expanding a raw camera count by
h_open alone assumes each counted object contributes exactly one angler-hour
per hour open — a calibration of 1 that was never measured. Reaching that path
now requires saying so. The calibration component becomes
present-and-unknown rather than absent, because the correction genuinely
applies and simply was not measured. COMP-05 asserted the opposite and is
inverted deliberately, with the reason recorded in the test.
Breaking (numeric): estimate_effort_aerial() standard errors move upward
wherever a boat count was expanded by derive_angler_count(). The function
never called compute_expansion_var_contribution(), so a count carrying a
party_size_se reached svytotal() with its multiplier's uncertainty
discarded — the carrier columns survived add_counts() and were simply not
read. Measured on a fixture, the dropped component was 560 against a reported
standard error of 236: the missing term was larger than the entire standard
error being reported.
Breaking (numeric): impute_camera_counts() returns a different object when
m > 1. It now yields a camera_imputations object of m completed data
sets rather than one. m = 1 is unchanged and still returns a plain data
frame.
Breaking (numeric): estimate_mr_harvest() confidence intervals are no
longer built by scaling the endpoints of the abundance interval when the
harvest rate is estimated. That identity is exact only while the rate is a
known positive constant; once it is estimated the endpoints are themselves
random. An estimated rate now falls back to a symmetric interval built from
the full product standard error.
visibility_correction gains visibility_se (#135). The correction is
estimated from paired air-ground counts and the standard field method reports
its standard error as routine output (Smucker et al. 2010, eq. 6–7);
tidycreel had no argument that could accept that number. The delta term
E * se_v / v is added once at the total, never per stratum: v is a
shared multiplier, perfectly correlated across flights, and summing it per
stratum in quadrature would treat it as independent and understate it. On the
GLMM bootstrap path v is resampled once per replicate, outside the model
refit, for the same reason — drawing it per flight would shrink its
contribution like 1/sqrt(n_flights).
Aerial designs gain angler_ratio and angler_ratio_se (#158). Smucker et
al. (2010) apply two corrections to a raw observer count — a visibility
correction and an angler-to-people ratio of 0.404 — and tidycreel implemented
only the first, overstating shore effort by roughly 2.5×. The two push in
opposite directions (0.404 down, 2.69 up), so applying only the visibility
correction is not conservative: it is biased in the direction of the
correction that was kept.
estimate_mr_harvest() gains harvest_rate_se (#138). It computed
se_H <- harvest_rate * se_N, which is product_total_variance() with
r_se = 0 — the package already implemented Goodman (1960) and made it the
default in all three creel-estimates-total-*.R files; this function simply
never called it. Rasmussen et al. (1998) draw the distinction in the package's
own cited literature: the subtractive form is for terms "estimated from a
sample", and differs from the population formula "used when the terms in the
product are known, not estimated". The bootstrap path now draws the rate once
per replicate rather than holding it fixed.
estimate_exploitation_rate() gains reporting_rate_se and the third delta
term (u/lambda)^2 var(lambda) (#139), since d(u)/d(lambda) = -u/lambda. It
enters once at the total: lambda is a single estimate dividing every
stratum, so adding it per stratum and summing in quadrature would treat a
shared divisor as independent. On the stratified path it is applied to the
aggregate, deliberately not inside var_u_h.
Its reporting_rate = 1.0 default is kept, unlike the aerial corrections.
It is a visible, documented default on an exported argument the caller opts
into adjusting, not a value substituted invisibly inside an estimator. That
asymmetry is recorded in the @param text rather than left to be
rediscovered.
Multiple imputation for camera outages (#137). impute_camera_counts() filled
every outage row with the model's fitted mean and returned one completed data
set. Inside svytotal() those predictions are indistinguishable from
observations, so the imputation model's own error was dropped; and fitted means
are smoother than real counts, so the between-day component shrank as well. The
reported standard error was biased downward twice over. Each of the m
completed data sets is now drawn from the model's predictive distribution —
coefficients drawn from their sampling distribution, then counts drawn from the
fitted family. Both draws are needed: drawing only the count treats the
coefficients as known, and drawing only the coefficients still yields a smooth
mean where a real count has sampling noise.
est_effort_camera_mi() estimates once per completed data set and pools by
Afrifa-Yamoah et al. (2020) eq. (5): the within-imputation mean variance plus
the (M+1)/(M(M-1)) between-imputation term — the quantity a single completed
data set structurally cannot have. Their factor is Rubin's (1 + 1/M)
inflation written over the raw sum of squares; MI-04 pins that the two forms
agree. M = 5 follows the paper's stated bias-variance balance. Components are
reported as within_imputation / between_imputation so a reader can see how
much of the uncertainty came from imputing.
validate_shared_multiplier() gives the four shared-multiplier arguments one
validation shape — required, "none" opt-out yielding NA rather than 0,
all-or-none standard error. The rule is documented once there rather than
restated at each call site.
visibility_correction is named a detection probability and documented as
the reciprocal of the published ground-truthing ratio (#157). Field studies
report r = ground/aerial, which exceeds 1 exactly when the correction matters
(r = 2.69 for shore anglers); tidycreel wants v = 1/r = 0.372. The > 1
abort branch now names the conversion, since that is where a reader of the
source paper lands.
The aerial GLMM's variance composition is documented as tidycreel's own
reasoning and is not attributed to Askey et al. (2018). That paper, this
estimator's cited source, was read in full while specifying this work and
contains no visibility correction and no bootstrap — it does not speak to v
at all, and propagates uncertainty by cross-validation rather than
analytically. Its nAGQ = 0 is likewise not carried over: the paper warns the
option is less accurate and used it only because their data set exceeded
250,000 observations.
Breaking (numeric): the per-section totals from estimate_total_catch(),
estimate_total_harvest(), and estimate_total_release() no longer aggregate
to the .lake_total row as though the sections were independent when one
party-size estimate spans them (#145). This is #144 on a second partition: the
sections path builds its frame by hand instead of routing through the shared
stratum helper, so the strata correction never reached it. A multiplier
estimated once and applied across sections is a single random quantity common
to all of them, so its contributions add before squaring. The lake-row
standard error moves upward on affected designs; the per-section rows and
every point estimate are unchanged.
The structure is now classified against the section partition rather than
the strata, because sections may cross-cut strata — a group can be nested
within strata while spanning sections. One consequence is visible: a
party-size estimate keyed by a stratum (for example one per day_type) is
nested within strata but straddles sections unevenly, so the lake row now
reports se = NA with a warning rather than a number that quietly assumed
one geometry or the other. As elsewhere in the package, an unknown standard
error is NA, never a zero and never a plausible substitute.
A sections total now reports the party-size component its standard error
carries, per row, instead of NULL (#145, completing #134). NULL means the
component was never propagated, and the sections constructor was saying that
while its se demonstrably contained the term.
Breaking (numeric): add_counts() now keys the sampling unit on the PSU
crossed with the section and site, not on the PSU column alone (#155). Four
places needed to know what "the same unit" means — duplicate detection,
within-day aggregation, the supplied within-day-variance key, and the
party-size constancy check — and they had drifted into three different
answers, none of which carried the section. They now share one
psu_key_cols() definition. Period enters through the strata, which is where
this package models it (strata = c(day_type, day_period)).
A day sampled in two sections was treated as one unit, with three consequences:
58, still labelled
with the first section's name, with the other section's rows absorbed into
it. This moved the point estimate — the daily total came out 58 where
the truth was 116 — and nothing downstream could detect it, because the
result looked like a clean frame with one section missing.ss_d was
dominated by the difference between sections rather than the spread
within a day: 8888 where the true within-section sums of squares were 50
and 2.expansion_se varies within a single PSU and blaming two derive_angler_count() calls,
on a single coherent call. Under sections the unit is the day within a
section, and each such unit carries exactly one estimate.The CNT-06 warning also stops firing on ordinary multi-section days and now names the key it judged the repeat on. A genuine repeat — the same unit counted twice with no count time — still warns, and two different party-size estimates inside one unit still abort.
Affects designs with sections or sites. Bus-route designs are untouched: they
hold counts in design$bus_route$data, which never reaches these checks.
The "partial" party-size geometry now returns a standard error instead of
NA (#150). When one party-size estimate spans some parts of the partition
being summed over and another sits inside one, the combination needs the
group-by-part decomposition — and compute_expansion_var_contribution() was
squaring and summing the group index away before returning, so
add_expansion_covariance() had nothing to combine and correctly refused.
The decomposition is now carried alongside the scalar component, and the
exact combination is Var = Σ_g (Σ_p rate_p × basis_{g,p} × se_g)²:
contributions from one estimate add before squaring because its error is
common to every part it covers, while contributions from different groups
come from disjoint interview subsets and add as variances.
The "nested" and "shared" numbers do not move. Both are special cases
of that formula, but each keeps its own arithmetic rather than being
re-derived through it, so their results are unchanged bit-for-bit. Only the
case that previously returned NA produces a new number, and it lands
strictly between the two shortcuts the old code refused to choose between —
quadrature understates it, the linear sum overstates it.
This matters most on the sections path introduced in #145, where "partial"
is ordinary rather than exotic: sections cross-cut strata, so a party-size
estimate keyed by day_type straddles sections unevenly and forced the
.lake_total row to NA. The refusal is retained for the case where no
decomposition was carried, since a combination that cannot be computed still
must not be guessed.
prep_counts_boat_party() gains mean_party_size_se, and emits the
expansion_* carrier columns when it is supplied (#143). This function
performs the same boat-to-angler expansion as derive_angler_count(), but
wrote no carriers and had no argument through which a party-size standard
error could be given — so on this path the component was not merely omitted
by default, it was unreachable, and no user action could recover it. Because
this is the pipeline the documentation calls preferred, the two documented
routes to one expansion were not statistically equivalent and nothing said
so: the prep path reported the pre-3.2.0 understated standard error with
se_expansion = NULL as the only signal.
The emitted basis is boat_count * correction_factor, not the bare boat
count, because this function applies the correction to the product — a bare
basis would be the derivative of a quantity it never produces and would trip
the #131 desync guard. expansion_of is "daily_effort" for the same
reason. Omitting the argument still leaves the component absent rather than
zero.
The creel_error_expansion_basis_desync message now states that correct
hand-rescaling reaches it too (#148). expansion_of records a column name
rather than a scale factor, so a basis correctly rescaled alongside its count
is indistinguishable from one left behind, and both are refused. Refusing
both remains the conservative and correct choice, but the message described
only the mistake — and every instructional example in the companion book met
it with arithmetic that was right. The wording changed; the check did not.
Relatedly, the four carrier columns are now documented as package-written and
not user inputs. Overwriting expansion_of to name a transformed column
silences the guard whether or not the basis was actually rescaled, which
re-enables the defect the guard exists to catch. Use period_length_col,
which scales count and basis together and can be verified, rather than
asserting the rescale.
Breaking (error): est_effort_camera()'s ratio-calibration path now
refuses a counts table that holds more than one row for the same day, rather
than silently double-counting it (#142). The calibration pairs interview days
to count rows by date membership and reads the day's effort total once per
matching row, so a repeated date entered rho = sum(E_d) / sum(C_d) twice on
both sides, and the survey total of raw counts counted it again. This moved
the point estimate, not only the standard error — 16 to 19.5 on the
package's own five-day test fixture, a 22% shift produced by a duplicated row
carrying no new information. add_counts() only warns about repeated PSU rows
(CNT-06), so such a table reached the estimator intact.
The table is refused rather than averaged because two counts on one day are
either sub-period snapshots or a data error, and nothing on this path can tell
which. Callers with genuine sub-daily counts should pass count_time_col to
add_counts(), which already collapses them to one row per day; the error
names the offending dates and says so. The raw-count path (h_open, no
interviews) is deliberately unchanged: expanding a duplicated PSU row through
svytotal() has the same shape in every design, and that is a wider question
than this fix.
Three cases where a quantity that was unknown, malformed, or modelled reached an estimator as though it were observed. All three were found by the statistical seam audits of 2026-08-14; none produced an error, a warning, or an implausible number.
Breaking (numeric): estimate_total_catch(), estimate_total_harvest(),
and estimate_total_release() no longer combine a shared party-size estimate
across strata as though the strata were independent (#144). The stratified
total variance adds per-stratum variances because strata are sampled
independently (Pollock, Jones & Brown eq. 3.12–3.13); a multiplier estimated
once and applied to every stratum is not stratum-independent error, and the
covariance the sum omitted is 2 Σ_{h<k} R_h R_k s_h s_k. Standard errors
were understated by up to sqrt(H) on the expansion term for H strata. This
is the default configuration, since mean_party_size() without by returns
one estimate. Reported standard errors move upward on affected designs;
point estimates are unchanged, and designs whose party-size estimate is
per-stratum are unchanged bit-for-bit. Where expansion groups straddle strata
unevenly the combination is not recoverable from per-stratum components, so
the standard error is NA with a creel_warning_expansion_structure_unknown
warning rather than a silently chosen formula. The correction reaches the
ungrouped, grouped, and per-species totals; the per-section path
aggregates its lake row separately and is still affected — see #145.
The three totals now report the party-size component they carry, as
se_expansion (#134). They routed through the effort estimators, whose
standard error includes the term, but passed no se_expansion to their
constructors — so a totals object whose se demonstrably contained the
component reported NULL, the value documented to mean "never propagated".
Anyone applying that test to a total drew the opposite conclusion from the
truth. The reported number is now produced by the same code that folds the
term into the variance, so the two cannot drift apart. The per-section
constructor is not covered; it builds its result frame by hand and still
reports NULL (#145).
print() on a creel_design now shows the count column and whether the
party-size term is carried (#124). Counts whose expansion carriers were
dropped by an ordinary select() are indistinguishable from counts that never
had them, so the design print is the last point at which the loss can be
surfaced while the user can still act on it. Both lines print whenever counts
are attached. This also closes the older note that the design never showed
which column it used as the count.
tidy() is documented as lossy for uncertainty components, with the reason:
a tibble column cannot hold the NULL-versus-NA distinction the component
contract depends on. se_between and se_within are likewise documented as
not reconstructing se on expansion designs.
derive_angler_count() now writes a fourth carrier column, expansion_of,
naming the column the expansion basis is the derivative of, and add_counts()
aborts when the count column is not that column (#131). expansion_basis is
d(count)/d(party_size), so a count transformed between the two calls — the
documented mutate(angler_hours = angler_count * shift_hours) pattern, for
one — scales the count and leaves the basis in the old units. The party-size
variance component then came out understated by exactly the scale factor while
remaining present and non-NULL, so it read as propagated: on a six-day
design with a ×12 shift length, se_expansion was 3 where the same physics
expressed through period_length_col gives 36. Point estimates were
unaffected. Supply the untransformed count and period_length_col, which
scales the count and the basis together. Breaking: pipelines that
premultiplied the count while retaining the carriers now abort.
mean_party_size() now names its "se" attribute by the group key, and
derive_angler_count() addresses it by name (#133). The attribute was matched
by row order while the means were joined by key, so any length-preserving
reordering of the lookup — an arrange(), most habitually — gave every
stratum another stratum's standard error, silently and with the point
estimates unchanged. On a two-stratum design the weekday and weekend standard
errors swapped outright. A by-form lookup whose "se" attribute has no
names is now refused rather than matched positionally; single-row lookups and
the scalar form are unaffected, having no order to go stale. The
expansion_group attribute was checked for the same hazard and does not have
it: it is built from the counts rows, never indexed into the lookup.
add_counts() now aborts when counts carries some but not all of the
expansion_* carrier columns (#132). They are written together by
derive_angler_count(), so a proper subset can only come from partial
deletion. The gate previously required the full set and otherwise took the
no-carriers path, which left an expansion_se sitting visibly in the table
while the party-size variance component silently went missing. Point estimates
were unaffected; se_expansion came back NULL. Dropping all of them is
still undetectable at this seam — see #124.
Camera ratio calibration reports NA rather than an exact ratio when a
stratum has a single paired interview/count day (#136). The calibration ratio
has no measurable spread from one pair, so its variance is unknown, not zero;
the delta term T² × var(ρ) previously vanished and the maximally uncertain
calibration was reported with the same standard error as a perfectly known
one. The NA propagates into the combined standard error and the confidence
interval, and a warning names the stratum. Strata with two or more paired days
are unchanged.
The single-day test counts distinct paired dates rather than matched count
rows, so a counts table holding two rows for one date — which add_counts()
only warns about — cannot present one day's information as two and restore
the false-precision path. The variance denominator is unchanged, so no
existing standard error moves. That such a table also shifts the point
estimate is a separate and older defect, filed as #142.
Camera effort estimation now warns when the counts carry .imputed rows
(#137), naming how many days contain imputed counts and what share of the
total they are. impute_camera_counts() flags rows it filled with model
predictions, but nothing downstream read the flag: inside svytotal()
predictions are indistinguishable from observations, so the imputation model's
prediction uncertainty is dropped and the between-day variance is further
understated because predictions are smoother than real counts. The reported
standard error is a lower bound. Propagating the prediction variance is still
open under #137.
.imputed now survives within-day aggregation by collapsing with any(),
alongside the existing mean-collapse for the count and expansion_basis. A
day is imputed if any of its sub-counts was; taking the first sub-count's
value, as every other column does, let a day whose first count was observed
report itself as fully observed, and the warning above never fired for
designs using count_time_col.
creel_estimates objects now carry se_components, a named list of the
standard-error contributions that make up se, and print() reports each one
with its relationship to se (#141). The contract is the one the party-size
component has followed since 3.2.0, generalised: an absent name means the
component does not apply to that path or was never propagated, NA means it
applies and is unknown, a finite value is a contribution and never se
itself, and none of them is ever 0 — a zero cannot be told apart from a
component that never propagated. se_expansion is unchanged and still
supported; the constructor now mirrors it into se_components[["party_size"]]
so a reported component and the standard error containing it cannot drift
apart, which is the defect #134 was.
Camera effort estimation reports its two delta-method terms separately as the
count_sampling and calibration components (#141). Since 3.3.0 a stratum
with one paired interview/count day gives its calibration ratio an unknown
variance, which correctly makes the whole standard error NA (#136) —
Var(E) = Σ_h [ρ_h² Var(T_h) + T_h² Var(ρ_h)] is unknown if any Var(ρ_k)
is, and reporting the measurable part as the standard error would publish a
lower bound under the name of the real thing. That NA stays. What changes is
that the count-sampling half is now reported as a finite component alongside
it, so one thin stratum no longer hides everything that is known. The
reported components reconstruct se exactly, and the raw-count path omits
calibration entirely rather than reporting it as NA, because that path has
no calibration ratio at all. No standard error changes value.
tidy() remains lossy for these components, for the reason already
documented: a tibble column collapses an absent component and an unknown one
into the same NA.
The sampling error of an estimated party size now reaches the effort standard error (#121). A mean party size taken from interviews multiplies the boat component of every count, so its error is one error applied many times rather than fresh noise per count: it does not shrink as counts accumulate. Treating it as known made every count-expanded effort standard error too small.
mean_party_size() now returns that standard error as a "se" attribute, and
derive_angler_count() reads it, so the usual pipeline propagates the term
with no extra argument:
counts |> derive_angler_count(
bank = bank_anglers,
boat_count = angler_boats,
party_size = mean_party_size(interviews, n_anglers, angler_type = angler_type)
)
Supply party_size_se directly to override it, in any of the three shapes
party_size accepts (scalar, column, lookup).
The component is reported as se_expansion on the returned estimates object
and is included in se, so it reaches catch, harvest, and release totals as
well. The estimates tibble keeps its existing seven columns.
est_biomass() can now propagate the length-weight regression error (#117).
a and b are point estimates from a regression, and a * L^b multiplies
every length bin, so their error is perfectly correlated across bins and does
not shrink as bins are added.
Supply alpha_se, b_se, and L0 together — all three or none:
est_biomass(ld, a = 0.0088, b = 3.1, alpha_se = 0.05, b_se = 0.03, L0 = 250)
The allometry is rewritten about a pivot length L0 as
W = alpha * (L / L0)^b, and the delta method applied in (alpha, b). The
parameter covariance is then absent by construction rather than by
assumption: on the raw (a, b) scale the two are typically correlated below
-0.99, so dropping their covariance there would overstate the variance
severalfold. L0 should be the geometric mean length of the calibration
sample, and alpha_se the intercept SE from a regression centred there — not
the standard error of a.
Reported as attr(x, "biomass_se_params") and included in biomass_se.
Absent — NULL, not 0 — when the arguments are not supplied.
mean_party_size() output, because a variance component that was previously
dropped is now carried. Estimates themselves are unchanged; only their
uncertainty moves. Designs that pass a bare number or a column as party_size
are unaffected, since no standard error is available for those.When no party-size standard error is available the component is omitted, not
set to zero. se_expansion is NULL rather than 0, because a zero would
produce a standard error identical to an unpropagated one while appearing to
have been propagated. A party size estimated from a single interviewed party
yields NA, which propagates to an NA standard error rather than being read
as certainty.
derive_angler_count() writes three further columns — expansion_basis,
expansion_se, and expansion_group — when a standard error is available.
add_counts() recognises all three and excludes them from count-column
detection, so they cannot make an otherwise unambiguous counts table look
ambiguous.
derive_angler_count() builds the single angler-count column add_counts()
needs from the columns a clerk actually records. Two forms, matching the two
ways boat anglers reach the form:
# Anglers aboard were counted directly
counts |> derive_angler_count(bank = bank_anglers, boat_anglers = boat_anglers)
# Boats were counted; anglers aboard were not
counts |> derive_angler_count(
bank = bank_anglers,
boat_count = angler_boats,
party_size = mean_party_size(interviews, n_anglers, angler_type = angler_type)
)
party_size accepts a single number, a column of counts, or a lookup table
keyed by stratum, so a party size that differs between weekdays and weekends
can be applied per group rather than averaged away.
boat_count and boat_anglers are separate arguments deliberately.
boat_count counts hulls, and adding it to an angler total is a units
error that produces a plausible-looking number; requiring party_size
alongside it makes that impossible to do by accident. Supplying both boat
forms is an error, since they are two routes to the same quantity.
Components are added with na.rm = FALSE: a count that was not taken and a
count of zero anglers are different observations and stay different.
Until now this derivation was available only on the sampled-day
prep_counts_* seam, via prep_counts_boat_party(). The raw-count pipeline —
the one that takes a within-day count schedule through count_time_col and
derives the within-day variance component itself — had no equivalent, so
callers there built the total by hand. Closes #119.
mean_party_size() returns the mean anglers per boat party from an interviews
table, optionally by stratum. It filters to boat parties, and errors rather
than returning NaN when no row matches — a silent NaN would propagate into
every expanded count.
Bus-route estimators no longer report a confidence bound below zero. Every
ci_lower produced by estimate_effort(), estimate_total_catch(),
estimate_total_harvest(), estimate_total_release() and
estimate_harvest_rate() on a bus-route design is now clamped at zero, in
both the ungrouped and by-grouped paths and in the bootstrap columns
(ci_lo_boot). Bus-route was the last family of estimators in the package
without this clamp; the product totals, exploitation rate and length
compliance already had it.
This changes reported numbers only where the old bound was outside the
parameter space. Angler-hours, fish and fish-per-hour cannot be negative, so a
symmetric Wald bound below zero was never a possible value for the quantity.
It is reached whenever the coefficient of variation exceeds roughly 0.51 —
routine for a bus-route survey with few sites, unequal inclusion
probabilities, or catch concentrated in one interview. The package's own
bootstrap snapshot fixture was already in that regime: estimate_total_harvest()
reported an estimate of 115 with an SE of 78.3 and a lower bound of
-38.8, which is now 0. On a deliberately skewed two-site design with
p_site of 0.05 and 0.95 the excursion is larger, with the total-catch bound
moving from -15999.40 to 0 and the harvest-rate bound from -54.85 to 0.
A clamped bound of exactly zero means the interval is wide relative to the
estimate. It is not a statement that the quantity could be zero, and the
clamp does not narrow the interval or change the estimate or the standard
error. See ?creel_confidence_intervals. Closes part of #95.
New topic ?creel_confidence_intervals states the two conventions the package
follows when building intervals: transform where a principled transform for
the quantity exists (logit for exploitation rate, Sadinle's transformed logit
for mark-recapture abundance, optional log for product totals) and clamp at the
feasible limit otherwise; and use a t-quantile where an estimator has a design
degrees-of-freedom to appeal to, a normal quantile where it does not. Written
down so a new estimator does not have to pick by coin flip. Closes #95 and #99.
est_biomass(), est_mean_length(), est_compliance() and est_mean_age()
now record why they use a normal rather than a t quantile. Their standard error
is propagated from the per-bin standard errors of a length or age distribution,
so there is no local sample size to key degrees of freedom to: the row count is
the number of bins, which is the caller's binning choice, and the row totals are
expanded estimates rather than counts of measured fish. Keying a t-quantile to
either would make the interval narrow as bins got finer, with no additional
fish measured. Closes #99.
est_biomass() now states that the length-weight parameters a and b are
treated as known constants, so biomass_se omits their estimation error and
should be read as a lower bound. Because a * L^b multiplies every bin,
that error is perfectly correlated across bins and does not shrink as bins are
added. Measured on the documented example it adds roughly 2–11% to a
coefficient of variation of 40–65% — minor there, but material for a survey
precise enough to reach a count CV near 10%, or when a/b are borrowed from
a system whose fish differ in size. Propagating the term needs an API that can
accept the regression's standard errors and their covariance; tracked in #117.
Argument guards on truncate_at and conf_level now reject a value whose
length is not 1, rather than letting it reach the comparison. Passing
truncate_at = c(0.5, 1) to estimate_catch_rate() raised base R's
'length = 2' in coercion to 'logical(1)', and numeric(0) raised
missing value where TRUE/FALSE needed — both of which name neither the
argument nor the constraint it violated. The intended error, which cites the
argument and its default, now fires instead. Affects estimate_catch_rate(),
the bus-route incomplete-trip path, and est_effort_camera().
A conf_level or truncate_at of NA_real_ still reaches base R's
"missing value where TRUE/FALSE needed". That gap predates this change and is
shared by the six other guards written to the same pattern; it is left for a
single pass over all of them rather than fixed at three sites only.
The first major bump since the package adopted semantic versioning. It closes the dimensional seam audit opened 2026-08-07: 27 findings, ten of them breaking changes to what an estimator returns. Estimates now carry the unit of the quantity they report, derived from the arithmetic the package performed rather than declared by the caller.
Read the Breaking changes section before upgrading — bus-route, ice, instantaneous, aerial and camera designs all report different numbers than 2.5.0 did, because 2.5.0's numbers were wrong.
Estimates now carry the unit of the quantity they report. creel_estimates
objects gain a unit field, print() shows a Unit: line, autoplot() puts
it on the y-axis, and write_estimates() records it in the CSV header. This
replaces hardcoded axis and header strings, which could not tell that the
number underneath them had changed dimension.
The unit is derived, never declared. A unit the caller types is exactly as
trustworthy as the axis label on the poster — a second place to write the wrong
thing — so tidycreel asserts one only where it performed the arithmetic that
produces it: angler-hours on the count side when add_counts() multiplied by
T_d, angler-hours on the interview side when add_interviews() multiplied trip
hours by a supplied party size, and party-hours when it did not.
Everywhere else the unit is NA, meaning unknown — deliberately not
"angler-days". A bare numeric count column may be an instantaneous head count
or effort the caller already expanded, and example_counts is the latter;
guessing between them would put a confident label on a number that may be in
either unit, which is the failure this machinery exists to prevent. An absent
Unit: line is the claim that tidycreel does not know, which is a different
statement from a default.
est_effort_camera() gains n_anglers, which makes the ratio-calibration
path's unit derivable instead of unknown. The calibration ratio is a ratio of
sums, so the camera counts cancel and the estimate inherits whatever unit the
interview effort column holds — angler-hours and party-hours were
indistinguishable, a factor of roughly two apart on the shipped example and
reported identically. Passing n_anglers, either a column in interviews or a
constant party size, makes the function perform the normalisation itself, which
is what earns the angler-hours label.
Omitting it now warns and names the ambiguity. That warning is only worth raising because the argument exists to answer it: before, it would have reported a gap the caller had no means to close.
The party-size rule is not reimplemented. This path calls the same exported
compute_angler_effort() that add_interviews() uses, so a party size of zero
is refused at both seams for the same reason, and they cannot drift apart.
n_anglers here takes a column name or a constant rather than a tidyselect
symbol, matching its neighbouring effort_col and intercept_col arguments.
Unit propagation now reaches the rate and total estimators outside the
standard CPUE spine. Species, sections, grouped, bus-route and regression
rates carry fish/<denominator>; species, sections and bus-route totals
carry fish. These paths reach different constructors than the ungrouped
ones, which is why they were still reporting NA after the first pass.
NA is not a neutral default: it reads as "tidycreel does not know what this
number is", and it suppresses the unit from print(), autoplot() and the
CSV header, so the number travels bare. Saying NA where the package does
know is as much a false claim as guessing.
The denominator is a property of the interviews rather than of which rate was asked for, so every rate estimator on one design now reports the same one — asserted between estimators in the tests rather than against a hardcoded string, since a wrong constant can satisfy a literal but cannot make two independent estimators agree.
Visible change: autoplot() y-axis labels on these paths now read e.g.
"Total Catch (fish)" where they previously read "Total Catch".
Unit propagation now covers the effort family, where the same quantity is derived three different ways and so takes its unit from three different places.
Bus-route effort reports the interview denominator, not the count side:
E_hat = sum(e_i / pi_i) is built entirely from interview contributions, so
labelling it from the counts would assert a provenance the number does not
have. Aerial effort is angler-hours unconditionally — an aerial design refuses
period_length_col, which makes h_open the sole period source.
Camera effort splits by path. The raw-count path is angler-hours for the same
reason as aerial. The ratio-calibration path is NA: its ratio carries the
unit of the effort_col column in a caller-supplied data frame, which nothing
normalises by party size, so angler-hours and party-hours are
indistinguishable there. Unknown is the honest answer, and the same one
add_counts() gives a bare count column.
estimate_angler_trips() and estimate_effort_per_acre() now carry units,
and both inherit rather than assert them. These two take a
creel_estimates rather than a design, so they cannot ask a design what
anything is in; each transforms a quantity whose unit it was handed.
Trips are effort divided by mean trip length, and the divisor is hours per
trip, so the count comes back in whichever actor the effort was measured in:
angler-hours give angler-trips, party-hours give party-trips. The method
name is "angler-trips" for every caller, which is precisely why the unit
cannot be read off it — a bus-route design with no n_anglers produces a
party-level count that a fixed label would have reported as angler trips.
Effort per acre composes its unit from the effort's, keeping
party-hours/acre distinguishable from angler-hours/acre. An unknown
effort unit stays unknown through both: dividing an unknown quantity does not
make it known, and "NA/acre" would read as a real unit on a plot axis.
Unit propagation now reaches the mark-recapture and exploitation-rate
estimators, the last group without units, and the honest answer for most of
them is NA.
estimate_exploitation_rate() reports "proportion" on both the stratified
and unstratified paths. It is the one estimator in the package whose unit no
input can change: \eqn{u = (C/T) \times (m/n)} divides fish by fish twice, so
both actors cancel for every design.
estimate_angler_n() reports NA, not "anglers". Its M, n and m
arrive as bare numerics that nothing inspects, and the arithmetic divides
counts by counts, so \eqn{\hat{N}} carries whatever actor the marking protocol
marked — anglers on some surveys, boats or parties on others. Asserting
"anglers" would restate the function's name rather than derive anything.
estimate_mr_harvest() inherits that unknown for the same reason: its product
is in fish only if \eqn{\hat{N}} counted anglers.
estimate_total_catch(), estimate_total_harvest() and
estimate_total_release() abort with class creel_error_unit_mismatch when
the effort unit and the rate's denominator are both known and disagree. Their
product is not a catch.
Two seams are deliberately excluded. A per-party-hour rate meeting angler-hour
effort keeps warn_party_hours_product()'s existing warning rather than
becoming an error, since that would break every caller who omits n_anglers.
An unknown effort unit is reported by the T_d warning below rather than a
second message, so one defect produces one diagnosis.
day_length() computes hours between sunrise and sunset for a latitude and
date using the CBM model of Forsythe et al. (1995). Closed form — no lookup
table, no network access, no location database. Only latitude is needed:
longitude and time zone shift when sunrise and sunset occur, not the interval
between them. horizon selects the depression angle, by name ("sunset",
"civil", "nautical", "astronomical") or in degrees. Days inside the
polar circles saturate at 0 or 24 hours rather than returning NaN.
Day length is astronomical and is not the same quantity as the estimators'
\eqn{T_d}, which is the period the counts were randomised within — a property
of the survey design, set by regulation, access hours, or field protocol. Use
day_length() for simulation and planning; pass the period your protocol
actually used to add_counts().
simulate_creel_data() gains lat and daylight_hours, either of which adds
daylight_hours and angler_hours columns to the simulated counts table.
lat derives the daily period per date via day_length(); daylight_hours
sets it directly, as a scalar or a named monthly vector, for surveys whose
fishing day is fixed by regulation. Supplying both is an error.
Supplying neither leaves both columns off, so the default output is unchanged. There is no honest default latitude, and substituting one would put a plausible number where the caller gave none.
add_lengths() warns when a binned release row carries a fractional count.
The guard's own error message had always said "a positive integer count" while
nothing checked integrality, so count = 3.5 was accepted silently and reached
estimate_length_distribution(), which aggregates that column as a per-bin fish
count. A fraction of a fish then entered the distribution and every proportion
computed from it.
Warned rather than rejected, matching how n_anglers treats the same category
error: a fractional count of discrete things signals the wrong column was
supplied, not that the data are unusable, and aborting would break tables that
have always been accepted. The NA message now says "a positive count;
non-integer values warn", so what it claims and what it enforces agree.
estimate_effort() warns, once per session, when an instantaneous design
carries no period_length_col. Without T_d the estimator expands the count
column to the season and returns it, which is not angler-hours. The warning
states the reading rather than asserting the unit: tidycreel cannot tell an
instantaneous head count from a column that already holds angler-hours, since
both arrive as a numeric column, so it says that if the column is a count the
result is in angler-days. Numbers are unchanged for these callers.
The three product totals raise the same warning. They call
estimate_effort_total() directly rather than estimate_effort(), so without
this a caller who only ever asks for a total never heard that the count column
had no T_d applied.
Output from the prep_counts_*() helpers is exempt. That seam resolves counts
into sampled-day effort before add_counts() sees them, so there is no
instantaneous count left to expand and no T_d to ask for — warning there would
fire on the documented preferred workflow. The marker is carried as an
attribute, so a table piped through intervening dplyr verbs degrades to
"unknown", which is the safe direction.
estimate_total_catch(), estimate_total_harvest() and
estimate_total_release() now accept by = species on bus-route and ice
designs, and answer on the Horvitz–Thompson path. All three resolved by
against the interview columns, which carry no species column, so the call
aborted with Column `species` doesn't exist on both design types — six
combinations, none of them reachable. The species-level total estimators they
would otherwise have reached are stratum product sums built on the standard
interview survey, so routing there instead would have reproduced the previous
entry's defect in the totals: a species total contradicting the all-species
total on the same object.
The falsifier is the same partition identity, and it is exact for a
Horvitz–Thompson sum because that sum is linear in its numerator. All six
combinations now satisfy it, and each species' total over the HT effort equals
that species' rate to machine precision — the cross-check tying the totals to
the rates. The reported method names the estimator and the quantity
(ht-total-release-species).
use_trips = "all" is still rejected: the completed-trip guard runs ahead of
the species branch, because an incomplete trip contributes catch-so-far under a
completed trip's inclusion probability whether or not the numerator is one
species.
Species-level rates (by = species) now take the Horvitz–Thompson path on
bus-route and ice designs. estimate_cpue_species() and its harvest and
release siblings build a per-species interview table and hand it to the
standard interview-survey estimators, so on these two design types they
ignored .pi_i and .expansion — the defect the previous entry removed from
the all-species rates, one estimator over. Fixing the all-species side first is
what made it visible: one design object then returned both answers, each under
a method string naming the same quantity.
The falsifier is a partition identity rather than a reference value. Species partition the catch and every species shares the same effort denominator, so the species rates must sum to the all-species rate exactly. Before the fix the species sum matched the standard-path rate to the last digit:
| design | rate | all-species | species sum | gap | | ------ | ---- | ----------- | ----------- | --- | | bus-route | CPUE | 0.748339 | 0.937805 | +25.32% | | bus-route | RPUE | 0.421378 | 0.494953 | +17.46% | | ice | HPUE | 0.919685 | 0.862944 | −6.17% | | ice | RPUE | 0.909720 | 0.964467 | +6.02% | | ice | CPUE | 1.829405 | 1.827411 | −0.11% |
All five now reconcile exactly. Per species the estimator repoints the
numerator at that species' counts and delegates to the bus-route estimator the
all-species rates already use, so the two can no longer drift apart, and the
reported method gains the -species suffix on both trip paths
(ratio-of-means-rpue-species where the standard path still reports
ratio-of-means-rpue). use_trips = "diagnostic" is refused with species
grouping: the diagnostic pair returns two estimates per species, and returning
either half under one label is the mislabelling this release is removing.
Also fixes a regression introduced by the previous entry: that dispatch
resolved by against the interview columns, where there is no species column,
so by = species aborted on ice designs where it had previously worked, and on
bus-route designs where it had never worked.
Breaking: every species-level rate on a bus-route or ice design moves.
The three rate estimators now dispatch to the Horvitz–Thompson path on ice
designs as well as bus-route ones, and estimate_catch_rate() gains the
bus-route dispatch it never had. estimate_effort() and all three totals
already treated ice as the degenerate bus route it is documented to be; the
rate estimators were the outliers, so a single design object returned a rate
that its own totals contradict. Both paths reported the same method string,
so nothing in the returned object distinguished them.
A ratio of HT totals must equal total ÷ effort exactly, which is what says which of the two answers was wrong rather than merely that they differed:
| design | rate | before | totals imply | after | | ------ | ---- | ------ | ------------ | ----- | | ice | HPUE | 0.514328 | 0.478561 | 0.478561 | | ice | CPUE | — | — | reconciles exactly | | bus-route | CPUE | 0.466438 | 0.433603 | 0.433603 |
Ice designs consequently take the bus-route use_trips set — "complete",
"incomplete", "diagnostic" — instead of the standard path's, so they now
accept the two values their own design type is built on and reject "all",
which is not an estimator on this path. For estimate_catch_rate() the roving
auto-route to "all" + MOR does not apply on these designs.
Breaking: ice HPUE, ice RPUE, ice CPUE and bus-route CPUE all move.
estimate_harvest_rate() and estimate_release_rate() now validate
use_trips on the bus-route path. The bus-route dispatch runs before the
standard path's check and handed the string straight to the estimator, which
branches on "diagnostic", then "complete", then "incomplete" with no
final else — so an unrecognised value reached the complete-trip code with
the trip-status filter switched off and returned the all-trips answer under
the complete-trip method string, silently. The dangerous input was not a
nonsense string but a valid value typed with the wrong case: on a fixture of
four complete and four incomplete trips, "Complete" returned 2.816514 over
all eight rows where "complete" returns 2.642202 over four. The standard
path rejected the same input, so whether a typo aborted depended on the design
type.
The valid set on the bus-route rate path is "complete", "incomplete" or
"diagnostic", as documented. It is deliberately not the standard path's set:
"incomplete" is a legitimate rate here (Hoenig et al. 1997) and is not
offered there, and "all" is legitimate there and is not an estimator here,
because pooling the two kinds of trip applies the complete-trip ratio of
Horvitz–Thompson totals to numerators that are catch so far. Matching is
exact — "comp" is an error, not "complete".
The product totals now warn when the rate and the effort they multiply are in
different units. Without n_anglers, add_interviews() leaves .angler_effort
equal to the raw effort column, so every rate is fish per party-hour while
count-derived effort is angler-hours; both operands are individually correct but
the product is not, unless every party is a single angler. add_interviews()
informed at construction, but design$angler_effort_col was ".angler_effort"
either way, so nothing downstream could tell the two apart and nothing spoke up
where the units actually collide. Designs now carry n_anglers_supplied, and
estimate_total_catch(), estimate_total_harvest() and
estimate_total_release() warn on the product path when it is FALSE.
Bus-route and ice designs are unaffected: their totals are Horvitz–Thompson sums
over interviews with no rate multiplication. The package's own examples now pass
n_anglers (#112).
estimate_total_release() and estimate_release_rate() had no bus-route
dispatch, so on a bus-route or ice design they ran the count-based product
path and ignored the inclusion probabilities entirely. The interview-derived
release counts were divided by a svytotal() over count rows — a different
effort basis from the one estimate_effort() reports for the same design,
with no warning. On a fixture whose catch records set the released count equal
to the harvest column interview by interview, so that the true release total
equals the true harvest total, estimate_total_harvest() returned 465.4 and
estimate_total_release() returned 51.1; the two now agree to machine
precision. Bus-route designs carrying no counts aborted demanding
add_counts(), which they do not need. estimate_total_release_br() had been
correct and unreachable since it was written (#110).
estimate_release_rate() on a bus-route design reaches the same estimators as
estimate_harvest_rate(). use_trips accepts "incomplete" — the truncated,
Hájek-weighted mean of ratios of Hoenig et al. (1997), reported as
method = "mean-of-ratios-rpue" — and "diagnostic", alongside the existing
complete-trip ratio of Horvitz–Thompson totals
(method = "ratio-of-means-rpue"). Both are releases per angler-hour (#110).
prep_counts_daily_effort() and prep_counts_boat_party() emitted
n_counts and within_day_var columns that add_counts() never read, so a
within-day variance component supplied through the documented preferred seam
was silently dropped and the reported SE omitted it entirely — biased
downward, the dangerous direction. On an eight-day fixture with three
counts per day the prep seam reported SE 6.93 where the equivalent
add_counts(count_time_col = ) route reported 9.52. add_counts() now reads
both columns into design$within_day_var, and the two seams agree exactly
(#109).
The columns are also rescaled into daily_effort squared units on output —
by correction_factor^2, and additionally by mean_party_size^2 in the boat
path. daily_effort is scaled by those factors but the sum of squares was
passed through untouched, so wiring the slot up without rescaling would have
left the within-day term a factor of cf^2 away from the between-day term it
is added to.
within_day_var is now documented unambiguously as a sum of squares, not
a variance: the estimator supplies the divisor itself, forming
sum(ss_d) / (n_sampled * (k_bar - 1)), so a variance understates the
component by a factor of k_d - 1. To make that contract enforceable,
within_day_var now requires n_counts, must be non-negative, and must be
0 wherever n_counts is 1. Supplying the component through both the
columns and add_counts(count_time_col = ) is an error rather than a
double count. Counts tables carrying neither column are unaffected.
estimate_angler_n() now defaults to Sadinle's (2009) 0.5 transformed logit
confidence interval on the Chapman and Petersen branches, via a new
ci_method = "logit". Every Chapman and Petersen bound moves. Pass
ci_method = "delta" to reproduce the previous symmetric Wald interval
exactly. estimate_mr_harvest() inherits the change, rebuilding its interval
from the same capture table.
The Wald interval is symmetric while \eqn{\hat{N}} is a ratio with a small
integer denominator and is strongly right-skewed, so it leaves the parameter
space in the regime Chapman exists for. At M = 200, n = 50, m = 3 it
reported ci_lower = -2124.8; at m = 5 it reported 48.7 against 245
individuals actually observed. Evans et al. (1996) measured Wald coverage
failing on one side 27.9% of the time against a 2.5% nominal rate.
Sadinle compared nine intervals and found the 0.5 transformed logit "the best of the intervals reported here", with near-nominal coverage even for small populations and capture probabilities near 0 or 1, where profile-likelihood and Monte Carlo intervals both degrade. Its lower limit cannot fall below the number of individuals observed. It is closed-form, deterministic, and adds no dependency.
| m | before | after |
| --- | --- | --- |
| 2 | [-17486.6, 24318.6] | [1319.1, 14085.6] |
| 3 | [-2124.8, 7248.3] | [1143.1, 8259.6] |
| 5 | [48.7, 3366.3] | [903.9, 4211.2] |
| 10 | [406.9, 1454.9] | [605.4, 1715.0] |
The Schnabel branch is unchanged — it already inverted Poisson quantiles and could not produce a negative bound.
One boundary behaviour is worth knowing: when m == n, every individual in
the second sample was already marked, the estimator saturates at
\eqn{\hat{N} = M}, and the logit lower limit sits fractionally above the
point estimate because the data imply \eqn{N > M}. Use ci_method = "delta"
if a bound that brackets the point estimate matters more than coverage.
estimate_mr_harvest() now derives its interval from the angler-population
interval instead of rebuilding a symmetric one, so a positive angler bound can
no longer become a negative harvest bound. On the ci_method = "delta" path
this is not a numeric change: the old code used the same degrees of freedom
and se_H = harvest_rate * se_N, so its bounds already equalled the scaled
angler bounds to machine precision.
estimate_angler_n(method = "schnabel") now builds its large-sample confidence
interval on \eqn{S - 1} degrees of freedom, where \eqn{S} is the number of sampling
occasions. It previously used \eqn{\sum m - 1}, the recapture total. Every
Schnabel interval with \eqn{\sum m \geq 50} widens; the point estimate and se
are unchanged.
Hansen & Van Kirk (2018) eq. (A.5) uses \eqn{t_{\alpha/2,,S-1}}, as does
fishmethods::schnabel(), the implementation they modified. The estimator has one
observation per occasion regardless of how many recaptures land in it, so keying df
to \eqn{\sum m} treats recaptures within an occasion as independent and understates
the interval. On five occasions with \eqn{\sum m = 52} the reported interval was
[1504.28, 2665.02] where the source gives [1388.48, 3127.07] — 33% too narrow.
The se itself was checked against the same sources and is correct as it stands.
Only the quantile changed.
estimate_angler_n(method = "schnabel") now applies Chapman's (1952)
small-sample correction by default, dividing by \eqn{\sum m_k + 1} instead of
\eqn{\sum m_k}. Every Schnabel point estimate falls, by exactly
\eqn{1/(\sum m_k + 1)} in relative terms: 33% at \eqn{\sum m_k = 2}, 1.9% at 52,
0.2% at 500. Pass bias_adjust = FALSE for the previous form, which is also
what fishmethods::schnabel() computes.
Dettloff (2023) eq. (6) simulated both forms across population sizes from
\eqn{10^2} to \eqn{10^6}. The unadjusted estimator turns biased high at
moderate sample sizes before settling, which propagates into an inflated
estimate_mr_harvest(); the adjusted form's bias "approaches zero as the sample
size increases without ever becoming positive", at lower variance and no cost in
large samples. He recommends the adjusted estimators "in place of the originals
in all scenarios".
The consistency argument is the other half. Schnabel reduces exactly to
Lincoln-Petersen at \eqn{K = 2}, so the unadjusted form meant that
method = "schnabel" on two occasions returned the estimator the package
already declines to default to at method = "petersen" — bias handling depended
on how many occasions had been sampled rather than on the data.
The se moves only through the delta-method Jacobian, which is evaluated at the
reported \eqn{\hat{N}}. \eqn{1/\hat{N}} shifts by the constant \eqn{1/\sum M_k n_k},
so \eqn{\mathrm{Var}(1/\hat{N})} is unchanged and invSE still matches
fishmethods. The Poisson interval (\eqn{\sum m_k < 50}) inverts the distribution
of \eqn{\sum m_k} rather than centring on \eqn{\hat{N}}, so its bounds do not
move; the large-sample interval is built around \eqn{1/\hat{N}} and does.
estimate_angler_n() gains method = "schumacher", the Schumacher-Eschmeyer
regression estimator, for \eqn{K \geq 3} occasions. It takes the same inputs as
"schnabel" and fits \eqn{m_k/n_k} against \eqn{M_k} through the origin with
slope \eqn{1/N}, giving \eqn{\hat{N} = \sum n_k M_k^2 / \sum m_k M_k}. The
interval is Seber (1982) eq. (4.17) on \eqn{K - 2} degrees of freedom, and
bias_adjust (default TRUE) applies Dettloff's (2023) eq. (8) small-sample
correction. With bias_adjust = FALSE the point estimate, invSE and both
bounds match fishmethods::schnabel()'s Schumacher-Eschmeyer row to printed
precision, and the formulas were checked against Seber's own worked example
(Ricker's red-ear sunfish: \eqn{\hat{N}} = 423, \eqn{\hat\sigma^2} = 0.1935).
Two details differ from the Schnabel branch on purpose. Degrees of freedom are \eqn{K - 2}, not \eqn{K - 1}: Seber excludes the first occasion because \eqn{y_1} is identically zero when \eqn{M_1 = 0} and so "is not strictly a random observation". And Dettloff's eq. (8) numerator sums from \eqn{k = 2} explicitly — \eqn{(M_k + 1)^2 (n_k + 1)} is the one term here that does not vanish at \eqn{M_1 = 0}, so occasion 1 has to be dropped by hand rather than by the algebra.
tidycreel deliberately does not implement the "pick the narrower CI" rule. Hansen & Van Kirk (2018) computed both estimators and "selected the mark-recapture estimator that produced the smallest 95% CI". Choosing the narrower of two intervals after seeing them conditions on the luckier draw, so the reported interval does not have its nominal coverage. Choose between the estimators on design grounds, or report both.
estimate_angler_n(method = "schnabel") no longer returns
ci_upper = Inf when the recapture total is very small. The Poisson interval
divides by the lower quantile \eqn{q_{\alpha/2}(\sum m_k)}, which is zero for
\eqn{\sum m_k \leq 3} at the 95% level. Hansen & Van Kirk (2018) eq. (A.4)
substitute Ilienko's (2013) continuous Poisson,
\eqn{F(x) = \Gamma(x, \lambda)/\Gamma(x)}, in exactly that case; it is positive
there and yields a finite bound. The substitution fires only where the discrete
quantile is zero — from \eqn{\sum m_k \geq 4} the continuous quantile sits just
above the discrete one, so the bound stays monotone across the seam.
The bound is an interpolation, and the warning that announces it is deliberate. It rests on a continuous interpolation of a discrete distribution at one to three total recaptures; it stands in for "the data do not bound this above" rather than measuring anything.
Implementers should note two traps. The quantile lives in the shape argument
of pgamma(), so it must be root-found — there is no qgamma() call that
produces it. And the source paper's own worked example is wrong: it reports
the 0.025 quantile at \eqn{\lambda = 2} as 0.24 and draws Figure A.1 to match,
but 0.24 is qgamma(0.025, shape = 2), a Gamma(2, 1) quantile. Inverting their
eq. (A.4) gives 0.3292. Equation A.4 transcribes Ilienko's Definition 3.1
faithfully; the example does not. Tests pin the implementation against Ilienko's
eq. (1) identity with ppois(), never against the printed example.
estimate_mr_harvest() now keys its Wald interval to the number of sampling
occasions when the input came from method = "schnabel", matching the change to
estimate_angler_n() above. It read angler_n$estimates$n, which for Schnabel is
\eqn{\sum m}, so the degrees-of-freedom defect fixed in the estimator survived one
function downstream: with five occasions and \eqn{\sum m = 52} the harvest interval
used \eqn{t = 2.008} where \eqn{t_{0.975,,4} = 2.776}, 28% too narrow. Schnabel
harvest intervals widen; Chapman and Petersen are unaffected and still use
\eqn{m - 1}.
add_counts(count_type = "progressive") now errors when a day's shift is
shorter than circuit_time, with condition class
creel_error_circuit_exceeds_shift. It previously warned and then returned an
estimate anyway.
The progressive estimator is Hoenig et al. (1993) eq. 3,
\eqn{\hat{E}_d = C \tau K} with \eqn{K = T_d/\tau} the number of whole circuits
in the day. When \eqn{T_d < \tau} no circuit completed, so the count is not a
progressive count of that shift and there is nothing for \eqn{K} to expand.
generate_progressive_start() already refused such a design, so the only way to
reach the old warning was a hand-built schedule — precisely the case with no
other guard in front of it.
\eqn{\tau = T_d} (\eqn{K = 1}) is unaffected: that is Robson's (1961) all-day circuit and remains valid.
estimate_mr_harvest(harvest_rate = ) is harvest per angler, in fish per
angler, and is no longer bounded above. It was documented as the "proportion
of anglers that harvested fish" and guarded to (0, 1].
Those two readings produce different quantities from the same arithmetic.
\eqn{\hat{H} = \hat{N} \times r} with a dimensionless proportion is a count of
anglers who kept a fish; the function returns it as total_harvest, from
estimate_mr_harvest(), with method = "mark-recapture-harvest". The
per-angler-rate reading is the one the output has always claimed, and the one
that makes the product fish.
The (0, 1] guard did more than mislabel — it enforced the wrong reading. A
fishery averaging 1.4 kept fish per angler is ordinary, and the guard made
total harvest unreachable for exactly those fisheries by erroring on the
correct input.
No numeric result changes. Every previously legal call returns what it always did, because the multiplication is untouched. What changes is which quantity you are told to supply, and that values above 1 are now accepted. If you were passing a proportion of anglers, your input was answering a different question than the output claimed to ask, and it should be replaced with mean fish kept per angler.
add_counts() now aborts with class creel_error_aerial_period_length when
period_length_col is supplied on an aerial design, and
est_effort_camera() aborts with class creel_error_camera_period_length
when its raw-count branch is handed counts that already carry T_d.
Both estimators already have a period-length term of their own: aerial scales
the count by h_open / v (Pollock Eq. 15.4) and camera's raw-count fallback
scales by a supplied h_open. Once add_counts() began applying
period_length_col for any count type (see the previous entry), a design
carrying both multiplied by time twice — on a 4-day fixture with h_open = 14
and T_d = 2 the aerial total went from 1400 to 2800, and the unit spine
labelled that 2800 "angler-hours".
This is a regression in the development version only; no released version
applied T_d on those paths, so callers of released tidycreel are unaffected.
Anyone who added period_length_col since that change should remove it and
set the period length through h_open instead. Camera's ratio-calibration
path is deliberately unaffected: it divides by mean(count) before
multiplying by count, so a constant T_d cancels out of the estimate.
add_counts() now applies period_length_col to instantaneous counts instead
of discarding it. Supplying the column on an instantaneous design used to be
accepted, recorded in design$period_length_col, and then ignored — the
estimate came back as the bare counts summed over days, with the T_d column
left sitting unread in design$counts. Effort estimates move for anyone who
passed it: on an 8-day fixture with T_d of 8–14 hours the total went from 140
to 1780.
An instantaneous count is a snapshot of how many anglers were present at one
moment, not effort. Effort is that count times the length of the period it was
randomised within, Ê_d = C̄_d × T_d (Hoenig et al. 1993). The multiplication
happens per PSU at attach time, so the ungrouped, grouped, sectioned and
within-day-variance paths all inherit it, and multi-count PSUs get their ss_d
scaled by T_d² so the within-day variance stays in effort² units.
Applying T_d per date rather than after aggregation is deliberate: the collapsed form computes C̄ × T̄ where the target is the mean of C × T, and the two differ by Cov(C, T). Anglers fish more on long days, so that covariance is positive and the collapsed form biases low. Multiplying per date makes the term exactly zero at any stratum width, which removes the constraint on stratum design that would otherwise follow from T varying within a stratum.
The positive-and-finite check on period_length_col now runs wherever the
column is supplied. It previously lived inside the progressive-only block, so a
zero or negative period passed unchecked on an instantaneous design.
n_anglers now means a party size, not a tidyselect column position. It was
resolved through tidyselect::eval_select(), where a bare integer selects a
column by position, so n_anglers = 1L — the literal in add_interviews()'s
own signature — selected column 1 and multiplied effort by whatever it held.
On interviews whose first column is numeric that silently produced
.angler_effort = hours × <that column>; on the shipped column order it failed
with * not defined for "Date" objects, naming neither the argument nor the
column it chose. Which of the two you got depended on your column order. It
also set n_anglers_supplied = TRUE, switching off the warning that exists to
catch exactly this mismatch.
A bare number is now a constant party size: n_anglers = 1 states that every
interview is a single angler, and n_anglers = 3 that every party held three.
Bare column names are unaffected. This is also the only way to declare a
genuinely solo-angler survey, and therefore to silence the party-hours warning
above without inventing a constant column.
Party sizes are now validated wherever they come from. Zero, negative and
non-finite values abort — a party of no anglers would silently zero out that
interview's effort — missing values abort as a stated constant but warn as a
column, and non-integer values warn. compute_angler_effort() follows the same
contract; it is the other exported entry point that writes .angler_effort.
Bus-route and ice totals now count completed trips only, in all three
quantities. estimate_total_harvest() already filtered; estimate_total_catch()
and estimate_total_release() did not, so on one design the three totals were
computed over different row sets and could not be compared. On a 24-interview
fixture split 12 complete / 12 incomplete, total catch was 1089.81 over 24 rows
where the completed-trip figure is 512.31 over 12 — a factor of 2.13.
These are access-point estimators (Malvestuto 1996, §20.3.1.2), and §20.5.1
builds them by summing completed-trip quantities over interviews. An uncompleted
trip breaks that in two directions at once: the observed count is catch so far
rather than the trip's catch, biasing the sum down, while \eqn{\pi_i} is the
inclusion probability of a completed trip and an uncompleted one is intercepted
with probability proportional to its length (length-of-stay bias, §20.3.1.1),
biasing it up. The two do not cancel predictably. Incomplete trips support a
rate — the truncated Hájek mean of ratios of Hoenig et al. (1997), reachable via
estimate_catch_rate(use_trips = "incomplete") — never a total (#112).
estimate_total_catch(use_trips = "all") now aborts on bus-route and ice
designs. It was previously accepted and silently discarded: "all" and
"complete" returned the same unfiltered number, so the argument documented as
selecting trips did nothing at all on these designs. "complete" is the default
and is unaffected, so callers passing nothing see no change beyond the
completed-trip filter above (#112).
estimate_angler_trips() and estimate_effort_per_acre() now reject any
creel_estimates whose method is outside the effort family ("total",
"total-sections"). Both are documented as taking angler-hours but guarded only
on class, so a CPUE object passed straight through: fish per hour divided by
hours per trip, relabelled "angler-trips", no warning. A fish-valued bus-route
total was accepted the same way (#112).
estimate_total_catch(), estimate_total_harvest() and
estimate_total_release() on a bus-route or ice design now report
method = "ht-total-catch", "ht-total-harvest" and "ht-total-release"
respectively, in place of the bare "total" all three returned. "total" is
the string the labelling code maps to effort, so a fish-valued total plotted
with a y-axis and title reading "Total Effort" and exported a CSV whose
provenance header read Method: total — nothing in the returned object said
which quantity it held. On an eight-day bus-route fixture the catch total of
1089.81 fish and the harvest total of 464.77 fish both plotted as "Total
Effort" beside a genuine effort total of 2513.38 angler-hours. The estimates
themselves are unchanged; only the method string and the labels derived from
it move. estimate_effort() still returns "total", which was correct for it
all along.
The ht- prefix names the estimator as well as the quantity, following the
existing product-total-* convention, so a bus-route Horvitz–Thompson total
is no longer indistinguishable from the standard design's effort × rate
product in either the object or the exported file (#111).
estimate_release_rate() gains truncate_at, defaulting to 0.5 hours, with
the same meaning, units, and NULL behaviour it has on
estimate_harvest_rate(). It applies only to the bus-route incomplete-trip
path (#110).
estimate_total_release(design, by = species) and
estimate_release_rate(design, by = species) on a bus-route or ice design now
abort with Column 'species' doesn't exist rather than returning a number
from the standard path. The bus-route Horvitz–Thompson estimators take no
species argument, and by resolves against the interview table, where a
species column does not exist. estimate_total_harvest() and
estimate_harvest_rate() have behaved this way since their own dispatches
landed; per-species release on these designs was never estimated from the
sampling frame (#110).
estimate_harvest_rate() on a bus-route or ice design now returns a rate. It
dispatched to the Horvitz–Thompson harvest total of Jones & Pollock (2012)
Eq. 19.5 and returned it with method = "total", so it produced a number
identical to estimate_total_harvest() under a function documented as
returning fish per angler-hour (#107).
Jones & Pollock give bus-route effort and harvest as HT totals and define no
rate estimator, so the rate this design supports is the ratio of those two
totals, H_hat / E_hat — the ratio-of-means form, and the same quantity and
method string ("ratio-of-means-hpue") the standard designs already return.
The ratio is computed with survey::svyratio() rather than by dividing two
separately estimated totals: the numerator and denominator come from the same
interviews and are strongly correlated, and treating them as independent
overstates the SE by roughly eightfold on the package's own fixture.
Grouped results no longer carry a proportion column. A share-of-total is
meaningful for a total and meaningless for a rate.
estimate_harvest_rate(use_trips = "incomplete") on a bus-route design now
returns a rate. It computed a per-angler ratio, divided that ratio by the
inclusion probability, and summed. Inverse-probability weights apply to
totals, not to ratios, so the result was neither the population rate nor a
total: it grew linearly with the number of interviews. On a fixture where
every angler harvests at 1 fish per angler-hour it returned 19.2, 38.3, and
76.7 as the same population was sampled with 4, 8, and 16 interviews. It also
dropped the .expansion factor the complete-trip path applies, and divided by
the party's elapsed hours rather than angler-hours, so the underlying ratio
was fish per party-hour (#108).
The path now returns the estimator this trip type supports: the truncated
mean of ratios of Hoenig, Jones, Pollock, Robson & Wade (1997, Biometrics
53:306–317), reported as method = "mean-of-ratios-hpue". For anglers
intercepted mid-trip they show ratio-of-means weights individual rates by the
square of completed trip length and so "does not provide an estimate of
catch rate that can be used with an independent estimate of total effort to
provide an unbiased estimate of total catch"; the mean of ratios has the
correct expectation. Because interviews are not equally likely under a
bus-route design, the mean is weighted by .expansion / .pi_i — a Hájek mean
rather than the paper's plain average — and computed with survey::svyratio()
so the variance is linearised over numerator and denominator together.
estimate_harvest_rate() gains truncate_at, defaulting to 0.5 hours.
The mean-of-ratios estimator has infinite asymptotic variance, because
1/L has infinite expectation as trip length approaches zero; Hoenig et al.
(1997) recommend discarding trips shorter than 30 minutes. The threshold
applies to elapsed trip duration, not to angler-hours — it is the short clock
interval that makes the reciprocal explode, and a large party fishing briefly
clears an angler-hour threshold while still being the unstable case.
truncate_at = NULL disables truncation and warns. The argument is ignored on
every other path, including use_trips = "complete".
use_trips = "diagnostic" on a bus-route design now compares like with like.
Its two slots held a harvest total and a quantity that grew with sample size,
so the gap read as enormous incomplete-trip bias when it was a change of
physical units. Both slots now report fish per angler-hour. They remain
different estimators — ratio of HT totals for complete trips, truncated mean
of ratios for incomplete ones — because each is the estimator its trip type
supports. A design carrying only one trip type now aborts with a clear message
instead of failing inside survey with "all arguments must have the same
length", and the verbose dispatch message names the estimator actually used
rather than always announcing the complete-trip one.
Bus-route and ice estimate_effort() now return angler-hours. They read the
raw per-party trip duration, so the estimate was party-hours reported under an
angler-hours label — invariant to party size, and understated by exactly the
mean party size in any boat fishery. They now read the angler-effort column
(duration × n_anglers) that every other rate estimator already used. On the
same design CPUE is fish per angler-hour, so the old behaviour also mixed
denominators in any effort × CPUE product (#106).
Surveys recording one angler per party are unaffected: with no n_anglers,
angler-effort equals the raw effort, and add_interviews() already warns.
Anything with parties larger than one will see totals rise by roughly the mean
party size. The ice output column total_effort_hr_on_ice is affected on the
same terms.
add_counts() gains a count_col argument and no longer picks the count
column by position. Previously the count variable was taken as the first
numeric column that was not design metadata, so a counts table carrying more
than one numeric column could have a row index, a daylight-hours column, or a
boat count silently expanded and reported as "Total Effort" — off by an order
of magnitude, with no warning. When more than one numeric column qualifies,
add_counts() now aborts and lists the candidates; name the intended column
with count_col. Tables with a single count column are unaffected (#105).
The resolved name is stored on the design as $count_col and used by
estimate_effort(), the sections and grouped effort paths, the aerial and
aerial-GLMM estimators, camera effort, audit_strata(), and autoplot(),
all of which previously repeated the same positional guess.
Callers of tidycreel.connect::fetch_counts() are affected: it returns
bank_anglers, angler_boats, and non_ang_boats, so add_counts() now
requires count_col to be named.
The progressive count articles now state the conditions under which
\eqn{\hat{E}_d = C \times T_d} is unbiased. vignettes/progressive-count-surveys.Rmd
gains a Conditions for an Unbiased Estimate section covering Hoenig et al.'s
(1993) three requirements — random starting location, randomly chosen direction of
travel, and an observer who outpaces the anglers — plus two cautions from the same
paper: do not interrupt the circuit to interview, and do not read the count as a
number of trips, which "results in a negative bias that can be severe."
None of these are checkable from the counts table, which is why they belong in prose rather than in a guard.
vignettes/effort-pipeline.Rmd previously derived the \eqn{\tau} cancellation
through an unmotivated \eqn{\times\tau} that returned the expression to
\eqn{C_d \times T_d}. It now follows the source's two-step argument — expand the
sampled block by \eqn{\tau}, then scale by the \eqn{K = T_d/\tau} blocks in the
day — which reaches the same formula and shows why \eqn{\tau} cancels: it defines
the blocks the count was scheduled within, so it has done its work before the
estimator runs.
Also corrected: the "circuit time < 30% of \eqn{T_d}" rule of thumb was not from Hoenig et al. and is not the paper's condition.
estimate_mr_harvest() attributed its known-constant harvest rate to Hansen &
Van Kirk (2018), which does the opposite: both factors of that rate are
estimated there, given log-normal sampling distributions, and resampled
alongside \eqn{\hat{N}} in the bootstrap behind every harvest CI. The
simplification is this package's, so the reported se is a lower bound on the
true uncertainty, and @details now says so rather than crediting a source.
harvest_rate also gains the period it was missing. In the paper's
\eqn{H = N \cdot D \cdot V} the multiplier on the angler population is
\eqn{D \times V} — days fished per angler times daily harvest per angler — so
the argument must cover the same period angler_n counts anglers for. "Fish
per angler" alone did not pin that down, and the daily rate is the wrong one.
estimate_angler_n() documents that its Chapman and Petersen confidence
intervals are symmetric and can fall below zero. \eqn{\hat{N}} is a ratio with
a small integer denominator, so it is right-skewed; at M = 200, n = 50,
m = 3 the reported ci_lower is -2124.8, and estimate_mr_harvest()
inherits the shape. Chapman is recommended precisely when recaptures are few,
so the docs now direct small-\eqn{m} users to ci_method = "bootstrap", whose
percentile bounds respect the skew. The Schnabel branch already inverts
Poisson quantiles and is unaffected. The interval arithmetic is unchanged in
this release — correcting it moves every shipped Chapman and Petersen bound.
estimate_exploitation_rate() described C as a harvest total while pointing
at estimate_total_catch() to produce it. Catch includes released fish, which
were never removed from the tagged cohort, so a catch total inflates
\eqn{\hat{u}} by the release fraction. @param C, @param strata and
vignettes/mark-recapture.Rmd now point at estimate_total_harvest() and say
why. The estimator is unchanged; only the cross-reference was wrong.
Noted in the docs because no check can catch it: both totals are counts of
fish and both carry unit = "fish", so the expression is dimensionally
coherent. The actor matches and the quantity does not.
vignettes/flexible-count-estimation.Rmd: the instantaneous baseline built an
open_hours column that no tidycreel function reads, so the example looked
like it accounted for the length of the fishing day while reporting 135 where
its own stated formula gives 1350 — a 10x understatement in the vignette
teaching this exact topic. The inert column is removed and the units of the
instantaneous estimate (angler-days, not angler-hours) are now stated
explicitly (#113).
vignettes/progressive-count-surveys.Rmd: the "Multiple Periods per Day"
example built open_hours and shift_hours and passed neither, so it
demonstrated the instantaneous multi-count path inside the progressive
article. Both inert columns are removed and the text now says the chunk shows
the within-day variance decomposition only, without the progressive T_d
expansion (#113).
Inert open_hours calendar columns are removed from the six remaining places
they appeared — vignettes/progressive-count-surveys.Rmd,
vignettes/effort-pipeline.Rmd and vignettes/temporal-extrapolation.Rmd.
creel_design() reads only the date and the strata, so the column was never
consulted anywhere it was written. The progressive article additionally listed
the calendar's open_hours as the \eqn{T_d} the estimator applies; the real
\eqn{T_d} travels with the count data and is passed as period_length_col
(#113).
vignettes/tidycreel.Rmd: two reported values had drifted from what the
chunks print. The total effort estimate is 372.5 angler-hours, not the 358
claimed, and the grouped estimates are 201.9 weekend / 170.6 weekday, not
250 / 108. The grouped comparison now notes that the calendar holds 10
weekdays to 4 weekend days, so a weekend total 18% higher is a per-day rate
about three times higher. The article also called example_counts
"instantaneous count observations" when the column holds angler-hours already
accumulated over the day (#113).
example_counts and example_sections_counts documented their
effort_hours column as an instantaneous count of angler-hours, which is
two different quantities at once. Both now state that the column holds
angler-hours, and that estimate_effort() expands whatever column it is
given without converting units — raw counts in, angler-days out (#113).
vignettes/glossary.Rmd sanctioned the same ambiguity by defining count data
as "the observed angler count or angler-hours". It now states that both are
accepted, that no conversion happens, and which unit each choice returns
(#113).
vignettes/ice-fishing.Rmd described estimate_total_catch() as CPUE times
effort over all interviews. On ice designs it is a Horvitz–Thompson sum with
no CPUE term and no effort term, over complete trips only — 60 of the
vignette's 72 interviews — and use_trips = "all" is refused. The standard
error comes from Taylor linearization, not the delta method the text credited
(#113).
generate_progressive_start() schedules randomised circuit start times for
progressive count surveys following Hoenig et al. (1993). Two strategies
supported: "discrete" (start drawn from valid τ-aligned offsets; avoids
mid-day bias from the common U[0, T−τ] error) and "wraparound" (start
drawn from U[0, T) with wrap detection). Returns a creel_schedule with
circuit_start, circuit_end, is_wrapped, and direction columns.add_counts() with count_type = "progressive": multi-circuit designs
(multiple counts per day via count_time_col) were previously blocked with
an error. Now supported — daily effort is estimated as mean(C_k) × T_d
across circuits.
add_counts() multi-circuit progressive: within-day variance ss_d was
in count² units but compute_within_day_var_contribution() requires effort²
units. ss_d is now scaled by T_d² per PSU before the progressive effort
computation, correcting variance estimates for multi-circuit designs.
add_counts() progressive: period_length_col was incorrectly included in
the numeric column scan used to auto-detect the count variable, causing it to
be misidentified as the count. Now excluded from the scan.
simulate_creel_data(): minimum trip effort floor raised from 0.05 h to
0.1 h to reduce implausibly short simulated fishing trips.
est_age_distribution() estimates proportional age structure with SE and
confidence intervals from age-frequency interview data, fully integrated with
the creel_design workflow. Stratified and grouped estimation supported.
est_mean_age() estimates mean age (± SE, CI) from structured interview
data. Complements est_age_distribution() for reporting age-structured
harvest results.
example_ages — new built-in dataset of simulated age observations for use
in examples and tests.
estimate_harvest_rate() gains species-level dispatch: pass a species column
and the function routes harvest-rate estimation independently per species,
returning a tidy multi-species result in a single call.
creel_design() gains open_start parameter for GLMM aerial designs,
allowing the survey window to be anchored to the count time rather than
requiring a fixed open time.
estimate_total_catch(), estimate_total_harvest(),
estimate_total_release(): strata with effort but no interview coverage were
silently dropped by an inner join in compute_stratum_product_sum(), biasing
season totals low. Fixed to warn and retain all effort strata (#Tier1-Bug1).
estimate_angler_trips(): stats::sd() on a single-interview stratum
returned NA, propagating silently into SE and CI. Guard added for n < 2;
emits cli_warn() and returns NA_real_ for SE so the point estimate
remains usable (#Tier1-Bug2).
estimate_effort(): finite population correction (FPC) was not applied to
the expanded effort svydesign, causing inflated SE for designs with high
sampling fractions. Fixed (#Tier1-Bug5-adjacent).
optimal_n(): named cost_ratio vectors were applied positionally instead
of by stratum name, producing wrong allocations when stratum order differed.
Zero variance (all s2_h = 0) and zero total (all ybar_h = 0) produced
silent NaN; both now abort with informative errors. n_total floored at 1
to prevent degenerate zero-sample result.
adjust_nonresponse(): method = "calibrate" was accepted and matched but
silently ignored — both methods used direct weight rescaling. Now aborts with
an informative error directing users to survey::calibrate() directly
(#Tier1-Bug4).
adjust_nonresponse() replicate-design path: svy$scale (a variance
formula constant) was multiplied by mean(wt_multipliers), affecting only
variance and using an average instead of per-observation values. Fixed to
scale svy$pweights per-observation and svy$repweights row-wise
(#Tier1-Bug5).
new_creel_validation(): all(logical(0)) == TRUE caused a 0-row results
object to silently report passed = TRUE. Fixed with nrow > 0 guard;
empty validation results now correctly return passed = FALSE.
design-validator: ybar_h, s2_h, and n_proposed were consumed
positionally against named N_h, producing wrong stratum indexing when order
differed. All three now rekeyed by strata_names before indexing.
validate_incomplete_trips(): perform_tost() crashed or silently passed
when se_diff == 0 (identical SEs) or df <= 0 (n = 1 group). Early-
return guards added for both degenerate cases; isTRUE() used in grouped-
passed aggregation to prevent NA propagating into if().
schedule_generators(): inclusion_prob could silently exceed 1 when
p_site * (crew / n_circuits) > 1. Now aborts with an actionable message.
creel_palette(n): modular recycling used 0-based index at palette-length
multiples, returning NA at those positions. Fixed to 1-based modular
arithmetic.
coerce_schedule_columns(): unconditional as.integer(period_id) silently
coerced character labels ("AM" / "PM") to all-NA, filtering all
downstream rows. Now only coerces when all non-NA values are numeric
strings; character period labels are preserved unchanged.
compare_variance(): Taylor and replicate SEs were paired by row position
rather than stratum key. If the two estimators returned rows in different
orders, divergence ratios were computed for mismatched strata. Fixed with a
keyed join; group-column detection now derived from x$by_vars rather than
a hardcoded exclusion list that would misclassify new output columns.
validation_report(), standardize_species(), hybrid_design(): second
positional string to cli_abort() / cli_warn() was silently dropped by
cli's argument handling. Merged into single message or named vector.
est_age_distribution() and est_length_distribution(): per-group n was
reporting the global interview count (nrow(design$interviews)) instead of
the within-group count. Fixed to nrow(wide) per group, consistent with
estimate_total_catch() and estimate_total_harvest().
est_age_distribution() and est_length_distribution(): left_join() was
called inside the per-group loop against the full interviews table (constant
across iterations). Replaced with match() lookup and direct column
assignment, eliminating repeated dplyr overhead.
inst/quarto/templates/creel-report/. The template demonstrates a full
season workflow — design, validation, estimation, and plotting — using
tidycreel helpers end to end.estimate_harvest_rate() and estimate_release_rate() now default to
use_trips = "complete" (previously use_trips = "all"). For standard
(non-bus-route) designs that supply trip_status, HPUE and RPUE are now
estimated from completed-trip interviews only. This is the statistically
preferred default: incomplete-trip rates underestimate harvest and release
when anglers keep or release additional fish after being interviewed (Hansen &
Van Kirk 2010). The previous all-interview behavior is no longer the default
but remains fully available.
To restore the previous behavior, pass use_trips = "all" explicitly:
estimate_harvest_rate(design, use_trips = "all")
estimate_release_rate(design, use_trips = "all")
Designs without a trip_status column are unaffected (the argument has no
effect). Bus-route designs already defaulted to "complete" and are
unchanged. Closes #69.
tidycreel design, validation, summary, and plotting helpers end to end.simulate_creel_data() now returns a $schedule component — a full-season
calendar (one row per season day) with columns date (Date), day_type
(character), and sampled (logical). Pass directly to creel_design() as
the calendar argument for a complete round-trip simulation pipeline with no
manual column construction. Unsampled days receive a day_type drawn
proportionally from the day_types distribution. Closes #68.
sim <- simulate_creel_data(params = my_params, day_types = c(weekday = 5/7, weekend = 2/7))
design <- creel_design(sim$schedule, date = date, strata = day_type) |>
add_counts(sim$counts) |>
add_interviews(sim$interviews,
catch = "catch_total", effort = "hours_fished", harvest = "catch_kept",
trip_status = "trip_status", n_anglers = "n_anglers", interview_type = "roving")
Note: this changes the return structure from three components
(interviews, counts, catch) to four (schedule, interviews,
counts, catch). Code that checks names by position should switch to
name-based access.
simulate_creel_data() day_types parameter now explicitly documents that
the argument must be a named numeric vector (not a character vector), with
a worked example showing the correct form c(weekday = 5/7, weekend = 2/7).@examples block expanded with a multi-stratum simulation and the full
round-trip pipeline from simulate_creel_data() through creel_design(),
add_counts(), and add_interviews().standardize_species(): added custom_codes argument (named character vector
applied as a second AFS-NA pass), expanded AFS lookup table with Freshwater
Drum ("FRD"), and corrected misleading "supply a custom code map"
documentation that implied a non-existent function argument. Closes #66.estimate_harvest_rate() / estimate_release_rate(): added use_trips
argument ("all" default, "complete" to restrict) with cli_inform notice
showing trip-status breakdown. Documented livewell-observable rationale and
downward-bias risk (Hansen & Van Kirk 2010). Closes #65. Future default flip
to "complete" tracked as #69.estimate_catch_rate() now auto-routes roving designs: when
add_interviews(..., interview_type = "roving") is set and use_trips /
estimator are not explicitly supplied, the function defaults to
use_trips = "all" and estimator = "mor" (Hoenig et al. 1997), using all
interviewed trips via mean-of-ratios rather than restricting to complete trips.
Access-point designs (interview_type = "access", the default) are unaffected.
Explicit use_trips or estimator arguments always override the auto-route.
Closes #67.
New use_trips = "all" option for estimate_catch_rate(): uses every
interview (complete + incomplete) with the MOR estimator. Previously only
"complete", "incomplete", and "diagnostic" were accepted.
estimate_catch_rate(by = species) returned all-zero estimates when catch
data contained only "harvested" and "released" rows (no "caught" rows).
Fix was in source since v2.0.0 but the installed binary at the site-library
was stale; reinstalling now picks up the correct aggregation logic. Closes #64.add_interviews() interview_type parameter description corrected: now
accurately states that "roving" triggers automatic estimator routing rather
than carrying the false claim that the flag was "stored metadata only".estimate_catch_rate() use_trips parameter and Details section updated to
document "all", roving auto-routing, and the access vs. roving distinction.Starting with this release, tidycreel follows semantic versioning (MAJOR.MINOR.PATCH) and names each MINOR release after a fish species native to Nebraska or the Great Plains. v2.1.0 is named for the Sauger (Sander canadensis), a walleye relative common in Nebraska's large rivers.
estimate_angler_trips() — estimates angler trip counts (angler days) from effort and mean trip length using Delta Method variance propagation.estimate_effort_per_acre() — computes effort density (angler-hours per acre) by stratum from an extrapolated effort estimate and supplied acreage.summarize_boat_composition() — returns percent angler boats by month and day type, computed from the angler-boat and non-angler-boat count columns.summarize_by_zip() — tabulates interview count and percentage by zip code from the interview zip code column.summarize_by_county() — maps zip codes to counties via zipcodeR and returns interview count and percentage by county; emits an informative error when zipcodeR is not installed.cli_abort() sites, formal lifecycle
badges on experimental APIs, a valid inst/CITATION, and removal of the
scales dependency from the package surface.lubridate from Imports to Suggests and added runtime install
guards at user-facing schedule entry points.rlang::caller_env() through the top-level bus-route estimator
internals and relocated get_site_contributions() into the estimation layer
to tighten call-frame quality and layering.@family tags across the exported surface so the pkgdown reference is
grouped by workflow topic rather than a flat function list.print.creel_design(),
print.creel_estimates_mor(), and print.creel_schedule().quickcheck-based property tests and generator helpers covering the
highest-value implemented invariants: INV-01, INV-02, INV-03, INV-04, and
INV-06.86.27%,
Codecov configuration, and a project target of 85%.estimate_catch_rate() now accepts estimator = "mortr" for truncated
mean-of-ratios (MORtr), which applies truncate_at as a mandatory threshold
and labels the method "mean-of-ratios-truncated-cpue".estimate_catch_rate() gains a targeted argument (default TRUE). Setting
targeted = FALSE excludes zero-catch trips before MOR/MORtr estimation for
incidental species workflows.power_creel() provides a unified tidy entry point for pre-survey
sample-size planning, wrapping creel_n_effort(), creel_n_cpue(), and
creel_power() into a single consistent interface with mode = "effort_n",
"cpue_n", or "power".compare_designs() compares multiple survey designs side by side from a
named list of creel_estimates objects. An autoplot() method renders a
forest plot of point estimates with confidence intervals.as_hybrid_svydesign() constructs a hybrid access + roving survey design
from combined access-point and roving-route count data.compare_variance() computes Taylor linearization vs. replicate (bootstrap
or jackknife) standard errors side-by-side for any creel_estimates object.adjust_nonresponse() applies nonresponse weighting to a creel_design and
records per-stratum diagnostics.est_effort_camera() adds ratio-calibrated camera/time-lapse effort indexing.est_length_distribution() adds weighted catch-at-length / size-structure
estimation from attached length data.autoplot.creel_length_distribution() adds a plotting surface for weighted
size-structure estimates.theme_creel() and creel_palette() add package-standard plot styling.validate_creel_data() adds field-level schema validation for creel inputs.standardize_species() adds canonical species-code standardisation helpers.validation_report() adds formatted validation summaries that can be exported
alongside other report-ready outputs.creel_counts_toy and creel_interviews_toy are now bundled example datasets
for examples, tests, and documentation.inst/rmarkdown/templates/creel-dashboard/.plot_design() now supports multi-strata designs.autoplot() methods now support opt-in
theme = "creel" styling without changing default behavior.survey:::onestrat message.aerial-glmm vignette downstream estimation chunk where
example_aerial_interviews was paired with the wrong design object.Imports to support the autoplot.* methods.Suggests for the optional report template.summary.creel_estimates() converts any estimate object to a creel_summary
with human-readable column names (Estimate, SE, CI Lower, CI Upper,
N). Includes print.creel_summary() and as.data.frame.creel_summary()
methods. Works for effort, CPUE, harvest rate, total catch, and grouped
variants.
flag_outliers() identifies extreme values in a numeric column using
Tukey's IQR fence (k = 1.5 default). Returns the input data frame with
is_outlier, outlier_reason, fence_low, and fence_high columns
appended, and emits a cli summary of flagged rows. Handles n < 4,
empty input, and zero-row data frames gracefully.
ggplot2::autoplot.creel_estimates() produces a point-and-errorbar plot
from any creel_estimates object. Ungrouped estimates show a single point
with confidence interval; grouped estimates show one point per group level,
colour-coded.
ggplot2::autoplot.creel_schedule() produces a monthly tile calendar from
a creel_schedule object. Sampled dates are coloured by day type (weekday
blue / weekend red); unsampled dates are shown in grey. Multiple months are
displayed as vertically stacked facet panels.
Single-PSU strata now produce a structured, actionable error instead of an
opaque survey:::onestrat message. The error names the problematic stratum
and suggests increasing the sampling rate or combining sparse strata.
Fixed a bug in the aerial-glmm vignette downstream estimation chunk where
example_aerial_interviews was paired with the GLMM design (built from
example_aerial_glmm_counts). The chunk now uses the correct matching
dataset (example_aerial_counts + example_aerial_interviews).
Imports to support the new autoplot.* methods.generate_count_times() adds three sampling strategies for allocating
interview periods within a survey day: random, systematic, and
fixed-interval. Supports a seed argument for reproducibility; returns a
creel_schedule object compatible with write_schedule().
The survey-scheduling vignette now covers the full pre- and post-season
planning workflow: generate_count_times() through validate_design(),
check_completeness(), and season_summary().
GitHub issue templates now use structured forms with
blank_issues_enabled: false, routing how-to questions to GitHub Discussions
to keep answers searchable for all users.
CONTRIBUTING.md has been rewritten with current workflow guidance,
contribution types, and community norms for the v1.x release line.
Launched the pkgdown documentation site at https://chrischizinski.com/tidycreel with a custom Bootstrap 5 theme, full function reference index (46 exports + 15 datasets), and a workflow-driven navbar.
Added a GitHub Actions CI/CD workflow to deploy the pkgdown site automatically on every push to main.