Changes in version 1.0.1 (2026-07-07) Bug fixes - DuckDB batch auto-tuning no longer probes the operating system for total RAM (sysctl on macOS, /proc/meminfo on Linux). It now reads the memory budget straight from the connection's own memory_limit setting, which defaults to 80% of system RAM and reflects any user override. This removes the shell call that broke on systems without sysctl on the PATH (the CRAN M1mac check failures, where batch planning errored with "argument is of length zero"), and makes the budget detection work identically on all platforms, including Windows. Changes in version 1.0.0 (2026-07-07) First stable release. The token core, the DuckDB backend, embedding matching, diagnostics, and calibration are feature-complete and the public API is stable. This release adds the documentation that makes the package usable end to end. Documentation - Reference site built with pkgdown: a grouped function index, a getting-started vignette, a concept glossary, and five how-to articles (fuzzy and exact strategies, matching across years and sources, calibration, embeddings, and working at scale with DuckDB). - Runnable examples on every entry-point verb. Example data - workshop_register, workshop_listings, workshop_panel, match_labels_example: synthetic woodworking-workshop data with planted difficulty tiers and ground-truth links, used throughout the articles. Each tier (containment, movers, phonetic twins, hub tokens) has a minority that measurably benefits from the feature it exercises. Changes in version 0.9.0 Phase 0.9: Staged linkage and region-free matching Staged entity resolution, region-free linking across blocks, and an always-on cost guard, plus embedding reuse and faster preparers. Staged entity resolution Run strategies in order, carry residuals forward, resolve entities once at the end. - multi_stage_dedup() and multi_stage_search(): run an ordered list of strategies as successive passes. multi_stage_dedup() finds duplicates within one table; multi_stage_search() links records across tables, or across years of one pooled table with self = TRUE. Both take a mix of exact, fuzzy, and embedding strategies; multi_stage_search() supports collapse-and-continue so a slowly drifting name links one step at a time. Renames multi_stage_match(). - exact_strategy(): identical-token-set matching as a strategy, for a cheap first pass. Runs through detect_duplicates() and search_candidates() like any strategy. Optional containment matches a subset rather than an exact set, with a per-column min_containment_tokens floor. - resolve_entities(): group an edge list into entities (connected components) and pick a representative per group. - materialize_records(): fetch the original rows for a set of ids, the complement of extract_unmatched(). - plan_strategy(): compare blocking keys before matching. Reports each candidate's block sizes, comparison cost, and how many true twins stay co-blocked, without computing any scores. - rarity_distribution(): report a column's token frequency and rarity before matching, with a suggested min_rarity. - find_stopwords(): list a column's high-frequency, low-information tokens for filter_stopwords(). - duckdb_control(): one object for DuckDB execution tuning (batch size, scoring chunk key, per-chunk failure policy, progress), passed as control =. Replaces the loose batch arguments. Region-free linking Follow an entity across geographic blocks (movers, name drift, year to year) without giving up block-based cost control. - block_on_tokens(): block on a record's own rare name tokens instead of a fixed key, so two records sharing any rare token are compared wherever they sit. Mix it with plain column names in block_by. - rarity_scope = "global": measure rarity across the whole corpus, so a distinctive name reads as strong evidence in any block and a common one stays weak. Fan-out guard - max_fanout / on_fanout: an automatic ceiling on comparison cost. When a hot or boilerplate token would fan a dense block into a near-quadratic join, joinery drops the offending tokens with a warning ("cap", the default) or stops ("abort"). On by default. Replaces max_comparisons. Embedding reuse - Embed once: the data.table and tibble backends cache embedding vectors per session, so a multi-stage run no longer re-embeds a record on every pass. Keyed by model and text. Set joinery.embedding_cache_dir to persist across sessions, or joinery.embedding_reuse = FALSE to opt out. - clear_embedding_cache(): empty the cache, optionally on disk too. - Faster scorer: score_embeddings() scores all pairs in a block as one matrix product, dropping a few hundred thousand pairs from seconds to a fraction of a second. Preparers - drop_short_tokens(): drop tokens below a length, useful after phonetic encoding. - Phonetic encoders on tokens: as_cologne(), as_soundex(), as_metaphone(), and as_nysiis() now encode token lists as well as raw strings, so you can encode after tokenizing. - normalize_street() gains drop_house_numbers and drop_stopwords to strip address noise. - Faster preparers: word_tokens(), filter_stopwords(), drop_numeric_tokens(), token_shapes(), and extract_initials() now run group-wise over token tables. Scoring and validation - Missing-column reweighting: when a column is empty for a record, its weight is shared among the present columns rather than dropped, so scores stay in range. - Earlier errors: search_candidates() rejects overlapping id spaces and prepare_search_data() rejects duplicate ids, both of which corrupt results silently otherwise. Bug fixes - resolve_entities() no longer drops singletons when ids mix integer and double forms. - summarise_matches() (DuckDB) no longer produces an out-of-range histogram bin for scores just above 1.0. - drop_joinery_temp_tables() is now exported. Changes in version 0.8.0 Phase 0.8: Stability and quality Internal consolidation after the calibration work, plus fixes surfaced by a full-scale Yellow-Pages panel build. Output schemas unchanged. - Token-set scoring: a token repeated within one record no longer inflates a pair's score; scores stay within [0, sum(weights)]. - DuckDB at scale: connected components run per block instead of one global recursion that exhausted memory at corpus scale; empty dedup results carry the full schema; filtered lazy inputs (tbl |> filter(...)) are accepted everywhere. - summarise_matches(entity_cols =): count duplicate groups whose listed columns are single-valued, separating real stopword clusters from cardinality artefacts. - Consistent errors: unified on cli::cli_abort() with rlang argument checks across exported verbs. - File layout: R/ reorganised under an eight-prefix naming scheme (see CLAUDE.md). Changes in version 0.7.0 Phase 0.7: Error calibration An optional post-match filter that learns to drop false positives from a small labelled sample. The same verb works on token and embedding strategies. - match_features(): build a one-row-per-pair feature table from a match result, with token-overlap counts, auxiliary-side informativeness (aIP, after Doherr 2023), and string similarities. - fit_filter() / apply_filter(): fit a logistic false-positive filter and apply it, choosing a threshold by Youden's J unless you set one. - calibrate_matches(): one verb composing features, fit, and apply. - calibrate(): evaluate a fitted filter on a labelled set; returns reliability, Brier score, log-loss, confusion matrix, and a threshold sweep. - Labelling helpers: sample_matches() stratification, plus export_for_labelling() / import_labels() for a CSV round-trip. - Tidymodels support: pass a parsnip spec or workflow to fit_filter() via joinery_recipe(). All tidymodels packages are optional; the glm path needs none. Changes in version 0.6.0 Phase 0.6: Diagnostics Verbs to answer four questions about a strategy and its results: will it work, did it work, why this pair, and where to look. - audit_strategy(): grade a strategy before matching. - summarise_matches(): overview of a dedup or candidate result, unified across backends. - explain_match(): per-token attribution of a single pair's score. - sample_matches(): draw pairs by mode (high, low, borderline, ambiguous, top-gap, random). - compare_stages(): per-stage coverage for multi-stage workflows. - Diagnostic plots: a family of pipe-composable tinyplot functions, one per view. - Recommendations: strategies and results surface inline advice from a signal-driven catalog, also available via recommendations(). Changes in version 0.5.0 Phase 0.5: Embedding-Based Matching Optional semantic matching that complements rather than replaces the token core. Use embeddings for fields where word-overlap fails (paraphrases, multilingual variants, fuzzy free-text descriptions) and combine them with token strategies via multi_stage_match(). New Features - embedding_strategy(): declarative strategy for embedding-based linkage, mirroring the ergonomics of search_strategy(). Specify one or more embedding columns, an optional block_by, an optional threshold, and an optional weights vector across embedding columns. - Cosine-similarity scoring between record-level embedding vectors, with optional pre-normalization so cosine reduces to a fast inner product at scoring time. Strategies expose a normalize flag for users who want to keep raw magnitudes. - Drop-in compatibility with the existing verbs: detect_duplicates(), search_candidates(), and extract_unmatched() all accept an Embedding_Strategy and return the standard joinery output schemas (duplicate_group / match_id, score, rank, original columns). - Multi-stage token + embedding workflows: multi_stage_match() accepts a sequence of mixed Search_Strategy and Embedding_Strategy objects, threading residuals between stages and stopping early when either side is exhausted. Useful pattern: cheap token stage first, then embedding stage on the residual. - block_by support for embeddings so cosine search runs within blocks (e.g. country, year bucket) instead of across the whole table. - Backend parity: full implementation on data.table, DuckDB, and tibble / data.frame, with the same call signatures across backends. DuckDB scales embedding search to large tables via the existing batch infrastructure. - Embedding generation via tidyllm (optional Suggests dependency): provider-agnostic helpers for Ollama, OpenAI, and other tidyllm-supported backends, so users can move from raw text to a matchable embedding column without leaving R. - Embedding-aware diagnostics groundwork: strategy-class dispatch in place so Phase 0.6 diagnostics can specialise to embedding strategies without API churn. Bug fixes - DuckDB block_by SQL bug fixed. - DuckDB lazy-query bug in multi-stage match fixed. Changes in version 0.4.0 Phase 0.4: Stability & Test-Quality Hardening A maintenance release with no new user-facing features. The goal was to harden the test suite and close coverage gaps before resuming feature work on embeddings and diagnostics. - methods_duckdb.R coverage raised from 34% to 90%; full behavioural parity with the data.table backend now exercised by tests. - embedding_methods_* coverage raised to 95%+ on both data.table and DuckDB backends. - Small-table batch_duckdb brittleness diagnosed and fixed (see notes/batch_duckdb_brittleness.md). User-facing impact: small inputs no longer hit pathological batching behaviour. - Total package coverage: 87.25%. Remaining low-coverage files are intentional: S7 dispatch boilerplate, interactive-only progress paths, and live-embedding paths reserved for local_tests/. Changes in version 0.3.1 - Fix batch_duckdb small-table brittleness. Changes in version 0.3.0 Phase 3: SearchEngine Heuristics This release implements advanced matching heuristics that significantly improve accuracy and robustness. New Features - rIP Smoothing: Four smoothing methods for token weights: - smoothing(method = "log"): Log transformation - smoothing(method = "softmax", temperature = 1.0): Softmax with temperature - smoothing(method = "offset", alpha = 0.1): Additive smoothing - smoothing(method = "none"): No smoothing (default) - Containment: Control maximum matches per record: - max_candidates parameter limits top-N matches - Prevents one-token overmatching - Works with threshold filtering - Feedback Weighting: Penalize low token overlap: - feedback_strength parameter (0-1) controls intensity - Reduces noise in partial matches - Rewards comprehensive token overlap DuckDB Backend - Unified .score_pairs_sql() helper consolidates scoring logic - All Phase 3 features supported in DuckDB backend - Used by both detect_duplicates() and search_candidates() Backend Improvements - Both data.table and DuckDB backends support all Phase 3 features - Full test coverage for all smoothing, containment, and feedback methods - 454 tests passing Changes in version 0.2.0 Phase 2: DuckDB Backend - Full DuckDB backend implementation - Scalable processing of datasets up to 50M rows - Batch-based processing with R preprocessing pipeline - Feature parity between data.table and DuckDB backends - All core generics working on both backends Changes in version 0.1.0 - Initial release - data.table backend - Token-based record linkage - Basic preprocessing pipeline - S7 class system