Package: vennabers 1.0.0

René-M. Kruse

vennabers: Venn-Abers Classification and Regression Calibration

Calibrates the predictions of classification and regression models with Venn-Abers predictors. A Venn-Abers predictor turns the score of any model into a pair of probabilities that contains a calibrated value when the observations are independent and identically distributed (Vovk and Petej, 2014, <doi:10.48550/arXiv.1211.0025>); Inductive and Cross Venn-Abers predictors make this practical for large data sets (Vovk, Petej and Fedorova, 2015, <doi:10.48550/arXiv.1511.00213>). The package serves analysts and researchers who need calibrated probabilities or prediction bounds for binary classification, multiclass classification by one-vs-one coupling (Manokhin, 2017, <https://proceedings.mlr.press/v60/manokhin17a.html>) or one-vs-rest, and regression (Petej and Vovk, 2026, <doi:10.48550/arXiv.2605.06646>). Predictions of an existing model can be calibrated directly, or optional 'tidymodels' workflows fit the model, create the calibration splits and tune the complete procedure. Diagnostics include the Brier score, log loss, plots of fitted calibration maps and reliability diagrams with the CORP (consistent, optimally binned, reproducible and pool-adjacent-violators based) method of Dimitriadis, Gneiting and Jordan (2021) <doi:10.1073/pnas.2016191118>.

Authors:René-M. Kruse [aut, cre, cph]

vennabers_1.0.0.tar.gz
vennabers_1.0.0.zip(r-4.7-any)vennabers_1.0.0.zip(r-4.6-any)
vennabers_1.0.0.tar.gz(r-4.7-any)vennabers_1.0.0.tar.gz(r-4.6-any)
vennabers_1.0.0.tgz(r-4.6-emscripten)
manual.pdf |manual.html✨
DESCRIPTION |NEWS
card.svg |card.png
vennabers/json (API)

# Install 'vennabers' in R:
install.packages('vennabers', repos = c('https://cran.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/rmkruse/vennabers/issues

Pkgdown/docs site:https://rmkruse.github.io

On CRAN:

Conda:

4.77 score 118 scripts 25 exports 0 dependencies

Last updated from:9fde43b6ae. Checks:6 OK. Indexed: yes.

TargetResultTimeFilesSyslog
linux-develOK390
source / vignettesOK231
linux-releaseOK328
windows-develOK276
windows-releaseOK427
wasm-releaseOK103

Exports:brier_scorecalibrate_workflowcalibrated_workflowcalibration_plotcorp_decomposition_plotcorp_plotcross_venn_aberscross_venn_abers_regressionfit_vennabersfit_vennabers_proxylocal_calibration_splitslog_lossmodel_adapterpredict_vennabers_classpredict_vennabers_numericpredict_vennabers_probregression_plotvenn_abersvenn_abers_decodevenn_abers_detailsvenn_abers_encodevenn_abers_one_hotvenn_abers_regressionvenn_abers_schemavenn_abers_tuning

Dependencies:

Calibrating supplied predictions
Binary Inductive calibration | Binary Cross calibration: retain the fold models | Multiclass: one-vs-one and one-vs-rest | Regression: moderated Inductive and Cross calibration | Keeping the complete predictor

Last update: 2026-11-07
Started: 2026-11-07

Diagnose and assess calibration
Fit diagnoses: the stored state | Assessment graphics: held-out pairs | CORP reliability diagrams | CORP bands are not Venn-Abers bounds | Brier decomposition | Regression predictions | Further reading

Last update: 2026-11-07
Started: 2026-11-07

Example scripts: Grid and Bayes selection
Choose an example | Running an example | Binary Grid selection | Shared multiclass Grid selection | Shared multiclass Bayes selection | Pairwise OvO Grid selection | Pairwise OvO Bayes selection | Regression Grid selection | Regression Bayes selection | Grid selection with postprocessing | Bayes selection with postprocessing

Last update: 2026-11-07
Started: 2026-11-07

Example scripts: learned postprocessing
Choose an example | Running an example | Binary Sigmoid holdouts | Binary Tailor holdouts | Shared multiclass Tailor holdouts | Regression Tailor holdouts | Regression isotonic holdouts | Binary postprocessing ensembles | Shared multiclass postprocessing ensembles | Regression postprocessing ensembles | Pairwise OvO postprocessing | Full-model postprocessing compositions

Last update: 2026-11-07
Started: 2026-11-07

Example scripts: manual calibration
Choose an example | Running an example | Binary precision, loss and event selection | Binary Cross calibration with retained models | Inductive multiclass one-vs-one | Inductive multiclass one-vs-rest | Cross multiclass calibration | One-hot class decisions | Inductive regression | Cross regression | Fit diagnoses and assessment graphics

Last update: 2026-11-07
Started: 2026-11-07

Example scripts: training and frozen workflows
Choose an example | Running an example | Training binary and shared multiclass workflows | Encoding binary names | Encoding multiclass and regression names | Reference-style data roles | Independent Inductive training and calibration sizes | Own fit and predict functions | Fixed engine options and internal validation | Binary full-model transfer | Shared multiclass full-model transfer | Regression full-model transfer | Pairwise Inductive OvO training | Pairwise Cross OvO training | Calibrating a frozen binary workflow | Calibrating frozen multiclass and regression workflows | Keeping the original schema for frozen adoption

Last update: 2026-11-07
Started: 2026-11-07

Training and freezing calibrated workflows
Choose a learning path | A complete binary workflow | Cross, matched models and full-model transfer | Multiclass: calibration method versus model training | Regression workflows | Frozen workflows: calibrate without retraining | Learning a postprocessor before Venn-Abers | Engine options and model-internal validation | Names, assessment and next steps

Last update: 2026-11-07
Started: 2026-11-07

Tuning and assessing the complete procedure
Data roles and search strategies | Joint Grid selection | Two-stage Grid selection | A small Bayes model search | Outer resampling and final assessment | Extending the same pattern

Last update: 2026-11-07
Started: 2026-11-07

Venn-Abers methods and their guarantees
Calibration as a problem | Inductive Venn-Abers | What the validity guarantee says | One probability from two | Cross Venn-Abers | Multiclass calibration | Regression | In practice | References

Last update: 2026-11-07
Started: 2026-11-07

Venn-Abers, Platt and isotonic calibration compared
Data and their roles | Three calibration maps | Results | A contrast: a well-specified model | What this does and does not show | References

Last update: 2026-11-07
Started: 2026-11-07

Readme and manuals

Help Manual

Help pageTopics
Venn-Abers Classification and Regression Calibrationvennabers-package vennabers
Score Classification Probabilitiesbrier_score log_loss
Calibrate a Frozen Workflowcalibrate_workflow predict.vennabers_frozen print.vennabers_frozen
Train a Calibrated Classification or Regression Workflowcalibrated_workflow predict.vennabers_workflow predict._vennabers_fit
Plot Classification Probability Calibrationcalibration_plot
Assess Calibration with CORP Reliability Diagramscorp_decomposition_plot corp_plot
Fit a Binary or Multiclass Cross Venn-Abers Calibratorcross_venn_abers
Manual Cross Venn-Abers Regression with Matched Fold Modelscross_venn_abers_regression predict.cross_venn_abers_regression
Resolve Prepared Local Calibration Splitslocal_calibration_splits
Own Fit and Predict Functions as a Workflow Modelmodel_adapter
Plot the Calibration Map of a Fitted Venn-Abers Calibratorplot.cross_venn_abers plot.cross_venn_abers_regression plot.vennabers_fit plot.venn_abers plot.venn_abers_regression
Predict with Matched Fold-Model Probabilitiespredict.cross_venn_abers
Predict with a Venn-Abers Calibratorpredict.venn_abers
Print a Cross Venn-Abers Calibratorprint.cross_venn_abers
Print a Venn-Abers Calibratorprint.venn_abers
Assess Regression Predictions on Held-Out Pairsregression_plot
Fit an Inductive Venn-Abers Calibratorvenn_abers
Inspect Calibrated Workflow Predictionsvenn_abers_details
One-Hot Matrices of Class Decisionsvenn_abers_one_hot
Moderated Inductive Venn-Abers Regressionpredict.venn_abers_regression venn_abers_regression
Reversible Names for Calibrated Workflowsvenn_abers_decode venn_abers_encode venn_abers_schema
Inspect Classification or Regression Grid or Bayes Selectionvenn_abers_tuning
Find and Run Venn-Abers Example Scriptsvennabers_examples
Postprocessing Before Workflow Calibrationworkflow_postprocessing
Calibrated Workflow Training Referenceworkflow_training