Package: multiRL 0.4.5

YuKi

multiRL: Reinforcement Learning Tools for Multi-Armed Bandit

A flexible general-purpose toolbox for implementing Rescorla-Wagner models in multi-armed bandit tasks. As the successor and functional extension of the 'binaryRL' package, 'multiRL' modularizes the Markov Decision Process (MDP) into six core components. This framework enables users to construct custom models via intuitive if-else syntax and define latent learning rules for agents. For parameter estimation, it provides both likelihood-based inference (MLE and MAP) and simulation-based inference (ABC and RNN), with full support for parallel processing across subjects. The workflow is highly standardized, featuring four main functions that strictly follow the four-step protocol (and ten rules) proposed by Wilson & Collins (2019) <doi:10.7554/eLife.49547>. Beyond the three built-in models (TD, RSTD, and Utility), users can easily derive new variants by declaring which variables are treated as free parameters.

Authors:YuKi [aut, cre], Xinyu [aut]

multiRL_0.4.5.tar.gz
multiRL_0.4.5.tar.gz(r-4.7-arm64)multiRL_0.4.5.tar.gz(r-4.7-x86_64)multiRL_0.4.5.tar.gz(r-4.6-arm64)multiRL_0.4.5.tar.gz(r-4.6-x86_64)
multiRL_0.4.5.tgz(r-4.6-emscripten)
manual.pdf |manual.html
card.svg |card.png
multiRL/json (API)

# Install 'multiRL' in R:
install.packages('multiRL', repos = c('https://cran.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/yuki-961004/multirl/issues

Pkgdown/docs site:https://yuki-961004.github.io

Uses libs:
  • c++– GNU Standard C++ Library v3
Datasets:
  • MAB - Simulated Multi-Arm Bandit Dataset
  • TAB - Group 2 from Mason et al.
  • WMT - Data from Collins and Frank

On CRAN:

Conda:

cpp

1.48 score 2 scripts 158 downloads 31 exports 31 dependencies

Last updated from:6810622e23. Checks:6 OK. Indexed: no.

TargetResultTimeFilesSyslog
linux-devel-arm64OK144
linux-devel-x86_64OK150
source / vignettesOK209
linux-release-arm64OK159
linux-release-x86_64OK167
wasm-releaseOK133

Exports:engine_ABCengine_RNNengine_RNN3estimate_0_ENVestimate_1_LBIestimate_1_MAPestimate_1_MLEestimate_2_ABCestimate_2_RNNestimate_2_SBIestimation_methodsfit_pfunc_alphafunc_betafunc_deltafunc_epsilonfunc_gammafunc_zetaprocess_1_inputprocess_2_behruleprocess_3_recordprocess_4_output_cppprocess_4_output_rprocess_5_metricrcv_drpl_eRSTDrun_msummaryTDUtility

Dependencies:clicodetoolscpp11digestdoFuturedoRNGfarverforeachfuturefuture.applyggplot2globalsgluegtableisobanditeratorslabelinglifecyclelistenvparallellyprogressrR6RColorBrewerRcpprlangrngtoolsS7scalesvctrsviridisLitewithr

Readme and manuals

Help Manual

Help pageTopics
Algorithm Packages (MLE, MAP)algorithm
Behavior Rulesbehrule
Column Namescolnames
Controls of Estimation Methodscontrol
Dataset Structuredata
The Engine of Approximate Bayesian Computation (ABC)engine_ABC
The Engine of Recurrent Neural Network (RNN)engine_RNN engine_RNN3
Estimate Methodsestimate
Tool for Generating an Environment for Modelsestimate_0_ENV
Likelihood-Based Inference (LBI)estimate_1_LBI
Estimation Method: Maximum A Posteriori (MAP)estimate_1_MAP
Estimation Method: Maximum Likelihood Estimation (MLE)estimate_1_MLE
Estimation Method: Approximate Bayesian Computation (ABC)estimate_2_ABC
Estimation Method: Recurrent Neural Network (RNN)estimate_2_RNN
Simulated-Based Inference (SBI)estimate_2_SBI
Estimate Methodsestimation_methods
Step 3: Optimizing parameters to fit real datafit_p
Function: Learning Ratefunc_alpha
Function: Probabilityfunc_beta
Function: Biasfunc_delta
Function: Exploration or Exploitationfunc_epsilon
Function: Utilityfunc_gamma
Function: Decay Ratefunc_zeta
Core Functionsfuncs
Layers and Loss Functions (RNN)layer
Simulated Multi-Arm Bandit DatasetMAB
Model Parametersparams
plot.multiRL.replayplot.multiRL.replay
Policy of Agentpolicy
Density and Random Functionpriors
multiRL.inputprocess_1_input
multiRL.behruleprocess_2_behrule
multiRL.recordprocess_3_record
multiRL.outputprocess_4_output_cpp
multiRL.outputprocess_4_output_r
multiRL.metricprocess_5_metric
Step 2: Generating fake data for parameter and model recoveryrcv_d
Dimension Reduction Methods (ABC)reduction
Step 4: Replaying the experiment with optimal parametersrpl_e
Risk Sensitive ModelRSTD
Step 1: Building reinforcement learning modelrun_m
Settings of Modelsettings
summarysummary,multiRL.model-method
Cognitive Processing Systemsystem
Group 2 from Mason et al. (2024)TAB
Temporal Differences ModelTD
Utility ModelUtility
Data from Collins and Frank (2012)WMT