Package: MDP2 2.2.2.0

Lars Relund Nielsen

MDP2: Markov Decision Processes (MDPs)

Create and optimize (semi) MDPs with discrete time steps and state space. Both hierarchical and ordinary-traditional MDPs can be modeled.

Authors:Lars Relund Nielsen [aut, cre]

MDP2_2.2.2.0.tar.gz
MDP2_2.2.2.0.tar.gz(r-4.7-arm64)MDP2_2.2.2.0.tar.gz(r-4.7-x86_64)MDP2_2.2.2.0.tar.gz(r-4.6-arm64)MDP2_2.2.2.0.tar.gz(r-4.6-x86_64)
manual.pdf |manual.html✨
card.svg |card.png
MDP2/json (API)
NEWS

# Install 'MDP2' in R:
install.packages('MDP2', repos = c('https://cran.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/relund/mdp/issues

Pkgdown/docs site:https://relund.github.io

Uses libs:
  • openblas– Optimized BLAS
  • c++– GNU Standard C++ Library v3

On CRAN:

Conda:

openblascpp

4.01 score 41 scripts 4 downloads 23 exports 24 dependencies

Last updated from:06cec2781a. Checks:5 OK, 1 FAIL. Indexed: yes.

TargetResultTimeFilesSyslog
linux-devel-arm64OK170
linux-devel-x86_64OK165
source / vignettesOK255
linux-release-arm64OK176
linux-release-x86_64OK180
wasm-releaseFAIL129

Exports:binaryActionWriterbinaryMDPWriterconvertBinary2HMPconvertHMP2BinarygetBinInfoActionsgetBinInfoStatesgetHypergraphgetInfogetPolicygetRPOgetSteadyStatePrgetWIdxhmpMDPWriterloadMDPmemoryMDPWriterplotHypergraphrandomHMDPrunCalcWeightsrunPolicyIteAverunPolicyIteDiscountrunValueItesaveMDPsetPolicy

Dependencies:clicpp11diagramdplyrgenericsgluelifecyclemagrittrpillarpkgconfigpurrrR6RcppRcppArmadillorlangshapestringistringrtibbletidyrtidyselectutf8vctrswithr

Building an MDP model

Lars Relund [email protected]

Rendered frombuilding.Rmdusingknitr::rmarkdownon Jun 12 2026.

Last update: 2026-06-12
Started: 2023-01-31

Solving a finite-horizon semi-MDP

Lars Relund [email protected]

Rendered fromfinite-mdp.Rmdusingknitr::rmarkdownon Jun 12 2026.

Last update: 2026-06-12
Started: 2023-01-31

An infinite-horizon HMDP

Lars Relund [email protected]

Rendered frominfinite-hmdp.Rmdusingknitr::rmarkdownon Jun 12 2026.

Last update: 2023-01-31
Started: 2023-01-31

Solving an infinite-horizon semi-MDP

Lars Relund [email protected]

Rendered frominfinite-mdp.Rmdusingknitr::rmarkdownon Jun 12 2026.

Last update: 2026-06-12
Started: 2023-01-31

The MDP2 package

Lars Relund [email protected]

Rendered frommdp2.Rmdusingknitr::rmarkdownon Jun 12 2026.

Last update: 2023-01-31
Started: 2023-01-31

Readme and manuals

Help Manual

Help pageTopics
Function for writing actions of a HMDP model to binary files. The function defines sub-functions which can be used to define actions saved in a set of binary files. It is assumed that the states have been defined using 'binaryMDPWriter' and that the id of the states is known (can be retrieved using e.g. 'stateIdxDf').binaryActionWriter
Function for writing an HMDP model to binary files. The function defines sub-functions which can be used to define an HMDP model saved in a set of binary files.binaryMDPWriter
Convert a HMDP model stored in binary format to a 'hmp' (XML) file. The function simply parse the binary files and create 'hmp' files using the 'hmpMDPWriter()'.convertBinary2HMP
Convert a HMDP model stored in a hmp (xml) file to binary file format.convertHMP2Binary
Info about the actions in the HMDP model under consideration.getBinInfoActions
Info about the states in the binary files of the HMDP model under consideration.getBinInfoStates
Return the (parts of) state-expanded hypergraphgetHypergraph
Information about the MDPgetInfo
Get parts of the optimal policy.getPolicy
Calculate the retention pay-off (RPO) or opportunity cost for some states.getRPO
Calculate the steady state transition probabilities for the founder process (level 0).getSteadyStatePr
Return the index of a weight in the model. Note that index always start from zero (C++ style), i.e. the first weight, the first state at a stage etc has index 0.getWIdx
Function for writing an HMDP model to a hmp file (XML). The function define sub-functions which can be used to define an HMDP model stored in a hmp file.hmpMDPWriter
Load the HMDP model defined in the binary files. The model are created in memory using the external C++ library.loadMDP
Function for building an HMDP model directly in memory.memoryMDPWriter
Plot the state-expanded hypergraph of the MDP.plot.HMDP
Plot parts of the state expanded hypergraph.plotHypergraph
Generate a "random" HMDP stored in a set of binary files.randomHMDP
Calculate weights based on current policy. Normally run after an optimal policy has been found.runCalcWeights
Perform policy iteration using the average expected-weight Bellman operator on the MDP.runPolicyIteAve
Perform policy iteration using the discounted expected-weight Bellman operator on the MDP.runPolicyIteDiscount
Perform value iteration on the MDP.runValueIte
Save the MDP to binary filessaveMDP
Modify the current policy by setting policy action of states.setPolicy