Title: | Shadow-Test Approach to Computerized Adaptive Testing |
---|---|
Description: | As an advanced approach to computerized adaptive testing (CAT), shadow testing (van der Linden(2005) <doi:10.1007/0-387-29054-0>) dynamically assembles entire shadow tests as a part of selecting items throughout the testing process. Selecting items from shadow tests guarantees the compliance of all content constraints defined by the blueprint. 'RSCAT' is an R package for the shadow-test approach to CAT. The objective of 'RSCAT' is twofold: 1) Enhancing the effectiveness of shadow-test CAT simulation; 2) Contributing to the academic and scientific community for CAT research. RSCAT is currently designed for dichotomous items based on the three-parameter logistic (3PL) model. |
Authors: | Bingnan Jiang [aut, cre], ACT, Inc. [cph] |
Maintainer: | Bingnan Jiang <[email protected]> |
License: | CC BY-NC 4.0 |
Version: | 1.1.3 |
Built: | 2024-11-15 06:46:15 UTC |
Source: | CRAN |
An S4 Class to represent parameters of the CAT configuration.
solverConfig
an instance of the S4 class SolverConfig
for the
MIP solver configuration.
initialTheta
the initial ability theta value.
scalingConstant
the constraint to scale a discrimination coefficient. estimated with the logistic model to the normal metric.
itemSelectionMethod
a character string specifying the item selection method.
scoreMethodConfig
a rJava jobjRef
object for CAT scoring method
configuration. It is generated by the function scoreMethodConfig
.
exposureControlType
a character string specifying the exposure control type. "None" to disable exposure control, "Item" for item-level exposure control, and "Passage" for passage-level exposure control.
exposureControlRate
an integer value specifying the exposure goal rate.
lValue
a non-negative integer specifying the nuumber of items to be randomlized.
An S4 class to represent expected A posteriori (EAP) scoring algorithm configuration.
An instance of this S4 class can be applied to the generic function
scoreMethodConfig
to create an Java object for scoring method
configuration.
numQuad
a positive integer specifying the number of quadrature points
minQuad
a numeric value specifying the minimum quadrature point
maxQuad
a numeric value specifying the maximum quadrature point
priorDistType
a character string specifying the prior distribution of ability. "Normal" for Normal distribution and "Uniform" for uniform distribution.
distParams
a numeric vector specifying parameters of the prior distribution. (mean, sd) for the Normal distribution, (a, b) for the uniform distribution.
Launches the shiny app to confiure and run CAT simulations.
launchApp()
launchApp()
if(interactive()){ launchApp() }
if(interactive()){ launchApp() }
Creates a simulation result CSV file.
result2CSV(simResult, file)
result2CSV(simResult, file)
simResult |
an instance of S4 class |
file |
a writable connection or a character string naming the file to write to. |
runSim
runs CAT simulations based on the provided configurations and
returns the simulation result.
runSim(catConfig, testConfig, simConfig)
runSim(catConfig, testConfig, simConfig)
catConfig |
an instance of the S4 class |
testConfig |
an instance of the S4 class |
simConfig |
an instance of the S4 class |
This function calls the Java helper method org.act.util.RHelper.runSim
via rJava to exceute CAT simulation.
the simulation result in the instance of SimResult
.
if(interactive()){ ## Defines item attributes types itemNumericColumn <- c(FALSE, FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, FALSE, FALSE, FALSE, TRUE, FALSE, TRUE, FALSE, FALSE,FALSE) ## Specifies the item pool file itemPoolCSVPath <- system.file("extdata", "itempool10Items.csv", package = "RSCAT") ## Specifies the constraint table file constraintCSVPath <- system.file("extdata", "constraintSet1.csv", package = "RSCAT") ## Configures solver parameters solverConfig <- SolverConfig(absGap = 1e-3, relGap = 1e-3, intTol = 1e-6) ## Configures the EAP estimaition eapConfig <- EAPConfig(numQuad = 6L, minQuad = -2, maxQuad = 2, priorDistType = "Normal", distParams = c(0,1)) ## Configures CAT catConfig <- CATConfig(solverConfig = solverConfig, scoreMethodConfig = scoreMethodConfig(eapConfig), lValue = 3L) ## Configures test specifications testConfig <- TestConfig(testConfigID = "Test1", testLength = 6L, itempoolPath = itemPoolCSVPath, constraintPath = constraintCSVPath, itemNumericColumn = itemNumericColumn) ## Configures the simulation simConfig <- SimConfig(simID = "Sim1", numExaminees = 8L) ## Runs CAT simulation simResult <- runSim(catConfig, testConfig, simConfig) }
if(interactive()){ ## Defines item attributes types itemNumericColumn <- c(FALSE, FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, FALSE, FALSE, FALSE, TRUE, FALSE, TRUE, FALSE, FALSE,FALSE) ## Specifies the item pool file itemPoolCSVPath <- system.file("extdata", "itempool10Items.csv", package = "RSCAT") ## Specifies the constraint table file constraintCSVPath <- system.file("extdata", "constraintSet1.csv", package = "RSCAT") ## Configures solver parameters solverConfig <- SolverConfig(absGap = 1e-3, relGap = 1e-3, intTol = 1e-6) ## Configures the EAP estimaition eapConfig <- EAPConfig(numQuad = 6L, minQuad = -2, maxQuad = 2, priorDistType = "Normal", distParams = c(0,1)) ## Configures CAT catConfig <- CATConfig(solverConfig = solverConfig, scoreMethodConfig = scoreMethodConfig(eapConfig), lValue = 3L) ## Configures test specifications testConfig <- TestConfig(testConfigID = "Test1", testLength = 6L, itempoolPath = itemPoolCSVPath, constraintPath = constraintCSVPath, itemNumericColumn = itemNumericColumn) ## Configures the simulation simConfig <- SimConfig(simID = "Sim1", numExaminees = 8L) ## Runs CAT simulation simResult <- runSim(catConfig, testConfig, simConfig) }
This is a gneric function to create a scoring method configuration from a speicfic estimation algorithm configuration.
scoreMethodConfig(object) ## S4 method for signature 'EAPConfig' scoreMethodConfig(object)
scoreMethodConfig(object) ## S4 method for signature 'EAPConfig' scoreMethodConfig(object)
object |
an S4 object for the estimation algorithm configuration |
the object of scoring method configuration which is an instance of
org/act/rscat/cat/ScoringMethodConfig
if(interactive()){ eapConfig <- EAPConfig(numQuad = 6L, minQuad = -2, maxQuad = 2, priorDistType = "Normal", distParams = c(0,1)) scoreMethodConfig <- scoreMethodConfig(eapConfig) }
if(interactive()){ eapConfig <- EAPConfig(numQuad = 6L, minQuad = -2, maxQuad = 2, priorDistType = "Normal", distParams = c(0,1)) scoreMethodConfig <- scoreMethodConfig(eapConfig) }
Defines server logic to configure and run CAT simulations.
shinyAppServer(input, output)
shinyAppServer(input, output)
input |
an object that stores the current values of all of the widgets in the app. |
output |
an object that stores instructions for building the R objects in the app. |
Defines UI for CAT simulations.
shinyAppUI
shinyAppUI
An object of class shiny.tag.list
(inherits from list
) of length 3.
An S4 class to represent CAT simulation configuraiton.
simID
a character string as the identifier of the CAT simulation.
numExaminees
a positive integer specifying the number of simulated examinees.
trueThetaDistType
a character string specifying the distribution of true ability of simulated examinees. "Normal" for the Normal distribution and "Uniform" for the uniform distribution.
trueThetaDistParams
a numeric vector spcifying parameters of the prior distribuution. (mean, sd) for the Normal distribution, (a, b) for the uniform distribution.
An S4 class to represent CAT simulation results.
numExaminees
a positive integer representing the number of simulated examinees.
trueThetas
a numeric vector representing the true theta values of simulated examinees.
finalThetas
a numeric vector representing the final theta estimates of simulated examinees.
finalThetaSEs
a numeric vector representing the final theta estimate standard errors (SEs) of simulated examinees.
estThetas
a list of length numExaminees
. Each element of the
list is a numeric vector representing theta estimate at adaptive
stages for the simulated examinee.
estThetaSEs
a list of length numExaminees
. Each element of the
list is a numeric vector representing theta estimate standard error (SE)
at adaptive stages for the simulated examinee.
scores
a list of length numExaminees
. Each element of the
list is a numeric vector representing scores at adaptive stages for the
simulated examinee. 0 for an incorrect repsonse and 1 for
a correct respone.
itemsAdministered
a list of length numExaminees
. Each element
of the list is a character vector representing identifiers of adminsitered
items at adaptive stages for the simulated examinee.
shadowTests
a list of length numExaminees
. Each element of
the list is also a list representing the shadow test assembled at each
adaptive stage.
engineTime
a list of length numExaminees
. Each element of
the list is a numeric vector representing the engine time at each adaptive step.
the engine time includes time consumed by CAT algorithms and shadow test assembly.
An S4 Class to represent parameters of the MIP solver configuration.
absGap
the absolute gap target to terminate the MIP solving.
relGap
the relative gap target to terminate the MIP solving.
intTol
the integer tolerance for the MIP solving. if the solution x is between -intTol and intTol, x <= 0 is true if the value of x is at most intTol. x > 0 is fulfilled if x > intTol.
Generates CAT simulation summary
## S4 method for signature 'SimResult' summary(object)
## S4 method for signature 'SimResult' summary(object)
object |
an object of |
An S4 class to represent test blueprint and specification.
testConfigID
a character string as the identifier of the test configuration.
testLength
a positive integer specifying the test length.
itempoolPath
a character string specifying the location of the item pool csv file.
passagepoolPath
a character string specifying the location of the passage pool csv file.
constraintPath
a character string specifying the location of the constraint csv file.
itemNumericColumn
a boolean vector indicating whether item attribute columns in the item pool table are numeric or not.
passageNumericColumn
a boolean vector indicating whether passage attribute columns in the passage pool table are numeric or not.
enableEnemyItem
a boolean indicator to specify if enemy item constraints defined by in the item pool is enabled or not.
numPassageLB
an integer specifying the minimum number of passages in the test.
numPassageUB
an integer specifying the maximum number of passages in the test.
numItemPerPassageLB
an integer specifying the minimum number of items in a passages in the test.
numItemPerPassageUB
an integer specifying the maximum number of items in a passages in the test.