Package: validateIt 1.2.1

Luwei Ying

validateIt: Validating Topic Coherence and Topic Labels

By creating crowd-sourcing tasks that can be easily posted and results retrieved using Amazon's Mechanical Turk (MTurk) API, researchers can use this solution to validate the quality of topics obtained from unsupervised or semi-supervised learning methods, and the relevance of topic labels assigned. This helps ensure that the topic modeling results are accurate and useful for research purposes. See Ying and others (2022) <doi:10.1101/2023.05.02.538599>. For more information, please visit <https://github.com/Triads-Developer/Topic_Model_Validation>.

Authors:Luwei Ying [aut, cre], Jacob Montgomery [aut], Brandon Stewart [aut]

validateIt_1.2.1.tar.gz
validateIt_1.2.1.tar.gz(r-4.5-noble)validateIt_1.2.1.tar.gz(r-4.4-noble)
validateIt_1.2.1.tgz(r-4.4-emscripten)validateIt_1.2.1.tgz(r-4.3-emscripten)
validateIt.pdf |validateIt.html✨
validateIt/json (API)

# Install 'validateIt' in R:

install.packages('validateIt', repos = c('https://cran.r-universe.dev', 'https://cloud.r-project.org'))

Datasets:

R4WSItasktest - Example R4WSI0 Tasks
allR4WSItasktest - Example R4WSI Tasks with Regular and Gold-Standard Tasks
goldR4WSItest - Example Gold-Standard R4WSI0 Tasks
heldouttest - An Example Heldout Test Set
keypostedtest - Example Answer Keys
masstest - An Example of the Combined Mass for Words with the Same Roots
modtest - An Example Topic Model
record - Reform tasks to facilitate sending to Mturk
resultstest - Example Results Retrieved from Mturk
stmPreptest - An Example Object of Prepared Documents

On CRAN:

This package does not link to any Github/Gitlab/R-forge repository. No issue tracker or development information is available.

1.70 score 2 scripts 152 downloads 17 exports 22 dependencies

Last updated 2 years agofrom:ced3d803eb. Checks:1 OK, 1 NOTE. Indexed: yes.

Target	Result	Latest binary
Doc / Vignettes	OK	Feb 20 2025
R-4.5-linux	NOTE	Feb 20 2025

Exports::=.data as_label as_name checkAgree combMass enquo enquos evalResults getResults mixGold pickLabel plotResults record sendTasks validateLabel validateTopic

Dependencies:BH cli curl here jsonlite lattice Matrix NLP png pyMTurkR rappdirs Rcpp RcppTOML reticulate rlang rprojroot slam SnowballC tm withr XML xml2

Help page	Topics
Example R4WSI Tasks with Regular and Gold-Standard Tasks	allR4WSItasktest
Check Agreement Rate between Identical Trails	checkAgree
Combine the mass of words with the same root	combMass
Evaluate results	evalResults
Get results from Mturk	getResults
Example Gold-Standard R4WSI0 Tasks	goldR4WSItest
An Example Heldout Test Set	heldouttest
Example Answer Keys	keypostedtest
An Example of the Combined Mass for Words with the Same Roots	masstest
Mix the gold-standard tasks with the tasks need to be validated	mixGold
An Example Topic Model	modtest
Pick the optimal label from candidate labels	pickLabel
Plot results	plotResults
Example R4WSI0 Tasks	R4WSItasktest
Reform tasks to facilitate sending to Mturk	record
Example Local Record of the R4WSI Tasks	recordtest
Example Results Retrieved from Mturk	resultstest
Send prepared task to Mturk and record the API-returned HIT ids.	sendTasks
An Example Object of Prepared Documents	stmPreptest
Topic_Model_Validation Repository Overview	Topic_Model_Validation_Overview
Create validation tasks for labels assigned to the topics in the topic model of choice.	validateLabel
Create validation tasks for topic model selection	validateTopic

Package: validateIt 1.2.1

validateIt: Validating Topic Coherence and Topic Labels

Citation

Readme and manuals

Help Manual

Usage by other packages (reverse dependencies)