NEWS
cNORM 3.4.0 (2024-11-04)
Date: 04.11.2024
Changes:
- new function 'compare' added to visually compare models
- inclusion of 10-fold subsampling and averaging of model coefficients in model estimation
which produces more stable results
cNORM 3.3.1 (2024-10-16)
Date: 16.10.2024
Changes:
- added check on score data in cnorm.betabinomial
- switch to rankByGroup in cnorm in case age variable is plausibly grouping variable
- bug in subtitles of plotPercentileSeries fixed
- adjusted output and model recommendations in cnorm.cv
- fixed legend in plotDensity()
- bug in predictRaw fixed, which caused plotDerivative to plot wrong results
- improved initial starting and control parameters in cnorm.betabinomial2
- bugs in diagnostics.betabinomial fixed
- vignettes revised
cNORM 3.3.0 (2024-08-26)
Date: 2024.08.26
Changes:
- checkConsistency performance improvement; now runs 100 times faster
- visualization improvement in 'plotDerivative'
- bug in subheadline of plotPercentileSeries fixed
- reduced number of messages
- Starting work on inclusion of regularized Taylor models. Since simulation
studies with glmnet did not show improvement, we generate much more models
and preselect those, which pass an initial consistency check.
Now, the a consistent model with the highest R2 is selected. R^2 and terms
can of course still be specified as usual.
- plotSubset improved: now indicates, which models did not pass the initial
consistency check via empty circles
- Plots improved generally, now prints formula and statistical indicators
in Greek letters and R^2 with the 2 uppercase
cNORM 3.2.0 (2024-08-17)
Date: 2024.08.17
Changes:
- Parametric modelling with beta binomial functions now fully implemented
- S3 functions predict, plot and summary added for bet a binomial models
- Fixed input validation in getNormScoreSE
- Started intensive work on regularization in Taylor models (planned for v4.0)
- Vignette on modelling with beta binomial distribution
- Transition from lattice to ggplot2
- Code on covariates removed from the complete package
- cNORM-Demo vignette revised
- code simplification in plotting functions, parameters removed
- new parameter to plot raw scores in plotPercentiles (default FALSE)
- 'buildCnormObject' function added to help with compatibility (joins data and model
to cnorm object)
- cNORM.GUI() updated
- datasets life, mortality and EPM removed
- performance optimization
cNORM 3.1.0 (2024-07-19)
Date: 2024.07.19
Changes:
- Added parametric continuous norming with beta binomial family and new functions
betaCoefficients, betaTable, betaByGroup and betaContinuous; beware - still experimental (Jan. 2024)
- Fixed bug in bestModels function when predictors are specified
cNORM 3.0.4 (2023-10-08)
Date: 2023.10.08
Changes:
- Added warning in 'cnorm' in case, both age and group is specified
- extended cnorm.cv for use of weights and sliding window
- added silent option to several functions to reduced the number of messages
- revised function documentation
cNORM 3.0.3 (2023-05-22)
Date: 2023.05.22
Changes:
- fixed regression bug in the internal predictNormByRoots-function for R4.3.0
- added new references
- new results in printSubset and plotSubset: F-tests on consecutive models
- internal improvements in calcPolyInLBase2 for retrieving regression function coefficients at specific
age. This speeds up norm score retrieval by 40%, leading to vast performance improvements in
large datasets and in cross validation by cnorm.cv
- Added WPS publisher as a funder. WPS helped financing the weighting procedure for post stratification
based on iterative proportional fitting ("Raking")
- citeEntry replaced by bibentry in inst/citation
cNORM 3.0.2 (2022-06-12)
Date: 2022.08.18
Changes:
- fix for bug in normTable function when ranking order is reversed
- added option to apply conventional norming in 'cnorm' by leaving
out the grouping variable
- extended plotPercentiles, plotNorm and plotRaw for usage with conventional
norming
- vignette extend for explaining conventional norming
cNORM 3.0.1 (2022-04-11)
Date: 2022.04.11
Changes:
- t parameter added to data preparation in the shiny GUI
- default paremeters in cnorm now k = 5 and t = 3
- error in shiny GUI corrected: Download data
- WeigtedRegression vignette extended
- Additional descriptive information in modeling when using weights
- Vignette cNORM-Demo revised
cNORM 3.0.0 (2022-03-28)
Date: 2022.03.28
Changes:
- Major version: Includes weighting functions to overcome biased norm samples,
by providing marginal means factor levels of stratification variables in the
population as a data frame
New function: computeWeights()
- Newly developed, highly performant and unbiased weighted ranking procedure
- New vignette: 'WeightedRanking'
- Modelling returns info on range of weights if post stratification is used
- automatically remove cases with missings in 'cnorm' function
- ppvt dataset exchanged with unstratified sample with additional background
variables (migration, region, sex)
- Documentation updated
- Author sequence changed. Alex is now first and corresponding author. Please
direct questions to [email protected]
- minor changes: if(class(x) == "cnorm") exchanged with if(inherts(x, "cnorm"))
throughout package
cNORM 2.1.1
Date: 2021.10.13
Changes:
- normList parameter in plotNormCurves not working in non T score scales fixed
- lower CI not reported correctly in normTable and rawTable corrected
- documentation in normTable and rawTable extended
- internal prettyPrint function corrected; now it displays interval in the middle of the tables
correctly in rawTable and normTable function
cNORM 2.1.0 (2021-08-12)
Date: 2021.08.10
Changes:
- add remarks on decrease of age power parameter in computePowers when R2 is low
- add parameter for powers of a in computePowers, prepareData, bestModel and cnorm
- predictNorm is now able to handle NA
- count, how often terms had been selected in cnorm.cv
- pretty print option added to normTable and rawTable to collapse intervals and round
to meaningful precision
- Bug corrected in normTable when using age vector to compute series of norm tables
cNORM 2.0.4 (2021-07-24)
Date: 2021.07.24
Changes:
- Fixed bug in setting getNormScoreSE and added option to calculate RMSE (now default)
- Corrected y axis label in plotDerivative
- changes header in plotNorm from SE to RMSE
- exceptions catched in predictNorm
cNORM 2.0.3 (2021-04-10)
Date: 2021.04.10
Changes:
- Fixed bug in setting minNorm and maxNorm in predictNorm, if attribute is missing
- Aligned function in predictNorm for single scores and vectors
- Code simplification
- suboptimal model selection when leaps.setups dependencies found; bug fixed
cNORM 2.0.2 (2021-01-30)
Date: 2021.01.30
Changes:
- Fixed bug in rankBySlidingWindow due to ranking
- New function for building groups and assign group means: getGroups
- display errors fixed in plotPercentiles, function optimized
- Fixed regression: Clipping of minRaw and maxRaw in predictRaw
cNORM 2.0.1 (2021-01-06)
Date: 2021.01.05
Changes:
- Fixing errors in the context of weighted percentile modelling
- Code change of weighted rank estimation from https://aakinshin.net/posts/weighted-quantiles/
code by Andrey Akinshin
- Additional message for plotting, when weighted percentiles are used
- Use weighted percentiles in plotPercentiles
- automatic weighting deactivated in bestModel, since it is already applied in ranking
- suppressWarnings in weighted ranking
cNORM 2.0.0 (2020-12-04)
Date: 2020.12.04
Version 2.0.0 features many fundamental improvements both relating to the procedure but as well to the
package itself. It introduces weighted percentiles and thus helps in correcting violations of
representativeness in the norm sample. There is a new main function 'cnorm()' that returns a
cnorm object. Most functions now accept this cnorm object and do not require separate data objects and
statistical models. And finally S3 methods plot(), summary() and print() have been introduced.
Changes:
- Preparing for next major release with complete redesign of S3 method structure and weighting
- New function cnorm() that does all the data preparation and modelling in one step
It returns a cnorm object, which can be used in all model check, plotting and prediction functions
- New S3 functions: print, plot, summary
- Vignette revised
- All functions have been extended to accept a cnorm object instead of data and / or model
- prepareData, rankByGroup and rankBySlidingWindow no have the option to provide a
weighting parameter to compensate for imbalances. The percentiles are weighted
accordingly. The weighted ranking is based on an adaption of wtd.rank of the Hmisc
package, provided by the courtesy of Frank Harrell
- bestModel automatically uses the weighting parameter from the ranking (if applied)
- prepareData, rankByGroup and rankBySlidingWindow can now directly handle vectors
instead of a data frame, e. g.
rankByGroup(raw = elfe$raw, group = elfe$group)
- If no group is provided and only a raw vector is present e.g. ranByGroup(raw=elfe$raw),
traditional ranking of a single group is done
- Power parameter k added to prepareData
- New convenience function modelSummary
- New method getNormScoreSE added: Compute SE for regression based norm scores sensu Oosterhuis
van der Ark & Sijtsma (2016)
cNORM 1.2.4 (2020-11-16)
Date: 2020.10.14
Changes:
- Improvements in Shiny GUI: download buttons for data and model, introduction page, CI for norm tables
- Descending order bugs corrected in Shiny GUI
- Option to automatically compute confidence intervalls in rawTable() and normTable() via CI and reliability parameters
- repeated cross validation cnorm.cv now calculates RMSE for norm scores
- repeated cross validation cnorm.cv can now use a prespecified formula
- data cleaning in output of cnorm.cv
cNORM 1.2.3 (2020-06-18)
Date: 2020.06.18
Changes:
- Error in citation fixed
- Improved explanations in the Shiny GUI
- Additional 'Update'-Buttons in norm and raw scores plot in visualization tab of shiny gui
- removed unnecessary import askYesNo
- spell checking in diverse function descriptions
- correction for monotonicity in rawTable and normTable (now default)
- predictRaw can now return matrices for list of norm x age
- new dataset added on the basis of the EPM paper
- Bug in plotPercentiles fixed for datasets with descending ranking order
- 'descend' parameter added to prepareData
- rawTable can now return matrices
cNORM 1.2.2 (2019-09-19)
Date: 2019.09.18
Changes:
- Error in rankByGroup and rankBySlidingWindow when covariate variable name was used
- warning added to rankBySlidingWindow in case, age and group do not correspond
- rankBySlidingWindow accepts age variable in addition to age variable name
- plotPercentiles now allows plotting both degrees of binary covariate
- code cleaning in plotPercentiles
- bestModel automatically does plotPercentiles if parameter plot set to TRUE
- Citation file added
cNORM 1.2.1
Date: 2019.08.01
Changes:
- prepareDate issues warnings if age and group values do not relate
- modified message in bestModel function if R2 is not reached
- if R2 and terms are not specified in bestModel function, fall back to model 5
in case R2 does not reach .99 in most complex model
- Additional message in bestModel in case of high number of terms
- Code simplification in computePowers by using 'poly' function
- Performance improvement and code cleaning in checkConsistency
- Multiple R2 output added to computePowers
- new option to add index labels to data points in plotSubset
- cnorm.cv now respects sliding window ranking
- silent parameter added to prepareData and computePowers
- rankBySlidingWindo and rankByGroup add width parameter to data preparation
- pCutoff in cnorm.cv now adjusts for sample size
cNORM 1.2.0 (2019-07-26)
Date: 2019.07.26
Changes:
- Preparing for the inclusion of a binary covariate. The package is been
rewritten by larger parts
NOTE: The inclusion of a covariate is currently still experimental and not
optimized. Please use carefully! If covariates are central for your research
question, consider packages like GAMLSS or quantreg
- Entering systematic testing
- BUG fixed: retrieving normtables and norm scores in large datasets could
produce outlier (hashing function to remove duplicates was flawed)
- Corrected keywords for datasets
cNORM 1.1.9
Date: 2019.07.07
Changes:
- weighting added to bestModel-function
- cast to data.frame in prepareData method to prevent SPSS import failing
- Ordering of raw table when using descending values
- Warning message added to computePowers function in case the multiple R2 between the explanatory variable
and the raw score is below .05; modelling norm scores in dependence of age is questionable in that case
- prepareData, rankByGroup and rankBySlidingWindow now accept variables instead of variable names as well
- rankByGroup and rankBySlidingWindow display warning in case of small groups
cNORM 1.1.8 (2019-03-15)
Date: 2019.03.15 (mainly testing and cleaning minor errors)
Changes:
- fixing exceptions
- group and age can now be deactivated, resulting in conventional norming procedure, based in ranking + regression over powers of L
cNORM 1.1.7
Date: 2019.02.28
Changes:
- bestModel function now accepts a formula as a predictors object
- plotPercentile now accepts descending ranking
- rawTable and normTable adapted for descending values
- rangeCheck prints additional information
- Leaner GUI with more options
- predictNorm now much faster through using lookup tables, large speed gains as well for depending functions
- Setting age = FALSE in computePowers prevents computation of powers of age and interactions. All plotting and modelling functions changed accordingly. cNORM in this case models norm score tables simply based on regression without computing different groups
- normTable automatically chooses default values for minNorm, maxNorm and step
cNORM 1.1.6
Date: 2019.02.07, third release on CRAN
Changes:
- Improvement to cv function in GUI and in package
cNORM 1.1.5 (2019-02-06)
Date: 2019.02.06, third release on CRAN
Changes:
- Cross Validation added to shiny GUI
- cnorm.cv documentation improved
- added information to BestModel output
cNORM 1.1.4
Date: 2018.12.18
Changes:
- scale parameter added to prepareData function
- fix for plotNorm by group with missing values
cNORM 1.1.3
Date: 2018.12.09
Changes:
- rmarkdown moved from imports to suggests
- cnorm.cv info added to README
cNORM 1.1.2 (2018-12-08)
Date: 2018.12.08
Changes:
- deleted code in vignette needing to much build time
- removed UTF-8 attributes from ppvt dataset and cleared all datasets from non ASCII signs
- deleted code in vignette needing to much build time
- additional tests run on R-hub
- added rmarkdown to imports
cNORM 1.1.1
Date: 2018.12.01
Changes:
- Parameters added to cv.norm: Significance level for stratification process
- Additional plot in cv.norm: delta R2 in norm score validation
- Example in readme improved
- CDC data: group variable set to center of interval
- descend parameter removed from plotPercentileSeries, plotPercentiles, checkConsistency,
rawTable & normTable; instead take default from model; vignette updated accordingly
- stop criterion added to data sampling in cnorm.cv
- cv.norm: lines added to R2 delta plot
- normTable and rawTable can now produce list of tables
cNORM 1.1.0
Date: 2018.11.23
Changes:
- Cross validation added: new function: cnorm.cv() for assessing RMSE for raw data and R2 and CROSSFIT for norm data
- Data table output for cnorm.cv
- rankBySlidingWindow now accessible via prepareData()
- group, raw, age and width can now be provided in cnorm.cv
- parameter for full cross validation (separate ranking for train and validation)
- Additional NA checks and warning messages
- plotPercentiles now with R2adjr in title
cNORM 1.0.3
Date: 2018.11.16
Changes:
- Additional instruction on series section of visualization tab in Shiny GUI
- Code cleanup in bestModel function
- SE added to plotNorm based on Oosterhuis, van der Ark & Sijtsma (2016)
- RMSE added to model object (m$subsets), to plotRaw and to plotSubset
- additional plotting options added to GUI:
- plotting of differences in raw and norm plot
- RMSE in model selection information function
cNORM 1.0.2
Date: 2018.11.16
Changes:
- Improvements in precision of plotPercentiles
- error corrected in ppvt dataset: groups did not represent group means
- function description in 'ranBySlidingWindow' updated
- checking for missing packages in shiny GUI improved
- user menu asking to install missing packages added
- derive-function: more general approach with "order" parameter
- plotDerivative function can now plot derivatives of higher order
- exclude cases with missing values in rankByX functions
- percentile columns added to rawTable and normTable
- additional data cleansing for data objects imported from Excel file format
cNORM 1.0.1 (2018-11-14)
Date: 2018.11.03
Changes:
- Improvements in the GUI: Waiting circle shown to indicate ongoing computation
- Additional help texts on best model in GUI
- Additional plotting options in cNORM.GUI(): Raw Score and Norm Score plots
- User input asking for missing suggested packages to install
cNORM 1.0.0
Date: 2018.10.26
Changes:
- Final polishing finished; releasing first major version
cNORM 0.9.20
Date: 2018.10.24
Changes:
- GUI with Shiny finished
- ... now working on finally releasing the package
cNORM 0.9.19
Date: 2018.10.20
Changes:
- API changed: predictNormValue renamed to predictNorm
- Shiny GUI enhanced
- Additional plotting options in plotNorm and plotRaw
- less strict warning messages in predictNormValue function and checkConsistency
cNORM 0.9.18
Date: 2018.10.08
Changes:
- First shiny prototype (many thanks to Sebastian Gary); please use cNORM.GUI() to start user interface
cNORM 0.9.17
Date: 2018.10.01
Changes:
- predictNormValue fixed and optimized (many thanks to Sebastian Gary)
- API change with respect to predictNormValue, rawTable and plotNorm
- plotNorm: norm score boundaries guessed by min and max score from modelling
cNORM 0.9.16
Date: 2018.09.21
Changes:
- bug in predictNormValue partly fixed (further optimization necessary)
- API change: plotValues renamed to plotRaw
- new function: plotNorm
cNORM 0.9.15
Date: 2018.09.18
Changes:
- plotDensity function added
- attributes added to data.frame to increase usability
cNORM 0.9.13
Date: 2018.09.16
Changes:
- predictNormValue with higher precision and effectivity
- rawTableQuick removed from source code
cNORM 0.9.12
Date: 2018.09.11
Changes:
- 'simulateRasch' to simulate test data was added
- old sim functions removed
- documentation improved
- new parameters to bestModel in order to force covariates into regression
- additional checks in box cox functions
cNORM 0.9.9
Date: 2018.09.06
Changes:
- Enhancements to the 'prepareData' function
cNORM 0.9.8
Date: 2018.09.05
Changes:
- Life expectancy dataset of the World Bank added
- Mortality of infants per 1000 life birth from 1960 to 2017 added
- Minor changes in functions to check data integrity and exceptions
- Vignette updated
cNORM 0.9.7
Date: 2018.08.31
Changes:
- License changed to AGPL
- Capitalizations in labels of plots
- min and max renamed to minRaw and maxRaw (where appropriate)
- terminology: standard or normal score instead of norm; score instead of value
- new function for model validation: plotPercentileSeries
- many functions now draw the default values from the model (plotting and predicting)
cNORM 0.9.6
Date: 2018.08.28
Changes:
- Minor improvements in function descriptions
- API of plotSubset changed due to new plotting options
cNORM 0.9.5
Date: 2018.08.25
Changes:
- New, large dataset for BMI centile estimation from CDC included, type ?CDC for explanation
- Extensive documentation available via https://www.psychometrica.de/cNorm_en.html (in progress)
cNORM 0.9.4
Date: 2018.08.23
Changes:
- Generating group variable in rankBySlidingWindow
- parameters in plotPercentile to restrict age range
- ppvt dataset restricted
cNORM 0.9.3
Date: 2018.08.20
Changes:
- plotNormCurves enhanced (Thanks to Sebastian Gary)
- new function to plot semi parametric analyses via box cox power transformation: plotBoxCox
- variable "explanatoryVariable" and "normVariable" in computePowers function renamed for easier API
cNORM 0.9.2
Date: 2018.08.18
Changes:
- Additional dataset: vocabulary development (PPVT4)
cNORM 0.9.1
Date: 2018.08.16
Changes:
- Added predictRawBC and predictNormBC for computing norm and raw values based on the parametric box cox power function parameters
- New contributor: Sebastian Gary, welcome to the team!
- Missing raw variable definition in plotValues corrected
cNORM 0.9.0
Date: 2018.08.14
Changes:
- Box Cox power transformation for regression model at specific age: optional parametric modelling for non-parametric regression model
cNORM 0.8.9
Date: 2018.08.13
Changes:
- Convenience method for selection best model added: 'printSubset'
- predictNormValue now accepts lists of values as well
cNORM 0.8.8
Date: 2018.08.12
Changes:
- parameter checks added
- new parameter 'descriptives' added to rankByGroup and rankBySlidingWindow added to retrieve descriptive statistics for each observation
- improvements in the documentation
- errors in bestModel and plotPercentiles corrected, when variable names are not as in example sample
cNORM 0.8.6
Date: 2018.08.11
Changes:
- new function: 'rankBySlidingWindow' which can be used for data sets with continuous age variables
- error corrected for data being loaded from SPSS files
- improvements in the documentation
cNORM 0.8.5
Date: 2018.08.06
Changes:
- new function for simulating data
cNORM 0.8.3
Date: 2018.08.03
Changes:
- Code cleaning and formatting
cNORM 0.8.2
Date: 2018.08.02
Changes:
- new internal function: rawTableQuick for speeding up generating norm tables
Still has to be checked for working with descending values.
Works only, if model assumptions are valid
cNORM 0.8.0
Date: 2018.08.01
Changes:
- new function: rawTable allows creating norm tables with assignment of raw -> norm values solves inverse function of regression model with brute force
cNORM 0.7.11
Date: 2018.07.31
Changes:
- improved 'prepareData' function
cNORM 0.7.10
Date: 2018.07.28
Changes:
- Description for computePowers improved
- option in plotPercentile to use percentile scale or self defined c(mean, sd)
cNORM 0.7.9
Date: 2018.07.28
Changes:
- 'descend' parameter added to consistencyCheck and normTable
- dependency rColorBrewer removed; plotPercentiles changed accordingly
- latticeExtra moved to 'suggests'
cNORM 0.7.8
Date: 2018.07.27
Changes:
- 'descend' parameter added to consistencyCheck and normTable
cNORM 0.7.7
Date: 2018.07.27
Changes:
- Small changes to error messages in bestModel
- printing of min value in plotDerivate removed
cNORM 0.7.6
Date: 2018.07.27
Changes:
- parameter 'predictors' added to allow self defined regression functions, e. g. for the inclusion of other ranking parameters like sex
- 'type' parameter added to 'plotPercentile' to allow selection of quantile algorithm. Please consult help(quantile) for further information on 'type'
cNORM 0.7.5
Date: 2018.07.26
Changes:
- dependency dplyr removed: rankByOrder and plotPercentiles rewritten
- API-change: derivationPlot renamed to plotDerivate
- small changes to vignette and readme
- parameter "raw" added to rankByGroup to specify raw value variable
- rawVar and groupVar in plotPercentiles renamed to raw and group to make API more coherent
cNORM 0.7.4
Date: 2018.07.25
Changes:
- rankByOrder: ranking in descending order added
cNORM 0.7.3
Date: 2018.07.25
Changes:
- Additional ranking algorithms: Filliben, Levenbach, Yu & Huang; API changed to index
- scale can be specified as double vector with c(mean, sd)
- vignette updated accordingly
cNORM 0.7.2
Date: 2018.07.24
Changes:
- None. This is the first release