NEWS

---

datagrade 0.2.0 (2026-10-29)

Scoring now follows the ISO/IEC 25012 data quality model: five inherent characteristics, fifteen quality properties, each measured as a ratio A / B over counted data items in the form ISO/IEC 25024 measurement functions take. Every statistic the package computes is unchanged; what changed is how those statistics are organised into scores.

New

• dg_spec() declares what the data is supposed to look like — required columns, permitted domains, valid ranges, regular expressions, referential sets, per-record rules, credibility criteria, provenance and update expectations. Nine of the fifteen properties have a denominator that counts the items for which an expectation is defined, which no data set contains; this is where it comes from. Every argument is optional.
• report$measures is a fifteen-row data frame giving each property's ISO code, characteristic, value, A, B, whether it applied, and — when it did not — which dg_spec() argument would make it apply.
• Six properties are computable with no spec at all: COMP_REG, COMP_VAL_ESP, FAL_COMP_FICH, RIES_INCO, CONS_FORM and CONV_ACT. FAL_COMP_FICH catches cells that are occupied but empty ("N/A", "unknown", "-999"); CONS_FORM measures each column against its own dominant format, excluding free text rather than scoring it near zero.

Breaking

• report$scores now holds the five ISO characteristics — accuracy, completeness, consistency, credibility, currentness — where it held four dimensions.
• timeliness is renamed currentness, which is ISO's word for it. report$timeliness still resolves and weights still accepts the old name.
• accuracy is NA unless a spec declares an interval, pattern or reference. All three ISO accuracy properties require a declared expectation. The old measure — outliers against an interval derived from the data — is unchanged, reported as report$plausibility, and excluded from the overall score, because an outlier need not be wrong and a wrong value can sit exactly on the mean.
• Completeness counts cells, not the average of per-column missing rates. The old quantity remains derivable from report$missing$columns.
• Scores are ratios in 0--1, where they were on the dissertation's 0--10 scale. Percentages are unchanged, so every reported percentage reads exactly as before. Code comparing raw scores against 0--10 thresholds needs its cut-offs divided by ten; the good/warning/serious/critical bands now break at 0.9, 0.7 and 0.5.
• A characteristic with no applicable property is NA and is dropped from the aggregate rather than scored zero. NA is a statement that B could not be counted, not a claim about the data.
• summary()$scores names its first column characteristic, not dimension, and summary() gains a measures table.

Correctness

• dg_assess() no longer aborts on its own default of verbose = TRUE. Progress messages are interpolated by cli in the frame of the caller, which was the internal say() helper rather than dg_assess(), so every message referring to a local variable failed.

datagrade 0.1.0

First release. Implements the data quality framework from Demirhan (2024), MSc dissertation, University of Greenwich.

Interface

• dg_assess() assesses a data frame or file and returns a dg_report object, with print(), summary(), plot() and as.data.frame() methods. data_quality_assessment() is kept as an alias.
• The data argument works. The dissertation documented it in three places and implemented it in none.
• dg_plots() returns figures as a named list of ggplot objects; dg_save_plots() writes them to disk.
• All thirteen checks are exported and usable on their own.
• Dependencies reduced to cli, ggplot2 and base R. dplyr, car, moments, reshape2, DescTools, mice and caret are no longer needed; mice and caret were never used by the original code at all.

Correctness

• Consistency is scored as a ratio in 0--1 like every other dimension. It was previously multiplied by ten as though it were on a 0--10 scale, which is why the dissertation reports 2.5% for iris where 25% was meant.
• flag_redundant_columns() no longer reports the literal strings "row" and "col" as redundant data columns.
• Figures are actually produced. ggplot(...) + theme_minimal() %>% print() printed the theme and discarded the plot.
• Kurtosis is excess kurtosis, matching the dissertation's stated formula rather than moments::kurtosis().
• Identifier detection runs before the storage-type tests, so the "ID" branch is reachable. A numeric column qualifies only if its values are whole numbers, so continuous measurements are not mistaken for keys and dropped.
• Dates are parsed before text normalisation, so timeliness is measurable from a CSV.
• Text normalisation is Unicode-aware and non-destructive by default.
• Outliers are counted on observed values rather than mean-imputed ones.
• Nothing is written to the global environment.
• The caller's data frame is never modified.

Robustness

Regression tests cover constant columns, single-column frames, all-NA columns, zero-row frames, perfect collinearity and single redundant pairs — each of which produced an NA, an error or a silently wrong answer before.

Performance

• Duplicate detection rewritten from duplicated.data.frame to integer-coded row hashing: 10 million rows went from 494 s to 38 s, bit-identical result.
• VIF computed as diag(solve(cor(X))), which is independent of row count.
• Figures built from precomputed bin counts and five-number summaries, so they render in constant time and the report object stays at ~83 KB regardless of input size.
• Shapiro-Wilk and the anomaly model use deterministic subsampling that neither depends on nor disturbs the caller's random state.

Known limitations

• Not ISO/IEC 25024 measure-conformant. Dimension names follow ISO/IEC 25012; the measures are this package's own. Genuine X = A/B measures require metadata that zero-configuration assessment does not have. See vignette("methodology").
• credibility, the fifth ISO/IEC 25012 inherent characteristic, is not measured.
• The accuracy score substitutes a statistically derived interval for a specified one, so it measures plausibility rather than accuracy in the ISO sense.
• Cost grows as O(p^3) in the column count through the VIF matrix inversion. Fine for hundreds of columns, not for tens of thousands.