Large speed-ups when the histograms have different bins (the default
simplify = FALSE), with results identical to 1.0.9 (up to differences of
the order of 1e-13 in covariances and fuzzy c-means, due to the order of the
sums). Times on a matrix of 300 histograms x 3 variables with different bins:
| Function | 1.0.9 | 1.0.10 |
|---|---:|---:|
| WH_adaptive_fcmeans() | 117 s | 4.1 s |
| WH_2d_Kohonen_maps() | 106 s | 4.6 s |
| WH_2d_Adaptive_Kohonen_maps() | 101 s | 4.4 s |
| WH.var.covar(), WH.correlation(), WH.SSQ() | 4.8 s | 0.1 s |
| WH.vec.mean() | 2.3 s | 0.08 s |
| WH_fcmeans() | 7.8 s | 1.6 s |
| WH_kmeans() | 6.8 s | 2.5 s |
| WH_adaptive.kmeans() | 7.5 s | 3.1 s |
| WH_MAT_DIST(), WH_hclust() | 3.0 s | 1.2 s |
| registerMH() | 5.9 s | 2.1 s |
SSQ_RCPP() (C++, used by WH.SSQ(), WH.var.covar(),
WH.correlation()) no longer builds a dense diagonal matrix of the weights
(of size G x G, G being the number of points of the common grid of cdf
values) for each individual; the sum of the weights is now stored in a
double instead of a float.WH.SSQ() on the already prepared data: they now use the fast
c_ComputeFASTSSQ().WH_kmeans() and WH_adaptive.kmeans(): the prototypes are computed without
copying the columns of the data of each cluster (new c_ComputeFASTSSQ_sel()).PREPARE_A_VEC_MAT() (C++, puts a set of histograms on a common grid of
cdf values) computes the quantiles with a single ordered scan of the grid;
COMP_Q_VECT() scans the cdf only once for sorted levels; c_Prepare() no
longer copies and transposes the data.registerMH(), the arithmetic of histograms with numbers and the regression
with simplify = TRUE avoid repeated validity checks and quantile
computations.WH.1d.PCA(): INERTIA (total inertia) was the sum of the eigenvalues
returned by FactoMineR::PCA(), which since FactoMineR 2.15 are only the
first ones; it is now computed from the percentage of inertia of the first
axis.
HTS.moving.averages(): each average was assigned the timestamp of the
element following the central one of its window; weights of length
different from k are now an error.
HTS.predict.knn(): with ties in the distances, which() returned several
positions (with a warning) and could select wrong neighbours; k larger than
the available history is now an error.
WH_2d_Adaptive_Kohonen_maps() with init.weights = "PROP": the condition
schema == 2 || 4 was always true, so the initial weights of schemas 1, 3
and 5 were overwritten; for these schemas the vector of initial weights
also had twice the needed length.
d1 - d2 failed (object 'tmp' not found) when the two distributions share
the same cdf, e.g. after registerMH().
a * d and d * a with a < 0, and a - d, returned (or failed to
create) objects with a decreasing x; they now return the distribution of
a * X and a - X.
plot() of a MatH (type "HISTO") cut the densities at the fixed value
2: variables with a narrow range were drawn as flat, truncated histograms.
The heights are now normalised within each variable.
WH.plot_multiple_indivs() drew the histograms of the individuals as flat
lines (the normalised domain and densities were computed but not used). The
size of the histograms is now set automatically: strx (default 0.1) is
the average width of the histograms and stry (default 1/8) the height of
the highest density, as fractions of the ranges of the coordinates of the
individuals on the two axes. Each histogram is centred on its mean, and
histograms with very narrow bins (e.g. quantiles with ties) are drawn on 30
equal-width bins to avoid spikes.
The printout of a histogram with many bins used two different formats for the bins.
[ on a MatH object: columns or rows selected by name got NA names;
negative and logical indices are now allowed (e.g. BLOOD[-1, ],
BLOOD[, "Cholesterol"]).
Five vignettes have been added:
vignette("HistDAWass-01-data"): creating histograms (distributionH,
data2hist()) and matrices of histograms (MatH), also from a data frame
of raw data;vignette("HistDAWass-02-basic-statistics"): statistics of histograms and of
histogram variables (means, quantiles, distances, covariances, correlations);vignette("HistDAWass-03-clustering"): k-means and fuzzy c-means, also with
adaptive distances, and hierarchical clustering;vignette("HistDAWass-04-pca"): principal component analysis of one or more
histogram variables;vignette("HistDAWass-05-regression"): the two-component linear regression
of histogram variables.c_WH_ADPT_KMEANS_TOTALSSQ() (used by WH_adaptive.kmeans() and the
adaptive Kohonen maps): the vector of cluster weights assigned to each row
of the weight matrix was allocated with k (number of clusters) elements
instead of vars (number of variables). When k < vars the remaining
variables read uninitialised memory. This made the examples fail with the
forthcoming Rcpp release (Rcpp PR #1513, which checks the length of vectors
assigned into matrix rows). Thanks to the Rcpp team for the report and the
patch.1e-8 is assigned to each empty bin and the cdf is renormalised.
This happens when a distributionH (or TdistributionH) object is created,
in data2hist() (that used a weight of 0.001 counts), in checkEmptyBins()
and in the internal C++ registration of distributions (that used 1e-14).
Up to 1.0.8 a distribution with an empty bin produced misaligned quantiles
when several distributions were put on a common grid of cdf values, with
errors in the computed distances (up to 7% in the China_Seas data).China_Seas, China_Month and Age_Pyramids_2014 datasets contained
683 distributions with empty bins: they have been fixed accordingly.compQ() returned wrong quantiles for distributions with repeated values
in x (e.g. compQ(distributionH(c(1,2,2,3), c(0,.3,.6,1)), 0.7) returned
2 instead of 2.25).compP() returned wrong values for distributions with empty bins.distributionH objects never checked that p is
not decreasing; distributionH(x) with a single value now creates an
impulse distribution as documented.whichLowerEQThanX() (C++) could read one element beyond the end of the
vector; c_DISTA_M2() (C++) accessed the second element of a list of length
one when data had a single variable.distributionH, TdistributionH, MatH, HTS) and
plotPredVsObs() now draw the plot on the current graphic device and return
the ggplot object invisibly. Up to 1.0.8 the distributionH and MatH
methods returned the object without drawing it, so nothing was shown inside
loops, functions or source().WH.1d.PCA() no longer opens new graphic devices with dev.new(): the four
plots of each factorial plane are drawn on the current device (2 x 2
panels), restoring the graphical parameters on exit. In the plot coloured by
the standard deviations the densities were centred on the standard
deviation instead of the mean.WH.MultiplePCA() gains a graph argument (default TRUE) passed to
FactoMineR::MFA().WH.plot_multiple_Spanish.funs() and WH.plot_multiple_indivs() no longer
print their plots twice; WH.plot_multiple_indivs() failed with
stat different from "mean", and its cut of the highest densities was
applied to the wrong column.VIOLIN plot of HTS objects showed the data of the first block of
timestamps in all the rows of the plot: fixed. HTS plots are now a single
ggplot with one facet for each block of maxno.perplot timestamps.plotPredVsObs() returns a single ggplot (faceted by object) instead of
drawing several plots in a grid of viewports.linewidth instead of size for lines, guides(fill = "none"), aes()
with the .data pronoun instead of aes_all() and with().rbind() in loops):
summaryHTS() and the HTS violin plot are about twice as fast.data2hist(algo = "base") no longer draws a histogram as a side effect.browser() calls from the code of the adaptive Kohonen
maps and of the adaptive weights.Config/roxygen2/version replaces RoxygenNote): descriptive titles, a
single usage for each method (no duplicated generic/method usage, no
initialize methods in the usage of the classes), accurate return values,
documented slots of the classes, dataset formats and examples, and an
overview of the package in ?HistDAWass.?"Arith-distributionH"), WH.SSQ/WH.var.covar/WH.correlation and
their two-matrix versions, WH.vec.sum/WH.vec.mean, WH.bind*,
registerMH/is.registeredMH, get.cell.MatH/set.cell.MatH and the
get.MatH.* accessors.\dontrun{} examples replaced by \donttest{} (they all run).checkEmptyBins() has a method for MatH objects, useful for objects
created with previous versions of the package.c_MM_L2_SQ_WASS_D(), c_DISTA_M(), c_Fast_D_Mat(),
c_STEP_2_ADA_KMEANS(), c_ComputeFast_L2_SQ_WASS_DMAT()) no longer
allocate temporary vectors nor access S4 slots in the inner loops, and
c_WH_ADPT_KMEANS_TOTALSSQ() computes cluster-invariant distances once.
Results are numerically identical to 1.0.8; on a 300 x 4 matrix of
histograms WH_MAT_DIST() and WH_hclust() are about 20 times faster,
WH_kmeans() about 6 times, WH_adaptive.kmeans() about 4 times and the
Kohonen maps about 3 times faster.Minor bug fixes fo compatibility isseues with ggridges.
Added the WH_WASS_MAT_DIST for computing a distance matrix
Fixed some minor bugs. Added the names of columns and the rows in weights matrices of adaptive distances based methods.
Fixed some minor bugs.
Fixed some visualizations accordingly to the new ggplot2 release.
#HistDAWass 1.0.2 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
Soelf Organizing maps have been improved for speed and aptions.
#HistDAWass 1.0.1 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
Some bugs have been resolved when processing data with a single variable.
#HistDAWass 1.0.0 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
Now, a set of methods for MatH (WH.SSQ, WH.vec.mean, and all the methods using them) are much faster due to RCPP implementation.
#HistDAWass 0.1.7 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
Some general improvements have been added for reducing elaboration times.
unsupervised learning
WH_kmeans now returns more informative outputs
principal components analysis
Multiple factor analysis has new functions for plotting results
#HistDAWass 0.1.6 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
unsupervised learning
fuzzy c-means and kohonen maps are 10 times faster
#HistDAWass 0.1.5 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
some minor bug fixed
#HistDAWass 0.1.4 What’s new ##(Histogram-valued Data analysis using Wasserstein metric)
some minor bug fixed
#HistDAWass 0.1.3 What’s new Some fixes to the worksapce
#HistDAWass 0.1.2 What’s new ##(Histogram-valued Data analysis using Wasserstein metric) > From raw data to histograms
A new function was added for generating a histogram from a set of raw data
Visualization
Now for plotting, ggplot2 package is used generally.
A new plot for visualizing Histogram Time Series was added.
A new plot for comparing obversed vs predicted histogram was added.
A new plot for showing the differences between an observed and a predicted histogram was added.
Regression and predictive techniques
A new function for computing Goodness of Fit of a model was added