--- title: "1. Getting started with STEM models" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{1. Getting started with STEM models} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", fig.width = 6.5, fig.height = 4 ) ``` ```{r load} library(Stem) ``` ## The model `Stem` estimates hierarchical spatio-temporal models of the form $$ z_t = X_t \beta + A y_t + e_t, \qquad e_t \sim N(0, \Sigma_e), $$ $$ y_t = G y_{t-1} + \eta_t, \qquad \eta_t \sim N(0, \Sigma_\eta), \qquad y_0 \sim N(m_0, C_0), $$ for $t = 1, \dots, T$. The observation vector $z_t$ collects the measurements at $d$ spatial locations; $y_t$ is a $p$-dimensional latent temporal process shared by the network and loaded on the locations through the $d \times p$ matrix $A$; $X_t$ holds $r$ known covariates. The spatial covariance is $$ \Sigma_e = \sigma^2_\varepsilon I + \sigma^2_\omega C(h; \theta), \qquad C(h; \theta) = \exp(-\theta h), $$ so that $\sigma^2_\varepsilon$ is a nugget and $\sigma^2_\omega$ the variance of the spatially correlated component. The parameter vector is $\phi = (\beta, \sigma^2_\varepsilon, \sigma^2_\omega, \theta, G, \Sigma_\eta, m_0)$, estimated by maximum likelihood through Kalman filtering and smoothing inside an EM algorithm, with $C_0$ held fixed. ## Data The package ships two datasets. `pm10` is the small example of Fasso and Cameletti: 22 stations and 366 days of log PM10, with emissions and altitude as covariates. `povalley` is a larger, real network of 36 background stations in the Po Valley observed daily over 2019--2023. ```{r data} data(pm10) str(pm10, max.level = 1) coordinates <- pm10$coords * 1000 # from km to meters covariates <- pm10$covariates z <- pm10$z dim(z) # T x d dim(covariates) # (T * d) x r ``` The covariate matrix stacks the $T \times r$ block of each station on top of the next: rows $1, \dots, T$ belong to the first station, rows $T+1, \dots, 2T$ to the second, and so on. Keeping this convention is essential, because every routine of the package relies on it to slice the data by location. ## Building a model object `STEM_Model()` assembles the data and the starting values into an object of class `STEM_Model`. The starting values are supplied through the list `phi`, which must have exactly the components of the parameter vector. ```{r model} phi <- list( beta = matrix(c(3.65, 0.046, -0.904), 3, 1), sigma2eps = 0.1, sigma2omega = 0.2, theta = 0.01, G = matrix(0.77, 1, 1), Sigmaeta = matrix(0.3, 1, 1), m0 = as.matrix(0), C0 = as.matrix(1) ) A <- matrix(1, ncol(z), 1) # a single latent process, loaded on every station mod <- STEM_Model( z = z, covariates = covariates, coordinates = coordinates, phi = phi, A = A ) class(mod) ``` ## Estimation `STEM_Estimation()` runs the EM algorithm on the state-space form of the model (the Kalman filter and smoother make the E-step), accelerated by default by SQUAREM; `STEM_control(algorithm = )` chooses among EM, ECME, SQUAREM and SQUAREM on ECME, which reach the same maximum. The iterations stop when the largest relative change of a parameter or the absolute change of the log-likelihood falls below its tolerance, `1e-3` by default; `precision`, used below to keep the example short, replaces the first. Progress messages are off by default and can be turned on with `verbose = TRUE`. ```{r estimation} fit <- STEM_Estimation(mod, precision = 0.01, distance = "euclidean") round(unlist(fit$estimates$phi.hat), 5) fit$estimates$loglik fit$estimates$convergence.par[c("conv.log", "conv.par", "iterEM")] ``` Use `distance = "geo"` when the coordinates are longitude and latitude, as in `povalley`; the geodesic distance is then computed with `geodist`. The smoothed latent process is returned as a `ts` object: ```{r smoothed, fig.alt = "Smoothed latent temporal process"} plot(fit$estimates$y.smoothed, ylab = "smoothed latent state", xlab = "day") ``` ## Uncertainty: the parametric bootstrap The EM algorithm returns point estimates but no standard errors. `STEM_Bootstrap()` simulates `B` datasets from the fitted model and re-estimates the parameters on each of them. ```{r bootstrap, eval = FALSE} boot <- STEM_Bootstrap(fit, B = 100, distance = "euclidean") npar <- 9 boot.estimates <- t(vapply(boot$boot.output, function(b) { p <- b$estimates$phi.hat c(p$beta, p$sigma2eps, p$sigma2omega, p$theta, p$G, p$Sigmaeta, p$m0) }, numeric(npar))) se <- sqrt(diag(stats::var(stats::na.omit(boot.estimates)))) round(se, 5) ``` ## Spatial prediction `STEM_Kriging()` predicts the process at unobserved locations, given their coordinates, covariates and loading matrix. ## Where to go next The second vignette, *Spatially-clustered STEM models*, shows what to do when a single global regression is not plausible over the whole network, and the coefficients themselves should be allowed to vary across spatial regimes: ```{r next, eval = FALSE} vignette("SCSTEM", package = "Stem") ``` A conceptual map of how the functions call one another is the third vignette: ```{r map, eval = FALSE} vignette("function-map", package = "Stem") ``` ## References Fasso, A., Cameletti, M. and Nicolis, O. (2007). Air quality monitoring using heterogeneous networks. *Environmetrics*, 18, 245--264. Fasso, A. and Cameletti, M. (2010). A unified statistical approach for simulation, modeling, analysis and mapping of environmental data. *Simulation*, 86, 139--153.