1. Purpose

Block 4 developed the asymptotic theory of Path 1 (hierarchical Bayesian estimation via Stan). This block develops the parallel theory for Path 2 (varying-coefficient models via penalized splines, implemented in mgcv), organized in three layers analogous to those of Block 4 but adapted to the frequentist penalized-spline setting:

• (L1) Pointwise consistency: \(\widehat{\beta}(z) \xrightarrow{P} \beta_*(z)\) at each fixed \(z\).
• (L2) Uniform consistency and convergence rate: \(\sup_z |\widehat{\beta}(z) - \beta_*(z)| \to 0\) at a rate \(r_n\) depending on smoothness, dimension, and penalty.
• (L3) Pointwise asymptotic normality: \(r_n^{-1}(\widehat{\beta}(z) - \beta_*(z) - \text{bias}(z)) \xrightarrow{d} \mathcal{N}(0, V(z))\).

The reference framework throughout is Stone (1985) for the optimal nonparametric rate, Fan and Zhang (2008) for varying-coefficient asymptotics, Wood (2017) for the practical penalized-spline implementation, and Hansen (2008) for uniform rates of kernel estimators (a sibling result for splines under analogous regularity).

Path 2 has no direct Bernstein-von Mises analog (BvM is Bayesian); its (L3) layer is the classical pointwise CLT for the spline estimator. We treat (L3) as the frequentist equivalent of Theorem 4C of Block 4, with the same caveats about scope (pointwise, not uniform; on the parametric component, not on the function class as a whole).

The block specializes (REG-EST) of Block 2 to Path 2: under the hypotheses of Theorem 5A (or 5B), (REG-EST) holds in the average-error form Lemma 2B requires.

Implementation status (gdpar 0.0.0.9001). Path 2 (varying-coefficient via mgcv) is not currently implemented in the package. A call to gdpar(..., path = "vcm") aborts with a gdpar_unsupported_feature_error (see R/gdpar.R). The asymptotic theory developed below is canonical and reference-grade for the framework; the implementation notes in §11 are written in prospective voice (“the planned implementation will…”) because the operational layer is queued for a future version of gdpar and the present block does not assume its availability.

2. Setting and Notation

3. Three Asymptotic Layers

Parallel to Block 4:

• (L1) Pointwise consistency. \(\widehat{\beta}(z) \xrightarrow{P} \beta_*(z)\) at each \(z\) in the support of \(\mu_z\).
• (L2) Uniform consistency and rate. \(\sup_z |\widehat{\beta}(z) - \beta_*(z)| = O_P(r_n)\) for some \(r_n \to 0\).
• (L3) Pointwise CLT. \(r_n^{-1}(\widehat{\beta}(z) - \beta_*(z) - \text{bias}(z)) \xrightarrow{d} \mathcal{N}(0, V(z))\) at each \(z\) where the design is regular.

The three layers are progressively stronger; each requires its own hypotheses, collected next.

4. Standing Asymptotic Hypotheses for Path 2

(DESIGN-2) Design conditions on \(z\). \(\mu_z\) admits a continuous density \(f_z(\cdot)\) on \(\mathcal{Z}\) that is bounded above and bounded away from zero on the interior of \(\mathcal{Z}\). The covariate \(X\) has \(\mathrm{Cov}(X)\) full rank.

(DESIGN-2) is the standard non-degenerate-design condition; the bounded-from-zero requirement on \(f_z\) is what guarantees that splines are estimable everywhere in the interior. At boundaries, additional care is needed (Stone 1985 gives boundary-corrected rates).

(KNOT) Knot placement and number. The spline space \(\mathcal{F}_a\) has \(J = J_n\) basis functions placed on a regular grid (or quantile-based grid) of the support \(\mathcal{Z}\). As \(n \to \infty\), \(J_n \to \infty\) at rate \(J_n \asymp n^{1/(2\beta + d)}\) for \(\beta\)-Sobolev smooth \(\beta_*\) in dimension \(d = \dim(z)\).

The choice \(J_n \asymp n^{1/(2\beta + d)}\) is optimal under (KNOT)+(SMOOTH) for the integrated \(L^2\) rate (Stone 1985); it is the rate that balances bias (which decreases with \(J_n\)) and variance (which increases with \(J_n\)).

(SMOOTH) Smoothness of true \(\beta(\cdot)\). The true coefficient function \(\beta_* \in \mathcal{H}^\beta(\mathcal{Z})\), the Sobolev (or Hölder) class of smoothness \(\beta > d/2\) on \(\mathcal{Z}\).

The condition \(\beta > d/2\) is needed for Sobolev embedding into continuous functions. If \(\beta_*\) is less smooth than this, the rates degrade.

(PEN) Penalty regularity. The roughness penalty \(\mathcal{P}\) is a quadratic form \(\mathcal{P}(\eta) = \eta^\top \Omega \eta\) with \(\Omega\) symmetric positive semi-definite, \(\mathrm{rank}(\Omega) = J_n - q^*\) where \(q^*\) is the dimension of the null space (typically \(q^* = q\), the polynomial space of degree below the penalty order).

(LAMBDA) Smoothing parameter rate. Either (LAMBDA-FIXED): \(\lambda\) is a fixed constant; or (LAMBDA-DATA): \(\widehat{\lambda}\) is selected by REML or GCV with \(\widehat{\lambda} \asymp n^{-2\beta/(2\beta+d)} \cdot J_n^{-(2\beta+d)/(2\beta+d) - \text{const}}\) (the rate consistent with the Stone-optimal balance under (KNOT)+(SMOOTH)).

(LAMBDA-DATA) is the version actually used in mgcv: \(\widehat{\lambda}\) is data-driven, and the rate above is achieved by REML in expectation (Wood, Pya, Säfken 2016) under standard regularity.

5. Theorem 5A: Pointwise Consistency

Theorem 5A. Stated under (R-RANDOM). Suppose:

• (DESIGN-2), (SMOOTH), (KNOT), (PEN), and either (LAMBDA-FIXED) with \(\lambda \to 0\) at rate \(\lambda \asymp n^{-1} J_n\), or (LAMBDA-DATA) with REML/GCV selection.
• The hypotheses of Theorem 1A of Block 1 hold for the AMM Level 1 (or 2) model being fit (in particular (LIN), (D-ID), and orthogonal-class construction for Level 2).

Then for each \(z\) in the interior of \(\mathcal{Z}\), componentwise (equivalently in vector norm \(\|\cdot\|\)): \[\widehat{\beta}(z) \;\xrightarrow{P}\; \beta_*(z) \quad \text{in } \mathbb{R}^p.\]

(R-COND) variant. Conditioning on \(\{(X_i, z_i)\}_{i=1}^n\), the same conclusion holds for \(\mu_{X, Z}^\infty\)-a.e. realization of the design.

Proof sketch. This is a specialization of Stone (1985, Theorem 1) and Fan and Zhang (2008, §3.1) to the AMM Level 1/2 context. The argument decomposes the error into bias and variance:

• Bias bound. Under (SMOOTH) and (KNOT), the spline approximation error satisfies \(|\Pi_{\mathcal{F}_a} \beta_*(z) - \beta_*(z)| = O(J_n^{-\beta/d})\). With \(J_n \asymp n^{1/(2\beta + d)}\), this is \(O(n^{-\beta/(2\beta+d)})\).
• Variance bound. Under (DESIGN-2), the variance of \(\widehat{\beta}(z)\) at fixed \(z\) is \(O(J_n / n) = O(n^{-2\beta/(2\beta+d)})\).

Combining, \(\widehat{\beta}(z) - \beta_*(z) = O_P(n^{-\beta/(2\beta+d)})\), which goes to zero. \(\square\)

Specialization to (REG-EST) of Block 2. Theorem 5A implies (REG-EST) of Block 2 for Path 2 in the average-error form. The argument follows the same three steps as Block 4 §5:

1. Pointwise consistency of \(\widehat{\beta}(z)\) at every \(z\) in the interior of \(\mathrm{supp}(\mu_z)\).

2. The map \(\eta \mapsto \theta_{\text{ref}} + \Delta(z, \theta_{\text{ref}})\) for fixed \(z\) is continuous in \(\eta\) under (LIN) and (DESIGN-2). Pointwise consistency of \(\widehat{\beta}\) implies pointwise consistency of the fitted \(\widehat{\theta}_i = \widehat{\beta}(z_i)\) (because \(\widehat{\theta}_i\) is a deterministic function of \(\widehat{\beta}\)).

3. Averaging over the empirical distribution of \(z_{1:n}\) via (IID) and uniform integrability gives \(n^{-1} \sum_i |\widehat{\theta}_i - \theta_*(x_i, \theta_{\text{ref}})| \to 0\) in probability, which is (REG-EST).

6. Theorem 5B: Uniform Consistency and Rate

Theorem 5B. Stated under (R-RANDOM). Under the hypotheses of Theorem 5A:

\[\sup_{z \in \mathcal{Z}_0} \bigl\| \widehat{\beta}(z) - \beta_*(z) \bigr\| \;=\; O_P\bigl(r_n\bigr),\]

where \(\|\cdot\|\) is the Euclidean norm on \(\mathbb{R}^p\), \(\mathcal{Z}_0\) is any compact subset of the interior of \(\mathcal{Z}\), and the rate is \[r_n \;=\; n^{-\beta/(2\beta+d)} \cdot \sqrt{\log n}.\]

Equivalently, componentwise, \(\sup_{z \in \mathcal{Z}_0} |\widehat{\beta}_j(z) - \beta_{*, j}(z)| = O_P(r_n)\) for each \(j = 1, \ldots, p\).

(R-COND) variant. Conditioning on \(\{(X_i, z_i)\}_{i=1}^n\), the same uniform rate holds for \(\mu_{X, Z}^\infty\)-a.e. realization of the design.

Proof sketch. Hansen (2008) gives the uniform rate \(n^{-\beta/(2\beta+d)} \cdot \sqrt{\log n}\) for kernel estimators under (DESIGN-2) and (SMOOTH); the analog for splines under (KNOT)+(PEN) is in Stone (1985) and refined by de Boor (2001) and Wood (2017). The \(\sqrt{\log n}\) factor is the cost of moving from pointwise to uniform via union bound on a fine grid of \(z\)-values. \(\square\)

The uniform rate is strictly slower than the pointwise rate by \(\sqrt{\log n}\) (no global equivalence between the two metrics, parallel to the \(d_H\) vs. \(d_{L^2}\) situation of Block 4).

Boundary behavior. Near \(\partial \mathcal{Z}\), the spline estimator has higher variance and larger bias due to the asymmetric design. Boundary-corrected estimators (P-splines with appropriate corrections; Wood 2017 §5.4) achieve the same \(r_n\) rate up to the boundary. In mgcv, this is automatic for default basis choices.

7. Theorem 5C: Pointwise Asymptotic Normality

Theorem 5C. Stated under (R-RANDOM). Suppose the hypotheses of Theorem 5A hold and additionally:

• The error \(\varepsilon_i\) has mean zero and variance \(\sigma^2 < \infty\).
• The design is sufficiently regular at \(z\) for a CLT to apply (formally: \(f_z(z) > 0\), \(\beta_*\) is twice differentiable in a neighborhood of \(z\)).
• The penalty parameter is chosen at the rate (LAMBDA-DATA) or (LAMBDA-FIXED) with \(\lambda \to 0\) but \(n \lambda \to \infty\).

Then for each fixed \(z\) in the interior of \(\mathcal{Z}\), the vector-valued estimator \(\widehat{\beta}(z) \in \mathbb{R}^p\) satisfies a multivariate CLT: \[\sqrt{n / J_n} \cdot \bigl(\widehat{\beta}(z) - \beta_*(z) - \mathrm{bias}_n(z)\bigr) \;\xrightarrow{d}\; \mathcal{N}_p\bigl(0,\; V(z)\bigr),\]

where \(\mathrm{bias}_n(z) \in \mathbb{R}^p\) has \(\|\mathrm{bias}_n(z)\| = O(J_n^{-\beta/d}) = O(n^{-\beta/(2\beta+d)})\), and \(V(z) \in \mathbb{R}^{p \times p}\) is the asymptotic covariance matrix derived from the spline estimator at \(z\), a positive-definite matrix depending on the design density \(f_z(z)\), the conditional design \(f(X \mid z)\), the error variance \(\sigma^2\), the basis \(\mathcal{F}_a\), and the penalty \(\mathcal{P}\).

(R-COND) variant. Conditioning on \(\{(X_i, z_i)\}_{i=1}^n\), the conditional CLT holds with \(V(z)\) replaced by the conditional asymptotic covariance, which converges to \(V(z)\) for \(\mu_{X, Z}^\infty\)-a.e. realization of the design under (R-EQUIV) and standard uniform-class conditions.

Proof sketch. This is the standard CLT for spline (or kernel) estimators; see Fan and Zhang (2008, Theorem 3.2) for the VCM context and Wahba (1983) for spline-specific results. The argument uses Lyapunov’s CLT applied to the spline coefficient estimates and translates the result to the function value at \(z\) via continuous mapping. \(\square\)

On the explicit form of \(V(z)\) (and why it is left generic here). Under the equivalent-kernel representation (Silverman 1984; Eubank 1999), the spline estimator \(\widehat{\beta}(z)\) admits a Nadaraya-Watson-like representation \(\widehat{\beta}(z) = \sum_i K^*(z, z_i) y_i\) with an equivalent kernel \(K^*\) determined by the basis and penalty, leading to a closed-form \(V(z)\) depending on \(f_z(z)\), \(f(X \mid z)\), \(\sigma^2\), and \(\int K^*(z, \cdot)^2 \, d\mu_z\). This closed form is specific to the equivalent-kernel approximation and the particular spline-penalty choice. The framework reports \(V(z)\) via the standard mgcv covariance estimator (which uses the Hessian of the penalized log-likelihood; Wood 2017 §6.10) without committing to a particular closed-form expression, both because the Hessian-based estimator is the practical object the user observes and because the closed-form equivalent-kernel expression introduces approximation error that is not always favorable in finite samples.

8. Proposition 5D: Adaptive Smoothing under Data-Driven \(\widehat{\lambda}\)

Proposition 5D. Suppose the hypotheses of Theorem 5A hold with (LAMBDA-DATA): \(\widehat{\lambda}\) selected by REML or GCV. Suppose additionally that the smoothness \(\beta\) of \(\beta_*\) is unknown but satisfies \(\beta > d/2\). Then under the regularity conditions of the cited literature (Wood, Pya, Säfken 2016 for REML; Wahba 1985 for GCV), \(\widehat{\lambda}\) typically adapts to the unknown smoothness and achieves rates that, in the cases treated by that literature, match or are close to the Stone-optimal rate \(n^{-\beta/(2\beta+d)}\) for the resulting estimator \(\widehat{\beta}(z; \widehat{\lambda})\).

Argument outline. REML for spline penalty selection corresponds to maximizing a marginal likelihood that, under the spline-smoothing equivalence with Bayesian Gaussian-process priors (Kimeldorf and Wahba 1970), is closely tied to the optimal posterior contraction rate. Wood, Pya, and Säfken (2016) establish the asymptotic behavior of \(\widehat{\lambda}\) under standard regularity; GCV achieves a comparable rate under additional conditions (Wahba 1985). The adaptivity is generally good in practice, but the precise rate depends on:

• the true smoothness class of \(\beta_*\) (e.g., Sobolev vs. Hölder; isotropic vs. anisotropic);
• the basis choice in \(\mathcal{F}_a\) (B-spline, thin-plate, tensor-product);
• the form of the penalty \(\mathcal{P}\) (curvature penalty, ridge component, etc.);
• the design density \(f_z\) (uniform-like vs. concentrated);
• the noise distribution of \(\varepsilon_i\).

The framework does not claim universal adaptivity —only that REML/GCV adaptivity is a well-supported heuristic in the cases covered by the cited literature, and that empirical verification of the achieved rate (§11.1 below) is part of the standard reporting.

Caveat: post-selection inference is delicate. Confidence intervals from Theorem 5C constructed using \(\widehat{\lambda}\) in place of \(\lambda\) have approximate rather than exact coverage in finite samples: the variability of \(\widehat{\lambda}\) contributes additional uncertainty not captured by the standard formula for \(V(z)\). Krivobokova, Kneib, and Claeskens (2010) and Wood, Pya, Säfken (2016, §4) give corrections; these are implemented in mgcv via the freq and Bayesian confidence interval options. Reported credible/confidence intervals in gdpar Path 2 carry an explicit note that they are post-selection.

9. Specialization to Block 3 Special Cases

Path 2 covers the cases involving smooth varying coefficients (3.4, 3.5). Cases 3.2 (purely parametric) reduce to the standard \(n^{-1/2}\) rate of OLS, which Path 2 recovers as a special case with no penalty needed. Case 3.6 with multiplicative \(b\) is naturally handled in Path 1 (Bayesian) because \(b\) is not a smooth function of a separate modifier \(z\) but a pointwise multiplicative interaction.

10. Open Questions

(O1-Path2) Uniform inference under data-driven \(\widehat{\lambda}\). Confidence bands valid uniformly across \(z\) post-selection of \(\widehat{\lambda}\) are an active research area. Krivobokova-Kneib-Claeskens (2010) give partial results under specific penalty structures; the general case for arbitrary AMM Level 2 with non-trivial \(W(\theta_{\text{ref}})\) remains open. The framework’s current default reports pointwise CIs with a clear note that simultaneous coverage requires additional adjustment.

(O2-Path2) CLT for non-Hölder smoothness classes. Theorem 5C requires \(\beta > d/2\) (Sobolev embedding into continuous functions). For coefficient functions with \(\beta \leq d/2\) (rough functions), the pointwise CLT may not hold and convergence rates are slower; the framework applies \(L^2\)-rate results from Stone (1985) but does not establish a CLT. Identifying the CLT-valid class for non-smooth \(\beta\) is open.

(O3-Path2) Joint CLT across multiple \(z\) and across components. Theorem 5C is pointwise. Joint CLT for \((\widehat{\beta}(z_1), \widehat{\beta}(z_2))\) at two distinct \(z\) values (joint inference on functional values) and joint CLT across the AMM components \(a\) and \(W\) (joint inference on multiple smooth functions) require additional regularity not stated here. Standard arguments give the joint CLT under regular design and bounded support; rigorous verification across all special cases of Block 3 is an exercise but not a proven theorem of this block.

(O4-Path2) Identifiability of orthogonal-class construction in finite samples. The orthogonal-class construction of Block 3 §7 (Theorem 3.5) is asymptotically clean but in finite samples, the projection \((\mathrm{Id} - \Pi_{\text{lin}})\) has empirical noise. The interaction between this projection error and the spline-fitting error is not fully characterized; the practical effect is small in typical samples but a theoretical bound is open.

11. Implementation Implications for Path 2 (mgcv)

The planned Path 2 implementation in gdpar will call mgcv::gam (or gamm for GLMM-style fits) under the hood, exposing the AMM structure through the framework’s function-class interface. The current release (0.0.0.9001) does not ship this path; the design notes below specify the contract the operational layer will honour when it lands.

12. Summary

This block has established for Path 2:

1. Three asymptotic layers —pointwise consistency (5A), uniform consistency and rate (5B), pointwise CLT (5C)— parallel to those of Block 4 for Path 1.

2. Standing asymptotic hypotheses for Path 2: (DESIGN-2), (SMOOTH), (KNOT), (PEN), (LAMBDA-FIXED) or (LAMBDA-DATA), each named and formally stated.

3. Theorem 5A (Pointwise Consistency) at the Stone-optimal rate \(n^{-\beta/(2\beta+d)}\), with explicit (R-RANDOM)/(R-COND) variants and bridge to (REG-EST) of Block 2.

4. Theorem 5B (Uniform Consistency) at rate \(n^{-\beta/(2\beta+d)} \sqrt{\log n}\), with the \(\sqrt{\log n}\) factor explicitly noted as the cost of moving from pointwise to uniform.

5. Theorem 5C (Pointwise CLT) with explicit scope: pointwise valid only; uniform inference is the subject of (O1-Path2).

6. Proposition 5D (Adaptive Smoothing) under data-driven \(\widehat{\lambda}\) via REML/GCV, with the caveat that post-selection inference requires correction.

7. Specialization to Block 3 special cases with a tabular summary of which Path 2 cases are covered and at what rate.

8. Four explicitly recognized open questions specific to Path 2 (uniform post-selection inference, CLT for non-smooth \(\beta\), joint CLT, finite-sample orthogonal-class identifiability).

9. Implementation diagnostics for Path 2 via mgcv, including dual CI reporting (pointwise frequentist + Bayesian credible) and (REG-EST) verification.

The block does not claim to close the open questions; it specializes existing nonparametric-spline asymptotic results to AMM and recognizes the open questions explicitly where they appear.

13. Connections to Subsequent Blocks

• Block 6 (Asymptotics for Path 3, hypernetwork) treats the partial results available for neural-network-based estimation, with explicit recognition of the open questions in BvM and contraction rate for the hypernetwork case.
• Block 7 (Empirical Bayes vs. Fully Bayes) interacts with this block via the spline-Bayesian equivalence (Kimeldorf-Wahba 1970): Path 2 with REML is asymptotically equivalent to a particular Bayesian fit with Gaussian-process prior, providing a bridge between Paths 1 and 2.
• Block 8 (CATE/ITE positioning) places Path 2’s varying-coefficient estimator in the context of heterogeneous treatment effect estimators (BART; causal forests; X-, R-learner), which are related but use distinct asymptotic frameworks.

Appendix A. Asymptotic Notation for Path 2

Appendix B. Asymptotic Hypothesis Table for Path 2

References Cited in This Block

de Boor, C. (2001). A Practical Guide to Splines, rev. ed. Springer.

Eubank, R. L. (1999). Nonparametric Regression and Spline Smoothing, 2nd ed. Marcel Dekker.

Fan, J., and Zhang, W. (2008). Statistical methods with varying coefficient models. Statistics and Its Interface, 1(1), 179–195.

Hansen, B. E. (2008). Uniform convergence rates for kernel estimation with dependent data. Econometric Theory, 24(3), 726–748.

Kimeldorf, G. S., and Wahba, G. (1970). A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Annals of Mathematical Statistics, 41(2), 495–502.

Krivobokova, T., Kneib, T., and Claeskens, G. (2010). Simultaneous confidence bands for penalized spline estimators. Journal of the American Statistical Association, 105(490), 852–863.

Silverman, B. W. (1984). Spline smoothing: The equivalent variable kernel method. Annals of Statistics, 12(3), 898–916.

Stone, C. J. (1985). Additive regression and other nonparametric models. Annals of Statistics, 13(2), 689–705.

Wahba, G. (1983). Bayesian “confidence intervals” for the cross-validated smoothing spline. Journal of the Royal Statistical Society, Series B, 45(1), 133–150.

Wahba, G. (1985). A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. Annals of Statistics, 13(4), 1378–1402.

Wood, S. N. (2017). Generalized Additive Models: An Introduction with R, 2nd ed. Chapman and Hall/CRC.

Wood, S. N., Pya, N., and Säfken, B. (2016). Smoothing parameter and model selection for general smooth models. Journal of the American Statistical Association, 111(516), 1548–1563.