Section navigation is enabled for this long article; the full text remains continuous.
BSTVC & Spatiotemporal Interpretability · Part 11 · Chao Song · 3 November 2026
Imagine hiding one year's observation from one district. The model still sees neighbouring districts and that same district in other years. Predicting the hidden value is a useful test, but its information advantage matters: it is a different task from predicting a new region or a future year.
For BSTVC and other spatiotemporal models, what remains available defines what a validation score can support.
What does a CPO measure?
The R-INLA documentation defines the conditional predictive ordinate for observation i as:
CPOᵢ = p(yᵢ | y₋ᵢ).
Here, y₋ᵢ means the other observed responses. The held-out response's likelihood contribution is excluded from the posterior used to predict it. Known covariates and the specified model are implicit in the notation. For a discrete outcome this is a predictive probability mass; for a continuous outcome it is a density, which need not be below one.
Log CPO values can be added to summarize pointwise leave-one-out predictive density. Compare models on the same outcome scale and evaluated observations, using the same scoring convention; larger log predictive density is better under that convention. This is different from scoring fitted values against data used to estimate them.
Separate the prediction target from the calculation
Leave-one-out, or LOO, specifies a validation task. Directly refitting after each omission is one way to evaluate it. INLA provides an approximation to CPO within its inference framework.
PSIS-LOO uses full-data posterior draws and Pareto-smoothed importance weights to approximate leave-one-out predictive evaluation. The smoothing stabilizes the importance-sampling calculation; it does not change which observation is supposed to be withheld.
An INLA CPO result therefore does not demonstrate that PSIS-LOO was run. Report the actual computation, its diagnostic checks and uncertainty in model comparisons. A small difference in estimated predictive performance need not establish a meaningful winner.
Two diagnostics, two different warnings
INLA's CPO failure indicator flags possible problems with its approximation. The official FAQ describes checking these indicators and recomputing affected CPOs. A flag is a reason to investigate the numerical calculation, not an instruction to discard an observation.
The Pareto-k diagnostic instead assesses the reliability of the PSIS importance-sampling approximation. Use its sample-size-aware guidance; posterior draw count is not the number of observations. A large k may call for direct refitting or another suitable evaluation approach. A different fold design also changes the training set and must be reported.
Neither diagnostic is a prediction-accuracy percentage. Passing a numerical check does not establish that the validation question matches the intended application.
Figure 1. AI-generated conceptual teaching illustration, not empirical results or software output. The panels show separate schematic split designs, not a common geographic dataset; grid axes differ between panels. Observation-level LOO retains other observations; a region holdout removes a location's observations; a forward-time split trains only on earlier data. The text gives the methodological limits.
Match the split to the intended use
Roberts and colleagues explain why spatial, temporal and hierarchical structure matters for cross-validation. A nearby training observation may be appropriate for interpolation, but unavailable when transferring a model elsewhere.
- A missing observation in a known setting: observation-level withholding may be relevant if the remaining information matches actual use.
- A new region: consider withholding entire regions or spatial blocks, with separation chosen for the intended prediction distance.
- Later years: train on the past and evaluate later observations, without using future information in preprocessing or model selection.
- A new region in the future: design both spatial and temporal separation explicitly.
These are design examples, not a claim that blocking is always superior. A block can create covariate extrapolation that is harsher than the interpolation task a model will actually face. All response-derived preprocessing must respect the split.
Before presenting a score, answer four questions: What was withheld? What information remained? Would that information exist when making the prediction? Was the numerical approximation reliable? Spatiotemporal interpretability benefits from prediction evidence whose scope is stated precisely.
中文速读
留一验证检验的，是“删去一条响应后，利用其余信息预测它”的能力。INLA 的 CPO 与 PSIS-LOO 可以面向相关的留一预测问题，但计算方法和诊断不同，不能混称。逐点留一仍保留邻近地区或其他年份的数据，因此不能自动证明新地区迁移或未来预测能力。验证划分应贴合实际用途；计算警示需要核查，不等于应删除观测。
Read, verify and use
This article explains validation principles, not a BSTVC experiment or a verified software feature. It credits the BSTVC team and the original methods authors. Explore the BSTVC research programme and BSTVC Desktop; check its current availability notice and licence before installation or reuse. No installer availability is implied here.
Primary reading: R-INLA FAQ: CPO and numerical checks; Vehtari, Gelman and Gabry (2017), Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC; Vehtari et al. (2024), Pareto Smoothed Importance Sampling; Roberts et al. (2017), Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure.
Continue the series: Part 10: STVPI, Bayesian R² and predictive accuracy · Part 5: coefficient uncertainty · Series introduction.