Can a Model Travel to a New Region?

BSTVC & Spatiotemporal Interpretability · Part 13 · Chao Song · 6 November 2026

A model predicts missing observations well across a familiar study area. Can it now predict a region whose outcomes have never been observed?

The answer depends on what was withheld during evaluation and what the new region changes. For BSTVC and other spatial models, filling gaps among known places and transferring to a new region require different evidence.

Begin with the destination

Suppose a health study contains annual observations for many districts. Randomly withholding district-year records may leave neighbouring outcomes, and other years from the same district, available for training. That can be useful for reconstructing missing observations within the study domain.

A new-region task asks more: how well can the fitted procedure predict where local outcome information is unavailable? If the target is an entirely unobserved district, its outcomes across all years should remain outside training. Forecasting in that district also requires a time cutoff, as explained in Part 12.

Roberts and colleagues (2017) explain why validation must respect the structure of the data and the intended prediction task. A model that accounts for spatial dependence still needs an appropriate validation design.

Figure 1. AI-generated conceptual teaching illustration, based on the validation distinction discussed by Roberts et al. (2017) and Valavi et al. (2019). Teal marks training outcomes; orange hatching marks held-out outcomes. The upper panel leaves scattered gaps among observed places. The lower panel withholds a contiguous region. These are fictional geometries, not study maps, fitted BSTVC output or measured performance.

Let the split resemble deployment

Spatial blocks, leave-region-out splits and buffers are ways to create different evaluation problems. Valavi and colleagues (2019) describe tools for spatially or environmentally separated folds.

Choose the design before inspecting the final scores:

• Nearby gaps: retain neighbouring observations if they would genuinely be available when making the prediction.
• An unobserved region: withhold that region’s outcomes together, including its repeated measurements.
• A distant destination: consider separation or a buffer that resembles the intended distance from training data.

There is no universally correct block size. Adjacent blocks can still contain nearby training and test observations. A buffer changes both separation and the amount of training data. Report these choices and why they represent the intended use.

Learned preprocessing and model selection must also respect the split. Estimate scaling, imputation rules and tuning choices from training data, with an inner validation design when needed. Final held-out outcomes should evaluate the selected procedure.

Check environmental support as well as distance

A nearby district can have unfamiliar combinations of population density, climate or healthcare resources. A distant district can have familiar predictor values. Geographic novelty and environmental novelty are related but different questions.

Meyer and Pebesma (2021) propose an area of applicability based on predictor-space dissimilarity and cross-validation. Their work highlights a useful check: a location can be outside the supported combinations of predictors even when each individual value falls within its training range.

Such diagnostics help identify where performance estimates may not apply. They do not prove that predictions are accurate, causal or well calibrated, and the public BSTVC documentation reviewed here does not establish them as a built-in feature.

Make the new-area assumptions explicit

BSTVC’s public methods overview describes structured spatial effects, dynamic prediction and missing-value imputation. These capabilities do not by themselves establish validated transfer to an entirely new region.

For a new area, specify how local coefficients, random effects and spatial relationships are defined. If the model uses an adjacency graph, explain how new nodes, neighbours and disconnected areas are handled. Whether refitting is needed depends on the model and implementation.

Withholding outcomes while retaining test locations in a known spatial graph evaluates inference on that declared graph. Adding previously absent locations after fitting is a different workflow. State which one was tested.

Posterior intervals remain conditional on the model and these assumptions. Narrow coefficient intervals alone do not establish reliable prediction of new observations or protection against distribution shift.

Report the journey, not just one score

Compare models and meaningful simple benchmarks on the same held-out regions. Report error by region, the weighting used for pooled summaries, and predictive interval coverage alongside width. Keep coefficient credible intervals separate from intervals for future or unobserved outcomes.

For spatiotemporal interpretability, the useful claim is specific: where was the model tested, what information was available, and how similar is that setting to the intended destination?

中文速读

在熟悉地区补齐缺失值，与把模型迁移到没有结局观测的新地区，是不同任务。新地区验证应整体留出该地区的结局，必要时结合时间截断、空间分块或缓冲；距离近也不代表协变量组合熟悉。还需说明新地区的局部系数、随机效应和空间邻接如何处理。BSTVC 的动态预测与插补能力不能直接当作跨地区迁移已经验证的证据。本文介绍验证设计，不报告新的实测结果。

Read, verify and use

• Roberts, D. R., et al. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography, 40, 913–929. Original paper.
• Valavi, R., et al. (2019). blockCV: An R package for generating spatially or environmentally separated folds for k-fold cross-validation of species distribution models. Methods in Ecology and Evolution, 10, 225–232. Original paper.
• Meyer, H., & Pebesma, E. (2021). Predicting into unknown space? Estimating the area of applicability of spatial prediction models. Methods in Ecology and Evolution, 12, 1620–1633. Original paper.

Continue with Part 11: what leaving one observation out tests or explore the BSTVC research programme and Desktop V2 documentation. Check the current documentation and licence before use. Credit belongs to the BSTVC team and the original methodological authors.