Modelling forecast uncertainty
Modelling prediction uncertainty
A Practical Guide to Model-Based, Bootstrap, and Conformal Prediction in R.
Who is the Course For?
This course is intended for public health researchers, epidemiologists, and practitioners who want to quantify uncertainty and measure “tail risks” in climate-sensitive disease forecasting. It assumes familiarity with basic time series forecasting and data manipulation in R.
Learning Objectives
Understand the critical flaw of point forecasts and why asymmetric risk matters in public health resource planning.
Distinguish between prediction intervals, forecast distributions, and sample paths.
Apply and extract Model-Based prediction intervals (e.g., ARIMA) as a baseline benchmark.
Simulate and visualize non-linear outbreak trajectories using Bootstrapping.
Construct model-agnostic prediction intervals with finite-sample guarantees using Conformal Prediction.
Link data characteristics (e.g., volume, skewness, climate drift) to the appropriate uncertainty quantification method.
Prerequisites
- Comfortable with basic probability (e.g., Normal distributions, mean, variance).
- Familiarity with R,
tidyverse, and basic forecasting workflows (e.g., thefablepackage). - No deep theoretical mathematics, proofs, or prior knowledge of conformal inference is assumed.
Course Topics
Section 1: The Illusion of Certainty (Foundations)
The point forecast fallacy and asymmetric risk in disease management
Defining Prediction Intervals vs. Forecast Distributions
Visualizing the bounds, shape, and dynamics of forecast risk
Section 2: Model-Based Uncertainty (The Baseline)
Parametric assumptions (Normality, Homoscedasticity)
The “fan chart” and how standard deviation scales over time
Implementing analytic formulas in R (
fable)The “symmetry trap” (predicting negative case counts) and fat tails
Section 3: Bootstrapping Sample Paths (The Simulation)
Overcoming the Normal distribution assumption using past residuals
Generating and plotting multiple “possible futures” (sample paths)
Capturing skewed risk and explosive outbreak dynamics
Introduction to IID vs. Block bootstrapping for autocorrelated climate data
Section 4: Conformal Prediction (The Guarantee)
Moving from distribution-based to empirical, model-agnostic intervals
The Split Conformal workflow: Training vs. Calibration sets
Calculating conformity scores (residuals) to guarantee valid coverage
The exchangeability assumption and the challenge of climate drift
Section 5: Decision Framework & Application
Evaluating trade-offs: Computational speed vs. flexibility vs. guarantees
Choosing the right method based on dataset size and outbreak volatility
Summary matrix: When to select Model-Based, Bootstrap, or Conformal methods