Modelling forecast uncertainty

A Practical Guide to Model-Based, Bootstrap, and Conformal Prediction in R.

Modelling forecast uncertainty

A practical guide to model-based, bootstrap, and conformal prediction in R.

Harsha Chamara Hewage | 2026-02-25

Slides GitHub Repo

Who is the Course For?

This course is intended for public health researchers, epidemiologists, and practitioners who want to quantify uncertainty and measure tail risks in climate-sensitive disease forecasting. It assumes familiarity with basic time series forecasting and data manipulation in R.

Learning Objectives

• Understand the limitations of point forecasts and why asymmetric risk matters in public health resource planning.
• Distinguish between prediction intervals, forecast distributions, and sample paths.
• Apply model-based prediction intervals, such as ARIMA, as a baseline benchmark.
• Simulate and visualize non-linear outbreak trajectories using bootstrapping.
• Construct model-agnostic prediction intervals with finite-sample guarantees using conformal prediction.
• Link data characteristics, including volume, skewness, and climate drift, to an appropriate uncertainty quantification method.

Prerequisites

• Comfort with basic probability, including normal distributions, means, and variances.
• Familiarity with R, tidyverse, and basic forecasting workflows such as fable.
• No deep theoretical mathematics, proofs, or prior knowledge of conformal inference is required.

Course Topics

1. The Illusion of Certainty

• The point forecast fallacy and asymmetric risk in disease management
• Prediction intervals versus forecast distributions
• Visualizing the bounds, shape, and dynamics of forecast risk

2. Model-Based Uncertainty

• Parametric assumptions, normality, and homoscedasticity
• Fan charts and how uncertainty scales over time
• Implementing analytic intervals in R with fable
• The symmetry trap, negative case counts, and fat tails

3. Bootstrapping Sample Paths

• Using historical residuals to relax distributional assumptions
• Generating and plotting multiple possible futures
• Capturing skewed risk and explosive outbreak dynamics
• IID versus block bootstrapping for autocorrelated climate data

4. Conformal Prediction

• Moving from distribution-based to empirical, model-agnostic intervals
• The split conformal workflow with training and calibration sets
• Calculating conformity scores to guarantee valid coverage
• Exchangeability and the challenge of climate drift

5. Decision Framework and Application

• Trade-offs between computational speed, flexibility, and guarantees
• Choosing a method based on dataset size and outbreak volatility
• When to select model-based, bootstrap, or conformal methods