6 The different types of uncertainty
“How uncertain am I?” is not one question but several. Asking which kind of uncertainty you’re dealing with is helpful, because they behave differently and some can be reduced while others cannot.
Uncertainty is a scientific concept and we need to be careful when using it outside of scientific settings, such as in science communication. For the general public uncertainty means “we don’t know anything”. For scientists it can mean the same thing, but more commonly it means “we aren’t sure but we can estimate how unsure we are”. Typical examples are confidence and credible intervals, which put a bound on our uncertainty.
Stochasticity is the technical term for this randomness: a process is stochastic if it can’t be predicted exactly, only described in terms of probabilities of different outcomes. This contrasts with a deterministic process, which always produces the same output from the same starting conditions. In ecological models, stochasticity is usually simulated by drawing random numbers from a probability distribution (e.g. adding random draws from a normal distribution to a population growth rate, or drawing survival events from a binomial distribution) and is the reason two runs of the same stochastic model, from the same starting point, can give different trajectories.
Some of the different types of uncertainty we encounter that you might want to model:
Process (or environmental) uncertainty: genuine randomness in the system itself. The world is stochastic: two identical populations in identical conditions won’t follow identical trajectories. This is irreducible (more data tells you about it but never removes it), and it’s what a process-error term in a dynamic model represents.
Observation (or measurement) error: the gap between the true state and what you managed to measure. Your survey miscounts; your instrument is noisy. This is a property of the observing, not the system, and it can often be reduced with better methods or more sampling. Keeping process and observation error apart is one of the central problems of state-space modelling, and confounding the two is a classic trap.
Parameter uncertainty: you’ve settled on a model structure, but you don’t know its parameters exactly; you have estimates with intervals. This is reducible: more (and more informative) data tightens it. This is also the uncertainty that estimation problems such as identifiability and confounding can inflate without you noticing.
Structural (or model) uncertainty: you’re not even sure the model itself is right, meaning which processes to include, which functional forms, which interactions. This is usually the largest and most neglected source, because it doesn’t show up inside any single model: a model is supremely confident about a world that may not be ours. It’s addressed by comparing models, model averaging, or ensembles, not by collecting more data under one structure.
Initial-condition and boundary uncertainty: for a dynamic model, uncertainty about where the system starts (or about the forcing pushed in at its edges) that then propagates forward. This is why forecasts can fan out even when the model and parameters are perfect.
Scenario uncertainty: uncertainty about the inputs you can’t predict at all, such as future policy, harvest, or climate forcing. You don’t normally put a probability on these; you explore them as scenarios (exactly the scenario modelling from earlier).
Reducible uncertainties (observation error, parameter uncertainty) can be shrunk by better data or better measurement, so they’re where “go and measure this” advice pays off.
Structural uncertainties can also be reduced, but usually by doing different types of science, not more of the same. An example would be designing a new type of experiment to test between two competing mechanisms.
Irreducible ones (process uncertainty) have to be represented if we want well-calibrated probability intervals. That means, if we say our interval is a 95% predictive interval, then 95% of predicted events should fall in that interval.
A model that reports tight intervals has usually only accounted for parameter uncertainty and is silently ignoring the rest. That gives a tidy-looking answer more confident than the world warrants.
The uncertainty topic is a minefield with a large literature (and a fair amount of inconsistent terminology), so treat the above as an orientation rather than the last word.
Uncertainty doesn’t stop at the model. The list above is framed around the model and its data, but in applied and management settings the most important uncertainties often lie outside the model entirely, in the human system it’s embedded in. Peterman’s (2004) account of fisheries is a good concrete example. He identifies five sources of uncertainty in a fishery system, only the first two of which are the “modelling” kinds we’ve discussed:
- Natural variability across space and time in the distribution, abundance, and productivity of the fish populations (our process uncertainty).
- Observation error: imperfect information arising from both measurement error and sampling error.
- Communication difficulties among scientists, managers, and stakeholders about technical scientific information and its associated uncertainties. Sometimes also called linguistic uncertainty, to differentiate it from the epistemic uncertainties we discussed above (Regan, Colyvan, and Burgman 2002)
- Unclear management objectives.
- Implementation error: the gap between a management goal and the actual realised outcome (e.g. the spawning-stock biomass or fishing-mortality rate that actually results).
The lesson for a new modeller is that a beautifully calibrated model addresses only sources (1) and (2). If your work is meant to inform a decision, sources (3), (4), and (5) can dominate the outcome, and no amount of modelling skill removes them. Can everyone understand and agree on what the model says? Is it even clear what we’re trying to achieve? Will the intended action actually be carried out? Knowing which of these you’re actually up against is half the battle.