The increasing pace of climate change and human impacts on ecosystems is pushing ecosystems beyond the envelope of past conditions. The pace of change makes predicting that change much harder. In a new study, Assessing predictive accuracy of species abundance models in dynamic systems, we show how to test model forecasts in rapidly changing ecosystems.
Good tests of model forecasts help us quantify uncertainty and are important for many management applications including fisheries stock forecasts, species extinction risk and climate-smart conservation planning.
The pace of change is a problem for making predictions because our models are trained on historical data. Model’s predict best when they are predicting to conditions they’ve ‘seen’ in the training data. Unprecedented environmental change means our models need to extrapolate. This is much harder.
One recent example is the deadly algal bloom across 500km of South Australia’s coast. No ones saw this coming. No-one yet knows if its likely to reoccur or not.
We developed a way to test models with historical data to get an idea of how well they extrapolate. This gives us a better measurement of model uncertainty that we can use to calibrate risk, e.g. to see how much we might be under-estimating extinction risk or overestimating fish stock size.
We illustrated our method with a 30 year timeseries of reef biota from Maria Island. We split this time-series into sections used for training and sections used for testing the model. The model is then fit to the training data and forecast to the testing data. The difference between its forecast and the test data is a measure of its errors.
Our new method forces the model to fit to ‘out-of-date’ data. Whereas, the typical method keeps the models contemporary to the testing data. g We confirmed that for a species with a population collapse our new method gives a much more pessimistic (broader) estimate of uncertainty than the typical method would do.
We suggest our method be used to better estimate risk measures when using models to forecast ecological change. Our method can also be used in meta-studies that are exploring how predictability varies across different species, places and times. This is a point we’ll pick up in our next study. Read more here.
Designed by Chris Brown. Source on Github