8 Understand your constraints
Once you know roughly what kind of model you want, the next thing that shapes it is what you’re able to build. Constraints come in several kinds.
8.1 Computational constraints
I like to think about these in terms of different types of computational resources; which one matters depends on the type of model:
- Processor speed (CPU): how fast you can do a single complex calculation. This is your bottleneck when each step depends on the last, e.g. a simulation stepping through time or the MCMC algorithms used to fit Bayesian models.
- Number of CPUs, “threads”: how many calculations you can run at the same time. This matters when steps can be run in parallel on different processors. Examples include repeat temporal simulations with stochastic behaviour, or multiple MCMC chains initialised at different starting points.
- Memory (especially RAM): how much data your computer can hold at once in its easily accessible memory. This matters for large data structures. It is often a limitation for global spatial computations, or models that need to store many intermediate calculations.
- GPU: the extreme of “many small calculations at once”, ideal for large-matrix work (this is why it powers so much of AI). GPUs require specialised programming packages to work with.
The general rule when you start out: work well within these limits. If you push your computational limits at the same time as you’re still developing the model, you’re trying to solve conceptual problems and technical problems simultaneously. That’s a difficult path to walk. Keep it simple at first.
8.2 Technical constraints
There are hard mathematical limits: some equations and integrals simply can’t be solved analytically. Some of these can be overcome with new mathematical techniques, or by borrowing a trick from another discipline. Increasingly we lean on numerical methods instead, which is good in that it opens modelling to people who are strong in ecology but not formally trained mathematicians. The trade-off: numerical approaches are great for specific cases but harder to turn into general solutions, and solid mathematical knowledge can still reveal analytical shortcuts that are just as accurate and save enormous amounts of computation.
Computer programs can also be a technical constraint, although this is increasingly rare. The languages of scientific computing (mostly R and Python) have many advanced packages available. Most tasks you will want to do as a modeller are already provided in a package or can be created by building on other packages if you have the technical programming ability.
The most important technical constraint, though, is often your own expertise in mathematics, statistics and programming. This is where good training and mentorship really matter. And the same principle applies as with computation: it’s hard to push your own technical limits at the same time as pushing the modelling itself. Expect an iterative back-and-forth of learning techniques and applying them.
8.3 Resource constraints
Time and money are important constraints to consider. Money buys people’s time and a good computer; time is often the real limit on a student project. It’s worth asking honestly and early whether the project is too ambitious for the time you have, given everything you’ll need to read and learn along the way.
8.4 Estimation and optimisation constraints
When you try to estimate parameters from data, a whole family of related issues appears: identifiability, exchangeability, separability, and confounding.
A concrete example is trying to estimate r and K in a logistic population-growth model: depending on your data they can be effectively exchangeable or confounded, so that the data simply can’t pin them down separately. A time-series that shows only the growth phase of a population won’t reveal much about K, and the model estimation algorithm will struggle to differentiate a high K from a high r (Szuwalski and Thorson 2017). The fix, where you have any control over it, is to get data that spans a wider range of population dynamics. Where the data don’t span these different dynamics prior information on population parameters can also help constrain the fit (Millar and Meyer 2000).
Another example of exchangeability occurs when two covariates are correlated when fitting a linear model to a response. When two variables are correlated there are many ways the estimation algorithm can add them together to get the same response value. This results in multiple parameter combinations that have equal probabilities (or likelihoods if you are frequentist). For example, temperature and precipitation are commonly positively correlated. A model of species abundance fit to these two variables could predict an equally high abundance for low temperature, high preciptation; high temperature low precipitation; or moderate temperature and precipitation.
This confounding leads to multiple constraints. It constrains how we interpret the effects of each covariate independently because we can’t say what one effect is without first defining the value of the other variable. It constrains our interpretation of causes (see Arif and MacNeil (2023)).
These estimation constraints also lead into technical constraints, algorithms can have trouble converging on an answer where there are many equally good answers available. A good principle here is the Folk Theorem of Statistical Computing
When you have computational problems, often there’s a problem with your model
I’ll extend to this ‘or a problem with your model-data fit.’
Finally, the title of this section is estimation AND optimisation constraints. Statistical estimation is optimisation to find parameters and their distributions. So all the above constraints also apply to any optimisation you might be doing. For instance, I have often run into exchangeability-type issues when trying to find optimal decisions for environmental decision analysis. The algorithms struggle when there are many decision combinations that give equally good results.
8.5 Start simple, then grow
The practical upshot of all of these constraints is the same advice: start well inside your limits and expand outward. There are four strategies I have for starting simple. They apply equally well to constraints in large datasets and constraints created by large complex models.
Subset the problem. Start work with a subset of the data, or a subset of the model’s equations. For data, this might simply be fewer samples, or all samples but from a single region. It may also mean working with fewer variables, such as picking five species to build a multivariate model, before progressing to all 50.
For a model’s equation you can also work with subsets. If your foodweb has 50 interdependent equations, start by developing one corner of the foodweb first.
Aggregate. You can often summarise your data, at least initially, so the overall number of samples is smaller. For instance, with spatial raster data, aggregate it onto a larger grid. This will mean computations are faster. Likewise, for a dynamic model you could develop it initially for annual time-steps, then work down towards daily time-steps.
Approximate. Approximations usually go hand-in-hand with aggregation. For a dynamic model you could approximate a daily process by time-averaging to years. Or there might be analytical approximations to some steps of the model, such as using an equilibrium solution for a dynamic process that moves much faster than your model time-step.
I use the approximation trick less often in statistical modelling. But it is hardwired into many statistical estimation algorithms. The INLA package, for instance, uses a very clever approximation trick to get some types of Bayesian models to fit much faster than other leading software.
Sacrifice precision. Simply rounding to fewer digits can often save a lot of computational demand and speed things up. In Bayesian estimation we might run fewer chains or fewer steps as we try to develop an error free model. We then increase the steps and chains for a final run to get precise parameter estimates.
I use one or a combination of these approaches when I’m working in unfamiliar territory, then I increase granularity by degrees. A life-stage model can start with fewer stages before you add more; a model with many dynamic parameters can start with just a few. Get something working, then make it more complex.
Just remember that model behaviour does not typically extrapolate intuitively as you progress to larger datasets or more complex models. New problems will arise as you go bigger, and the model’s predictions may well change (that’s what keeps modelling interesting and relevant). It’s just nice to start small, as it reduces the number of problems you face initially, and helps you break them down into discrete, solvable steps.