Making reliable forecasts the outcomes of a project is important for project planning. A typical example for a researcher is planning a collaborative new research paper. You have to coordinate a team and resources to pull it off. You will need to estimate how long different tasks will take, so you can coordinate the team. You will also need to forecast the likely outcome.
You would also typically make forecasts about the benefits on completion, such as the likely journal or other benefits such as policy decisions it will influence.
Below is a set of questions to test how you think about forecasting the outcomes of projects. First take the quiz. Then below is some discussion of how this self-evaluation can help you improve your decision making when planning projects.
The test is based on the Foxes and Hedgehogs dichotomy proposed by Philip Tetlock, a psychologist who specialises in how humans make forecasts.
A nice write up of the framework is in Nate Silver’s book Signal and the Noise.
Think about a project you planned and have been working on recently, such a research paper you are leading. Now scores yourself on each one of the below questions. Each question is in two parts. Score yourself 1-5, where 1 is a strong yes to the first question in each pair and 5 is a strong yes to the second question in each pair.
Try to be as honest with yourself as possible, don’t score yourself for what you think you should have done, score yourself for what you did (even better, get someone who knows you to score for you). The evaluation will be more useful that way.
Now add up your scores. If you scored less than 20 you are more like a ‘fox’ in your forecasts. If you scored more than 20 you are more like a hedgehog.
Foxes tend to be better at forecasting than hedgehogs. Foxes take into account diverse viewpoints, consider the complexity of the world, hold multiple theories for how the world operates at the same time, use data to inform forecasts, estimate the chances of different outcomes, adapt to changes in outcomes and acknowledge their mistakes.
Hedgehogs based their forecasts on a single theory, have simple theories for how the world works, use theories or their ‘gut’ to forecast outcomes, don’t draw on different perspectives, stick to a single plan or theory and tend to attribute forecasting mistakes to bad luck.
If you want to know more read the book or watch the video above.
It is not that hard to think more like a fox when you are planning and implementing your next research project. Just scan the questions above and try to action the first in each pair (e.g. seek multiple perspectives on your plan, use data from past projects to estimate how long this one will take).
Designed by Chris Brown. Source on Github