Papers › Assessing Human Judgment Forecasts in the Rapid Spread of the Mpox Outbreak: Insights...
Assessing Human Judgment Forecasts in the Rapid Spread of the Mpox Outbreak: Insights and Challenges for Pandemic Preparedness
Thomas McAndrew, Maimuna S. Majumder, Andrew A. Lover, Srini Venkatramanan, Paolo Bocchini, Tamay Besiroglu, Allison Codi, Gaia Dempsey, Sam Abbott, Sylvain Chevalier, Nikos I. Bosse, Juan Cambeiro, David Braun
In May 2022, mpox (formerly monkeypox) spread to non-endemic countries rapidly. Human judgment is a forecasting approach that has been sparsely evaluated during the beginning of an outbreak. We collected -- between May 19, 2022 and July 31, 2022 -- 1275 forecasts from 442 individuals of six questions about the mpox outbreak where ground truth data are now available. Individual human judgment forecasts and an equally weighted ensemble were evaluated, as well as compared to a random walk, autoregressive, and doubling time model. We found (1) individual human judgment forecasts underestimated outbreak size, (2) the ensemble forecast median moved closer to the ground truth over time but uncertainty around the median did not appreciably decrease, and (3) compared to computational models, for 2-8 week ahead forecasts, the human judgment ensemble outperformed all three models when using median absolute error and weighted interval score; for one week ahead forecasts a random walk outperformed human judgment. We propose two possible explanations: at the time a forecast was submitted, the mode was correlated with the most recent (and smaller) observation that would eventually determine ground truth. Several forecasts were solicited on a logarithmic scale which may have caused humans to generate forecasts with unintended, large uncertainty intervals. To aide in outbreak preparedness, platforms that solicit human judgment forecasts may wish to assess whether specifying a forecast on logarithmic scale matches an individual's intended forecast, support human judgment by finding cues that are typically used to build forecasts, and, to improve performance, tailor their platform to allow forecasters to assign zero probability to events.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections