Designing Multimodel Applications with Surrogate Forecast Systems

Smith, Leonard A.; Du, Hailiang; Higgins, Sarah

Abstract

Probabilistic forecasting is common in a wide variety of fields including geoscience, social science, and finance. It is sometimes the case that one has multiple probability forecasts for the same target. How is the information in these multiple nonlinear forecast systems best “combined”? Assuming stationarity, in the limit of a very large forecast–outcome archive, each model-based probability density function can be weighted to form a “multimodel forecast” that will, in expectation, provide at least as much information as the most informative single model forecast system. If one of the forecast systems yields a probability distribution that reflects the distribution from which the outcome will be drawn, Bayesian model averaging will identify this forecast system as the preferred system in the limit as the number of forecast–outcome pairs goes to infinity. In many applications, like those of seasonal weather forecasting, data are precious; the archive is often limited to fewer than 26 entries. In addition, no perfect model is in hand. It is shown that in this case forming a single “multimodel probabilistic forecast” can be expected to prove misleading. These issues are investigated in the surrogate model (here a forecast system) regime, where using probabilistic forecasts of a simple mathematical system allows many limiting behaviors of forecast systems to be quantified and compared with those under more realistic conditions.

Citation

Smith, L. A., Du, H., & Higgins, S. (2020). Designing Multimodel Applications with Surrogate Forecast Systems. Monthly Weather Review, 148(6), 2233-2249. https://doi.org/10.1175/mwr-d-19-0061.1

• Journal Article Type: Article
• Acceptance Date: Oct 31, 2019
• Online Publication Date: May 5, 2020
• Publication Date: Jun 30, 2020
• Deposit Date: Jun 3, 2020
• Publicly Available Date: May 5, 2021
• Journal: Monthly Weather Review
• Print ISSN: 0027-0644
• Electronic ISSN: 1520-0493
• Publisher: American Meteorological Society
• Peer Reviewed: Peer Reviewed
• Volume: 148
• Issue: 6
• Pages: 2233-2249
• DOI: https://doi.org/10.1175/mwr-d-19-0061.1