How Is AIFS Doing Vs. Other Forecast Models? Some Preliminary Results

Fig. 1: Mean average error scores (from CWG/SVWM) for 6-10-day forecasts from the past 30 days for all models averaging results from NAO, PNA, WPO, and EPO.

Discussion: To evaluate AI forecasts, some preliminary verification results are recently available. The verification presented is for climate signal forecasts for the 6-10-day period from the past 30 days (Fig. 1). Evaluated is the mean average error (MAE) for North Atlantic oscillation (NAO), Pacific North America (PNA) index, East Pacific oscillation (EPO), and West Pacific Oscillation (WPO). The MAE for each climate signal is combined for 5 operational models and 5 AI processes. Final skill scores are reasonably close. The Artificial Intelligence Forecast System (AIFS) ranked no. 1 overall followed closely by the Canadian Ensemble (CMC ENS) and Global Forecast System (GFS). The Forecast Net GFS ranked last in this comparison. While AIFS ranked No. 1, operational models ranked 2 through 4. Clearly, based on this sampling, AIFS is “competitive”. Combining all 10 models and their forecasts of NAO, PNA, WPO, and EPO for the 6-10-day period from the past 30 days, most skillful was the PNA forecast while least skillful was the NAO forecast (Fig. 2). Implied are the upper air forecasts across western North America is best, easter North America worst, and middle of the road results in-between as identified by WPO and EPO.

Fig. 2: Combining mean average error for 10 models and their forecasts of NAO, PNA, EPO, and WPO during the past 30 days.