Statistical analysis of forecasting models across the north slope of Alaska during the mixed-phase Arctic clouds experiment

The National Centers for Environmental Prediction's (NCEP) Eta Model, the models of the European Centre for Medium-Range Weather Forecasts (ECMWF) and the National Aeronautics and Space Administration's (NASA) Global Modeling and Assimilation Office (GMAO) models, and the Regional Atmospheric Modeling System (RAMS) model are all examined during the Mixed-Phase Arctic Clouds Experiment (MPACE) that took place from 27 September through 22 October 2004. During two intensive observation periods, soundings were launched every 6 h from four sites across the North Slope of Alaska (NSA): Barrow, Atqasuk, Oliktok Point, and Toolik Lake. Measurements of temperature, moisture, and winds, along with surface measurements of radiation and cloud cover, were compared to model outputs from the Eta, ECMWF, GMAO, and RAMS models using the bootstrap statistical technique to ascertain if differences in model performance were statistically significant. Ultimately, three synoptic regimes controlled NSA weather during the MPACE period for varying amounts of time. Each posed a unique challenge to the forecasting models during the study period. Temperature forecasts for all models were good at the MPACE sites with mean bias errors generally under 2 K, and the models had the fewest significant errors predicting temperature. Forecasting moisture and wind proved to be more difficult for the models, especially aloft in the 500-300-hPa layer. The largest errors occurred in the GMAO model, with significant moist biases of 40% and wind errors of 10 ms^-1 or more. The RAMS, Eta, and ECMWF models had smaller moist biases in this layer. Both the Eta and RAMS models overestimated the surface incident shortwave radiation, underestimated longwave radiation, and underestimated cloud cover fraction. Overall, the bootstrapping results coincided with findings from conventional statistical comparisons as model outputs with the largest errors were most likely to be captured and declared statistically significant in the bootstrapping process. The significant model errors during MPACE were predominantly traced to the inability of the models to simulate disturbances in synoptic regime I, warm or cold biases over higher inland terrain, a warm bias along the NSA coastal waters in the Beaufort Sea, and difficulty in forecasting the intensity of the explosive cyclone in synoptic regime III.