The Development of Forecast Confidence Measures Using NCEP Ensembles

• Other Authors: Durante, Andrew Vincent (authoraut)
• Format: Others
• Language: English; English
• Published: Florida State University
• Subjects: Meteorology
• Online Access: http://purl.flvc.org/fsu/fd/FSU_migr_etd-0627

Ensemble model data can provide a wealth of guidance to forecasters, especially in terms of forecast confidence. A model run where members diverge generally corresponds to a low confidence forecast, while a model run where members converge generally corresponds to a forecast of high confidence. The current NWS graphically based forecasts accessible to the public do not show this measure of uncertainty and thus communicate a false sense of confidence or precision. From August 2004 into 2006, approximately 2 years of individual GFS model ensemble data were analyzed. The result is a climatology of each ensemble member, which obviously does not match the observed climatology based on the NCEP reanalysis. The GFS model ensemble climatology was normalized so that there is a mapping between the current model ensemble value and the climatological value. Since there is only two years of data, the climatology is calculated on a 45 day temporal window. This method is similar, but more simplistic, to the method that is used in the FSU Superensemble (Krishnamurti 2000) of using temporal windows to increase climatology robustness in the training dataset. The variables analyzed here include 2-m temperature, 10-m wind speed and 10-m vorticity. Normalized climatology distributions have been calculated for each grid point within the ensemble member, with forecast confidence measures developed by comparing the normalized spread of the ensemble members to the model climatological spread, as described below. This normalized spread is compared to the typical spread for that time of year, location, and forecast length to arrive at a relative measure of forecast uncertainty. If the current model uncertainty is greater (less) than the uncertainty of the model climatology, then there is a lower (higher) than average confidence. Confidence graphics have been developed and analyses to see how confidence values behave with certain synoptic situations are ongoing. This overall behavior along with certain case studies will be featured. It has also been seen that there is a statistical significant difference in NWS forecast error between low confidence and high confidence regimes. Average NWS error for the below (above) normal GFS confidence forecasts was 5.20oF (3.08oF). A student t-test on these values revealed that there is a statistically significant difference to 95% confidence of the mean forecast error during low and high confidence GFS forecasts. That is, the mean WFO forecast error is significantly increased during times of low forecast confidence in the GFS ensemble. Therefore, forecasters have a-priori knowledge of the likely human forecast error when they see the GFS ensemble output-- before the NWS forecast even verifies. During cases of extreme low confidence where the current model standard deviation is greater than the 25-year observational standard deviation, a climatology forecast was found to be more accurate than the overall ensemble mean. Although the confidence graphics are only based on the GFS ensembles as of now, more models will be added in the future to see how they behave when compared to each other. The GFS ensembles and the corresponding confidence technique have been used in the FSU-MM5 to see how a mesoscale model affects the overall confidence for a specific case. Recent feedback from NWS employees suggests an additional development of confidence graphics based on the "poor man's ensemble", which is an ensemble of all the operational forecast models. Eventually these graphics of below and above average confidence may be implemented into the Graphical Forecast Editor (GFE) for use in the National Digital Forecast Database (NDFD). === A Thesis submitted to the Department of Meteorology in partial fulfillment of the requirements for the degree of Master of Science. === Degree Awarded: Summer Semester, 2006. === Date of Defense: June 29, 2006. === Hart, Weather, NWS === Includes bibliographical references. === Robert Hart, Professor Directing Thesis; Henry Fuelberg, Committee Member; T.N. Krishnamurti, Committee Member; Andrew I. Watson, Outside Committee Member.