In this paper, we develop a novel way to measure the accuracy of weather forecasts based upon the impact on air traffic flows. This method uses new techniques developed as part of the CWAM that consider the complicated interaction between pilots, air traffic controllers and weather. This technique, known as the blockage model (Martin et al., 2006), differentiates between minor deviations performed by pilots around convective weather and their larger deviations due to fully blocked air routes that require air traffic control interaction. This blockage model is being used by the automated Route Availability Planning Tool (RAPT) to predict route blockage for NYC departures. RAPT integrates the Corridor Integrated Weather Systems (CIWS) deterministic 0-2 hour forecasts of precipitation and echo tops into route specific forecasts of impact on air traffic in the congested east coast corridor. Applying the blockage model to the entire CIWS weather domain as a metric for scoring the performance of the forecast algorithms is shown to be an excellent approach for measuring the adequacy of the forecast in predicting the impact of the convective weather on air traffic operations.