We present a report on Recommended Practices (RP), specifically part 3: evaluation of forecasts and forecast solutions, from the International Energy Agency’s Wind Task 36 (Wind Power Forecasting). The purpose of this Task 36 is to improve the value of wind energy forecasts to the wind industry, and the present report contains recommendations designed to help forecast users make informed decision relating to the procurement and evaluation of forecast products and services. The RP have been complied by an expert group representing 53 organisations including forecast vendors, consumers and academia from 13 countries, though development of the RP has been led primarily by a small task force group collecting the information gained by experience, discussions and workshops organised within the Task.
The evaluation of forecasts and forecast solutions is a requirement for any forecast provider as well as end-users of forecasts as significant business decisions are often based on evaluation results. Therefore, it is crucial to design forecast evaluation exercises with this importance in mind and to ensure that results are both meaningful and representative. Additionally, forecast skill and quality has to be understood in the context of forecast value: that is to say that the quality of a forecast should be evaluated based on the value it contributes to decision processes.
The RP have been developed through the combined experience and expertise of Task 36 members and wide consultation with the wind and energy forecasting industries. This has included engagement at international conferences and a public consultation on a draft RP.
The RP in Part 3 are based on the principal that evaluation results should be:
A set of recommendations is made relating to the design of evaluation exercises (metric selection, evaluation period, fairness and transparency, quantification of uncertainty), data quality (treatment of non-weather effects, IT failures, data quality assurance) and interpretation of results (verification vs metric-based approaches, cost-loss models and the economic impact of “typical” and “extreme” forecast errors, types of error, e.g. phase vs level error).
The recommendations may be summarised as follows:
We present a summary of the RP and some practical information as to how best to incorporate the recommendations into practise.