Interfaces to Applications

The work of Gilbert Walker was driven not by the need to understand the climate of India but by the need to plan for droughts in a largely self-dependent agricultural country. Much of the solution to the food problem in India was ultimately obtained through improved management of food stocks rather than through climate prediction, but India has the distinction of having the longest sequence of operational seasonal predictions using modern empirical techniques anywhere in the world. Elsewhere the long-term drought of the 1980's in the Sahel brought increased attention to the impact of sea surface temperature variations on rainfall and lead, at the UK Met Office, to the development of an empirical forecast model for rainfall over the region (Ward et al., 1989), and then to experiments with numerical models (Folland et al., 1991). A second area subsequently considered by the Met Office team was the Nordeste of Brazil (Ward and Folland, 1991), and the Met Office has provided forecasts regularly for this region since 1987. The Nordeste is subject to El Nino-related severe droughts that impact on both food and local industrial production. Such is the level of predictability that the Nordeste is favoured by having numerous prediction models available created by various research groups.

Applications of seasonal forecasts go back many years and a review of estimates of value obtained through use of the predictions has been published by Nicholls (1996). However as the science has reached a level of maturity over the past few years so the number of projects in which forecasts are fed into decision making in applications has increased and the question of value has become more critical. In the following discussion agricultural applications will provide the focus, although activities are underway in many other applications sectors including water management, energy planning and trading, manufacturing, insurance and health.

One of the longest consistent research projects studying the use of seasonal prediction to agriculture has been that focussed on corn production in the US Mid West (e.g. Mjelde et al., 1997). This series of studies has illustrated the potential value of predictions provided these are designed to fit directly into the needs of the farmer. The basic concept of "designed to fit" was taken as perfect deterministic predictions of temperature and rainfall anomalies delivered at a lead sufficient for decisions to be taken and acted upon. In later studies the concept of partially unreliable predictions was introduced. Substantial benefits were to be gained should prediction quality attain the levels assumed. However experience in the US and elsewhere has indicated that, despite the positive theoretical position established, multiple impediments to the use of seasonal to interannual predictions exist. These impediments lie both on the forecast producer side - including the provision of imperfect forecasts, in probabilistic terms, using non-intuitive presentation formats, at too short leads, in insufficient detail, and through limited-access delivery methods - and on the user side - including misinterpretation of probabilistic information, lack of full understanding of the limitations of forecasting, and want of optimal methods of incorporating real forecast information into decision processes.

Measurements of forecast quality, and its interpretation as value ultimately obtained in applications, remain amongst the key issues linking and offering at least partial solutions to many of the impediments. The issue of measurement of forecast quality has already been covered in brief. Most currently-used quality measures have been derived from the perspective of the forecaster, with consideration for the needs of the user often attached only as an add-on. A noteworthy sequence of studies in which theoretical attempts were made to link forecast quality with ensuing value was provided by Alan Murphy and collaborators (e.g. Ehrendorfer and Murphy, 1992). One central methodology used in these studies was the Cost/Loss model, in which value for a particular binary event (as defined above under ROC) was calculated for the four combinations of event predicted/not predicted with event observed/not observed. Attempts to relate quality and value were not entirely successful, however, until the introduction of ROC, as the approach used by ROC for examining quality permits transfer directly into the Cost/Loss model without transformation of data. This ROC-Cost/Loss approach was first used at a practical level within a seasonal forecasting context during the first SARCOF and produced results that illustrated the potential value in southern Africa of forecasts of then-existing quality (Harrison and Graham, 2001). Other studies of ROC and its use in estimating value have been made (e.g. Richardson, 2001).

One important result that appears consistent across all studies to date relates to the fact that ROC can be used to measure quality in both deterministic and probabilistic systems in a common manner. As far as is known highest value always results, at least in theory, from use of probability forecasts according to the ROC approach.8 This theoretical result is a consequence of the balance between value gained from correct decisions against value lost in incorrect ones. No forecast system can consistently guide perfect decision making, and the evidence suggests that the management and consequences of those occasions on which incorrect decisions are taken determine the final overall value obtained across many years. In the view of the author no quality measure approach that fails to provide detail on prediction limitations sufficient to satisfy value estimates through the range of potential outcomes is adequate for applications needs. One rider to this is that deterministic forecasts, despite offering lower value, can be used provided adequate recognition of the possibilities and consequences of an erroneous prediction is taken. However, as the consequences as far as an application is concerned may depend on the magnitude of the error, a distribution of outcomes is still desirable even in this case for planning purposes. Despite their potential importance few studies of approaches to contingencies in decision making within the context of seasonal forecasts have yet been undertaken. These, plus additional theoretical and practical value studies, are needed in order to design improved decision processes in applications. While the ROC approach to value is a useful starting point, it handles only limited aspects of issues related to individual decisions, it implicitly assumes certain vital aspects of the decision process, and it ignores socioeconomic interactions on both the micro and macro scales. ROC's merit in providing guidance in the use of seasonal forecasts in agriculture has, nonetheless, been established.

One additional benefit of using ROC illustrated in the SARCOF project is that it can be employed to indicate optimal strategies for forecast use by helping identify the selection of model parameters that maximises benefit (Harrison and Graham, 2001). However this carries the vital rider that it is quite possible to use a given set of forecasts in a way that would provide negative value rather than extracting the available positive value, a point that is frequently not considered in planning applications. The benefit gained from the use of seasonal to interannual predictions is therefore not dependent upon the quality of the predictions alone but also relies on the manner in which decisions are made based upon the predictions. Limited research attention has so far been focussed on this latter aspect by comparison with the forecast quality issue.

Relative Operating Characteristics provided much of the basis for a 2000-2001 project in the United Kingdom in which the value of forecasts on a wide range of time scales for the complete food chain, from production to retail, was examined.11 Despite the relatively low level of seasonal predictability over the United Kingdom, certainly by comparison with many tropical regions, substantial benefit was shown to be achievable, and confirmed through a calibration process using ROC. One key ingredient in the project's success was the close collaboration undertaken between meteorologists and all those involved on the agricultural/food side in each of the four sub-projects (dealing with field vegetables, sugar beet, apples and tomatoes), and this stands as a prime example of the type of integrated approach needed. One important project conclusion was that optimal benefits were obtained by coordination of responses to predictions throughout the food chain rather than as a series of separate decisions.

A number of other approaches to the decision process have been examined in both theory and practice, although it is not currently clear what level of benefit is achievable through most. The simplest, and probably the most common, is the straightforward treatment of a deterministic prediction in terms of experience. The benefits and hazards of this approach were illustrated during the 1997-1998 El NiƱo event. A second approach, the use of analogues, is basically a development of the deterministic approach in that it identifies historical sequences similar to those recently observed and/or predicted that provide appropriate analogues to guide decision making. The technique can be extended to probabilistic predictions through the development of analogues across the full range of possibilities that can then be interpreted as a group of weighted decision options. Analogues require substantial historical data and resource to develop, and so are not appropriate in all situations. A third alternative is the approach of 'least regret'. Following careful examination of the needs, perceptions and restrictions upon the end user, a decision based on forecast information is created that will provide least future regret should it turn out to be incorrect. By this approach a buffer against the more risky decisions, often those with greater rewards if correct but at greater costs if incorrect, is created. This approach also acknowledges that some agricultural forecast users, particularly in developing countries, do not have the scope to accept incorrect decisions or the flexibility to develop contingencies in the manners that probability forecasts inevitably impose.

In recent years research projects have been developed to provide more direct approaches to decision making through use of climate model outputs to drive agricultural models in a variety of manners. It is expected in these projects that these methods will provide the types of direct information required by agriculture. Further it is hoped that these methods will handle the issue that agricultural responses depend as much, if not more, on the detailed sequence of weather events during a season than on the averages across the season. Statistical weather generators have been used as one method of providing realistic daily weather sequences to feed crop models, as have daily outputs from the climate models. There is insufficient knowledge as yet about the statistics of climate models on a daily basis to be certain that this option is an improvement over the statistical generators, and results will be awaited with interest. Major activities in this area include CLIMAG,12 PROMISE13 and DEMETER.14 As discussed earlier, regional climate models (RCMs) are also being examined as a means of providing climate information on spatial and temporal scales below those provided by the climate models and to provide information for agricultural decision making.

Most of the above discussion has focused on the use of forecast information alone, with limited reference to the pertinence of historical information in guiding the decision process. The substantial benefit that can be obtained, even by use of information alone without forecast input, has been well demonstrated for agriculture by the work in Australia at BoM15 and APSRU,16 both organisations having developed extensive information systems based on historical data to assist in decision making. These systems cover not only climate information but also pertinent agricultural, economic and financial information. A basic approach developed at APSRU, and based in part on observed phases of ENSO in recent months, has been successfully adopted in countries outside Australia (Meinke and Stone, 2004).

0 0

Post a comment