Validation and Intercomparison Methodology

The assessment methodology used ultimately for the GODAE intercomparison project is a direct heritage of the validation activity performed earlier in the framework of operational oceanography projects. It is based on two aspects (Crosnier and Le Provost 2007). First, «the philosophy»: a set of basic principles to assess the quality of OFS products/systems through a collaborative partnership:

• Consistency: verifying that the system outputs are consistent with the current knowledge of the ocean circulation and climatologies.

• Quality (or accuracy of the hindcast/nowcast): quantifying the differences between the system "best results" (analysis) and the sea truth, as estimated from observations, preferably using independent observations (not assimilated).

• Performance (or accuracy of the forecast): quantifying the short term forecast capacity of each system, i.e. Answering the questions "does the forecasting system perform better than persistence and better than climatology?"

• Benefit: end-user assessment of which quality level has to be reached before the products are useful for an application.

Second, «the methodology»: a set of sharable tools for computing diagnostics, and a set of sharable standards to refer to, for assessing the products quality. Both tools and standards should be subject to upgrades and improvements in an operational framework. This methodology has been built using "metrics": mathematical tools that compute scalar measures from systems outputs, compared to "references" (climatology, observations etc...). The metrics provide equivalent quantities extracted out of the different systems for the same geographic locations. Applied on different forecasting systems, they provide homogeneous and consistent sets of quantities that can be compared without depending to the specific configuration of each OFS (horizontal resolution, vertical discretization etc.).

"Share-ability" is mandatory and allows each forecasting center to perform intercomparison and validation independently, using results from other centres. Metrics, are computed in a standardized way, the NetCDF file format using the COARDS-CF convention is used, allowing time aggregation, easy and flexible manipulation, and self consistent meta-data representation. Distribution relies on internet communication protocols, basically through FTP. However, more user-friendly communication technologies based on OPeNDAP servers that can be visualized through a Live Access Server (LAS), through Dynamic Quick View portals or with similar

Fig. 23.6 Summary of Class 2/3 metrics. All existing and available moorings, tide-gauges, XBT lines, WOCE/CLIVAR lines and others have been selected in order to define virtual sections and mooring points implemented in ocean models

clients that have now been widely adopted (Blower et al. 2008). In practice, these technologies allow each forecasting centre to compute a considerable amount of diagnostics stored on the local servers of other centres. The total set of validation data do not need to be centralized requiring large storage capacities. Instead, for a given diagnostic, one can specifically gather the information spread across the different centres.

Metrics are defined in four types, or "classes" (see Hernandez et al. 2008 for more details):

• Class 1 metrics, i.e. 3D standardized grids of temperature, salinity, currents, mixed layer depth, sea ice quantities and fluxes, can be directly compared to climatologies, but also at the surface to satellite observations (e.g., SLA, SST, or ice concentration). By using similar Class 1 grids, several OFS can intercompare their ocean estimates with a given reference dataset (example is provided in Fig. 23.8 in next section).

• Class 2 metrics (virtual moorings and sections) are designed to match location of existing in-situ datasets as shown in Fig. 23.6. Then each time observations are provided (e.g., an XBT sections from a merchant ship), the Class 2 diagnostic can be performed routinely, and the model variable can be compared to "ground truth". Figure 23.7 illustrates the use of Class 2 diagnostic for intercomparison between five systems in the Gulf of Cadiz. Compared to older WOCE hydro-graphic transects, it also allows a consistency assessment. Finally, it helps to assess improvements from two generation of Mercator systems.

• Class 3 metrics concern derived quantities, like ocean transport, heat content, thermohaline circulation.

Fig. 23.7 Intercomparison of several ocean forecasting systems (Mercatorl, TOPAZ, FOAM, HYCOM) during the European Project MERSEA Strandl through a Class 2 salinity section averaged in September 2003 in the Gulf of Cadiz. Two WOCE lines are used as reference dataset. Further comparison was carried on when a new version of the Mercator system was developed (Mercatorl)

Fig. 23.7 Intercomparison of several ocean forecasting systems (Mercatorl, TOPAZ, FOAM, HYCOM) during the European Project MERSEA Strandl through a Class 2 salinity section averaged in September 2003 in the Gulf of Cadiz. Two WOCE lines are used as reference dataset. Further comparison was carried on when a new version of the Mercator system was developed (Mercatorl)

• But to get closer to data, both for hindcasts and forecasts, Class 4 metrics were designed to build up a dataset of "model values equivalent to observations" for all OFS outputs: hindcast, nowcast and forecast. Thus, forecasting skill of OFS can be objectively evaluated. Class 4 diagnostics have been implemented in several centres for temperature, salinity (observations from Coriolis in-situ data centre), sea-ice concentration (maps from OSI-SAF22), sea level (satellite altimetry from AVISO23) and currents (from the Global Drifter Program). For all these diagnostics, a particular attention is paid to use independent observations, i.e., preferably not assimilated. Ideally, instead of satellite altimetry assimilated in most OFS, tide gauge data for sea level, or drifter or ADCP24 velocities for current. Table 23.2 summarizes the list of ocean/sea-ice parameters that can be evaluated with Class 4, and the corresponding data set.

22 See Ocean & Sea Ice Satellite Application Facility at http://www.osi-saf.org/.

23 See http://www.aviso.oceanobs.com/.

24 Acoustic Dopler Current Profiler.

Table 23.2 Ocean and sea-ice physical quantities, and corresponding available observations for validation in real time (RT) or delayed mode (DM)

Data type

Measurement

In-situ temperature

CTD (DM), XBT (RT), buoy (RT), mooring (RT/DM), TSG

(DM), deep float (RT), glider (RT/DM)

In-situ salinity

CTD (DM), XCTD (DM), buoy (RT/DM), mooring (RT/

DM), TSG (DM), deep float (RT), glider (RT/DM)

Sea surface temperature

Satellite radiometer/radar (RT), TSG (DM), buoy (RT),

mooring (RT/DM)

Sea surface salinity

TSG (DM), buoy (RT), mooring (RT/DM) [SMOS, Aquarius]

(RT expected)

Horizontal currents

Drifters (RT), Current meter (DM), ADCP (DM)

Satellite altimeter (RT), SAR (DM), High Frequency radar

(DM), derived from SST (DM), derived from deep float

displacement (DM)

Sea level

Tide gauges (RT), satellite altimeter (RT), GPS (to be tested)

Ocean colour

Satellite imagery (RT/DM)

Sea Ice concentration, drift

Satellite (RT)

CTD conductivity temperature depth, XBT expendable bathythermograph, TSG thermosalino-graph, XCTD expendable conductivity temperature depth

CTD conductivity temperature depth, XBT expendable bathythermograph, TSG thermosalino-graph, XCTD expendable conductivity temperature depth

From Class 1, 2 and 3 metrics, the consistency and quality of each system could be deduced, or intercompared. For instance, daily section of operational run can be routinely compared to Class 2 historical section as illustrated in Fig. 23.7: in this case, the "general good looking" of the water masses distribution is verified against two historical WOCE lines: e.g., one expect that salinity signature of the Mediterranean waters appears at the proper depth.

A system's performance can be addressed using Class 4 metrics. The "benefit" could also be addressed using a set of Class 1, 2, 3 and 4 metrics. However, new "user-oriented" metrics might need to be defined to fully address this.

Was this article helpful?

0 0
Guide to Alternative Fuels

Guide to Alternative Fuels

Your Alternative Fuel Solution for Saving Money, Reducing Oil Dependency, and Helping the Planet. Ethanol is an alternative to gasoline. The use of ethanol has been demonstrated to reduce greenhouse emissions slightly as compared to gasoline. Through this ebook, you are going to learn what you will need to know why choosing an alternative fuel may benefit you and your future.

Get My Free Ebook


Post a comment