Criteria for Methodological Strength of Indicators

The development of indicators is a matter of concessions and compromise. The quality of indicators reflects the developers' dexterity in responding to and anticipating a number of constraints. The way an indicator is used and performs depends not only on how it responds to individual criteria applicable at the level of the individual indicator but also on whether it responds in a balanced way to the sum of criteria as an interconnection of constraints.

Criteria can be developed on a number of levels, from technical quality criteria intervening at the level of the statistical nature of the data to quality criteria related to the usability of the indicators. The errors in developing indicators are inextricably linked to the danger of measuring the wrong issues perfectly or the correct issues inadequately.

Constructing information on integrated and complex issues is bound by our imperfect understanding of reality (i.e., constraint by the measurable), and for issues where reasonable understanding exists, developers of indicators are bound by what is actually measured (i.e., data availability) and how it is measured (i.e., data quality). Epistemo-logically, our understanding of reality is limited by knowledge gaps and our inability to measure, simplify, and compare many of the complex factors involved in sustainable development. Technically, any representation of reality is limited by the quality, accessibility, and reproducibility of the background data. Societally, integrated assessments that are concerned with multidimensional issues are influenced by the way the issues involved are interpreted.

To help indicator developers overcome the constraints encountered during their work, relevant metacriteria can be set out to make the constraints on the level of the indicator selection and construction more transparent (Table 3.1):

Purpose: This refers to why an indicator exists, the appropriateness of scale, and the accuracy with which it links its purpose to the general concept of sustainable development. The quality of an indicator lies in the way it addresses its purpose and provides clear information on the state or trend of some aspect of sustainable development. The appropriateness of the scale at which an indicator is usable is a secondary consideration. Although some indicators may be used intentionally for cross-scale comparisons, generally the scale at which an indicator is to be used must be clearly defined. (See Box 3.1.)

Measurability: This refers to how the values in an indicator are measured and the extent to which it measures reality. Even though indicators necessarily limit themselves to the sphere of the measurable, their link to reality is imperfect to varying degrees because they use sample measures taken on specific days or at specific times and locations. They are also limited by the way the raw data reflecting reality are translated into quantitative data and measures. (See Box 3.2.)

Representativeness: This refers to the completeness and adequacy with which an indicator measures or expresses the phenomena with which it is concerned. Although many aspects of human—environment interactions and their associated complex feedback loops are not completely scientifically understood, enough is known about many issues for the scientific accuracy with which indicators represent reality to be assessed.

Reliability and feasibility: This reflects the truth and reproducibility of indicators and their robustness in statistical terms and the ability to develop the indicator in practical terms of data and cost-effectiveness. The quality of an indicator depends on the data from which it is derived and from the indicator construction method, which may increase uncertainty. Gaps in monitoring of human—environment interactions and deficiencies in the spatial coverage of many global and local data sets constrain the quality of many indicators. Ideally indicators should be built on existing data sets, but many desirable indicators lack suitable background data. Collection of new data and repeated data collection exercises may be costly and lengthy enterprises.

Communicability: This is the extent to which indicators are understood and the effectiveness with which they convey their purpose and meaning to their target audiences. Fundamentally, indicators are communication tools. An indicator that fails to do this is redundant. Because sustainable development is a multistakeholder project, indicators must be meaningful to many different actors (e.g., citizens, policymakers, decision makers). Thus the capacity of an indicator to reach its target audience ultimately determines its communicability and contribution to sustainable development.

Table 3.1. Case study: Applying assessment criteria to three economic headline indicators.

GDP per Capita

Purpose The virtues that help GDP per capita retain its utility in sustain-

ability assessments include its widespread acceptability and implementation in national policy as a measure of and influence on the level or vigor of exchange transactions and its influence on political survival.

Measurability Like any other highly aggregated measure used to represent the long-term quality of human activities, GDP per capita is far from perfect, and it must be considered in tandem with related developments represented by key social indicators such as life expectancy at birth, choice or freedom measures, security, access to education, and subjective well-being.

Table 3.1. Case study: Applying assessment criteria to three economic headline indicators (continued).

Representativeness The limitations of GDP per capita as a comprehensive measure of economic welfare of a nation's citizens stem primarily from problems associated with capturing and distinguishing all relevant sources of welfare (even within a given period). In addition, GDP levels do not embody information on the level of wealth, long-term sustainability, and quality of life because GDP represents an income flow for a short time and does not reveal whether this income was derived from qualitative or quantitative gains in productive capital stock (including natural resources) or from depletion of existing assets that will jeopardize future economic sus-tainability. There have been laudable attempts to adjust GDP per capita for environmental and other capital losses, unpaid labor, defensive expenditure, and so forth, but interpretive difficulties and their methodological inadequacies linked to data needs, questionable construct validity, and economic valuation limit their utility as viable headline indicators for the economic domain.

Feasibility GDP is supported by a long history of well-developed underlying data and methods.

Communicability This "social product" has many positive influences in terms of the choices, diversity, and access to resources that make up other major facets of long-term sustainability (e.g., adequate nutrition, health services, dematerialization, and related eco-efficiency— enhancing technologies and infrastructural and strategic change). However, trends in GDP per capita should be complemented by simultaneous consideration of the various components of overall expenditure that have significant effects on sustainability (e.g., social products committed to social indicators such as education, preventive eco-efficiency, natural capital protection and augmentation, and, arguably, the nature of production and consumption, examined in environmental pressure terms).

Income as a Measure for the Distribution of GDP

Purpose In addition to an account of the overall economic value added that is represented by GDP, another indicator is necessary that addresses the distribution within the socioeconomic system and thus shows the amount of income per person or household. Concerning sustainability, household income provides necessary information on the distribution of wealth among the population.


Table 3.1. Case study: Applying assessment criteria to three economic headline indicators (continued).




GDP per capita often is used as proxy for individual income. However, GDP is not evenly distributed among the population. Thus, other indicators are necessary to cover distributional issues (e.g., income per household).

Income data are difficult to acquire in economies where there is a large informal sector, and the data obtained will not adequately represent reality.

See above.


The limitations of income data to represent distributional issues are of diminishing importance because they are readily understood by a majority of users (unlike those of more complex indicators such as the Gini coefficient).

Material and Energy Intensities


Given that human activity relies on a natural system with finite resources, sustainable economic development has to account for environmental limits (both sink and source). Material and energy intensities relate economic data to indices of natural resource use (and waste output) per unit of economic output. These hybrid approaches are at the basis of the material and energy intensity measures, which can provide good indications of natural resource or eco-efficiency needed for dematerialization processes.

Measurability Economic indicators are not defined solely as those that use mon etary values. In recent years, there has been a substantial shift in interest (in the environment—economics nexus) toward depicting the economy in terms of the production, exchange, and demand of physical resources and goods providing welfare services. For environmental demands, the key factor becomes the material and energy flows (often effective consumption) generated by the overall economy and specific types or classes of economic activity and output (and associated technologies, inputs, and waste outputs).

Representativeness Dematerialization indicators and material or energy intensities are being used more often. However, any dematerialization observed can be derived in two different ways: through real reduction of resource input per unit of GDP or simply outsourcing of material-


Table 3.1. Case study: Applying assessment criteria to three economic headline indicators (continued).

or energy-intensive processes to other countries. The latter does not result in dematerialization on the global scale but only for the specific socioeconomic system observed. Any interpretation of dematerialization therefore has to take into account the global scale and thus consider effects on the international level.

See above.

Users of indicators do not necessarily think about energy, water, and material budgets in the same way as they think about monetary budgets. However, the mechanisms underlying material and energy intensity indicators are similar to thinking in terms of monetary values and therefore can be readily understood.

Was this article helpful?

0 0
Whole Health

Whole Health

Get All The Support And Guidance You Need To Be A Success At Better Total Health. This Book Is One Of The Most Valuable Resources In The World When It Comes To An Introduction To Your Overall Health For You And Your Loved Ones.

Get My Free Ebook

Post a comment