Quantitative analysis of the ACIA reports

To complement the qualitative text analysis, the computer software Concordance18 was used to identify the most commonly used words in each chapter and to count frequency of selected key words related to framing and knowledge base. Both approaches can be described as classical text analysis or a "fishing expedition." They do not rely on any interpretive coding of the text and can be called representational in that they are based on the assumption that the authors say what they mean. The weakness is that this

15 From the introduction of Chapter 4. Future Climate Change. Modeling and Scenarios for the Arctic, 101

16 From the introduction of Chapter 15. Human Health, 865

17 From summary of Chapter 3. The Changing Arctic Indigenous Perspectives, 62.

18 Concordance Version 3.2.

method does not reveal how the words are used and what intention the writer had using the words.19

In addition, selected key words were chosen to look for specific patterns in the knowledge base, for example to capture issues of special interest to different scientific disciplines (e.g. biological, economic, cultural), specific discourses (e.g. rights, vulnerability, sustainable), the scale in focus (e.g. global, regional, local), or methods used (e.g. model, case study, monitoring). The analysis only included the chapters by the scientific authors and not the report preface and similar material.

The Concordance program uses text files, which were created by taking the Adobe Acrobat Reader files of the individual ACIA chapters that were downloaded from the ACIA website and saving them as text-files.20 The files include all text, including those in figures, figure legends, and references. Except for text in figures, the graphics were not included in this analysis. Words without intrinsic meaning were either placed on a stop list, and thus not included in the word count, or deleted before creating lists of the most commonly used words for each chapter. Examples of deleted words are: and. as, be, for, which, and any number (1,2,3 etc.).

The program allows for looking at the context of each word usage, which would allow a more careful comparison. This feature was mainly used to check the major uses of words with possibly ambiguous meaning (e.g. radiation referring to UV radiation, solar radiation or ionizing radiation) and to further scrutinize "odd" results.

For the overview document, it was not possible to use the Concordance software because the document's format did not allow saving it as a text file. Therefore, some selected words were searched and counted via the search function in Adobe Acrobat Reader.

There are inherent limitations to representational text analysis. First, it requires further work to explore the context in which the words occur. Second, it cannot easily capture concepts that are not explicitly mentioned. This includes framings that are so taken for granted that the authors do not consider them important to mention. Third, the program cannot read between lines, the way an observant human reader can do. Moreover, it is difficult to capture the flow of logic in a text. Therefore, the quantitative analysis should be seen as a complement to the qualitative analysis and vice versa. The strength in the quantitative analysis is that it can assist in detecting patterns that were not seen in the initial qualitative analysis. It can also be used to cross check whether patterns identified in the initial qualitative analysis are also apparent via frequency of words.

When comparing texts by different authors, a quantitative text analysis may provide results that represent more the linguistic idiosyncrasies of the authors than a change in a major scientific framing of an issue, which is the central question to do this study. However, the scientific writing style is rather formalized and both the ACIA and the IPCC reports, which were written by groups of authors, have gone through editing, including editing for consistency. Major differences in the frequency of key words are thus likely to indicate differences in framing.

19 Carl W. Roberts, "A Conceptual Framework for Quantitative Text Analysis," Quality & Quantity 34, (2000): 259.

20 www.acia.uaf.edu. Accessed Nov. 2005 - Mar. 2006.

To assist the interpretation of the quantitative data and to help in pattern detection, the frequency with which different words were used were statistically analyzed using principal component analysis. This method is useful for detecting patterns in multivariate data (i.e. when there are three or more variables) and where many of the variables are dependent upon each other. The object of the analysis is to take the variables and find combinations that are uncorrelated. The lack of correlation reveals different dimensions in the data, where the first dimension displays the largest amount of variation.21 For the analyses in this chapter, the variables are the number of times certain words are used in a specific chapter or report and the cases are either different chapters or different reports. The results are presented in two-dimensional graphics that provide a perceptual map of the location of each case (e.g. chapter) in the relation to the first two principal components.

Was this article helpful?

0 0

Post a comment