The review

In the summer of 2003, the first official draft of the complete scientific report was sent out for review. The reviewers had been selected through nominations from the ACIA Executive Committee, AMAP, CAFF, IASC, the Indigenous Peoples' Secretariat, and several national organizations. After further solicitation of nominees from the lead authors, the final selection was made by the Assessment Integration Team based on "scientific balance, country balance and gender equity." All in all, 142 reviewers are selected of whom 79 returned reviews. In addition to the ACIA-initiated review process, several countries engaged national experts to review the science chapters. To collate all the review comments, the secretariat hired a person with experience from IPCC's re

68 Summary Report of the Ninth Assessment Steering Committee Meeting (ASC) 2-5 June, Asker Norway.

69 Summary Report of the Ninth Assessment Steering Committee (ASC) Meeting, 2-5 June 2002, Asker, Norway.

view process.71 In addition to general issues about each chapter, most authors received many pages of detailed comments. Each lead author was expected to reply to each of these comments, either accepting changes or providing some justification for not making the suggested change.72

The Assessment Steering Committee discussed the possibility of making the reviews available on the World Wide Web but there was resistance to this idea because of concern that some of the reviewers may not have wanted their strongly phrased critisicm so publicly accessible. While some argued that confidentiality was essential to successful peer review, other members of the Assessment Integration Team argued that the material had to be openly available for the process to be open and credible.73 A paper providing guidance to the authors states that "[w]e are expecting that comments and responses will be put on the Web,"74 but in the end, a decision was made that authors' responses to external reviews would not be posted for public viewing, but that they could be requested from the ACIA Executive Committee.75

The discussion surrounding the review process illustrates the contrasting perspectives when internal scientific norms confront other needs at the science-policy nexus. Even if the peer-review system has been challenged from time to time, including calls for more openness, there is more basic internal trust within the scientific community than in the policy arena. In this arena, science may be challenged and transparency in process becomes part of building (or safeguarding) scientific credibility. In highly contested science, someone is likely to ask what comments were made on drafts and how they were taken into account. This is also what happened at a discussion about ACIA-follow-up in the AMAP Working Group. This politically mandated body decided that the documentation, outlining how reviewers' comments were handled, should be put on ACIA's web site.76 They have since become available via AMAP's website, albeit hidden in the password area.77

During the last meeting of the Assessment Steering Committee, in October 2003, the authors had to give their summary response to the review comments and how they planned to deal with them. In addition, the Assessment Integration Team had chapter liaisons present a more independent assessment of the review comments. For some of the chapters, the exercise of discussing the review was straightforward, but for a few chapters, major tensions came to the fore.78 For example, comments on a chapter about

71 International Expert Review Process - ACIA Science Chapter Summary by Patricia Anderson, ACIA Secretariat via e-mail January 25, 2006, including supporting lists of nominations and selection of reviewers.

72 Guidance to ACIA Authors on Documenting Responses to Reviewers Comments. For Discussion at the ASC Meeting, 15-16 October 2003, London.

73 Observation notes October 14, 16 and 17, 2004. International Expert Review Process - ACIA Science Chapter Summary by Patricia Anderson, ACIA Secretariat via e-mail January 25, 2006.

74 Guidance to ACIA Authors on Documenting Responses to Reviewers Comments. For Discussion at the ASC Meeting, 15-16 October 2003, London.

75 The reviews were made available to me as background material in connection to the meetings of the Assessment Integration Team and Assessment Steering Committee October 2004.

76 Minutes of the 19th Meeting of the Arctic Monitoring and Assessment Programme (AMAP Working Group. S:t Petersburg, Russia, 12, 13, and 16 September 2005, Item 5.3.

77 Accessed Sep. 20, 2006 without any need for a password.

78 Observation notes London ASC meeting October 15-16, 2003.

the management and conservation of wildlife in the changing Arctic raised the question of whether climate impacts on conservation were really covered, leading to a decision to find an additional author.79 In this discussion CAFF became an important pusher. Later this led to the division of the chapter into two chapters - one focusing on wildlife management as it relates to the use of living resources and the other focusing on conserving Arctic biodiversity. This process shows how an author of a chapter was not allowed to get away from the original purpose and outline of the assessment as it was set out by the Assessment Steering Committee. It also shows that CAFF had enough clout in the process to get a perspective included that a selected lead author had not prioritized.

The chapter on assessing vulnerabilities also generated a heated discussion. One controversial point was a case study in which a local population in Greenland had not been involved, raising questions about the research ethics of that kind of study.80 The point was brought up by the indigenous peoples' representatives in the Assessment Steering Committee, the Danish spokesperson relating comments from Greenland, a lead author for another chapter, as well as by the chapter liaison, and illustrates that there are current sensitivities about outside researchers coming into the Arctic and doing studies without the approval and participation of locals. Another issue was whether this chapter was an assessment of available knowledge or a narrow research paper that had no place in the ACIA. The ACIA chair was one of the co-authors of this chapter, and although he did not take part in the discussion, this fact colored some of the dispute at the meeting. It was also pointed out that the process had been different for this chapter. For example, nomination of authors was not open.81 During the meeting, a small group discussed re-focusing the chapter, which also was renamed "Climate Change in the Context of Multiple Stressors and Resilience." This discussion illustrates that tensions can come up in an assessment process when the initial overall framing of the task is challenged. The dynamics and significance of the controversy will be further discussed in Chapter 6 of this dissertation.

Another assessment chapter that generated controversy was the summary, when lead authors of other chapters did not want to allow new material, which they had not included in their respective chapters, to be added to the summary. This debate continued at an Assessment Integration Team meeting half a year later.82 A specific issue was a table of economic impacts on industrial development. The lead author of the summary chapter expressed a clear wish to see these issues treated in the assessment and similar requests were voiced about lessons from the oil industry and the military. However, this was not covered in the relevant assessment chapter where they could have been brought up, i.e. the chapter on infrastructure, which created problems when the summary lead author wanted to bring them into the summary. This discussion illustrates the tension between the wish for credibility, based on the scientific independence of lead authors, and the wish of assessment leaders to cover aspects that were not explicitly included in the chapter outlines. In this case, scientific independence was clearly linked to the

79 Observation notes and Summary Report on the Tenth Assessment Steering Committee (ASC) Meeting, 15-16 October 2003, London, U.K.

80 Observation notes and Summary Report on the Tenth Assessment Steering Committee (ASC) Meeting, 15-16 October 2003, London, U.K.; Interview 30.

81 Observation notes October 14, 2003.

82 Observation notes March 24, 2004.

authority of the lead authors in the main part of the assessment rather than in ACIA's executive director as lead author of the summary. Not spoken out loud but underpinning the discussion may also have been developments in the policy process that made it clear that the United States was likely to question the scientific credibility of the ACIA. Scientific credibility had therefore become a more heavily weighted issue concerning more than only the reputation and credibility of individual authors.

The review also generated a discussion about some cross-cutting issues. They were raised by the ACIA chair and included discussions on the carbon cycle, whether Arctic climate change is anthropogenic in origin, and interactions between climate change and contaminants and biodiversity. The way in which cross-cutting themes were brought into the discussion illustrates the close interaction between the policy sphere and the scientific community more clearly than the treatment of specific chapters. These were questions raised by ACIA's chair and the AMAP and CAFF representatives, i.e. people who generally were closer to the policy sphere and more aware of its dynamics and needs, and who made specific references to what policy makers may want or need. Another specific example is a lively discussion on the Assessment Integration Team on how to place and phrase the overall conclusions and how they could be used in the policy document.83

There was not much time before the authors had to submit their final draft after the London meeting. By now, it had become clear that the content of the scientific document had to be "frozen" before there could be any approval of the overview and policy documents. In practice, they had only until January 2004 to get their chapters ready and forwarded for technical editing, graphics work, and layout to meet the aim of having the report in print by the Arctic Council ministerial meeting in the fall of 2004. As it turned out, the process was much prolonged. At an Assessment Integration Team meeting in March 2004, there were outstanding issues for several chapters, including major changes in chapter structure, i.e. splitting the chapter on conservation into two chapters, one with a new lead author. Also, there remained an extensive discussion on the summary chapter around what materials could or could not be included on economic and regional impacts with reference to what had been evaluated as scientifically credible by the lead authors in the preceding chapters. There were also some concerns raised during the review of the overview document that called for a new look at the material in the underlying scientific chapters (new scientific findings that needed to be included). It would, thus, be an overstatement to say that the content of the whole science document was frozen at this point. Later in the process, the technical editing and production were more time-consuming than the ACIA had allocated in its initial time plan and the scientific report was not ready by what was supposed to be its formal release at the ACIA Scientific Conference in early November 2004. It was not until the early spring of 2005 that the first chapters became publicly available on ACIA's website and not until late October 2005 that the whole report was published in its final printed form. ACIA's chair has explained that the delay was caused by a gross underestimate of the time needed for technical editing.84

83 Observation notes October 17, 2005.

84 Interview Robert Corell June 26, 2006. The delayed publication of the scientific report was not unusual compared to other scientific assessments from AMAP.

Was this article helpful?

0 0

Post a comment