International Conference

Disagreement in Peer Review

Humboldt-Universität Berlin
Topoi Building, Hannoversche Str. 6, Room: 1.03
Evening Lecture: Main Building, Unter den Linden 6, Room: 2094
May
19–20, 2016

Registration is free, but places are limited. Contact kerstin.helf@hu-berlin.de
Scientific organization: Geert Keil, Ralf Poscher, Martin Reinhart

Peer review is the most influential instrument of quality control in science. But peer reviewers often disagree. All too often, inter-rater reliability, i. e. the correlation between two or more reviews of the same research paper or grant proposal, is low. From a sociological perspective, disagreement among reviewers is to be expected and does not pose a serious threat to the rationale of the review process. In the sociology of science, the normative question of which reviewer (if any) is right, or closer to the mark, is less important than empirical questions about the factors that enhance or diminish agreement, the way dissent is dealt with in review procedures, and how those procedures are organized in order to reach a decision in the absence of unanimity.

In philosophy, disagreement among peer reviewers is a special case of the phenomenon of “peer disagreement”, which has received much attention in recent epistemology. The philosophical debate is concerned with the question of how epistemic peers, i. e. people with the same intellectual capacities, evidence and thoughtfulness in considering that evidence, should react when encountering disagreements. Is it rationally required for them to remain steadfast, or should they rather reconcile their views by according the judgment of the other some weight? Or should both parties even suspend judgment?

Assessing the quality of scholarly research may not be a matter of determining or overlooking the truth. But the mere fact that discordant members of review panels and editorial boards reason with one another indicates that quality assessment is also not simply a matter of subjective preference. The conference aims to explore the phenomenon of disagreement in peer review from various perspectives. It brings together sociologists, philosophers, legal scholars, journal editors and practitioners from funding bodies.

Thursday, May 19, 2016

13:30
Geert Keil (Berlin)
Introduction

14:00
Martin Reinhart (Berlin)
Reliable Disagreement. The (Non-)Issue of Interrater Reliability in Peer Review Research

Coffee Break

15:30
Stefan Hirschauer (Mainz)
Communication Benefits in Peer Review. Towards a Theory of an Evaluation Practice

Break

18:15
Michèle Lamont (Harvard)
The Blackboxing of Expert Judgment
Evening Lecture: Main Building, Unter den Linden 6, Room: 2094

Friday, May 20, 2016

9:30
Klaus F. Gärditz (Bonn)
Wissenschaftlicher Dissens als Rechtsproblem: Zwischen der Bewältigung epistemischer Kontingenz in Verfahren und institutionalisiertem Freiheitsschutz

Coffee Break

10:45
Michèle S. Garfinkel (Heidelberg)
Revealing Reviewer Agreement and Disagreement: A Transparent Black Box

11:30
Thomas Baldwin (York)
Editorial Judgment and Peer Review

14:30
Verena Graf (Berlin)
Peer Disagreement in Post-Publication Peer Review

15:30
Elke Brendel (Bonn)
Peer Disagreement and Third-Party Research Funding

Coffee Break

16:45
Stefan Hornbostel (Berlin)
Taking stock

19:00
Wolfgang Löwer (Bonn)
Begutachtung von Forschungsanträgen im Rahmen der Regeln guter wissenschaftlicher Praxis

  • In cooperation with Kiel University