Peer reviewing is the institutional activity by which scientific texts submitted for publication are first reviewed by scientists of the same scientific domain. This review typically involves comments on merits and weaknesses of the text submitted and a recommendation for the editor or publisher concerning its possible publication.
A specific variant of peer reviewing is double-blind reviewing, where the editor or publisher sees to it that both the author of the text submitted and the reviewer remain fully anonymous. Its purpose is to support frankness and objectivity. This variant is preferred by many, though by no means by all editors and publishers.
Anonymy of the author and the reviewer is often hard to guarantee. A responsible editor or publisher will preferably choose a reviewer who is a specialist in the field of the text in question. (To the extent that he himself is not such a specialist, he will typically look for a possible reviewer in the references list of the text submitted.) The reviewer will then likely be familiar with the author's work and recognize him even in an anonymous text.
Frequently, the reviewer will realize that her own publications have been insufficiently taken into account by the author. To some extent, she will be right in this assessment, and to another extent, it will be based on vanity. At any rate, the reviewer will frequently have occasion to refer to her own work in her suggestions for improvement. This, however, renders her recognizable by the author. This puts the reviewer in an awkward situation as she must do without such references even where they are appropriate just in order to remain unrecognizable. Consequently, the pressure to remain anonymous does not necessarily contribute to the quality of the review.
On the other hand, many a reviewer recognizes the author of the text under review and, remaining anonymous herself, uses her power to prevent publication of the text. In principle, a responsible editor can sense such a situation and then choose a different reviewer. Not seldom, however, the editor and the reviewer even form a cartel determined to suppress the publication of certain colleages' work.
The institution of reviewing scientific work before publication aims, of course, at improving its quality. To achieve this goal, criticism must not be completely destructive. That is, while the review can and must contribute to suppressing the publication of false and worthless matter, it can and must also help the author to correct his mistakes and to overcome his limitations. Many reviewers limit themselves to noting that something is wrong without telling the author what they would regard as correct or what he could do to improve his text. In doing so, they remain in a position of personal competition with the author instead of joining the common enterprise of advancing scientific research in their discipline (which generally is what they are being paid for).
The author of the text is required to either comply with the reviewers' demands or to prove them inappropriate. In the best of possible worlds, the latter action would contribute to advancing the state of the art, too. However, few editors and publishers are actually willing to control the observation of ethical standards by authors and reviewers. To reduce their own work load and to avoid dispute, they often publish a text without making sure that the author has taken the reviewer's comments duly into account.
A considerable percentage of peer reviews is fraught with such problems. There is, however, no alternative in sight to having research reviewed by colleagues who are active in the same scientific context. One cannot delegate this task to politicians or other laymen. Consequently, the question remains what could be done to improve the peer review system. It seems clear that there must be a quality control not only of scientific texts submitted for publication, but also of their reviews. There must be pressure on reviewers to strive for quality in their performance. This can be done by a reward system. Here is a possible procedure:
- The system remains anonymous, but full anonymy is controlled more efficiently.
- When submitting the final version of their work for publication, authors must fill in a short questionnaire about the quality of the reviews that they had received. These questionnaires are sent to a central institution which administrates this business.
- This institution produces a rank order of reviewers. It organizes rewards for the best of them and suggests that those at the bottom of the rank order be dropped from the list of reviewers.
- Appropriate awards include invitations to guest lectures, keynote speeches and the like. Incentives for such invitations are offered to authors who got their work reviewed and published.