Peer Review of Reviewers: The Author's Perspective (original) (raw)
Related papers
2006
Objective: To study the characteristics, evaluate the performance of reviewers and quality of their reviews as being helpful for the authors and the editors to make a final decision on the manuscripts. .The information collected included age, sex, academic affiliation, number of publications, training local or overseas, punctuality, hand written notes or typed, participation in peer review workshops etc. Rating scale of 1-5 (1 meant casual and 5 detailed comments) was used to evaluate their comments regarding originality of manuscripts, comments regarding strong and weak points, tables, usefulness of the manuscript and interpretation of results. Results: A total of sixty eight reviewers were evaluated. Majority of the reviewers 46(67.7%) were between the age of 40-60 years. Only three were female. Four (5.9%) reviewers had more than hundred publications to their credit while twelve (17.6%) had 51-100, thirty two (47%) had between 25-50 and twenty (29.4%) had less than twenty five publications. Fifty one (75%) were affiliated with academic institutions. Those who attended peer review workshops (38 out of 68) did a better review. Eleven (16.2%) did not wish to disclose their identity for various reasons. Based on their consistency and quality of reviews, fifteen reviewers were rated excellent by the Editors which included eight retired medical teachers, thirty three good and twenty as average reviewers.
REVIEWER AND EDITOR DECISION MAKING IN THE JOURNAL REVIEW PROCESS
Personnel Psychology, 1997
Much research on the journal review process has found little consistency among reviewers' evaluations of manuscripts. We propose theoretical explanations for these differences related to gatekeeping and particularism phenomena and generate hypotheses regarding influences on initial editorial decisions. A sample of 823 original submissions to the Journal ofApplied Psychology were analyzed with respect to author and paper characteristics, reviewer evaluations, and editor decisions. Support was found for gatekeeping functions in that reviewers and editors appeared to pay particular attention to the adequacy of the research design, operationalization of constructs, and theoretical development. Evidence was found for variable gatekeeping in reviewer evaluations, and the impact of reviewer evaluations on editor decisions was moderated by this variability across reviewers. Little evidence was found for social particularism (i.e., favoritism based on gender or affiliation) or content particularism (preference for or against particular research settings or methodologies) .
Analysis of Three Factors Possibly Influencing the Outcome of a Science Review Process
Accountability in Research, 2014
We analyzed a process for the annual selection of a Federal agency's best peer-reviewed, scientific papers with the goal to develop a relatively simple method that would use publicly available data to assess the presence of factors, other than scientific excellence and merit, in an award-making process that is to recognize scientific excellence and merit. Our specific goals were (a) to determine if journal, disease category, or major paper topics affected the scientific-review outcome by (b) developing design and analytic approaches to detect potential bias in the scientific review process. While indeed journal, disease category, and major paper topics were unrelated to winning, our methodology was sensitive enough to detect differences between the ranks of journals for winners and non-winners. Award. This science award seeks to recognize the premier science conducted by CDC scientists, or in collaboration with scientists around the world, and the award attests to scientific excellence via the published work of CDC's scientists. The award has been made annually since its inception in 1986. The awardees, one from each of three public health science categories-Assessment and Epidemiology (AE), Laboratory and Methods(LM), or Prevention and Control (PC)-are selected via formal nomination by the agency's organizational units. The review process is conducted by a cross-cutting matrix of agency scientists who carry out reviews in one of each of the three public health science categories. Their objective is to identify the best representation of scientific excellence among a heterogeneous set of papers within each of these public health science categories based on specific criteria related to scientific merit (originality, difficulty, efficiency or quality, and clarity) and impact (importance and significance). This means that no one reviewer will be a peer with recognized expertise (regarding merit and impact) among all the heterogeneous cadre of papers within a public health science category; even so, there is little doubt that identifying excellence is aided by assembling the agency's preeminent scientists to serve as reviewers. We leveraged 10year publication patterns of the published papers nominated for this award and among which the three annual winners are selected. We analyzed publicly available data (i.e., publication metadata) and constructed comparisons from these naturally occurring data that appeared to support reasonable, parsimonious conclusions about the susceptibility of the process to potential bias that could compromise the integrity of this review process. Our goals were to develop a relatively simple method that would use publicly available data and is able to assess the presence of factors, other than scientific excellence and
2015
Aim: To investigate the relationship between time taken for peer review, publication decision, and level of agreement among reviewers. Methods: The average time for eight stages of the peer review process was estimated for 369 peer review processes of three international Chilean journals published in Spanish in the fields of the humanities, engineering and university teaching. According to the combination of recommendations made by reviewers, each process was classified as having total, partial or low agreement. Data for each stage were grouped according to level of agreement and decision type. Results: Total peer review time was greater for articles that were accepted. For all three of the journals examined, publication period was the longest stage, and time taken to select referees was longest for the humanities journal. Partial agreement between reviewers was related to longer publication times in the university teaching journal, while there was no relationship between reviewer a...
Information Theory & Research eJournal Vol 4, Issue 38, August 03, 2023
Pre-publication Peer review has long been seen as a crucial component of academic publishing. Many people still view this examination and critical evaluation by specialists as vital, despite numerous objections and flaws. This chapter explains what may be reasonably expected from peer review and what constitutes excellent practice, putting a special focus on the editor's crucial role. Additionally, it describes the various modifications being made to traditional peer review, the new models that are emerging, and the growing importance of openness and transparency. Several issues are addressed, such as the difficulty in locating reviewers, the global imbalance between publication output and peer review participation, the 'wastage' of reviews that results from the repetitive submission of manuscripts to journals after journals after rejection, and the mounting pressure on researchers to publish not only in general but also in high-impact journals. The burden of editors and journal peer-review processes are both impacted by all of these. Researchers eager to publish their work without unnecessary delay are accepting the latest innovation to focus exclusively on the examination of the soundness of research methodology and reporting prior to publishing, and an increasing number of publishers are adopting it as well. The evaluation of interest, importance, and possible influence is postponed in this model until after publication. In the online environment, there are many opportunities for post-publication assessment and evaluation, but there are also many difficulties. It is evident that approximately three and a half centuries after the first journals were published, there are more options than ever for peer review innovation and experimentation. The study covered a variety of peer review sources and their significance, standardization implementation, journal format processes, and transparent peer review cascade with future recommendations for educational experiments, new educational review policies, and creative ideas with a focus on research output.
The profile of evaluators of a medical publication in relation to the response
Neurología (English Edition), 2010
Introduction: The expert is essential in the external evaluation process and for this reason it is necessary to know the profi le and characteristics of the best evaluators. Material and methods: We have retrospectively analysed the external review process of the journal from the 1st of January 2005 until the 30th of June 2009, with the aim of knowing the profi le of the experts in relation to the response to the requests. The response rate, mean delay time and responder rate were evaluated, using, sex, age and forming part of the editorial committee as variables. Results: The response rate fell as the number of evaluations increased. Women had a higher response rate, lower delay time and better performance than males. The response rate showed a tendency to decrease with age and the large majority of responders were between 29 and 39 years. Being a member of the journal committees was not associated with a better response rate, although there was less delay. The response rate and the delay time are similar, although it may increase with the number of requests to a reviewer. Conclusions: Lower age and being female are associated with a better response. No fatigue effect was observed in good responders, but if there is a fall in the response rates the number of evaluators should be increased.