I was reading this essay about peer review (I have attached the link at the bottom), and there are some interesting “behind-the-scenes” explanations of the process of finding a reviewer and what should be considered before a scientist should accept an offer to peer review a colleague’s work. Some of the points that are brought to light seem intuitive, but it amazes me to see everyday that the educated individuals this effects do not take them into account.
For example, the author asks questions such as, “Do you have the expertise the editor is looking for?” or “Is it too close to your own work?” For the last point point, she says that the selected reviewer should not review the paper because it is a no-win situation. If the reviewer does critique the paper, there is a risk that it could interfere with their own research and subsequent submission of their own paper. If the work of the two scientists are similar and the reviewer approves of other scientists paper, then the reviewer may have a difficult time getting his own research published in the future. So…what should the selected reviewer do? According to the author, the reviewer should read the title and abstract, and if it is similar to their research they should not review the rest of the paper. This seems like the best way to approach the situation, but most times the abstract does not contain enough information to make an educated decision to continue the review. There may be a point while reading the paper that the reviewer realizes that they have similar areas of research. The ethical reviewer should contact the editor and decided to cease the review. Unfortunately, there may be a situation where there are only a few scientists qualified to review the paper because there are very few people in that field of research. When this situation arises, what is the editor supposed to do? I think it will be very difficult to find a reviewer that has enough knowledge in the area of study while not being in direct competition with the author of the paper under review, especially since there will be at least two or three reviewers for each paper. The odds that one of them has a conflict of interest in one of many areas (highlighted in the article) is very likely.
It seems that there are too many “case-by-case” conflicts that can arise in the peer review process. Even the guidelines that are provided leaves room for questions on whether one should review a certain paper. In most cases, contacting the editor seems like the most logical and ethical course of action. From that point, the editor can act as a neutral party that can make the final decision on who reviews which papers based on the information at hand.
http://radonc.yale.edu/pdf/Ethical_Issues_in_Peer_Review.pdf
Showing posts with label Peer Review. Show all posts
Showing posts with label Peer Review. Show all posts
Sunday, March 8
Plagiarism in Science
I was looking over the current issue of Science and noticed an article about scientific integrity and plagiarism ("Responding to Possible Plagiarism" by Tara C. Long, Mounir Errami, Angela C. George, Zhaohui Sun, Harold R. Garner -- you can access the article through the library's website), and it really has made me question the level of responsibility in which scientists are held to.
The paper begins by stating:
"The peer-review process is the best mechanism to ensure the high quality of scientific publications. However, recent studies have demonstrated that the lack of well-defined publication standards, compounded by publication process failures, has resulted in the inadvertent publication of several duplicated and plagiarized articles. "
After a discussion about varying search engines/databases that are available for helping detect plagiarism, the authors' note that after their own search they found at least 212 cases of potential plagiarism. They then notified original and later authors and editors about the potential plagiarism. Response ranged from shock, anger, and disappointment to at least one of the later authors stating that they were not involved in the article. In conclusion the authors' note that there needs to be:
- Authoritative oversight by the scientific community
- Authors need to commit to the novelty and accuracy of the report
- Peer reviewers must make informed and thorough reviews
- Verification of originality by editors
However, my question is how is one to implement and/or enforce rules or guidelines that address these conclusions? For example, in respect to authors needing to commit to the accuracy of their report, isn't this what journals/editors/etc. already ask of the authors?
Also, how are editors to verify the originality? Are they to ask the authors to submit proof of all data? But wouldn't individuals already intent on copying or making up data find a way to do the same with "their" data?
In general there is an obvious need for the scientific community to start holding scientists to higher levels of responsibility and scientific integrity. I think that one way to help start this process is to start with younger/newer research scientists. Many of these scientists are under pressure to get a certain number of papers published, and unfortunately (as has been noted) this causes a quite a bit of "crap" to be published. This pressure needs to be lessened to allow these younger/newer scientists a chance to develop and produce real research. Much of this lessening of pressure needs to come from the universities and research insititutes.
Also, in respect to scientists who are well established and have larger labs in which they oversee, these scientists need to actually be overseeing and be apart of all research. I am not saying that all such scientists are not involved, but that involvement needs to be more than just being the person who gets all the grant money and signs off on projects and papers--they need to be involved in the development and carrying out of the projects.
Though there are other ways in which the responsibility and scientific integrity of scientists can be increased, I think that the scientific community needs to actually start doing something about it--instead of just going "oh well" or giving a simple slap on the hand in cases like plagiarism or "crap" publications.
The paper begins by stating:
"The peer-review process is the best mechanism to ensure the high quality of scientific publications. However, recent studies have demonstrated that the lack of well-defined publication standards, compounded by publication process failures, has resulted in the inadvertent publication of several duplicated and plagiarized articles. "
After a discussion about varying search engines/databases that are available for helping detect plagiarism, the authors' note that after their own search they found at least 212 cases of potential plagiarism. They then notified original and later authors and editors about the potential plagiarism. Response ranged from shock, anger, and disappointment to at least one of the later authors stating that they were not involved in the article. In conclusion the authors' note that there needs to be:
- Authoritative oversight by the scientific community
- Authors need to commit to the novelty and accuracy of the report
- Peer reviewers must make informed and thorough reviews
- Verification of originality by editors
However, my question is how is one to implement and/or enforce rules or guidelines that address these conclusions? For example, in respect to authors needing to commit to the accuracy of their report, isn't this what journals/editors/etc. already ask of the authors?
Also, how are editors to verify the originality? Are they to ask the authors to submit proof of all data? But wouldn't individuals already intent on copying or making up data find a way to do the same with "their" data?
In general there is an obvious need for the scientific community to start holding scientists to higher levels of responsibility and scientific integrity. I think that one way to help start this process is to start with younger/newer research scientists. Many of these scientists are under pressure to get a certain number of papers published, and unfortunately (as has been noted) this causes a quite a bit of "crap" to be published. This pressure needs to be lessened to allow these younger/newer scientists a chance to develop and produce real research. Much of this lessening of pressure needs to come from the universities and research insititutes.
Also, in respect to scientists who are well established and have larger labs in which they oversee, these scientists need to actually be overseeing and be apart of all research. I am not saying that all such scientists are not involved, but that involvement needs to be more than just being the person who gets all the grant money and signs off on projects and papers--they need to be involved in the development and carrying out of the projects.
Though there are other ways in which the responsibility and scientific integrity of scientists can be increased, I think that the scientific community needs to actually start doing something about it--instead of just going "oh well" or giving a simple slap on the hand in cases like plagiarism or "crap" publications.
Wednesday, March 4
More Peer Review Sketchiness
A new book, How Professors Think: Inside the Curious World of Academic Judgment apparently paints a not-so-nice portrait. Hmmm. . . . don't know how I feel about this. . . .
Discussed here.
Discussed here.
Monday, February 23
Peer Review Symposium
Below is an email about a Peer Review Symposium that is to be held. I thought that the quotes and the purpose (which is bolded) of the meeting was interesting:
From: ISPR/KGCM 2009 [mailto:ispr@mail.ICTconfer.org]
Subject: Invitation to a Symposium on Peer Reviewing
Only 8% members of the Scientific Research Society agreed that "peer review works well as it is". (Chubin and Hackett, 1990; p.192).
"A recent U.S. Supreme Court decision and an analysis of the peer review system substantiate complaints about this fundamental aspect of scientific research." (Horrobin, 2001)
Horrobin concludes that peer review "is a non-validated charade whose processes generate results little better than does chance." (Horrobin, 2001). This has been statistically proven and reported by an increasing number of
journal editors.
"Peer Review is one of the sacred pillars of the scientific edifice" (Goodstein, 2000), it is a necessary condition in quality assurance for Scientific/Engineering publications, and "Peer Review is central to the organization of modern science...why not apply scientific [and engineering] methods to the peer review process" (Horrobin, 2001).
This is the purpose of the International Symposium on Peer Reviewing: ISPR (http://www.ICTconfer.org/ispr) being organized in the context of The 3rd International Conference on Knowledge Generation, Communication and Management: KGCM 2009 (http://www.ICTconfer.org/kgcm), which will be held on July 10-13, 2009, in Orlando, Florida, USA.
From: ISPR/KGCM 2009 [mailto:ispr@mail.ICTconfer.org]
Subject: Invitation to a Symposium on Peer Reviewing
Only 8% members of the Scientific Research Society agreed that "peer review works well as it is". (Chubin and Hackett, 1990; p.192).
"A recent U.S. Supreme Court decision and an analysis of the peer review system substantiate complaints about this fundamental aspect of scientific research." (Horrobin, 2001)
Horrobin concludes that peer review "is a non-validated charade whose processes generate results little better than does chance." (Horrobin, 2001). This has been statistically proven and reported by an increasing number of
journal editors.
"Peer Review is one of the sacred pillars of the scientific edifice" (Goodstein, 2000), it is a necessary condition in quality assurance for Scientific/Engineering publications, and "Peer Review is central to the organization of modern science...why not apply scientific [and engineering] methods to the peer review process" (Horrobin, 2001).
This is the purpose of the International Symposium on Peer Reviewing: ISPR (http://www.ICTconfer.org/ispr) being organized in the context of The 3rd International Conference on Knowledge Generation, Communication and Management: KGCM 2009 (http://www.ICTconfer.org/kgcm), which will be held on July 10-13, 2009, in Orlando, Florida, USA.
Subscribe to:
Posts (Atom)