After Tuesday’s discussion and thinking the Webster article it is rather clear that the current model for conducting, funding, and disseminating scientific research is far from perfect.
I think part of the problem stems from the necessity of quantifying the merit of researchers (for promotion, salary, etc.) and quantifying the merit of individual projects (for funding, publishing, validity, etc.). Because the current system uses publication numbers as a metric we have many “crap publications.” In truth, there are probably many different skills we are interested in measuring which aren’t really captured by the metric such as: meticulousness, creativity, work ethic, analytical ability, writing ability, logic, and so forth. So what might we do to change this system?
For arguments sake let us pretend we can ignore political issues and some of the funding issues and redesign the whole system using existing information technology. I envision online collaborative communities of researchers roughly divided by scientific domain. A typical researcher might belong to several different such communities. Each community would have guidelines dictating acceptable social behavior, and individuals not willing to comply could be asked to leave similar to how web forums are moderated.
Within each community individual projects would go through series of phases (the number of which would depend partly on the field of study and the standards set by the community). Examples phases might be design, implementation, analysis, write-up. During each phase a researcher could use the collaborative knowledge and experience of the community to work through problems and share successes with the community. Because everything is transparent collaborators get credit for helping researchers at competing universities or research labs. And because along every step of the way each project is being continuously critiqued and improved the science at the end is better. If people have differing ideas on how the project should go forward the project could be split and both ideas could be pursued similar to how open source software projects often split. If the project makes it through all the phases the final write-up is archived.
Credit would be divided between all individuals who actively contributed to the project providing more resolution into the actual contribution and capability of the individual researchers. Research money could go to the communities and the communities could decide how to use it as a group. Because of the collaborative nature they would focus on maximizing the utility of the money instead of just giving it to more prestiges institutions or based on nepotism.
Some elements are already taking place with scientific blogs, societies, and online publishing. The idea is to expand upon the positive elements of those trends, and create a system which fosters collaborative research and social transparency.
Really interesting idea. Would the natural extension of this idea be something like "Wiki-Science": give up trying to sort out the crap from the non-crap via peer-review and just make everything electronic. The scientific community as a whole would in a sense take over peer review by mere dint of what was taken to be influential, useful, &c. J.S. Mill who advocated the "marketplace of ideas" might have gone for this. The obvious trouble, however, is with the inefficiency that might well result, though again, as devotees of free-market thinking like to point out, the market is rather good at finding efficiencies (recent financial crisis notwithstanding).
ReplyDeleteI like this idea as well, but because other people/scientists have access to all these proposals via the internet, couldn't there be a huge increase in plagiarism, and one scientist taking another scientists idea? I may be wrong I was just merely wondering. And if this was to occur, what would be the penalty, I'm assuming there would be a huge ethical concern for "stealing" other people's ideas.
ReplyDeleteThanks for the feedback. It would be sort of like a Wikipedia meets Facebook meets ArsTechnica.
ReplyDeleteI think the argument could be made that this model is actually faster and more efficient than the conventional model because there is more automation, no printing delay, less peer review holdup, and less "crap" to sort through.
In regard to plagiarism the electronic system would keep precise time logs making it very evident who plagiarized whom. Filters could even tag possible plagiarizing as content was submitted similar to how email spam filters sort spam.
I think scientific fraud would also be discouraged because whole data sets could be submitted along with data analysis scripts. Because journal space is at such a premium a lot of stuff has to be left out.