After Tuesday’s discussion and thinking the Webster article it is rather clear that the current model for conducting, funding, and disseminating scientific research is far from perfect.
I think part of the problem stems from the necessity of quantifying the merit of researchers (for promotion, salary, etc.) and quantifying the merit of individual projects (for funding, publishing, validity, etc.). Because the current system uses publication numbers as a metric we have many “crap publications.” In truth, there are probably many different skills we are interested in measuring which aren’t really captured by the metric such as: meticulousness, creativity, work ethic, analytical ability, writing ability, logic, and so forth. So what might we do to change this system?
For arguments sake let us pretend we can ignore political issues and some of the funding issues and redesign the whole system using existing information technology. I envision online collaborative communities of researchers roughly divided by scientific domain. A typical researcher might belong to several different such communities. Each community would have guidelines dictating acceptable social behavior, and individuals not willing to comply could be asked to leave similar to how web forums are moderated.
Within each community individual projects would go through series of phases (the number of which would depend partly on the field of study and the standards set by the community). Examples phases might be design, implementation, analysis, write-up. During each phase a researcher could use the collaborative knowledge and experience of the community to work through problems and share successes with the community. Because everything is transparent collaborators get credit for helping researchers at competing universities or research labs. And because along every step of the way each project is being continuously critiqued and improved the science at the end is better. If people have differing ideas on how the project should go forward the project could be split and both ideas could be pursued similar to how open source software projects often split. If the project makes it through all the phases the final write-up is archived.
Credit would be divided between all individuals who actively contributed to the project providing more resolution into the actual contribution and capability of the individual researchers. Research money could go to the communities and the communities could decide how to use it as a group. Because of the collaborative nature they would focus on maximizing the utility of the money instead of just giving it to more prestiges institutions or based on nepotism.
Some elements are already taking place with scientific blogs, societies, and online publishing. The idea is to expand upon the positive elements of those trends, and create a system which fosters collaborative research and social transparency.