Gregory Petsko expounds eloquently on the "climate of fear" in science in a recent commentary in Genome Biology, titled "The right to be wrong." Drawing a provocative parallel to US politics, he describes how honest, intelligent people willing to admit their (almost always) understandable mistakes are turned on and burned at the stake for by their opponents, accused of lacking integrity and being "flip-floppers" . In science, the attacks are much less direct, but the attitude is still entrenched, and the vast majority of scientists are opting for "safe", incremental, "data gathering/discovery" based research as opposed to bold, hypothesis-driven science. The sentiment is echoed by funding agencies who do not want to risk funding anything that might "fail."
The commentary, although ominous at first, should inspire us all to behave as true scientists should - boldly but carefully, objectively and rationally.
Showing posts with label academia. Show all posts
Showing posts with label academia. Show all posts
Tuesday, March 18, 2008
Thursday, January 31, 2008
The scooping debate continues
Bora Zivkovic over at ScienceBlogs posted about a scooping story that was just published in Nature. The story itself is quite scandalous, since the individual in question doesn't have a great reputation as far as I could tell from reading various comments on blogs posting on the subject. Go see some of them at ScienceBlogs to get an idea. What I want to address in this post has to do with Bora's commentary on the story, since he suggests that in an Open Science world, scooping would be much more difficult to pull off, since everything is documented and associated with time stamps, and the community can rally behind the "first to blog". I'm not sure the picture is that simple.
Something that I did not fully appreciate before about scooping is that the "scooper" will claim that the discovery was made independently, which is difficult to disprove in many fields. Paleontology and archeology may have very slight advantage here in that some types of discoveries are singular and tangible - a fossil in the desert, artifacts under a dirt mound - physical things at physical locations. Someone would be hard-pressed to say they were at the same place someone else was, digging up the same object (though, as the aetosaur controversy makes clear, scooping of a related sort can still happen). In biology, you might slave away for years to discover that protein A regulates protein B, but if someone else publishes it first, you're out of luck. Sure, you have the reagents and the cell lines and the protein products - but so do your competitors, as long as they have the basic equipment and resources in place to reproduce what you did. So the fear that making your research public would increase your risk of getting scooped maybe is not that unfounded. Closed scientists could easily leech off the hard work of Open scientists with no one being able to prove anything.
Of course, that is an extremely pessimistic view, but unfortunately, it's a kneejerk reaction from many people in biomedical fields. Until Open Science becomes so widespread that anything Closed is viewed with suspicion, there will be the possibility of exploitation. And that statement itself smacks highly of Big Brother. I don't think we want a "tryanny of Openness" any more than we want scooping to happen. My conclusions from this rather sobering train of thought are that yes, scooping is a moral outrage and the fact that it is an issue is frustrating, but because it does happen, we need to think carefully about how to prevent it from escalating as more people go Open. Can we prevent it? Is collective disapproval enough?
Something that I did not fully appreciate before about scooping is that the "scooper" will claim that the discovery was made independently, which is difficult to disprove in many fields. Paleontology and archeology may have very slight advantage here in that some types of discoveries are singular and tangible - a fossil in the desert, artifacts under a dirt mound - physical things at physical locations. Someone would be hard-pressed to say they were at the same place someone else was, digging up the same object (though, as the aetosaur controversy makes clear, scooping of a related sort can still happen). In biology, you might slave away for years to discover that protein A regulates protein B, but if someone else publishes it first, you're out of luck. Sure, you have the reagents and the cell lines and the protein products - but so do your competitors, as long as they have the basic equipment and resources in place to reproduce what you did. So the fear that making your research public would increase your risk of getting scooped maybe is not that unfounded. Closed scientists could easily leech off the hard work of Open scientists with no one being able to prove anything.
Of course, that is an extremely pessimistic view, but unfortunately, it's a kneejerk reaction from many people in biomedical fields. Until Open Science becomes so widespread that anything Closed is viewed with suspicion, there will be the possibility of exploitation. And that statement itself smacks highly of Big Brother. I don't think we want a "tryanny of Openness" any more than we want scooping to happen. My conclusions from this rather sobering train of thought are that yes, scooping is a moral outrage and the fact that it is an issue is frustrating, but because it does happen, we need to think carefully about how to prevent it from escalating as more people go Open. Can we prevent it? Is collective disapproval enough?
Friday, January 25, 2008
Is the danger of being scooped field-dependent?
The students in my program get together once in a while for what we call "Researchome", also known fondly as "dinner-ome", originally conceived as a casual forum in which students could present their research or other topics of interest to other students, while getting dinner for free. But without someone to present, there is no justification to have Researchome, so rather than deprive a dozen grad students of free food, I threw together a quick presentation of Open Science and my proposal for PSB for our Researchome last night.
Biomedical informatics students are a smart bunch, so there was some great discussion. Naturally, the concern over getting scooped came up, and while I was quick to pooh-pooh it as a naive/narcissistic fear, the others were fairly adamant that it was a valid concern. Several gave personal anecdotes. And the picture that started to emerge was one where the danger of being scooped was highly dependent on the field you were in - theoretical vs. applied, basic vs. translational, science vs. medicine, all of which may put different emphasis on the idea vs. the implementation.
According to one of the students at the Researchome, in theoretical disciplines such as math or physics, credit is given as soon as an idea is recorded. But in fields like cell biology, just having the idea for an experiment or a hypothesis is not enough; instead, you must conduct the experiment and demonstrate successful results through a peer-reviewed publication before credit is given. Because of this, people in these fields are more reluctant to be open about their research before it has been published, and getting scooped can have real consequences for someone's career and funding. In my limited explorations of the world wide open science web, it seems as if a significant portion of those participating are chemists. Is getting scooped less of a concern in chemistry than it is in, say, molecular biology, and, if so, why? If there is a discrepancy between fields in the danger of being scooped, how should this be addressed as the open science community moves forward? Is it possible to change the standards by which success and intellectual credit are determined?
Aside from this interesting issue, some valid points were brought up concerning the proposal for an open science session at PSB. One is that the audience at PSB is by and large composed of scientists who don't generate their own data, but use the data generated by others. A lot of high-throughput, -omics, and bioinformatics-minded people. Therefore, open data and open source will probably be of greater interest to them than the more overarching idea of open notebook science. Focusing on standards, exchange formats, and tools and methodologies for conducting open science may be a good approach.
The other consensus that the students came to was that open science is so broad and important a topic that it should be featured as a session at a much larger conference, such as ISMB or AMIA. Many were bemused as to why I chose what is arguably a niche conference as a venue for open science. My rationale at this point is that it is the soonest we could possibly organize a meeting on open science jointly with an established conference, and I think the audience is relevant enough for it to be productive. Being smaller, it may also be a good stepping stone towards a bigger meeting, and I wouldn't be surprised if some efforts began for that before PSB 2009 comes around.
Many thanks to the students who attended the Researchome for their feedback. I'll be working on a draft of the proposal over the next couple days.
Biomedical informatics students are a smart bunch, so there was some great discussion. Naturally, the concern over getting scooped came up, and while I was quick to pooh-pooh it as a naive/narcissistic fear, the others were fairly adamant that it was a valid concern. Several gave personal anecdotes. And the picture that started to emerge was one where the danger of being scooped was highly dependent on the field you were in - theoretical vs. applied, basic vs. translational, science vs. medicine, all of which may put different emphasis on the idea vs. the implementation.
According to one of the students at the Researchome, in theoretical disciplines such as math or physics, credit is given as soon as an idea is recorded. But in fields like cell biology, just having the idea for an experiment or a hypothesis is not enough; instead, you must conduct the experiment and demonstrate successful results through a peer-reviewed publication before credit is given. Because of this, people in these fields are more reluctant to be open about their research before it has been published, and getting scooped can have real consequences for someone's career and funding. In my limited explorations of the world wide open science web, it seems as if a significant portion of those participating are chemists. Is getting scooped less of a concern in chemistry than it is in, say, molecular biology, and, if so, why? If there is a discrepancy between fields in the danger of being scooped, how should this be addressed as the open science community moves forward? Is it possible to change the standards by which success and intellectual credit are determined?
Aside from this interesting issue, some valid points were brought up concerning the proposal for an open science session at PSB. One is that the audience at PSB is by and large composed of scientists who don't generate their own data, but use the data generated by others. A lot of high-throughput, -omics, and bioinformatics-minded people. Therefore, open data and open source will probably be of greater interest to them than the more overarching idea of open notebook science. Focusing on standards, exchange formats, and tools and methodologies for conducting open science may be a good approach.
The other consensus that the students came to was that open science is so broad and important a topic that it should be featured as a session at a much larger conference, such as ISMB or AMIA. Many were bemused as to why I chose what is arguably a niche conference as a venue for open science. My rationale at this point is that it is the soonest we could possibly organize a meeting on open science jointly with an established conference, and I think the audience is relevant enough for it to be productive. Being smaller, it may also be a good stepping stone towards a bigger meeting, and I wouldn't be surprised if some efforts began for that before PSB 2009 comes around.
Many thanks to the students who attended the Researchome for their feedback. I'll be working on a draft of the proposal over the next couple days.
Wednesday, January 23, 2008
The blog of negative results
Magda, inspired by her group meeting today, decided to start a blog of negative results. We've all been there before, and it probably wasn't funny then, but what they say is true: you'll look back on those times and have a good laugh. Now, with the Worst Result Ever blog, you can appreciate the humor that much faster - it's the darker and the lighter side of academia all in one!
If you have your own results that are so bad it's funny, and don't mind sharing, feel free to contact Magda so she can add it to what will surely be a growing collection.
If you have your own results that are so bad it's funny, and don't mind sharing, feel free to contact Magda so she can add it to what will surely be a growing collection.
Friday, January 18, 2008
New meaning to "publish or perish" - an opening for Open Science?
The saying "publish or perish" is well-known in academia, and typically both actions refer to the same subject - you, the aspiring/struggling grad student/post-doc/fellow/assistant professor. A recent correspondence in Nature puts a new and bracing spin on the phrase.
I think at some point most academic researchers have experienced the conflict that can arise when it is time to write a paper. On the one hand, you're getting a chance to reward those months or years of hard work with some exposure and a line on your CV, and invest in the potential for future collaborations. On the other, maybe you've just gotten started on a really promising or exciting research direction, are in a groove, work-wise, and to have something like writing suddenly vying for your attention just means that both activities suffer. You feel that you can't drop what you're working on to write the paper, but the paper writing is distracted and unfocused because you're still trying to conduct research half the time (and thinking about it more than that). But we march on to these two seemingly competing drummers, fueled somewhat by the vague hope that our work, once it is in the public domain, will also contribute a drop in the bucket that is scientific advancement of our species.
But what about other species? In conservation biology, "publish or perish" can take on new, and frighteningly literal, meaning. Time spent working on publications is time taken away from research on ecosystems and endangered wildlife. In the meantime, earth's natural resources and diversity suffer. To prevent this from happening, the authors of the letter suggest (only slightly ironically) the adoption of a new impact factor:
Even if this proposal was made half in jest, it does highlight some important questions. The first sentence essentially asks: how much faster could research be conducted (and, by translation, medical or scientific advances be developed) if there was less emphasis on publication? The second sentence is quite a bit more complex, since it seems it would bring in value judgments on the worth of specific research questions - something that would be hard to define objectively and is easily influenced by prevailing trends, funding, and big talk.
So let's talk about the first idea - that the pace of scientific advancement suffers from the emphases placed on publication. Obviously, research needs to be disseminated if it is going to contribute. But here is where Open Science comes in. Suppose Open Science and Open Notebook Science became the norm rather than the burgeoning, but still fringe, movement that it is now. Two big questions immediately come to my mind: Would publication matter as much as it does now? Would research proceed faster? I say no and yes.
With most, if not all, of your methods, data, and results made public, formal publication would not be necessary for others to learn of and benefit from your work. Peer-review may become an intrinsic part of the entire research process. Of course, a formal summary of your work adds great value and would be indispensable for someone searching for information on your field of study, but much of the pressure to publish could be alleviated. Add to that the increased exposure to the entire community and you get enormous potential to speed up your research in addition to research in general. You can learn what is working and not working in your experiments, get useful feedback and suggestions, and meet people who may be able to help you, all on a much faster timescale. At the same time, new ideas may be spawned, collaborations fostered, and interesting connections made between concepts.
Obviously, the future of Open Science is not going to be as rosy as that, at least in the early stages of its evolution (issues like patenting and privacy are valid and worth lengthy discussion in their own right, but are beyond the scope of this post). In fields like conservation biology, however, the shadow cast by "publish or perish" has terribly real implications, and the move towards Open Science will help to lift it. Can anyone really argue that Open Science is a bad thing?
I think at some point most academic researchers have experienced the conflict that can arise when it is time to write a paper. On the one hand, you're getting a chance to reward those months or years of hard work with some exposure and a line on your CV, and invest in the potential for future collaborations. On the other, maybe you've just gotten started on a really promising or exciting research direction, are in a groove, work-wise, and to have something like writing suddenly vying for your attention just means that both activities suffer. You feel that you can't drop what you're working on to write the paper, but the paper writing is distracted and unfocused because you're still trying to conduct research half the time (and thinking about it more than that). But we march on to these two seemingly competing drummers, fueled somewhat by the vague hope that our work, once it is in the public domain, will also contribute a drop in the bucket that is scientific advancement of our species.
But what about other species? In conservation biology, "publish or perish" can take on new, and frighteningly literal, meaning. Time spent working on publications is time taken away from research on ecosystems and endangered wildlife. In the meantime, earth's natural resources and diversity suffer. To prevent this from happening, the authors of the letter suggest (only slightly ironically) the adoption of a new impact factor:
This impact factor would be based on an estimation of how much worse the conservation status of an endangered species or ecosystem might be in the absence of the candidate's research. It would select for targeted investigation that should help to fill in 'the great divide', and would exclude opportunistic ecology papers claiming to be of conservation significance.
Even if this proposal was made half in jest, it does highlight some important questions. The first sentence essentially asks: how much faster could research be conducted (and, by translation, medical or scientific advances be developed) if there was less emphasis on publication? The second sentence is quite a bit more complex, since it seems it would bring in value judgments on the worth of specific research questions - something that would be hard to define objectively and is easily influenced by prevailing trends, funding, and big talk.
So let's talk about the first idea - that the pace of scientific advancement suffers from the emphases placed on publication. Obviously, research needs to be disseminated if it is going to contribute. But here is where Open Science comes in. Suppose Open Science and Open Notebook Science became the norm rather than the burgeoning, but still fringe, movement that it is now. Two big questions immediately come to my mind: Would publication matter as much as it does now? Would research proceed faster? I say no and yes.
With most, if not all, of your methods, data, and results made public, formal publication would not be necessary for others to learn of and benefit from your work. Peer-review may become an intrinsic part of the entire research process. Of course, a formal summary of your work adds great value and would be indispensable for someone searching for information on your field of study, but much of the pressure to publish could be alleviated. Add to that the increased exposure to the entire community and you get enormous potential to speed up your research in addition to research in general. You can learn what is working and not working in your experiments, get useful feedback and suggestions, and meet people who may be able to help you, all on a much faster timescale. At the same time, new ideas may be spawned, collaborations fostered, and interesting connections made between concepts.
Obviously, the future of Open Science is not going to be as rosy as that, at least in the early stages of its evolution (issues like patenting and privacy are valid and worth lengthy discussion in their own right, but are beyond the scope of this post). In fields like conservation biology, however, the shadow cast by "publish or perish" has terribly real implications, and the move towards Open Science will help to lift it. Can anyone really argue that Open Science is a bad thing?
Subscribe to:
Posts (Atom)