The future of science, gradical change, and tools for the people

Maybe you've felt it - the buzz in a room, the tension in the air, the accelerating pace at which people are connecting and the realization that we're all in this together, even if we don't quite know what "this" is. At least in my small pocket of the world (wide web), something is brewing.

That something is The Future of Science. Michael Nielsen has written about this at length in preparation for his forthcoming book of the same name, with a lively discussion in the comments following. At BioBarCamp this past weekend (many thanks to John Cumbers and Attila Csordas for organizing!), the future of science became a recurring theme, with an impromptu discussion on open science the first day and spirited sessions on open science, web 2.0, the data commons, change in science, science "worship", and redefining "impact" and "failure" the second. Each of these topics could be their own blog series, and, in fact, many of them are. Even if people didn't always agree on the details, it was clear that everyone there (a biased group, inarguably) agreed that change is necessary, and inevitable. The question is, what will that change look like, and how will we get there?

The creators of Labmeeting.com put forth the following thesis:

Science relies on trust. Trust only remains intact when change occurs through consensus. Change through consensus is inherently gradual. (Therefore change in science must be gradual to succeed.)

Though you could agree or disagree with each statement, there are two things I'd like to discuss in particular. One is the issue of trust. Science relies on trust, right? I would say instead that science could be built on trust, if people weren't so worried about it! The most popular argument made against radical openness in science is based on the fear that other people will not act in good faith, i.e. if you make your lab notebook public, you could get scooped. And yet it is exactly this current climate of secrecy and cutthroat competition that encourages scooping and offers little recourse when it happens. If all research were open, digital, and timestamped, there would be an indisputable record of work and ideas that could be used to argue precedence.

Of course, this all starts to sound a little chicken and egg after a while. How do we assuage the fear of scooping enough for things to get sufficiently open so that scooping really isn't a problem? This brings us to the next point - that change must be gradual. Let me add the session leaders' conclusion to this: "the first step is to create incentives for scientists to voluntarily start doing the same everyday things on the same web platform." I think this is a valuable statement to keep in mind as more and more web 2.0 tools and platforms keep cropping up - that in some sense, the best way to enact satisfied change is to make it beneficial to the individual researcher, and allow them to discover this on their own terms. Scientists are a skeptical lot by training; the fact that they are also generally time-strapped and resource-starved makes them, ironically, reluctant to experiment, at least with the way they do their work. They neither need, nor want, another social networking tool.

The key that some groups have discovered (Labmeeting, Epernicus, and OpenWetWare among them) is to discover what people need, and then build something they will want. For Labmeeting, it is online paper management, for Epernicus it is effective question answering and resource finding (no more wild goose chases looking for someone who can help you with a specific problem), and for OWW it is tools for managing group websites and sharing protocols. Although Epernicus does rely on there being a social/professional network in place, the other two provide services that are useful even if you're the only one using it; the online community therefore can build itself without pressure. And Epernicus along with the others recognizes that in order to be successful among scientists, you need to provide them with something useful. In other words, you need to make tools for the people, rather than tools that need people.

So what about change? How will it happen and when? Well, I'm hoping Michael's book will tell us. ;) But I have a feeling it will be "gradical" - gradual at first, and then...

Labmeeting.com - "Make science easier"

Stanford students are nothing if not entrepreneurial. Only a couple months after reporting on Ologeez (out of the Genetics department), I receive an email about Labmeeting.com, the product of a group of students in Vijay Pande's lab, headed by a graduate student in the Physics department created by three folks who knew each other from their undergrad days at Harvard, one of whom is now a biophysics PhD student at Stanford.

The website grew out of their desire to "solve some of the organizational problems we've encountered while doing our Ph.Ds", and includes a PDF organizer, a space for labs to share protocols and files, and mechanisms for discovering and recommending papers.

After registering and doing a brief tour, it appears to be a well thought-out and executed entrant into what is starting to become a crowded market for tools that help you organize, search for, and share papers, or tools that help your lab share data and files. Because of the many offerings, however, some with much earlier and more widespread adoption, I'd be surprised if Labmeeting gathers much of a foothold. Then again, it's not claiming to be another OpenWetWare, so if it does what it does well, perhaps those looking for a smaller feature set (and a bit of relief from social networking) will find it just right.

Epernicus - 2 steps closer to a science exchange?

This morning I received an email from Mikhail Shapiro inviting me to join Epernicus, a science networking site which he helped launch. He'd visited the blog and (quite rightly) deduced that I would be interested in trying out a web community that aims to do some of the very things discussed here regarding a science exchange. So after the slightest hesitation (so many networking sites, so little time...), I decided to give it a go. I populated my profile (inspiring me to update my CV), uploaded a picture and a shameless plug for the Open Science Collective shop, checked out other people's profiles (has anyone invented the term "profile envy" yet?), clicked on a little link next to the "Launch BenchQ" icon for "What is BenchQ?", and generally muddled around until I ran out of narcissistic things to do.

So far, I'm rather pleased with Epernicus. It has a nice, aesthetically pleasing interface (as do SciLink and Laboratree, which I've mentioned before) and I didn't run into any bugs like others have experienced with other networking sites. When you first sign up and fill out your affiliations, it automatically populates your network with members of Epernicus with the same affiliations - in my case, 2 students in my PhD program and 60 or so people from my university. Once you've signed up, you can create groups, send people messages, and invite others to join - fairly standard social networking fare. The two distinguishing features of Epernicus, however, are Assets and BenchQ.

As part of your Epernicus profile, you can list your "assets" - topics (Phosphoprotein signaling, gene expression analysis, etc), materials (anything from chemicals, cell lines, equipment, and kits), and methods (protocols, algorithms, techniques) in which you can claim knowledge or experience ranging from novice to expert. Through their search bar, you can identify people who know something about your topic of interest. Taking this one step further is BenchQ, their version of a discussion board, which lets you post a question to a person or network. The questions can be geared towards methods ("How do I stain live cells?", "What is the best software for hierarchically clustering microarray data?") but can also be requests like "I need to do assay Y on protein X. Does anyone have purified protein X available?" or "My [equipment] is broken, can I use someone else's this weekend?"

I posed two of the major issues that came up during the science exchange discussion to Mikhail. Issue 1 is Motivation - what is the incentive to help in science? Issue 2 is making connections rapidly and effectively - given the passive nature of BenchQ, were people getting answers to their questions? Issue 1 can be mitigated by having networks of people who are more likely to help each other because they have an actual professional relationship. Regarding issue 2, Mikhail said that he'd seen a number of successful interactions within the MIT and Harvard networks, and that almost every BenchQ question gets answered, which sounds like a great start. He also mentioned that an algorithmic notification targeting system is being developed, sort of like what human supernodes might do, which I'm interested in seeing when it comes out.

One thing Epernicus does not seem to support yet is some kind of "Projects" feature.* This is kind of a loose concept but generally it could be described as a place where you could list projects you work on and include pages with more information or interactive areas for each project. Project management, sort of. A mini-blog type thing could also be a useful feature, though at some point you need to decide what features contribute to the overall scientific mission and which ones distract. If Epernicus is more about science networking and less about the actual collaboration and project management aspects, maybe it's fine for it not to go down that road.

For a site that does go down that road, Laboratree tasks itself as a research management system, so in addition to the shallow networking aspects, it also allows you to upload documents, write blog entries, and create projects with group members, messages, and project-specific blogs.

In my limited experience, I've been most impressed with Laboratree and Epernicus out of the various science networking sites I've seen (though Laboratree does have some bugs of its own - notably the search bar, and empty Help page). They independently provide different aspects of the science exchange idea and do so with style without being over the top. A question now is whether we want to have everything but the kitchen sink in one website? Or do multiple networking sites (like Laboratree for research management and Epernicus for question-answering) serve their different functions? Is it simply an issue of critical mass?

Check out my public profile at Epernicus.

* Update: After posting this, Mikhail pointed out that the "Group" feature does accommodate some project management through document uploads and a "whiteboard". Thanks for the clarification and apologies for the lack of research on my part!

Envisioning the scientific community as One Big Lab

The blogosphere has been abuzz recently, or, at least, it seems that way if you've only been checking up on it sporadically the last few weeks. Jennifer Rohn's post about lab notebooks has spurred over 100 lively comments spanning electronic lab notebooks, peer-review, openness in science, and the reward system in science, making for an engrossing peek at the social science of science. Cameron's own musings on that discussion. Pawel Szczesny writes about what it means to be a freelancing scientist. All of this is fascinating and it is exciting to contemplate both what the future of science holds and the obstacles we will need to overcome; the fact that there are indeed stubborn obstacles (technological as well as cultural) and potentially tremendous rewards makes the anticipation of that future all the more heightened.

Emboldened by the collective fervor, I would like to propose an idea - an idea with the same name as this blog. But first, the back story.

About 8 months ago, one of my lab mates was writing up a short paper for submission to a translational bioinformatics conference. The work she was submitting revolved around a powerful literature-search tool tailored for pharmacogenomics called Pharmspresso. Although Pharmspresso had features lacking in existing search methods and was thus useful, the intent was for it to recognize genes, drugs and polymorphisms in free text, and so she needed a way to evaluate its performance. The evaluation task would be straightforward: given a set of pharmacogenomics papers, what percentage of the mentions of genes, drugs, and polymorphisms does Pharmspresso capture? Getting the list of recognized entities from Pharmspresso would be easy, just give it the documents and set it running. But what would be the gold standard?

Typically, gold standards are created by humans. In this case, it would be the list of entities recognized by human readers with the appropriate knowledge to make the distinctions, in the same set of papers. To get her gold standard then, she essentially asked favors of her colleagues in the lab and the department, which translated to a number of them reading papers and doing data entry during free time (or during faculty talks) at a departmental retreat in early fall - not exactly fun, but done out of a sense of duty to science and the goodness of their hearts.

Afterwards, while socializing during one of the poster sessions, this task came up, and the discussion (in which Samuel Flores, Magda Jonikas, Yael Garten, Alain Laederach, and Bernie Daigle all participated) quickly turned to alternative solutions for tackling this and similar problems in science - those requiring knowledge and resources external to your own. As another example, many bioinformaticians work on problems that produce predictions of functions which would benefit from experimental tests of their validity. Conversely, a wet lab may benefit greatly from someone with computational expertise guiding or leading the data analysis, or even providing the hypotheses for experimental studies (in the form of predictions). This is the stuff from which many collaborations are born, but it may be difficult to find the right people in the first place, or the task at hand might seem not quite collaboration-worthy.

In essence, the problem boils down to this: you or your lab possesses a certain collection of skills, knowledge, and resources (hereafter referred to as simply resources), but your needs may not be fully addressed by what you possess. The solution lies in this simple proposition: some other person or lab has what you're looking for.

While it makes sense for a lab or individual to grow their resources and be mostly self-sufficient, at some point it becomes more economical to outsource certain tasks - to companies for antibody development, software for data analysis, supercomputers for high-throughput computing, etc. In some cases, the exchange takes place directly at the academic level, for example, with some labs maintaining and sharing specific cell lines or mouse strains for use by other researchers, or less directly through the use of published and available tools for all sorts of tasks in bioinformatics. So it would seem that outsourcing is common and accepted. But aside from these sorts of established avenues, what other needs do scientists have in conducting their research that are not easily solved? How often is a line of inquiry abandoned or slowed because of a lack of necessary skills, knowledge, or material resources?

The idea behind One Big Lab is that the scientific community should act as, well, one big lab, sharing resources when it makes sense, and everyone, especially the community as a whole, benefits.

During that discussion at the departmental retreat, the solution boiled down to some form of online transaction service built around a credit system. Scientist X would like 5 gold standard outputs for a certain task, so she posts a description of the task along with some credit attached. Other users can then sign up to complete the task, after which they receive the stated number of credits. Of course, in order to post tasks, you need to have a balance of credits you can draw from - which you earn by doing other people's tasks. Getting credits into the system to start needs to be figured out (give everyone N credits? Money for credits?), but assuming there's some baseline of credit floating around amongst the various users, an equilibrium should eventually be reached (at least, that's the hope).

Variations on this theme are natural - have a peer rating system, have the final credit payment be subject to a bidding system (based somehow on user ratings, e.g. highly rated users can ask for more credits to complete a task and the task-poster may select which user to "hire" based on the user ratings as well as how much each user is asking), have some kind of mechanism for taking transactions "offline" into serious collaborations, etc. Tasks may run the gamut from routine and rote to intellectually stimulating and scientifically rewarding. Obviously, guidelines will have to be set for what transactions may be appropriate for this forum and which ones might be more suited for formal, collaborative relationships - but even here, a forum such as this could be very useful for finding collaborators.

In addition to the scientific transaction system, there could be other features that build on the community aspect, such as journal clubs, informal manuscript review, resources for students, and discussion forums. There could be repositories for knowledge or links to existing ones, informal or formal consulting, and casual exchange of ideas which could stimulate research or professional development. All of this should reinforce the idea that science is strengthened by community and the scientific community should not be held back by insufficient allocation of resources.

Although there are a number of websites out there that tackle some of these aspects, especially the community-building ones, I haven't really seen much resembling the transaction system, which is really the core of the idea. Pawel's freelance science comes close, and what I'd like to see is a formalized community-wide online service for essentially that. Maybe this is technically infeasible right now right the way grants work (it may be difficult to justify spending time or resources on other people's research) or with the way scientists work, but I would like to think that the basic premise - bringing together people with complementary skills and resources - makes sense and balances out in everyone's favor. (Whether this premise actually pans out in practice is up for debate - if we offered credits for cash, would anyone ever do someone else's tasks, or would demand outpace supply? By the same token, there could be "freelance" scientists like Pawel who primarily complete tasks, and could then have the option of "cashing out".) I'm sure there are a ton of tricky legal, IP, financial, organizational, etc not to mention social and cultural issues (would you trust someone you don't know to do work for you?), but I think the idea of having One Big Lab is worth exploring.

If I had the time, skills, and business acumen I would throw together a prototype and work out a business plan, but at the moment the most I can do is outsource it to the closest thing we have to One Big Lab - the blogosphere. ;)

Incidentally, Alain Laederach had come up with a similar idea about a year earlier and we thought about naming it "Experitrade" - an online system for trading experiments, essentially, but the name sounded too corporate and the grant he wrote never got off the ground. But the idea has persisted and inspired One Big Lab.

So, I'd welcome any thoughts, logical extensions, deal-makers or deal-breakers, important issues to consider, "prior art"... does anyone think this idea has legs? Will it work if it is completely altruistic? Does adding money into the equation detract from its mission or the science? What sorts of technical and organizational roadblocks are there? Clearly it makes the most sense, if any prototype is developed, to start small - with a couple participating labs or within a school or university, which helps with the trust issue as well. But I'd like to make sure I'm not completely missing the picture!

New paper-protocol-lab-knowledge sharing website out of Stanford

Stanford PhD student Jason Hoyt in the Department of Genetics was fed up with the inadequate presence of literature resources on the web, specifically good discussion surrounding papers, so he's set out to build his own website that would allow users to post, rate, and discuss papers, in addition to other features. Jason says,

Hey fellow colleagues and grad students. So, about a year and a half ago I got tired of the lack of good discussion around research literature online. For instance, what was the best review paper in the field of a new research project I was about to start? So, I started building a website.

What I ended up with was:
-A citation manager called 'My Libraries' (easily download papers to EndNote)
-A lab database called 'WikiGroups' for any lab in the world
-A protocols database
-A paper search that gives better results than PubMed (this depends on you adding more
papers)
-Import papers from PubMed
-Contact or colleague manager called 'Notes'
-A 'My World' page that gathers all the latest from your colleagues, lab group activities
and school seminars.

It's in beta, so please report any bugs or feature requests (form available on all pages).

It's called Ologeez. From the plural of the suffix "-Ology," it refers to every branch of learning. If you find it useful, let other departments or schools know.

After very briefly exploring Ologeez, it seems like a competent addition to the handful of other science oriented resource and knowledge sharing websites currently available. OpenWetWare offers lab websites and shared protocols, but doesn't have literature-oriented resources. PLoS ONE has a journal club feature, but just for PLoS ONE and PLoS doesn't host lab websites or protocols. Laboratree and SciLink offer nice networking and some content management features, but don't support lab websites and literature discussion is indirect at best. Although Ologeez has very few users and entries right now, people may find it useful to be able to set up a lab presence with shared protocols and papers, post and discuss interesting papers, and keep up to date with what their colleagues are doing, all in one website. It includes categories for all branches of science and research, including business/econ, law, and math.

Given its inclusiveness, it has the potential to spread school-wide, though it'll be interesting to see if it catches on enough for the discussion and search features to be useful.

Laboratree

Sean Mooney and his colleagues at the Indiana University Center for Computational Biology and Bioinformatics Core are developing a new professional networking site called Laboratree. It looks and feels like a more subdued version of Facebook, which is a good thing in my opinion - Facebook is all about Too Much Information and too many annoying alerts and requests, whereas I think (and hope) Laboratree will stay trim and relevant. It also integrates a blog into your account, which is nice for those who don't already keep one. While Laboratree may compete with sites like OpenWetWare in some of its features, it is less of a content management system and more meant to be a networking site. It is still being developed and so has some bugs and a limited user base, but you can sign up for account.

Open Students

A brand new blogsite devoted to Open Access sprang up about a week ago. This one is geared towards students and is sponsored by the Scholarly Publishing and Academic Resources Coalition (SPARC) as part of its student outreach campaign. It's called, naturally enough, Open Students.

Thanks to A Blog Around the Clock for the tip.

The blog of negative results

Magda, inspired by her group meeting today, decided to start a blog of negative results. We've all been there before, and it probably wasn't funny then, but what they say is true: you'll look back on those times and have a good laugh. Now, with the Worst Result Ever blog, you can appreciate the humor that much faster - it's the darker and the lighter side of academia all in one!

If you have your own results that are so bad it's funny, and don't mind sharing, feel free to contact Magda so she can add it to what will surely be a growing collection.