Thursday, March 13, 2008

Online collaborative manuscript annotation

While at the inaugural AMIA Summit on Translational Bioinformatics the first half of this week (stay tuned for another post summarizing that), I started thinking about some ideas for tools that could help make discussion of papers easier and more productive.

Currently, it seems that there are a few avenues for discussing a paper: 1) have an informal conversation in person, 2) hold a journal club where one person presents the paper and discussion ensues, or 3) blog about it and hope others comment. (You could argue that another avenue exists through some journals - especially open access ones - allowing comments on published articles, but this hasn't caught on as far as I can tell.) There are several disadvantages of the current systems. In-person conversations or journal clubs can be stimulating as they happen, but are transient and usually go unrecorded, resulting in little tangible benefit to others (or often even the participants); they also usually preclude remote participation without some sort of audio-visual setup. Going the blog route allows anyone to participate, but it's difficult to connect the comments back to the paper and the discussion may be less productive than hoped.

A group of students in my human-computer interaction class a few years ago developed an idea called Collaboread for their final project. In essence, it allowed multiple online users to markup a document, enabling collaborative annotation. I'm sure there are several products out there that allow either online markup of documents (Adobe, for one) or collaborative editing (Google Docs), but I haven't seen anything that resembles exactly what I envision.

Suppose you are viewing a document on the screen - maybe a full-text articles at BioMed Central, or a PDF. Clearly, things like web URLs and references should be hyperlinked already. But suppose you could create additional hyperlinks, such as to wikipedia pages, other papers that were not referenced but are relevant, blog posts, etc. You can also start individual discussion threads attached to a particular results, claims, or points made in the paper, or to tables or figures. Mousing over or clicking on the icon indicating such a thread would bring up a summarized view of the thread overlaid on the screen which you could browse more deeply or hide if you decide you're not interested. The idea is to make a richly annotated document that is easy to read but at the same time make it easy to see what other people thought or were confused by and respond if so inclined without too much disruption. When I envision this, I see a Google Maps-like navigation and manipulation style with lots of linked text, little colored balloons at the POIs - the discussion threads, and liberal use of tags to help with filtering and searching of the document and annotations.

A tool like this would be useful not just for journal club-style discussion of papers, but also as a teaching and editing tool. Authors could collaboratively comment on a paper, or learn from others' comments after it is published and made the focus of such a discussion. Readers and students would benefit from the additional linked resources and learn from the discussions how to critique a paper. And the annotated document would be available to everyone long after discussion has tapered off.

Of course, there are potentially many technical, legal, social, *al issues surrounding this, but I think some kind of tool along this vein would be useful and interesting. Does anyone know of any tools that do these things already? If not, I am already looking into what it would take to develop it, and would appreciate tips, suggestions, warnings...


Nadeem said...

Forgetting for a second all the technical things that are possible, I think it would be valuable just to get other peoples opinion / comments on papers. I.e. a simple forum discussing the results of a paper. As you say, some open access journal sites already do this -- but lots of interesting publications don't do this. The only place that I can think of that does something like this if -- but its closed -- i.e. you have to be an elite scientist to be able to leave a comment.

Bill Hooker said...

Marginalia springs to mind:

I'm at work, but there are probably other tools that might interest you under my Simpy/webtools tag:

shwu said...

@nadeem: I feel that simple discussion of papers could happen with existing tools - publisher websites or blogs. We just need to know where these are, maybe a central website where people could submit their impressions of papers open for comment. However, collaborative annotation would add benefit to such discussions by organizing them based on relevant passages in the document or by providing additional resources, knowledge, or interesting asides.

@bill: Marginalia looks good in that it allows linking of comments with the relevant text in the document, but I didn't see any capability for threaded discussion, or multiple comments for the same passage. I suppose one could just add to the annotation already linked to that passage, but I think it would be more useful if it was organized as a dialogue between readers.

Cameron Neylon said...

A lot of this seems to be about how you 'pivot' from one view to another. You have the paper but then you click through to a wiki, or a blog, or another paper, or the raw data. We can kind of do most of these things one at a time but what is missing is the framework that says 'now I want to look at this stuff from this other angle. Make it so...'

As I think I commented somewhere else, to make this work requires semantic search capabilities.

alf said...

Have a look at the collaboration tools at perhaps?

Michael Nielsen said...

The Django book project ( might be of interest. is the first attempt I know of to do this kind of thing on the web (from the late 90's), and generated a lot of interesting discussion. Now defunct, but still accessible using the wayback machine.

Benjamin Good said...

You should definitely have a look at the Annotea project

It may not have all the functionality you envision right now, but they've been working on the collaborative annotation problem for a long time now... Worth a chat.

Would such a service be solely focused on academic papers? Seems that it might be a nice, general-purpose application (and a great way to make use of lots of semantic web technology).

Fred Howell said...

[from one of the developers]

As Alf mentioned, A.nnotate does shared notes / discussions on PDF documents and web pages online - e.g. a couple of sample
notes on a snapshot of this blog entry. You can highlight text to write notes, and add tags; other people can reply to notes to start a discussion at a particular point in a paper.

It'll do shared notes on PDFs / Word docs in the browser too, without needing any plugins like flash or adobe reader - e.g.
sample PDF paper