I've always found the notion of peer review of philosophy journal articles a bit suspect. I also find it ludicrous that the phrase 'peer reviewed' is taken by some people to be a mark of quality assurance. A good editor is surely better than an incompetent anonymous peer in this respect.
One basic flaw in the peer review process is the selection of people to do the peer reviewing: are they competent, biased, awake, rivalrous, too busy with their own work to read to the end? The trouble is, we can't get reliable information on this sort of thing as they are almost always anonymous and unaccountable. Do we trust that editors always get great reviewers? It seems unlikely that they will be able to get many eminent thinkers to devote their time to sifting through the huge slush pile of articles generated by RAE pressure in the UK. Judging by the huge quantity of tedious and poorly written drivel that gets through the process of peer review and into print in philosophy journals, something isn 't working here.
Obviously if the reviews contain good arguments and criticisms these can be judged more or less independently of their source. But such reviews are not particularly common. Praise and acceptance from a mediocrity who doesn't completely grasp the topic is very likely as meaningless as condemnation and rejection by the same individual. But praise or rejection from someone who is a major player in the field - that might mean something. It might also sharpen up the process if named people could be held to account for their views.
Better still, why don't we bypass journals altogether and just publish all our views on the Internet as we choose. Let our peers review these after the event (if they feel like it) and in the open (or before, by private arrangement, if we are shy about making fools of ourselves). That might re-invigorate the subject and the ways in which we express ourselves.
A programme that just went out on BBC Radio 4,'Peer Review in the Dock' and which you can listen to again here for the next 7 days, revealed how much more dangerous faith in the peer review process is in areas such as medicine. It also refers to evidence that peer review is not a reliable mark of quality...far from it.
See also some useful links in Tim Crane's comments below.
Hi Nigel
Interesting post, thanks. I agree with you that we should not fetishise peer reviewing as the 'gold standard'. There are, and have been, some excellent philosophy journals which are not run on an anonymous peer review basis. And reviewers can be as bad as you say.
However, the idea of yet more 'accountability' makes my heart sink...
The British Academy actually published a quite interesting report on the merits of peer reviewing last year:
http://www.britac.ac.uk/reports/peer-review/index.html
Getting rid of paper journals is obviously the way to go, for many reasons; getting rid of journal publishers making huge profits out of universities is even better (see the discussion a few years ago on Leiter's blog: http://leiterreports.typepad.com/blog/2006/09/time_to_end_for.html).
But this would not necessarily getting rid of journals as such -- journals do provide *some* kind of quality filter of all the stuff that is out there; and also, some kind of guide for the perplexed -- imagine trying to start finding out what one should read by using Google alone.
To my mind, the ideal would be to have open access (free) online journals edited by people who know what they are talking about. Something like peer review would have to be involved too, I think. Universities could then spend some of the money they are now spending on over-priced journals on running these online facilities.
This wouldn't solve the problem of over-production which you mention, of course. I'd like to introduce a scheme in which academics were given research grants on the condition that they did NOT publish anything in (say) ten years. If they did publish something in that period, they would have to pay a portion of the grant back (a kind of 'set aside' subsidy).
all the best
Tim
Posted by: Tim Crane | August 05, 2008 at 01:09 AM
Thanks for this Tim.
Really interesting response. Thanks for the links too. I'm amazed at how readily academics have signed away their copyright (electronic rights bundled in often) to publishers who don't pay anything for it, and who 'generously' receive camera ready copy subsidised by universities and then sell back the journals at hugely inflated prices. The claim that the publishers add kudos or act as filters is implausible in philosophy: most of the filtering and kudos is added long before it gets to the publisher. In my experience journal publishers in philosophy rarely have much say about what goes in a journal and the selection criteria for inclusion. Now some publishers are even looking to charge for access to their digital vaults of the goodies they've collected for nothing over the years...Post the digital revolution academics have the means to publish and distribute electronic journals at virtually no production costs: so what is the added value a journal publisher now gives? It isn't obvious in our subject.
You're right that if Google were the only filter for quality in a free market of ideas we would be in trouble. But already there are many informal filters. For instance, were you to put up a webpage of recommended reading on consciousness, people who recognise your authority in this area would use your reading list as a filter. Look at how Arts and Letters Daily (www.aldaily.com) works - many people use it as a filter for quality now.
Best wishes,
Nigel (aka Virtual Philosopher)
Posted by: Nigel Warburton | August 05, 2008 at 08:29 AM
I agree with Tim that the British Academy report on peer review is interesting -- and I think spot on. One recommendation in the report that may interest you if you have not yet read it is that there should be some kind of training in reviewing for graduate students. An excellent report I highly recommend.
Otherwise, I do think -- at the very least -- that the better journals more often than not perform quite well at gaining the good (and free) advice of top persons in the field. Many journals also publish lists of referees for a year, including Ethics, Journal of Moral Philosophy and many others. Such lists with lists of editorial board members (who often do more work than many colleagues are led to believe) do demonstrate the quality of the review at work.
Posted by: Thom Brooks | August 05, 2008 at 10:33 AM