Historiann has been thinking a lot about peer review lately. It seems, as in the old nursery rhyme, that peer review is like the little girl with the curl right in the middle of her forehead: When it is good, it is very very good, but when it is bad, it is horrid.
At its best, peer review helps writers avoid making dunderheaded factual errors and points them to other sources to help bolster their arguments. When it’s done by generous and intellectually engaged reviewers, it helps writers sharpen their arguments, tone down (or rev up) their prose, and to see more big-picture connections and implications of their work that even the writers couldn’t see until someone slightly more expert than they are pointed it out. What’s not to like, with a fair and humane group of supportive senior scholars freely sharing their wisdom with their (usually junior) colleagues? Furthermore, having one’s work reviewed by supportive senior scholars is a really great way of making new friends and influencing influential people. I’ve had that experience a few times–and I’m truly grateful to the people who lent their time and expertise to make me a better historian.
Well, that vision of peer review is very much an ideal, in the way that a 2-2 load at a wealthy institution with brilliant students and lots of leave time is an ideal that most of us will never know outside of our dreams. Peer review is fraught with opportunities for abuse, deception, and caprice. And, when either getting or keeping a job is on the line, that means that the misuse of peer review is not just a playful game of Chutes and Ladders. Here are some of the major problems I’ve seen firsthand or heard about from friends and acquaintances:
- Manuscript reviews at history journals and presses are rarely double-blind, so the reviewers know who the author is, but the author does not know who the reviewers are unless they choose to out themselves. Little wonder, then, that Professor Famousface’s articles are rarely rejected, and that Janey Noname, ABD, undoubtedly doesn’t get the same deference. Now, it’s frequently the case that the Professor Famousfaces, because of their greater knowledge and experience, actually write better articlesthan most of the Janey Nonames out there, but let’s not kid ourselves that these decisions are based strictly on the merits of what’s on the page. (I’ve also been a little surprised to learn that journal editors are very status-conscious: they want to publish only or mostly articles by Professor Famousfaces, so they don’t reserve a lot of room for advancing the careers of the Janey Nonames.) I always thought that taking diamonds-in-the-rough and polishing them up would be the real challenge and great joy of being an editor, and that publishing articles by the already-acclaimed was something that any chump could do. Clearly, I’ll never be a journal editor, with my screwed-up priorities!
- (An aside, but related to the point above: the first time I gave a paper at a conference with the title “Associate Professor” next to my name in the program, I was contacted by the editors of two big journals and invited to submit the revised edition of the paper to them. While I was flattered by the attention, I also thought, “where the heck were you when I was an Assistant Professor trying to get tenure, when an article in your journal would really have been important to me?” Now that I’m a made man, sealed in the temple, a member of the guild, so to speak, you’re dying to publish my incredibly brilliant and insightful work? That kind of sucks!)
- A lot of worthy work by grad students and junior faculty gets rejected because peer review, whether single- or double-blind, serves as an instrument to discipline and punish. In the hands of an unscrupulous or intellectually un-self-confident person, peer review is a means to marginalize arguments and perspectives that appear to conflict with the reviewer’s body of scholarship or that challenge conventional interpretations. In one of my fields, which is still heavily dominated by white men in the senior ranks, I have seen and heard of peer review used to suppress scholarship written by feminist women and people of color. (I’ve also heard about this in many other fields, because most of them are dominated by older white men.) And again, because jobs and tenure are on the line, peer review can be a powerful tool for maintaining status hierarchies and the intellectual homogeneity of the in-group.
- At journals, the function of peer review is extremely unclear, because journal editors are sovereign, and authors should be aware that they’ll publish what they want to publish sometimes in spite of what the reviewers recommend. For the most part, I expect that journal editors who want to publish an article are sure to find friendly reviewers for it, and if they don’t want to publish an article, they are sure to find reviewers who are practically guaranteed to shoot it down. (Most editors, most of the time, are probably looking for some honest opinions to guide them–I should probably emphasize that point a bit, because this post may seem to be only about hatin’ on editors.) However, when reviewers report back unanimously (or nearly unanimously) that an article is great and should be published and the editor rejects the article anyway–what we to make of that? (This just happened to a friend of mine. Coincidentally, another friend of mine heard back from the same journal editor on an article that ze had submitted. This second friend received a very warm and friendly “revise and resubmit” letter from the editor, despite the fact that 2 out of 3 reviews that came back to hir recommended against publication, and one of the negative reviews was extremely short, nasty, and unhelpful. As they say in the military: Whiskey Tango Foxtrot? Oh, and I have to say that both letters explaining both of these, shall we say, eccentric decisions were so opaque, turgid, and contorted that I now have serious reservations about this editor’s judgment about what good writing looks like. The letters looked like the product of a very disordered mind.)
- Journals also seem to have no shared rules or system of peer review. Some journals I’ve published in have showed me the readers’ reports (which is what I always thought was the industry standard for peer review.) Other journals never showed me the readers’ reports, and instead sent me only an editor’s letter of acceptance with hir suggestions for revision. When I asked to see the readers’ reports, for the purposes of satisfying my senior colleagues that this was in fact a peer-reviewed article, I was told by this editor that “we just don’t do that.” Sing it with me: Whiskey Tango Foxtrot? Wev. It was an established journal in one of my fields more than eighty years old, so people just accepted that that article was legitimately peer-reviewed, but to this day I have no idea if any “peers” actually “reviewed” anything. I had another article published in a journal that is accepted as a peer-reviewed journal that never did more than copy-edit my work, so far as I know.
- From what I’ve heard from others, book publishers can be just as capricious, and they certainly are even more clearly sovereign than journal editors. (Historiann loves her editor and publisher, and is always raving about hir/that outfit.) Interestingly, I found publishing my book to be a much more straightforward and less mysterious process than publishing in journals, and so have most (but not all) of my friends.
So, should we all give up and just publish on the non-peer reviewed, open-source internets? Realistically, that’s only an option if you’re already tenured and don’t really care if you ever get another promotion. I guess what I’ve learned in my first decade of working with journal editors and a publisher is that you have to choose your venue for publication carefully. You have to grow a thick skin, because there is a significant minority of creeps who hide behind peer review to suppress, rather than foster, new scholarship. (Cowards.) And it helps to have a lot of friends–at the very least, they can console you when you get a negative verdict or review. Oh, and never do an anonymous review–I always sign my name and invite the author to contact me if ze has any questions. Mostly I do this because it forces me to make constructive criticism. I’m not the voice of God–I’m an individual that was trained in a particular way and have my own axes to grind, and the author should know that so that ze can put my advice in context. It’s a rigged system, but we can each make the process a little fairer and a little more transparent for each other.
What lessons have you all learned–whether you wanted to learn them or not?
Historiann 
It’s great that you sign your name to reviews. We should all write only what we are willing to own publicly.
All of the issues that you raise make good sense to me. Of course, the people who give out nasty, negative reviews don’t imagine themselves as standing in the way of somebody’s career or new knowledge. They think that they are “keeping up standards.”
LikeLike
This is a very interesting post, Historiann, and I agree with you about the potential for abuse in the peer-review system. I’ve had some experiences like the ones you’ve cited, and I would have to say that the book-publishing process has been a little less fulfilling for me than it appears to have been for you. And that process, along with the perils of peer review, has got me thinking about why we publish anything at all. Obviously, getting tenure is important, and metaphysically, I write history because I can’t think of what my life would be like without it. Those aren’t insignificant reasons. But when some book publishers are basically saying, “we’re thrilled to be publishing your book, but we really don’t think we’ll be able to sell any copies at all,” it’s a little depressing and hard to explain to outsiders why I’m so committed to such a selfish (because hardly anyone’s reading what I write) *and* unremunerative (because big advances/sales are a pipe dream) hobby. I’ll keep on keeping on, but to quote Dreiser’s Hurstwood, “what’s the use?”
LikeLike
Ahh, yes–“keeping up standards!” The last refuge of the scoundrel. One can also keep up standards by encouraging a junior scholar to read a few more books and cite more evidence in a constructive review, but it’s so much more satisfying for people with poorly integrated egos and no social skills to say, “Nope. You can’t join our club.!”
And regis–maybe you should skip the books and go directly to screenplay. Seriously, there are a lot of other non-history writers out there writing worthless crap that no one will read or ever see in movies or TV. Look on the bright side: a few grad students might one day be coerced into reading your book! So long as your books and articles please you, don’t harm anyone, and help you make your living doing what you want to do, then what’s the harm? And, what would you do otherwise if you weren’t always hunched over a computer? Develop a moderate drug habit? Take up Iron Man Triathalons? Become a mad scrap-booker?
LikeLike
All very good points historiann. I’ve noticed that the press I’m working with currently freely submitted my name with the manuscript to the reviewers, but has not reciprocated with the reviewers’ identities.
Another huge problem in this process is the time turn around. I once submitted an article and waited nearly 8 months for reader reviews (which I did actually get). The piece was published nearly a year and half after I originally submitted it. With another journal the initial response took even longer, with a terse email from the editor rejecting the piece without any further comment. Good times in our profession!
LikeLike
Hey, habitual historian–thanks for visiting and commenting. An idea I didn’t put in this post–because it is so dogawfully long already–is that we do away with this etiquette of submitting an article to only one journal at a time, and having to sit on our hands while we wait for the verdict. Why shouldn’t we submit the same thing simultaneously to 2 or 3 journals, and say that the first one who says “yes” gets it? (There are problems with this system–notably that reviewers might get irritated at being asked to review the same article 3 times–but that doesn’t seem so burdensome. If you’ve read it once, you can send off the same review again to another journal, with a few thoughts as to how the article does or doesn’t fit into the journal in quesiton.)
The fact is that this false etiquette is just a way of preserving the status quo of power relations. I can’t wait to see (or hear) of someone who gets an article acceptance 8 months later like you did, and says, “oh, sorry, the editor of Better Than You journal has already accepted this article. She told me last week–I was going to write you today to tell you that I was withdrawing my submission. Sorry!”
LikeLike
Hi Historiann. Thanks for writing another great post.
While preparing for my comprehensive exams I submitted an article to an OK journal in my field. I received three glowing reader reviews and a very positive letter from the editor urging me to “revise and resubmit.”
I followed most of the readers’ suggestions and resubmitted the article. I then received a nice rejection letter from the editor. One helpful and positive reader review, and another nasty reader review. The nasty reviewer signed hir name. The nasty reviewer recently finished a dissertation in my field and has only published a book review. This reader’s comments were written in the style of an “academic hit.” The young scholar’s tone suggested that s/he wished to knock me off until s/he gets published.
I stewed for a few days and resolved to send the article out again. I mailed it away yesterday to another journal. Since I thrive on conflict, it feels good to have an imagined academic nemesis. 🙂
LikeLike
Wow, Ortho–it’s a pretty agressive maneuver for someone to sign a nasty review when ze hasn’t published an article hirself yet. (What was the editor thinking, asking hir to do the review? I should think that the minimum qualification would be that ze had published an article!) Pretty bold, too, considering that you will likely be invited to review hir articles and book, either before or after publication. (Please remember this, dear readers: petty cruelties will be revisited upon your heads, and sometimes magnified!)
The scenario you describe happened to me almost exactly once upon a time. But, the journal I published it in was extremely grateful that other “peers” had reviewed it so thoroughly and had pushed me to do such a thorough and careful revision. It was accepted immediately, and in print in less than a year after getting rejected by the first journal. It still sucked–and (I will admit, not proudly) I am still honked off about the rejection, but I got the article published and that article did what I needed it to do in getting me a better job.
LikeLike
Pingback: More on why “peer review” isn’t code for “awesome” « info-fetishist
Historiann, the philosophy people have started up an interesting wiki:
http://wikihost.org/wikis/philjinfo/wiki/start
I don’t know if literature or history people have done the same yet, but it’s a nice way of demystifying a tiny bit of the publishing process by using open wiki technology.
They have listed their experiences with turnaround time and the outcomes of rejection or acceptance, along with any comments or horror stories, and thus, reading across all the journals, one can start learning about these journals’ publishing track records —- who is hopelessly behind, who is run by someone mean or incompetent, who takes a year and a half to respond to anything, who leans on articles with too heavy an editorial hand, etc. It’s a way for the philosophy grads who started it to learn something about both the quality of the journal and of the applicant treatment.
This wouldn’t solve any of the problems you list, of course, but it would solve other problems and give a tiny bit of control back to the applicant, which helps so much.
LikeLike
Sis–thanks so much. I agree with the spirit of the philosophy publication wiki very much, and admire the determination of junior people to demystify the process and help each other out. There are all kinds of history job wikis, which started out a year and a half ago as something positive, but this year they degenerated into some paranoid rants that had the effect of discouraging people from sharing information. (I posted on this a few months ago here, at http://www.historiann.com/2008/01/15/the-war-between-the-states-of-employment/)
But, article publication is not nearly as high-stakes as jobs, so I wish them luck, as I hope the experiment works as you suggest.
LikeLike
Nice post, Historiann. I’m wondering if the nasty review is a product of what I see as a tendency in some graduate programs to encourage students to trash other people’s work rather than engage in thoughtful, constructive criticism. Also, this person may have lousy role models. Still no defense, though.
As to slow turn around time — this is just sheer poor prior planning. If someone can’t read and review an article within the time frame set by the editor, s/he shouldn’t agree to do it in the first place.
LikeLike
KC–yes, agreed about the poor planning. Especially because reviewing an article takes what–1-3 hours? (And the upper time limit is only for the truly obsessive.)
But, take someone who has a poorly integrated ego and is already on a power trip–making someone wait 8 months to get their work shredded makes a perverse kind of sense, doesn’t it? (Fortunately, that’s only a minority of reviewers, but those experiences surely stick out in one’s mind.)
LikeLike
While in graduate school I worked for three years on the editorial staff of a prominent scholarly journal. Not only were we underpaid and understaffed, we were underfunded by the sponsoring professional organization and university. Nevertheless, it was a terrific gig–great training for my comps and for my career.
We rarely rejected a manuscript without sending it, through a double-blind process, to scholars. In the era before email, we used postcards to ask potential reviewers for their permission to send on the manuscript. We received between 250 and 300 manuscripts a year. Members of the editorial board were used as experts but also as third or fourth reviewers if we had conflicting opinions. Sometimes we simply could not find reviewers who were willing to take the work, and that takes time–how long, for example, do you wait for a response from a potential reviewer before you again begin again the process of matching reviewer to manuscript? And we gave reviewers six weeks in which to complete the task–yes, it may take 1-3 hours, but I’ve also had reviewers who took more time to fact-check, to read up again this theorist or that, etc., etc. (As a reviewer, I do this, too.) And taking on a review is, though generally expected, an unplanned addition to one’s schedule. I’ve declined to review books; I won’t decline reviewing manuscripts in my field. It’s a professional obligation of another, important sort.
Back to my journal editing experience: The editor was fantastic and democratic–two great reviews and the editor’s endorsement meant publication. We published a LOT of junior faculty members’ work. That meant, of course, that we had the proper year’s backlog. I still find it amazing that junior colleagues are angered, dismayed, or shocked when they learn that there’s a lag time in publishing. I don’t recall that anyone called our office to ask that question, though many called during the review process because they needed an answer about a nearing tenure application deadline. A quarterly journal has, basically 16-24 slots to fill annually; a special issue may reduce that number.
The journal was eventually transfered to another institution; the new editor instituted another editorial board to oversee decisions (keep count: two editorial boards). I have to say I disagreed with the decision: it adds a layer of decision making that may sully the product, in that board members have votes but no knowledge of the field or approach of a given essay, or may require changes for publication that dilutes the final essay. The year after the journal changed hands I received calls from colleagues asking me why, after two positive readers’ reports, the journal’s editorial board was hesitant to commit to publication or were asking for more changes than necessary. Given increasing scholarly interdisciplinarity, this is a real problem.
Such a structure (and I should add that editorial boards were greatly expanded in the 1980s, at a time when publishers sought income through the acquisition of scholarly journals) is fraught with problems–the primary one is that it weakens scholarship. Given the temperament and/or expertise and/or experience of the editor, the addition of another level of “peer” review (usually 12 board members) has undermined in various ways and to varying degrees the power of editors to make decisions, to guide a discipline or field, and to be an honest scholarly broker. Another colleague received two conflicting reviews; rather than seek a third reviewer or make a decision, the editor asked the colleague what to do. That’s a weak editor.
As for abuse: I admit that we did keep an informal list of scholars who were nasty, inept, careless, or habitually late in their reviews. The journal itself had to maintain its integrity and reputation, and we took seriously that charge.
We also learned to prepare for the onslaught of manuscripts in September, and tried mightily to get readers for any manuscript arriving in April–in the summer months it was pretty impossible to find readers and get reviews. Winter break, too, at times delayed reports.
We always sent the reader’s reports, though we did redact at times nasty comments, such as “this had to be written by a graduate student” when we knew it was senior scholar. Journal editors are stewards as well as brokers; all authors should receive those reports.
LikeLike
Ortho, watch out for those nemeses… sometimes they stick around for years:
http://squadratomagico.blogspot.com/2007/05/story-of-my-nemesis.html
Historiann, your proposal to shift towards multiple submissions of a single manuscript sounds dreamy, in some ways, though I imagine that in the long run it might have unintended consequences. It’s a lot more processing of mss, to begin with, since every journal would then be flooded with many times more submissions, and ever ms, would be processed multiple times. It’s creating a lot more work around the same number of actual scholarly productions. This, in turn, would further slow down the publication timeline, strain journal resources (not high to begin with) and potentially create room for lots of weird abuses on the part of “star” writers whose works were wanted in many places, and who don’t have enough grace to behave well. While I’ll admit that my immediate response to that proposal was to find it very appealing, in the bigger picture I think it might create more problems than it solves.
LikeLike
Interestingly, I’ve never been asked to review something where the author was identified — it’s always been double-blind. But I have in the past few years started to routinely say that I can be identified to the author. In one case it led to a really nice relationship with a junior scholar.
On the other hand, when I was in grad school back in the dark ages, I submitted an article to a prestigious journal (on the advice of a prestigious scholar with ties to the people at said journal). I got back a really snotty rejection, which made it pretty clear no one had read the article.
THe other thing about peer review is that it works against unconventional approaches to a subject. Or it does if not managed generously. The best advice I ever got was that when you got a rejection, just send it out: a friend suggested that you wait until you get three rejections before you spend time revising!
LikeLike
history maven–thanks so much for stopping by to leave such an informative comment. You give a great overview of the whole process, and you’re very fair to the good-faith actors! I should say, re: your comments and Susan’s comments, that upon reflection I have been part of double-blind review processes for journals as a reviewer. (Who knows what happened in the case of the journals that never sent me readers’ reports, though!) The single-blind reviews I’ve done were for book manuscripts and book proposals, not for journal articles. (I wonder why the difference?)
Another issue you brought up history maven, which I considered addressing but the post was getting long, is the growth and proliferation of the “special issue.” It feels like the number of articles published by any journal anymore is about half (or less than half) what it used to be. This practice again raises questions in my mind about the difference between publishing an article in a book versus in a journal, when these “special issues” are edited by a guest editor who undoubtedly has in most cases solicited manuscripts from friends and acquaintances, and who probably has a greater degree of liberty than editors who are abiding by the standard review process. (When I was a junior faculty member and first started publishing 11 years ago, senior faculty used to make a big deal over the difference between articles in books and articles in journals, but I think people have settled down and now accept them as equivalent.) Still, special issues seem to take up a lot of space that otherwise went to unsolicited articles.
And Sqadratomagico: you’re entirely right about my utopian author-friendly scheme! But, maybe it would make journals sharper about which articles they sent out to readers, and which they rejected out of hand. I think journals need to develop a new “prejection letter” saying, “thanks for your article, but we’re not really interested in stuff in this field right now.” (I think some journals do this.) That would seem to be more honest than putting an article through the review process when an editor clearly wasn’t interested in publishing an article. (Although, with the existence of e-mail, it would seem odd I suppose for someone to send an article somewhere that was wildly off-base. I’ve had good luck e-mailing editors and asking if they have any interest whatsoever in an article before sending it to them–sometimes you get a politely tepid letter encouraging you to send it on, but other times you get a stronger response, either way!)
Susan–a friend of mine was a student of Judith Bennett, and your “best advice” was her advice, too: she told my friend to have 3 copies of the article in addressed envelopes ready to go, so that if the first journal you sent it to rejected you, you send it on to the next journal unrevised and continue the process uninterrupted! (Was it Judith Bennett who gave you that advice!) I liked the moxie–“screw you if you don’t like my article, someone else surely will!”
LikeLike
Pingback: Peer Review: editors versus authors smackdown edition : Historiann : History and sexual politics, 1492 to the present
Pingback: Peer Review 2.0 (2.0) - Ontario Library Association Superconference « info-fetishist
Wow! I am currently a PhD candidate in history and have been publishing since my MA days, which were not that long ago. I started with local and regional journals first, then moved on to more prestigious and national ones. I found it easy to publish in peer-reviewed journal at the local historical society because, well, I could walk into their offices and sell the story. However, the national journals have been tougher to crack. First time I submitted to one I got a rejection in three weeks! And this was after the article had been revised at least ten times! Second time I submitted to a journal outside my home state I was told that my approach was “too sociological.”
Well, hopefully, third time will be the charm. I actually met the third editor in person and spoke with him for an hour at a conference I presented at. In addition, I had him read the article first before I formally submitted to the journal. He liked it so I gave it a shot. Finally, I had sent this article out for my own peer review process to several top scholars in my field, Jewish American history; most agreed that it was insightful and more than worthy of being published. Surprise, surprise! I emailed one of those scholars a few days ago with my latest work and he know that he wrote a “very, very positive readers report. This guy is a big deal so I hope the journal listens to him!
I will let you all know what becomes of it. But even at this very early stage in the game, and it is a game, I know not to give up.
LikeLike
Pingback: Hug an Editor Day: Journal of the History of Sexuality : Historiann : History and sexual politics, 1492 to the present
Update: The article was accepted for publication with minor revisions. Not bad for a first year PhD student! I have another article out for review and I will keep everyone posted. But I have a good feeling. Thank you for this blog.
LikeLike
I just stumbled on this while I should be working….
I find it frustrating that peer review does not allow me to request clarification from reviewers…..I’ve had reviewers make negative comments that don’t make sense. I think there should be a system in place in which you can request for a reviewers to restate a portion of their review if it if unclear.
I’ve found it difficult to dissect a few “word salads” that I have gotten back from reviewers.
LikeLike
Silly Wabbit: I feel your pain! This is another reason I always sign my reviews. It give the authors an opportunity (if they want it) to correspond with me about my review.
I think you’re well within your rights to correspond with the editor to ask for hir clarification, and/or to ask hir to forward your questions on to the reviewers. Then ze can leave it up to the reviewers as to whether or not they want to be in touch directly with you, or perhaps just convey the response through the editor.
But, disciplinary conventions vary–in my field this would be ok, but I would consult friends and colleagues in your field.
LikeLike
Pingback: From Peer Review to the Wisdom of Crowds? Open Access & Peer Review | History Workshop
Pingback: The IPCC: Bogus data on "Climate Sensitivity"!!! - US Message Board - Political Discussion Forum
“There are problems with this system–notably that reviewers might get irritated at being asked to review the same article 3 times–but that doesn’t seem so burdensome. If you’ve read it once, you can send off the same review again to another journal, with a few thoughts as to how the article does or doesn’t fit into the journal in quesiton.)”
I know this isn’t your (good) main point here, but I think I’d describe this problem differently. The victims, such as they are, are not repeat readers but rather the reviewers whose reviews end up being wasted because “their” journal is not the first one to accept. Judging from the complaints of my editor friends, the ecosystem wouldn’t survive if for some reason they collectively had to find 2 or 3 times as many reviewers for each article submitted. Then add to that the fear of being moot would further discourage reviewers from participating.
LikeLike
I just found this posting when a friend pointed me to it. It raises some interesting issues. While I am not in the field of history (I was in folklore studies before making a jump to artificial intelligence, but that is another story), I think I can also contribute something from the perspective of journal editors.
First, very often journal editors know what sort of review a piece will get in advance. They receive many contributions and they have to read them to know where to send them. Serious problems are usually apparent from the beginning. In other cases the submission may be perfectly good but the journal represents the wrong audience for it (e.g., it emphasizes certain perspectives and the authors didn’t do their homework before submitting something out of range for the journal).
In such cases it does nobody a service to send those pieces out for a lengthy review process just to confirm that the piece is unpublishable in that journal. Now of course this statement presumes that the editors are acting in good faith and actually can detect bad/misguided papers reliably. There will certainly be a certain number of false negatives—pieces rejected that should have been accepted—at this stage (as there will be for a full peer review process), but eliminating most true negatives reduces the burden spent on making sure that pieces that actually have a chance can be given proper attention.
In the case of the journal I worked at we would take these early negatives and decide how to approach them. In many cases we took a nurturing approach and gave preliminary feedback to the authors telling them what would be required to obtain a full review and consideration for publication. In other cases it was clear that the authors would not stand a chance with their articles in our journal, so we felt it a better service to deliver a polite rejection stating that the article was not a good fit for our journal.
Now we were very friendly and took many submissions, especially by those coming from outside the U.S. and western Europe, and worked extensively with authors even before peer review to help them submit something that would make it through peer review. I realize this approach is far from common, but in many cases we had authors who just needed a little help to navigate writing for an American audience and framing their arguments in appropriate terms. We also took some degree of pride in introducing non-Western perspectives to our audience.
For those that did go out for review, we were very careful in how we handled them. Many, many times we discarded reviews that were clearly biased or unfair. And rather than just handing over the comments, we would synthesize them into actionable summaries for the authors. So if we go “word soup”, we saw it as our responsibility to fix them for the authors. Often we would put authors in touch with reviewers where it made sense to do so (always with permission from both sides).
I suspect we were not typical in any of this. That being said, I think peer review can work, but it requires the commitment of the editors to be fair and to respect both sides and mediate between them.
Because of that experience, I try very hard in doing reviews to do two things:
(1) If there are factual problems, I point them out and, if relevant, supply references. For example, in a recent review some grad students were trying to pass statistically insignificant results off as positive and I had to explain to them why their results were statistically indistinguishable from noise. It won’t be pleasant for them to read, but because I took the time to do the statistical analysis they didn’t, they now know what they need to do. And if they don’t want to do it, I gave them guidance on how to rewrite their finding as negative findings (which would actually be quite valuable in the area).
(2) If there are problems I have with the theoretical perspective, I am careful to label those as my *preference* and offer criticisms that I think will help strengthen the article, even if I disagree with it. After all, tearing down helps nobody. So I try to build up, even those things I disagree with.
So peer review can work, but it requires good faith by all parties and editors willing to intervene to ensure that happens. If peer review fails, as it clearly does in many cases, it is ultimately the editors who are responsible.
LikeLike
Pingback: It’s Beginning to Look a Lot Like Monday Links | Gerry Canavan