Last spring when I posted on the vagaries and arbitrary nature of the peer review process for publishing journal articles and books, we had quite a conversation. Here are two articles that address peer review for journal publishing from both sides: an editor of a top journal in her field, and a professor who appreciated her helpful comments but wanted to remind editors of their responsibilities in upholding the integrity and professionalism of the process.
First, Lynn Worsham, the editor of a “quarterly journal of rhetoric, writing, culture, and politics,” gives a number of common-sense how-tos for submitting one’s work to a scholarly journal. I especially like her first point: make sure that your work is suitable for the journal in question, and (paraphrasing here) indicate that you’ve at least skimmed one or two recent issues. (I always hit the 10-year backlist of a journal heavily so that I can read and appropriately cite some of the journal’s articles I may have missed.) This jibes with my experience serving on and chairing search committees: having to evaluate totally inappropriate job applications that were clearly fired like grapeshot from a cannon, instead of tailored to address our specific job ad and qualifications. I totally get where she’s coming from. Worsham boils it all down helpfully to a list of bullet points:
Familiarize yourself with the types of articles that a journal publishes and only submit work appropriate for that journal. Pay close attention to the tone and style of work published in the journal and try to duplicate it in your own work. Follow, religiously, the style guide used by the journal. No hybrid styles! Only submit work that you believe to be final, publishable copy. A poorly proofread manuscript wastes your time and mine. Placing your work in the context of articles previously published in the journal is good scholarly practice and helps make your article a better “fit” for the journal. Follow the journal’s submission rules — exactly. Develop a healthy attitude toward rejection. You know from the outset that competition is fierce, so maintain a positive attitude.
Next, Kevin Brown in “What Professors Want from Journal Editors and Peer Reviewers,” compliments Worsham’s article, but addresses the frustrations on the other side. For the most part, his righteous complaints boil down to lengthy delays in the peer review process, and editors who don’t communicate with authors about the status of their manuscripts. But perhaps above all, he warns editors and manuscript reviewers to respond with constructive criticism instead of nastiness when rejecting articles. He writes, “[W]e know that editors will decide, for whatever reason, that our submissions should not be published in their journals. However, that does not give them license to insult either our work or us. In speaking to friends and colleagues, we all have horror stories about responses from editors and readers that are nothing more than ad hominem attacks or a dismissal of ideas because of the readers’ particular view of a work.”
He then relates a personal brush with evil, in the form of an anonymous (natch!) peer review:
This type of response can be especially problematic for graduate students and professors just beginning in a field. When I was in graduate school, I submitted an essay on Edith Wharton to a journal. The essay was the best one I had ever written, as far as I could tell, and I was eager to begin participating in what I hoped would be my future discipline. I attended a college, though, where professors never discussed publishing, so I had no knowledge of it before I entered graduate school. Not surprisingly, the journal turned down the essay and rightly so, as it was certainly not the caliber of writing that editors should expect. However, the response has stuck with me for years, as the reader simply wrote, “This is a good essay, for an undergraduate.” When I tell that to most people, they are surprised that I stayed in the profession and that I ever submitted anything again.
Ouch! That “undergraduate” comment was entirely gratuitous. What an a$$hole. Brown continues:
As professors we are not afraid of a healthy debate about ideas, and we seek honest feedback on our work. However, insults, whether directed at those ideas or at us personally, have no place in the critical debate. We would never allow our students to write essays using some of the responses I have seen from readers, nor would we write those comments on our students’ papers.
* * * * * * * * * * * *
What professors truly want is constructive feedback that will make them better writers, thinkers and researchers. If, especially in our early days, we have somehow overlooked a seminal work (or a work that a reader at least believes is seminal), or have faulty logic, then, please, tell us so, but do so in an effort to make us and, therefore, the discipline, stronger.
UPDATE, this morning: Um, Kevin Brown (and everyone)–please avoid use of the adjective “seminal.” It’s just, well, eeeewww.
Historiann will once again renew her call for all peer reviewers to sign their reviews rather than hide behind anonymity, although she’s sure she’s vox clamantis in deserto. (For those of you who don’t read Latin and/or didn’t go to a Bible college or Catholic school, the rough translation is “pissing up a rope.”) Anonymity on the part of reviewers is cowardly and encourages bad behavior (like that on the non peer-reviewed internets, by the way). Clearly, people write differently and give different advice if their identities are known, so cowgirl up and sign those reviews. As a very wise cowgirl once said, “[i]t’s a rigged system, but we can each make the process a little fairer and a little more transparent for each other.”
Historiann 
One other unfortunate aspect of this process, at least from the author’s perspective, is that the anonymity of peer review can lead one to incorrect conclusions about other scholars in the field. Despite the anonymity, we often (even if we try to restrain ourselves) develop suspicions about who the reviewers are and develop (irrational but nonetheless real) animosities or kind feelings towards these imagined authors.
For my book, I had strong suspicious about each of the reviewers’ identities, based on the tone and content of the reviews. When the press sent me the promotional material for the book, it contained blurbs using language drawn almost directly from the reviews that in had the effect of revealing the reviewers’ identities — two completely different scholars from the ones I had been imagining. Since the reviews were positive and useful, this was a relatively benign thing, but I’ve never quite given up my resentment of the person I imagine to be the extremely dismissive reviewer of the first article I ever submitted, even though I don’t know for sure.
Signed reviews would relieve a lot of that wasted emotional energy and speculation, and, as you point out, might force those dismissive reviewers to be more constructive and civil in their rejections.
LikeLike
JJO–welcome, and thanks for stopping by to comment! You’re entirely right–I hadn’t even considered that it would assist in preserving one’s mental health and relationships with colleagues. Great point!
LikeLike
I submitted an article to one of the top journals in my field last March.
I’m still waiting…
LikeLike
A 6+ month lapse is more than enough excuse to write to ask for an update on the status of your ms., I think.
LikeLike
Just a comment or two: I just (anonymously, I think) reviewed a book ms for a press (my turn-around was six weeks, as requested by the press) and I am frankly glad it was anonymous. I thought the manuscript needed a lot of work, and while I tried to say that clearly and nicely, it’s a tricky rhetorical situation: partly because I felt I was working for the press, and not for the author. If I had been communicating directly with the author, I believe I would have written a different kind of response.
I have also reviewed works submitted anonymously to me before–for which I was also grateful when I later found the essay had been written by a friend in the field. I am sure I would have responded differently had I known who the author was.
That’s not to say that readers’ reports shouldn’t be characterized by honesty and professionalism–of course they should. But there may sometimes be good reasons (and good consequences) for anonymous reviews, too.
[A major conference I attend with some frequency recently went to anonymous review of proposals, in an effort to ensure that major, famous, or plain-old friendly scholars were not always just put on the program on the principle that “So-and-so always delivers in the end,” even when their proposals this round were weak. Anonymous submissions, I guess I’m saying, can clearly have a payoff, if it opens doors for newer scholars.]
LikeLike
I’ll take your word for it that you delievered stiff medicine in a courteous fashion, Tom. I can see where remaining anonymous (especially in a smallish field) would be attractive. It’s more the abuse of anonymity that I don’t like, rather than the anonymity itself. (I have lots of anonymous or pseudonymous commenters at Historiann.com that I don’t know–at least I don’t think I know them–but so long as they follow the rules for commenting, all are welcome here!)
I hear what you’re saying about working for the press, not the author–but when readers reports are shared with the author and the author is expected to respond to your reader’s report and perhaps even to incorporate your suggestions and advice, well, then I think it’s more complicated than that you’re just working for the press. You are asked to render a final judgment to the press, but the context you provide for that judgment is very much for the author, it seems to me.
I think reviewing conference proposals anonymously is a good idea if you’re trying to combat cronyism or open up a conference to new participants with new ideas. This is smaller-stakes stuff than article and book reviewing–at least I’ve never heard of someone being denied tenure because ze didn’t get into a particular conference, but the number-one reason people don’t get tenure is an insufficient publication record.
LikeLike
This is a complex issue, but I lean toward agreeing with Tom on the anonymity question. It was developed for a series of basically legitimate reasons, and the fact that a tool can be abused is not generally thought sufficient to abolish that tool. The question would be balancing the pluses against the minuses from making a change. Reviewer self-disclosure is usually there as an option, and one that I’ve intermittently taken. I’m not sure, though, that for the personality type that for whatever reason just has to snark-out every point they choose to make, identification would remedy the ill. It might just make for more subtle snarkiness. And for some I could see it even energizing the abuse by giving the reader a bigger perceived stake in the contest. I think that abusive reviewers do coincide with those folks who somehow contrive to see every intellectual transaction as a form of competition.
This is an area where editors themselves might be asked to patrol more assertively; not that it’s easy to imagine a hard-pressed editor “rejecting” a reader’s report because it was too abrasive and then having to round up another. But maybe this should occasionally happen. I do think in the net it would be much harder than it already is to recruit stables of readers if abandonment of anonymity was a requirement rather than an option. The blind-review system developed in the sciences, where the supposed standard was whether a finding was “true” (as in replicable) more than whether reader A “liked” writer B’s “work.” I think the translation to the humanities has never fully worked.
If “stakes” or scope of impact are the guiding line, should we also consider abandoning what effectively amounts to anonymous search committees (save for the occasional chair named in the advert)? They don’t make “reports” to the applicant, as such, but the weight of their decisions exceeds those of publications and conferences put together. In these cases, too, the universe from which they can be drawn is almost always available information, so the point JJO makes about attributed sources of negative judgements and lasting ill will goes double. And the “grapeshot” apps. that Historiann cites may result as much from inept efforts to divine audience as about other things. Job ads are often models of useful (to the employer) ambiguity.
Maybe it should be transparency in or transparency out, categorically, across the entire range of judgements and decisions made in the industry at large? Just wondering, and hopefully neither pissing up nor down-rope!:}
LikeLike
I used to be the managing editor of a journal, which meant I was the go-between for the board, the reviewers and the authors. One of our board members was wonderful about editing reviewers comments. Any nasty or unnecessary comments were taken out of the reviews. I would retype reviews that were sent as a hard copy and just cut and paste from electronic copies. I thought this was a good policy, although not all of the board members practiced this policy. I had one friend send in an article which was butchered by a reviewer (it was the first time my friend submitted to a journal) and they asked me if this was typical of the process. Unfortunately, I had to say “yes it is.”
LikeLike
I think anonymous submission and review are one of the best parts of this profession. Unlike job searches and many other exchanges where reputation, school attended, connections, frienships, etc. can influence the outcome, anonymous review keeps the focus on the work. Too bad some people use it as an opportunity to be rude and belittle people. I think sitting for long periods (reading, writing, archiving) makes people cranky.
In my recent reviews I have tried to be very generous with suggestions for revision — to the point where perhaps I was too generous. That is because I am concerned that it is the work of a graduate student or beginning assistant prof. So in this case anonymity works in the other direction.
Liz raises an important question. Why didn’t an editor throw out that review that said “for an undergraduate”? I don’t think reviews should be edited, but they could be discounted if they do not meet the standards of a review.
LikeLike
Indyanna and Liz–you’re right that journals should try to strip out the nasty and/or unhelpful comments, whether or not they’re anonymous. It’s so easy now, with everything being sent digitally–no actual cutting and pasting or magic-marker redaction required.
The other reason for reviewers to sign their reviews, aside from serving as an exercise in restraint, is that it reveals the reviewer’s biases because of their training, research interests, etc. I’ve written reviews in which I’ve said that the author should play up the women’s and gender history in the article and explain the possibilities, but I think it’s also important to explain that that’s just what I do, so that they’re free to accept my brilliant advice, or decline it as tangential to their interests. (But perhaps I’m too modest. People hostile to feminist analysis have absolutely no shame about expressing their disgust for what they see as a trivial subfield in their anonymous reviews–maybe I’m putting my field at a disadvantage by not presuming that everyone should be writing women’s history and pretending that I speak as the authoritative voice of Godly erudition?)
And, Rad: I’m only suggesting that the review be single-blind, with the author unknown to the reader but the reader unmasked for the author. And yes, that line about the undergraduate should have been excised, clearly!
LikeLike
As I think I said in the spring, I increasingly often reveal my identity in reviews. But not always. I once reviewed a ms. of a book by someone who was senior in the field, and it was really second rate. Well, I was going to have to see this person again, probably; I was happy to make suggestions — and like Rad Readr I think I went beyond expectations in making positive suggestions — but not to sign my name.
As it happens, I don’t think the author ever revised the ms. satisfactorily — I did not want to review the revised one, but it’s never come out. (Which was too bad, because it could have been really interesting.)
Which is to say: negative but encouraging reviews are hard to write, and they would be even more difficult to write if you knew you would be identified. Not all of us are in big fields . . .
LikeLike
Personally, I am grateful for anonymous review. As a referee, I can express my honest opinions of papers. A lot of the papers I get to referee are of low quality and do not deserve to be published. (I am a theoretical physicist, so there is a more or less objective standard for papers being wrong.) Yet in many cases, they are authored by people that I know, who I have cordial personal relationships with. I don’t want to damage my relationships with these people, and I don’t want to antagonize anyone in my field. Most authors probably take it in stride when I point out fatal errors in their work, but I’d rather not rely on their goodwill.
I am an untenured junior faculty member. If providing candid feedback on bad papers could put my career at risk, I wouldn’t do it. Yet the physics community needs people to provide just that feedback, to weed out the rubbish. Anonymity is important to getting the best quality reports.
Lest you get the wrong idea, I do not revel in writing rude negative reports. When papers are borderline, I try very hard to help the authors out. Many errors are potentially fixable, and if I have any ideas how they might be fixed, I supply them. Sadly, they very rarely take such suggestions to heart; the authors are usually too wedded to their erroneous ideas to make more than a half-hearted effort to fix the problems.
LikeLike
The editor has to be the key figure here, if not in editing out extraneous or eggregious commentary (and that revelation from Liz made me a bit nervous, though in truth this is not “work” that we have any real ownership in), then in helping the author to address, contextualize, or if necessary even ignore cranky feedback. It’s interesting to reflect on how we “read” the veiled author behind the manuscript. I’ve on one or more occasions buffered a criticism with a side note to the editor that I thought the author was probably a neophyte scholar, only to be told that the author was no such thing. Conversely, the nicest thing a reader ever said about me–that the author in question was “obviously an old pro”–was as nuts as it was nice, since I was neither old, nor really yet a pro at that point. But you never forget a misreading like that!
LikeLike
Signing reviews is a great thing.
I had an article rejected from the top journal in my field and the reviewers all signed. That meant I knew their work, etc., and instead of wondering I was able to realize: of course these reviewers would say these things. They are writing from their known points of view.
For my infamous book, the first set of reviewers were anonymous but I could tell who they were. One was very negative and the editor thought he must have a personal animus against me. I said no: he’s read the manuscript seriously, and he has found the problems in it I have with it, it isn’t personal (although he knows me), he’s actually speaking seriously, which is one way of showing true respect. The editor, amazed, thought I was being masochistic or something, but I disagree.
LikeLike
I should clairfy that the editorial board of the journal I worked for regularly edited reviews – they edited them mostly for what should be fixed. For example, if a reviewer suggested the person include everything the reviewer had ever written and the editor didn’t think that was appropriate or necessary for the revision of the article, we cut that request out of the review. Having received reviews myself now, I realize that I can easily just decide to include some things and not others and justify what I fixed in my letter to the editors. However, we also had a big name person furious with us and withdraw their manuscript because a reviewer was rude in their review.
I guess my point is that you’re damned if you don’t and damned if you do. Editing a journal is on some levels a thankless task. You get blamed if peer reviewers don’t turn in the reviews on time. You get blamed if every person you ask refuses to review the manuscript so it takes forever to get it reviewed. If you send out nasty reviews authors are angry, if you edit the reviews authors get nervous…Seriously, this is an unpaid job in many cases. Unless you are running the AHR or other topic journal, you are editing the journal in addition to your other work. (And for the record, the editors of the AHR still have to teach at least 1-2 classes a year!)
LikeLike
I see this issue within a longer debate about the nature of expertise. Twenty years ago, journals sent manuscripts with a cover letter thanking the reviewer, setting a deadline, stating the journal’s editorial mission and policy, and offering some general guidelines. What I receive now is a list of specific questions that are to guide my analysis. Not a few instruct the reviewer to be nice. (It seems to me that this allows an editor to reject a nasty and useless review, and to edit a nasty but useful one.)
I’ve been in the humanities business long enough to know that such lists explicate what was earlier assumed. And that begs the question not only of what critical skills are being taught in graduate school and refined in one’s career, but what is being taught about professional courtesy in a profession increasingly defined as a competitive rather than a mutual enterprise. What has happened in the humanities (or academia, generally) that has required that editors remind reviewers of their scholarly and professional obligations? I work in an interdisciplinary field that is now more multidisciplinary–that is, there are now more fences than gates, and it’s difficult to find a collective willingness to jump those fences or even share in the effort of taking them down. Easier to dismiss perceived competitors or degrade what one doesn’t understand. Those commentators attempting to unmask negative reviewers are falling into the trap caused by competition (insecurity, positioning one’s self against or with someone rather than with a set of ideas or a school of thought, the “academic star” phenomenon). Presumably, the same submission earned support during its production from one’s colleagues, from positive responses when offered at a conference, or from the other journal reviewer. So why (like that lone nasty student evaluation) allow another’s poor etiquette or understanding or enmity toward an interpretation sway one’s opinion of his or her own work?
I daresay that another reason why editors began to remind reviewers to behave is because of the entrance into the profession in the U.S. of more women, persons of color, and foreign-born individuals—and with them, new fields and epistemologies. (By the way, any new approach makes it difficult for journal editors to find appropriate readers.) In theory, anonymous review allows both the reviewer and the “reviewee” the luxury of just being scholars discussing ideas. In practice, it has helped persons who may have faced or endured marginalization to enter a discipline’s or field’s discussion. If I feel I may be useful to an author as s/he revises a work, I tell the editor (if s/he hasn’t asked) to reveal my name if the author wishes to contact me for further discussion. Journal editors are more than judges or arbiters; they facilitate the discussions and scholarship that define fields and disciplines. Nevertheless, many editors have been restricted from having any vision for their respective journals because its community of potential and actual contributors are vying for acceptance—hence, the proliferation of even more journals in the last twenty or thirty years. We haven’t yet achieved that new model of scholarship (and its representation in books and journals, essays and reviews) introduced by the democratization of the academy in the last half of the twentieth century.
LikeLike
Liz–I really like your last paragraph. At the Berks this summer, I talked to the editor of a major journal in my field, and she said basically what you said, namely, that it’s thankless work, everyone feels slighted and/or pissed off, and no one appreciates it. So, guess what’s now last on my list of professional contributions I have yet to make? Mmmm-hmmmm.
And, Susan, Buzz, and Indyanna–you all make good points about the usefulness of anonymous peer review. So long as the system isn’t a cover for bad behavior, then it’s fine with me. I take Buzz’s point especially about being a junior person reviewing a senior person’s work–that hadn’t occured to me as a possibility, mostly because I think it’s fairly atypical in history for a junior person to review someone much more senior.
And Prof. Z–your experience with the journal article rejection is exactly what I’m getting at when I say that signing a review allows the author to understand the (unavoidable) biases that all reviewers bring to their work. (You have an “infamous” book? How exciting…!)
LikeLike
And History Maven–welcome back, and thanks for your compelling comments once again.
I agree with most of what you say except this:
I think it’s much more complicated than that. Feminist scholarship is still for the most part practiced by women. I have had my work treated nastily and dismissively by antifeminist readers because of its theoretical perspective, and I believe the majority of women’s historians have similar stories of stupid NEH grant reviews and readers’ reports like this. In my experience, the double-blind review process clearly worked so that hostile readers could engage in a$$hattery under the cover on anonymity, and they could be pretty certain that they were engaging in this aggression against a junior woman scholar. I guess what I’m saying is that identity is frequently (although not always) tied to the subjects we choose and the theoretical perspectives we employ, so the marginalization of those perspectives can be enabled by the blind review process, not limited by it.
LikeLike
Yours is an excellent point, Historiann. I certainly have been the recipient of pointed, anti-feminist reviewers (especially when I submit to disciplinary, rather than interdisciplinary or field-specific journals), but those were tempered by editors’ remarks expressing concern about tone and the lack of willingness to engage as a scholar another’s premises and politics. One colleague told me that her NEH grant application was rejected by one reviewer who asked the question “Why do we need another work on Anne Hutchinson?” and went on to dismiss the application. It’s hard to image the same question being raised about the Mathers (like Lincoln scholarship, its own cottage industry) in an NEH review round. It’s hard to imagine, too, that this one, content-less review led to the rejection (although it certainly didn’t help). And it’s just plain unprofessional and unethical to make those sorts of dismissive statements.
Anonymity may invite a reviewer’s baser biases, but I’ve witnessed no shortage of similar, full-on attacks at conferences and seminars. (I’ve been on the receiving end of those. I’ve seen big-name scholars cry.) Those folks are going to inscribe those biases in reviews whether they sign their names or not.
My larger point was that the same scholars who may receive unfair or hostile reviews are entering and changing the system because their insights and expertise are being tapped–and protected–as well. I have only anecdotal evidence of proving this positive, if only because of the rule of anonymity (all the informal discussions of “a recent journal submission I read”). One could only imagine what may happen if an angry author, knowing the names of the reviewers, thought to retaliate. (I was involved in one case where an author attacked a book reviewer without going through the journal’s letters page. Scary.) We, all the more, should appreciate excellent editors all the more when we realize that they foster scholarship and, through anonymous peer review, bring new perspectives into scholarship. If that isn’t happening, and if there’s a disjuncture between the editor’s vision, the diversity of a given editorial board (usually created with inclusiveness in mind), the profile of reviewers, and what appears on a journal’s pages, then those endowed with the power to shape scholarly discussion must bring to their annual editorial board meetings an agenda to discuss a real ethical dilemma. I have to say that I don’t know of any journal that has taken the time to make such an assessment.
Oh, I sound like a schoolmarm there. Sorry! (Class dismissed.)
LikeLike
Pingback: Peer Review 2.0, revised and updated « info-fetishist
Pingback: Hug an Editor Day: Journal of the History of Sexuality : Historiann : History and sexual politics, 1492 to the present