"Pennywise the Clown, combined with el chupacabra, wrapped in the Blair Witch."
Read the whole story
Read the whole story
Empirical analysis tells Reviewer 2: “Go F‘ Yourself”
- Thread starter JournalBot
- Start date
I hadn't thought about it that way. I found the headline and sub headline (whatever that's called) confusing, and then read the article twice. Really easy to understand when the context is clear, but without a good context going in and with the headline definitely not helping, this wasn't the easiest read.
But to your point, I read this article twice because of it. Most articles I don't read even once! So yes, very effective technique for getting the article read
Upvote
8
(9
/
-1)
I've always approached my reviews as objectively as possible... but to be honest, I never paid attention to what reviewer number I was until I got the editor response email.
Upvote
6
(6
/
0)
The question is: how many reviewers reviewed this paper?
Reviewers #2 and #3 — We blame Reviewer#1, he's too accommodating….
Reviewer #1 is the worst because he/she knows the paper is really bad, but it is letting the other reviewers take the hit.
Upvote
7
(7
/
0)
The question is: how many reviewers reviewed this paper?
Reviewers #2 and #3 — We blame Reviewer#1, he's too accommodating….
Reviewer #1 is the worst because he/she knows the paper is really bad, but it is letting the other reviewers take the hit.
You might be more correct than you think: https://www.wired.com/story/peer-review ... their-job/
Upvote
6
(6
/
0)
I've known some surprisingly reasonable editors in the past when one referee has been a major outlier. (Perhaps not that surprising, as many of them were also authors once, or indeed still are).
But otherwise yes...a year between finishing a manuscript and seeing it accepted somewhere is not unusual. And the worst ballache (other than, you know, actually getting the data and writing the paper) is reformatting it for a different journal. A few journals in the life sciences take a reasonable approach to this, others not so much.
But otherwise yes...a year between finishing a manuscript and seeing it accepted somewhere is not unusual. And the worst ballache (other than, you know, actually getting the data and writing the paper) is reformatting it for a different journal. A few journals in the life sciences take a reasonable approach to this, others not so much.
Upvote
4
(4
/
0)
A little inside perspective on how this came about. You've got about 70 characters (the title) to say what the story's about, and another 80 in the sub-title to add some elaboration or clarification for the title. We test two headlines, and i usually like to make one clever and one a bit more blandly descriptive to give the readers a chance to decide which is more compelling for a given story.
I hadn't thought about it that way. I found the headline and sub headline (whatever that's called) confusing, and then read the article twice. Really easy to understand when the context is clear, but without a good context going in and with the headline definitely not helping, this wasn't the easiest read.
But to your point, I read this article twice because of it. Most articles I don't read even once! So yes, very effective technique for getting the article read
So, 70 for what this story's about, when it's about peer review, scientific publishing, one rather amusing journal editor, the hassles of academic research... It was clear i wasn't going to summarize everything, so i tried to capture that there was a bit of juicy analysis wrapped in a crunchy coating of nuttiness. There was a more staid version that did more or less the same thing ("Why is your academic career faltering? New paper blames reviewer 3"), but it was obvious before this was published that people were going to find "Go F' yourself" the better choice.
So, that left 80 characters to elaborate. My first thought was to try to describe the over-the-top language of the paper, but it's always better to have actual examples rather than just ask you to take my word that it's pretty far out there. Typically with the subheads, i write things how i'd normally phrase them, and edit down to the character limit. Since i wanted a quote and a few of my own words, i started sampling quotes to see how many additional characters i'd have for my own words if i used it.
The chupacabra line and two flanking quotation marks was precisely 80 characters. I figured it was a sign from the gods that this was meant to be, so i skipped including any of my own words.
Upvote
48
(48
/
0)
IMHO that was a solid triple. Maybe an inside the park home run. Of course I read every science article ars publishes so there is that. I have to admit, having never been in the actual process I was unaware of the phenomenon but a close friend is a recovering botanist and journal editor, and reviewer.... and he has described some of the issues for me. Just not this particular one. I had to send a link to the paper to him as soon as I saw it, with the subject line Not reviewer 2, I hope....A little inside perspective on how this came about. You've got about 70 characters (the title) to say what the story's about, and another 80 in the sub-title to add some elaboration or clarification for the title. We test two headlines, and i usually like to make one clever and one a bit more blandly descriptive to give the readers a chance to decide which is more compelling for a given story.
I hadn't thought about it that way. I found the headline and sub headline (whatever that's called) confusing, and then read the article twice. Really easy to understand when the context is clear, but without a good context going in and with the headline definitely not helping, this wasn't the easiest read.
But to your point, I read this article twice because of it. Most articles I don't read even once! So yes, very effective technique for getting the article read
So, 70 for what this story's about, when it's about peer review, scientific publishing, one rather amusing journal editor, the hassles of academic research... It was clear i wasn't going to summarize everything, so i tried to capture that there was a bit of juicy analysis wrapped in a crunchy coating of nuttiness. There was a more staid version that did more or less the same thing ("Why is your academic career faltering? New paper blames reviewer 3"), but it was obvious before this was published that people were going to find "Go F' yourself" the better choice.
So, that left 80 characters to elaborate. My first thought was to try to describe the over-the-top language of the paper, but it's always better to have actual examples rather than just ask you to take my word that it's pretty far out there. Typically with the subheads, i write things how i'd normally phrase them, and edit down to the character limit. Since i wanted a quote and a few of my own words, i started sampling quotes to see how many additional characters i'd have for my own words if i used it.
The chupacabra line and two flanking quotation marks was precisely 80 characters. I figured it was a sign from the gods that this was meant to be, so i skipped including any of my own words.
Upvote
10
(10
/
0)
I remember the good old days when the journals had copy editors, and a submitted paper could come back a lot better written than it went, especially if one submitted to the British journals. Nowadays, one of the jobs of the referee is copy editor, and as you observe, for the low price of free.
Upvote
18
(18
/
0)
For me it was always Reviewed #2. Reviewer #1 provided a fairly reasonable and insightful critique and suggestions were useful and appropriate. Reviewer #3 was often overly positive, almost as if they were just happy the research was being done or anticipated the results of Reviewer #2 and were trying to offer encouragement. Reviewer #2 though... criticisms indicating they didn't read the dang article, lacked a fundamental understanding of the topic, and just wanted to tear someone down. Ugh. Reviewer #2 was always my nemesis!
Upvote
7
(7
/
0)
Upvote
10
(11
/
-1)
Upvote
11
(11
/
0)
Like some other Ars readers, I suspect, I've been on both sides of the fence (as author and reviewer) and I think peer review is definitely broken. If I had my way I'd make two changes immediately:
1) Blind reviewers to authors' names and institutional affiliations to reduce biases
2) Do *not* blind authors to the reviewers identities. Force reviewers to be accountable for the contents of their reviews.
Things have started to change, as I know some journals now *are* blinding reviewers to authors. But I don't think there's been much traction on making reviewers' identities public yet, unfortunately.
(Edited to correct typos as I notice them)
1) Blind reviewers to authors' names and institutional affiliations to reduce biases
2) Do *not* blind authors to the reviewers identities. Force reviewers to be accountable for the contents of their reviews.
Things have started to change, as I know some journals now *are* blinding reviewers to authors. But I don't think there's been much traction on making reviewers' identities public yet, unfortunately.
(Edited to correct typos as I notice them)
Upvote
19
(19
/
0)
Reviewer #1 is the toadie as either they were recommended by the submitting author or they responded immediately to the request to review.The question is: how many reviewers reviewed this paper?
Reviewers #2 and #3 — We blame Reviewer#1, he's too accommodating….
Reviewer #1 is the worst because he/she knows the paper is really bad, but it is letting the other reviewers take the hit.
Upvote
4
(4
/
0)
Misleading article title aside, finally someone gets it right!
I told you fuckers it wasn't my fault.
I told you fuckers it wasn't my fault.
Upvote
26
(26
/
0)
Upvote
21
(21
/
0)
Completely agree. If you don't already know what this all about, I daresay you'll have a difficult time understanding it from this article.
Upvote
2
(5
/
-3)
With respect to the journal I review for most frequently, I am almost always the second reviewer. The third reviewer is almost always the journal’s biostatistician, which is a good thing as I am in a field where many of the authors have little statistical training.
When I look at the reviewer comments at the decision stage, I’m also quite commonly the most supportive reviewer. I view the process of publication as part of a larger term conversation about the science and the field as a whole. I don’t expect the entire, definitive answer to be rolled up in one study. Barring dreadful errors in design (We compared the median values for each dataset to the control group previously described by Ourfriends et al (2008), using pair wise t-tests...) and obvious p-hacking, I usually work to make sure that the paper is readable, clearly states hypotheses, and is at least grounded in the body of knowledge that we have already accumulated. So I’m usually the knowledgeable copy editor, and I’m totally fine with that.
In its original form, peer review meant standing up in front of a group of your peers and presenting your thoughts, findings etc. More like the abstract presentation sessions at conferences, with robust discussion of results, methods and their implications.
When I look at the reviewer comments at the decision stage, I’m also quite commonly the most supportive reviewer. I view the process of publication as part of a larger term conversation about the science and the field as a whole. I don’t expect the entire, definitive answer to be rolled up in one study. Barring dreadful errors in design (We compared the median values for each dataset to the control group previously described by Ourfriends et al (2008), using pair wise t-tests...) and obvious p-hacking, I usually work to make sure that the paper is readable, clearly states hypotheses, and is at least grounded in the body of knowledge that we have already accumulated. So I’m usually the knowledgeable copy editor, and I’m totally fine with that.
In its original form, peer review meant standing up in front of a group of your peers and presenting your thoughts, findings etc. More like the abstract presentation sessions at conferences, with robust discussion of results, methods and their implications.
Upvote
8
(9
/
-1)
Can't speak to scientific journals, but a similar issue exists with conference talks and sessions, of which I present, organize, and sometimes review/select.
It's often hard for newer voices to get selected, due to the preference for people already well-known in the field/community (can be a type of celebrity). This may help on one hand by having recognizable names, which helps promote the event, but the downsides include creating a type of echo chamber of the same group of people rotating, which is quite off-putting to less-well-known people who have good ideas to share. As a result, there are some newcomers who decide not to deal with the hassle—ostensibly to the chagrin of organizers looking to broaden the appeal of the event.
In some of the events I'm involved with, we address these biases by stripping the author identity from submissions to review. This helps a great deal with avoiding overt biases creeping in.
Edited for clarity.
Upvote
6
(6
/
0)
That isn't clear to me either. In my field (planetary science), there are two reviewers for most journals, and the number is just based on the chronological order in which they agreed to review a paper. Sometimes, if the reviewers give very different opinions, the editor may call in a third reviewer, which would make ``Reviewer #3'' different. And some journals (including ApJ, I think) usually only have one reviewer and sometimes call in a second if iterations between the reviews and revisions aren't converging.
Upvote
4
(4
/
0)
From the article:
Put another way, Peterson wrote "Reviewer 2 is dismissive of other people's work, lazy, belligerent, and smug."
So imagine what happens when you get a bunch of people like that, assemble them into a government advisory committee and ask them to give good advice on managing a pandemic...
Put another way, Peterson wrote "Reviewer 2 is dismissive of other people's work, lazy, belligerent, and smug."
So imagine what happens when you get a bunch of people like that, assemble them into a government advisory committee and ask them to give good advice on managing a pandemic...
Upvote
7
(7
/
0)
You don't even need a bunch of people. You just need the one guy who should maybe--theoretically--listen to what advice they have.
Pity we are missing that one guy.
Upvote
6
(6
/
0)
As someone who deals with some pretty toxic chemicals on a daily basis, the image attached to this article gives me a large amount of anxiety.
Upvote
6
(6
/
0)
Some of this discussion is confusing me. It sounds like reviews in different fields work differently. In my field, you get two reviews, simultaneously, and without the reviewers having seen the other reviewer's comments. What you are describing seems to be a sequential process. One review goes over the paper and recommends some changes, then the second reviewer gets the revised paper and makes further recommendations, and then perhaps a third review gets the twice-revised paper to make further recommendations. Is that really how it works in some fields?
Upvote
2
(3
/
-1)
I was going to say that the journals to which I've submitted articles supposedly do blind reviewers to the authors; similarly when I've reviewed I had no idea who the author was. However, I'm also aware that not all journals do this and it baffles me why they don't.
Upvote
4
(4
/
0)
Again, I have to ask about the differences between different fields. In mine, you sometimes have a second author who did the actual work and a first author who just slapped his name on because he was the second author's advisor, supervisor or source of funding. But most people in my field consider that inappropriate and abusive. If the student or post-doc did the work and wrote the paper, he should the first author. Does it work differently in other fields?
Upvote
6
(6
/
0)
It's a social science paper - still I'm not sure how it got through peer review. It is based on an ordinal regression model of subjective scores, where the p values are not given and the data is "proprietary and cannot be made public". The majority of reviews recommend rejection, and that is what the model can clearly see above the error, but I dunno about any other effects. There is also some subtle confounds in the review process, which may need better models for the reviewers (if not the scores). So the analysis doesn't say me much if anything.
But maybe my problem is that I've never heard about a "Reviewer #2 effect". This part of science on science seems like a young field, and in need of more statistics.
But maybe my problem is that I've never heard about a "Reviewer #2 effect". This part of science on science seems like a young field, and in need of more statistics.
Upvote
2
(2
/
0)
That depends on how large your field and specialty is. In my case, we're talking about well under a thousand people, and based on the subject of the paper and who's presented what at conferences, it's hard to blind the reviewers to the authors' names. You can redact the names, but they are easy for a reviewer to guess. Honestly, even the figures can be a give away; the style and software used reveal the author. And the same thing replies in reverse. Although reviewers are anonymous, guessing who the reviewer is, based on the opinions and biases his comments reveal, is a common game. (And, reviewer anonymity is often waived by the reviewers, at the request of the editors, after the paper is accepted.)
Upvote
5
(5
/
0)
For a regular research paper, the first author should be the junior lab member (student, postdoc, whatever) who actually did the lion's share of the work/intellectual input, and the senior author the PI, no? I agree that things can get a bit spicy when the need for joint first authors arises amongst junior members, as it can do, not unreasonably. As for the list of authors in the middle...I'm not sure the funding agencies/hiring committees really care about that (although the authors do).
Upvote
2
(2
/
0)
That's definitely not how it works in planetary science or astronomy. There isn't any such thing as a ``senior author'' or ``joint first authors''. There is a first author, and then a list of co-authors. The list of co-authors may be short (one or two) or it can run into the dozens of names. For hiring and performance reviews, being the first author is taken more seriously, but being any one of the co-authors is also taken seriously. There are some games with long author lists. If it's clear that the first three or four co-authors are in some unclear order and then the rest are in alphabetical order, then people who notice understand that the later, alphabetically listed co-authors only made a minor contribution.
In that context, I don't understand what you mean about the junior scientist who did the lion's share of the work being the first author while the PI should be the ``senior author''. In my field, there is no such distinction.
Upvote
0
(2
/
-2)
Appealing to the editor is definitely a thing. I once had to do it when two of my reviewers (#2 and #3, incidentally) effectively got into an argument with each other through the review process. One insisted I cover a specific, important but tangentially connected issue; the other insisted I did not, as the issue is complete nonsense.
By the sixth revision - when I'd added, then removed the section twice - I emailed the editor directly. She responeded to me in less than a week, saying she'd "talked to" the reviewers (I think the choice of "to" was entirely deliberate); there would be no further review rounds and I was free to include the section or not as I wished (I did not).
Upvote
9
(9
/
0)
Since the subject of obnoxious reviews has come up, I'd like to post a famous joke from my field. Unfortunately, I can't because I can't find a way to attach an image to a comment, unless it is online. (I've only got a copy on my laptop.) So I'll just try to summarize it.
In the 1990s, the Journal of Geophysical Research was in the habit of sending Christmas cards to their regular reviewers. One editor, Chris Goertz, composed one which was based on actual reviewer comments he had received. It was one of those things where the text was formatted to take the shape of an image (in this case a Christmas tree). Some of the choice comments were things like:
``This is baloney''
``...which is a fantasy of the authors''
``I see even the authors themselves are confused''
``I know [X] well. He is smart but stubborn and getting old.''
``There is, however, a serious problem that I did not raise in my first comment''
``First of all, the analysis method is extremely primitive. There have been some advances made since the `eyeball' techniques of 1949.''
``This referee is totally confused.''
If someone cares and wants to help me figure out how to do so, I'll post the whole thing.
In the 1990s, the Journal of Geophysical Research was in the habit of sending Christmas cards to their regular reviewers. One editor, Chris Goertz, composed one which was based on actual reviewer comments he had received. It was one of those things where the text was formatted to take the shape of an image (in this case a Christmas tree). Some of the choice comments were things like:
``This is baloney''
``...which is a fantasy of the authors''
``I see even the authors themselves are confused''
``I know [X] well. He is smart but stubborn and getting old.''
``There is, however, a serious problem that I did not raise in my first comment''
``First of all, the analysis method is extremely primitive. There have been some advances made since the `eyeball' techniques of 1949.''
``This referee is totally confused.''
If someone cares and wants to help me figure out how to do so, I'll post the whole thing.
Upvote
6
(6
/
0)